2023-12-11T17:23:34.508 INFO:root:teuthology version: 0.0.1.dev216+g5f3731e 2023-12-11T17:23:34.508 DEBUG:teuthology.run:Teuthology command: teuthology --description fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/no 5-workunit/suites/blogbench}} --name rishabh-2023-12-11_15:37:57-fs-rishabh-2023dec11-testing-default-smithi --verbose --owner scheduled_rishabh@teuthology --archive /home/teuthworker/archive/rishabh-2023-12-11_15:37:57-fs-rishabh-2023dec11-testing-default-smithi/7487144 -- /home/teuthworker/archive/rishabh-2023-12-11_15:37:57-fs-rishabh-2023dec11-testing-default-smithi/7487144/orig.config.yaml 2023-12-11T17:23:34.563 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-11T17:23:34.657 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2023-12-11_15:37:57-fs-rishabh-2023dec11-testing-default-smithi/7487144 branch: rishabh-2023dec11 description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/no 5-workunit/suites/blogbench}} email: null first_in_suite: false job_id: '7487144' kernel: client: sha1: distro flavor: default kdb: true sha1: cd68252d7b4d7802f3cae7f5945b38e9b20dad4c ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: rishabh-2023-12-11_15:37:57-fs-rishabh-2023dec11-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: rishabh-2023dec11 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding sha1: a62fda73efba5520d409bb5a21bd8373289dda5d subvols: create: 2 subvol_options: --size 25000000000 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount - mds.dir_split install: ceph: flavor: default sha1: a62fda73efba5520d409bb5a21bd8373289dda5d kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=secure - wsync syntax: v1 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: rishabh-2023dec11 sha1: a62fda73efba5520d409bb5a21bd8373289dda5d owner: scheduled_rishabh@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 5791 sha1: a62fda73efba5520d409bb5a21bd8373289dda5d sleep_before_teardown: 0 subset: 209/512 suite: fs suite_branch: rishabh-2023dec11 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_a62fda73efba5520d409bb5a21bd8373289dda5d/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: a62fda73efba5520d409bb5a21bd8373289dda5d syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi059.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= smithi115.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= smithi181.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= tasks: - pexec: all: - sudo cp /etc/containers/registries.conf /etc/containers/registries.conf.backup - sudo dnf -y module reset container-tools - sudo dnf -y module install container-tools:rhel8 --allowerasing --nobest - sudo cp /etc/containers/registries.conf.backup /etc/containers/registries.conf - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - kclient: null - check-counter: {} - exec: mon.a: - ceph mgr module enable snap_schedule - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - workunit: clients: all: - suites/blogbench.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73 timestamp: 2023-12-11_15:37:57 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.232157 2023-12-11T17:23:34.657 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_a62fda73efba5520d409bb5a21bd8373289dda5d/qa; will attempt to use it 2023-12-11T17:23:34.658 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_a62fda73efba5520d409bb5a21bd8373289dda5d/qa/tasks 2023-12-11T17:23:34.658 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-12-11T17:23:34.660 INFO:teuthology.task.internal:Checking packages... 2023-12-11T17:23:34.684 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash 'a62fda73efba5520d409bb5a21bd8373289dda5d' 2023-12-11T17:23:34.684 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-12-11T17:23:34.684 INFO:teuthology.packaging:ref: None 2023-12-11T17:23:34.684 INFO:teuthology.packaging:tag: None 2023-12-11T17:23:34.684 INFO:teuthology.packaging:branch: rishabh-2023dec11 2023-12-11T17:23:34.684 INFO:teuthology.packaging:sha1: a62fda73efba5520d409bb5a21bd8373289dda5d 2023-12-11T17:23:34.684 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=rishabh-2023dec11 2023-12-11T17:23:34.906 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-119.ga62fda73 2023-12-11T17:23:34.908 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-12-11T17:23:34.915 INFO:teuthology.task.internal:no buildpackages task found 2023-12-11T17:23:34.915 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-12-11T17:23:34.936 INFO:teuthology.task.internal:Saving configuration 2023-12-11T17:23:34.956 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-12-11T17:23:34.962 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-12-11T17:23:34.988 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi059.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-12-11_15:37:57-fs-rishabh-2023dec11-testing-default-smithi/7487144', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-11 17:18:22.640935', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-12-11T17:23:35.013 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi115.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-12-11_15:37:57-fs-rishabh-2023dec11-testing-default-smithi/7487144', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-11 17:18:22.643994', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-12-11T17:23:35.035 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi181.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-12-11_15:37:57-fs-rishabh-2023dec11-testing-default-smithi/7487144', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-11 17:18:22.642543', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-12-11T17:23:35.036 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-12-11T17:23:35.042 INFO:teuthology.task.internal:roles: ubuntu@smithi059.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2023-12-11T17:23:35.042 INFO:teuthology.task.internal:roles: ubuntu@smithi115.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2023-12-11T17:23:35.042 INFO:teuthology.task.internal:roles: ubuntu@smithi181.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2023-12-11T17:23:35.043 INFO:teuthology.run_tasks:Running task console_log... 2023-12-11T17:23:35.157 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f0245123e50>, signals=[15]) 2023-12-11T17:23:35.158 INFO:teuthology.run_tasks:Running task internal.connect... 2023-12-11T17:23:35.164 INFO:teuthology.task.internal:Opening connections... 2023-12-11T17:23:35.164 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi059.front.sepia.ceph.com 2023-12-11T17:23:35.165 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi059.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-11T17:23:35.234 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi115.front.sepia.ceph.com 2023-12-11T17:23:35.235 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi115.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-11T17:23:35.306 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi181.front.sepia.ceph.com 2023-12-11T17:23:35.307 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-11T17:23:35.378 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-12-11T17:23:35.386 DEBUG:teuthology.orchestra.run.smithi059:> uname -m 2023-12-11T17:23:35.408 INFO:teuthology.orchestra.run.smithi059.stdout:x86_64 2023-12-11T17:23:35.408 DEBUG:teuthology.orchestra.run.smithi059:> cat /etc/os-release 2023-12-11T17:23:35.469 INFO:teuthology.orchestra.run.smithi059.stdout:NAME="Red Hat Enterprise Linux" 2023-12-11T17:23:35.469 INFO:teuthology.orchestra.run.smithi059.stdout:VERSION="8.6 (Ootpa)" 2023-12-11T17:23:35.469 INFO:teuthology.orchestra.run.smithi059.stdout:ID="rhel" 2023-12-11T17:23:35.469 INFO:teuthology.orchestra.run.smithi059.stdout:ID_LIKE="fedora" 2023-12-11T17:23:35.469 INFO:teuthology.orchestra.run.smithi059.stdout:VERSION_ID="8.6" 2023-12-11T17:23:35.469 INFO:teuthology.orchestra.run.smithi059.stdout:PLATFORM_ID="platform:el8" 2023-12-11T17:23:35.469 INFO:teuthology.orchestra.run.smithi059.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-12-11T17:23:35.469 INFO:teuthology.orchestra.run.smithi059.stdout:ANSI_COLOR="0;31" 2023-12-11T17:23:35.469 INFO:teuthology.orchestra.run.smithi059.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-12-11T17:23:35.469 INFO:teuthology.orchestra.run.smithi059.stdout:HOME_URL="https://www.redhat.com/" 2023-12-11T17:23:35.469 INFO:teuthology.orchestra.run.smithi059.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-12-11T17:23:35.470 INFO:teuthology.orchestra.run.smithi059.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-12-11T17:23:35.470 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-12-11T17:23:35.470 INFO:teuthology.orchestra.run.smithi059.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-12-11T17:23:35.470 INFO:teuthology.orchestra.run.smithi059.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-12-11T17:23:35.470 INFO:teuthology.orchestra.run.smithi059.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-12-11T17:23:35.470 INFO:teuthology.orchestra.run.smithi059.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-12-11T17:23:35.471 INFO:teuthology.lock.ops:Updating smithi059.front.sepia.ceph.com on lock server 2023-12-11T17:23:35.499 DEBUG:teuthology.orchestra.run.smithi115:> uname -m 2023-12-11T17:23:35.519 INFO:teuthology.orchestra.run.smithi115.stdout:x86_64 2023-12-11T17:23:35.520 DEBUG:teuthology.orchestra.run.smithi115:> cat /etc/os-release 2023-12-11T17:23:35.582 INFO:teuthology.orchestra.run.smithi115.stdout:NAME="Red Hat Enterprise Linux" 2023-12-11T17:23:35.583 INFO:teuthology.orchestra.run.smithi115.stdout:VERSION="8.6 (Ootpa)" 2023-12-11T17:23:35.583 INFO:teuthology.orchestra.run.smithi115.stdout:ID="rhel" 2023-12-11T17:23:35.583 INFO:teuthology.orchestra.run.smithi115.stdout:ID_LIKE="fedora" 2023-12-11T17:23:35.583 INFO:teuthology.orchestra.run.smithi115.stdout:VERSION_ID="8.6" 2023-12-11T17:23:35.583 INFO:teuthology.orchestra.run.smithi115.stdout:PLATFORM_ID="platform:el8" 2023-12-11T17:23:35.583 INFO:teuthology.orchestra.run.smithi115.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-12-11T17:23:35.583 INFO:teuthology.orchestra.run.smithi115.stdout:ANSI_COLOR="0;31" 2023-12-11T17:23:35.583 INFO:teuthology.orchestra.run.smithi115.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-12-11T17:23:35.583 INFO:teuthology.orchestra.run.smithi115.stdout:HOME_URL="https://www.redhat.com/" 2023-12-11T17:23:35.583 INFO:teuthology.orchestra.run.smithi115.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-12-11T17:23:35.583 INFO:teuthology.orchestra.run.smithi115.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-12-11T17:23:35.583 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-11T17:23:35.583 INFO:teuthology.orchestra.run.smithi115.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-12-11T17:23:35.583 INFO:teuthology.orchestra.run.smithi115.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-12-11T17:23:35.584 INFO:teuthology.orchestra.run.smithi115.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-12-11T17:23:35.584 INFO:teuthology.orchestra.run.smithi115.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-12-11T17:23:35.584 INFO:teuthology.lock.ops:Updating smithi115.front.sepia.ceph.com on lock server 2023-12-11T17:23:35.607 DEBUG:teuthology.orchestra.run.smithi181:> uname -m 2023-12-11T17:23:35.627 INFO:teuthology.orchestra.run.smithi181.stdout:x86_64 2023-12-11T17:23:35.627 DEBUG:teuthology.orchestra.run.smithi181:> cat /etc/os-release 2023-12-11T17:23:35.690 INFO:teuthology.orchestra.run.smithi181.stdout:NAME="Red Hat Enterprise Linux" 2023-12-11T17:23:35.690 INFO:teuthology.orchestra.run.smithi181.stdout:VERSION="8.6 (Ootpa)" 2023-12-11T17:23:35.691 INFO:teuthology.orchestra.run.smithi181.stdout:ID="rhel" 2023-12-11T17:23:35.691 INFO:teuthology.orchestra.run.smithi181.stdout:ID_LIKE="fedora" 2023-12-11T17:23:35.691 INFO:teuthology.orchestra.run.smithi181.stdout:VERSION_ID="8.6" 2023-12-11T17:23:35.691 INFO:teuthology.orchestra.run.smithi181.stdout:PLATFORM_ID="platform:el8" 2023-12-11T17:23:35.691 INFO:teuthology.orchestra.run.smithi181.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-12-11T17:23:35.691 INFO:teuthology.orchestra.run.smithi181.stdout:ANSI_COLOR="0;31" 2023-12-11T17:23:35.691 INFO:teuthology.orchestra.run.smithi181.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-12-11T17:23:35.691 INFO:teuthology.orchestra.run.smithi181.stdout:HOME_URL="https://www.redhat.com/" 2023-12-11T17:23:35.691 INFO:teuthology.orchestra.run.smithi181.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-12-11T17:23:35.691 INFO:teuthology.orchestra.run.smithi181.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-12-11T17:23:35.691 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-12-11T17:23:35.691 INFO:teuthology.orchestra.run.smithi181.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-12-11T17:23:35.691 INFO:teuthology.orchestra.run.smithi181.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-12-11T17:23:35.691 INFO:teuthology.orchestra.run.smithi181.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-12-11T17:23:35.692 INFO:teuthology.orchestra.run.smithi181.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-12-11T17:23:35.692 INFO:teuthology.lock.ops:Updating smithi181.front.sepia.ceph.com on lock server 2023-12-11T17:23:35.718 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-12-11T17:23:35.730 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-12-11T17:23:35.735 INFO:teuthology.task.internal:Checking for old test directory... 2023-12-11T17:23:35.735 DEBUG:teuthology.orchestra.run.smithi059:> test '!' -e /home/ubuntu/cephtest 2023-12-11T17:23:35.737 DEBUG:teuthology.orchestra.run.smithi115:> test '!' -e /home/ubuntu/cephtest 2023-12-11T17:23:35.740 DEBUG:teuthology.orchestra.run.smithi181:> test '!' -e /home/ubuntu/cephtest 2023-12-11T17:23:35.758 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-12-11T17:23:35.764 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-12-11T17:23:35.765 DEBUG:teuthology.orchestra.run.smithi059:> test -z $(ls -A /var/lib/ceph) 2023-12-11T17:23:35.798 DEBUG:teuthology.orchestra.run.smithi115:> test -z $(ls -A /var/lib/ceph) 2023-12-11T17:23:35.802 DEBUG:teuthology.orchestra.run.smithi181:> test -z $(ls -A /var/lib/ceph) 2023-12-11T17:23:35.824 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-12-11T17:23:35.903 INFO:teuthology.run_tasks:Running task kernel... 2023-12-11T17:23:35.917 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': True, 'sha1': 'cd68252d7b4d7802f3cae7f5945b38e9b20dad4c'} 2023-12-11T17:23:35.917 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2023-12-11T17:23:35.917 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2023-12-11T17:23:35.918 DEBUG:teuthology.orchestra.run.smithi059:> test -f /run/.containerenv -o -f /.dockerenv 2023-12-11T17:23:35.936 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-11T17:23:35.937 DEBUG:teuthology.orchestra.run.smithi059:> uname -r 2023-12-11T17:23:35.997 INFO:teuthology.orchestra.run.smithi059.stdout:4.18.0-372.9.1.el8.x86_64 2023-12-11T17:23:35.997 INFO:teuthology.task.kernel:Running kernel on smithi059: 4.18.0-372.9.1.el8.x86_64 2023-12-11T17:23:35.997 DEBUG:teuthology.orchestra.run.smithi059:> sudo yum install -y kernel 2023-12-11T17:23:37.229 INFO:teuthology.orchestra.run.smithi059.stdout:Updating Subscription Management repositories. 2023-12-11T17:23:37.229 INFO:teuthology.orchestra.run.smithi059.stdout:Unable to read consumer identity 2023-12-11T17:23:38.560 INFO:teuthology.orchestra.run.smithi059.stdout:Copr repo for python3-asyncssh owned by ceph 25 kB/s | 3.5 kB 00:00 2023-12-11T17:23:39.386 INFO:teuthology.orchestra.run.smithi059.stdout:Extra Packages for Enterprise Linux 20 MB/s | 16 MB 00:00 2023-12-11T17:23:43.103 INFO:teuthology.orchestra.run.smithi059.stdout:lab-extras 366 kB/s | 24 kB 00:00 2023-12-11T17:23:43.964 INFO:teuthology.orchestra.run.smithi059.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2023-12-11T17:23:44.015 INFO:teuthology.orchestra.run.smithi059.stdout:Dependencies resolved. 2023-12-11T17:23:44.016 INFO:teuthology.orchestra.run.smithi059.stdout:Nothing to do. 2023-12-11T17:23:44.016 INFO:teuthology.orchestra.run.smithi059.stdout:Complete! 2023-12-11T17:23:44.123 DEBUG:teuthology.orchestra.run.smithi059:> echo no | sudo yum reinstall kernel || true 2023-12-11T17:23:44.471 INFO:teuthology.orchestra.run.smithi059.stdout:Updating Subscription Management repositories. 2023-12-11T17:23:44.471 INFO:teuthology.orchestra.run.smithi059.stdout:Unable to read consumer identity 2023-12-11T17:23:44.745 INFO:teuthology.orchestra.run.smithi059.stdout:Last metadata expiration check: 0:00:01 ago on Mon 11 Dec 2023 05:23:43 PM UTC. 2023-12-11T17:23:44.790 INFO:teuthology.orchestra.run.smithi059.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-12-11T17:23:44.820 INFO:teuthology.orchestra.run.smithi059.stderr:Error: No packages marked for reinstall. 2023-12-11T17:23:44.868 DEBUG:teuthology.orchestra.run.smithi059:> sudo yum reinstall -y kernel || true 2023-12-11T17:23:45.215 INFO:teuthology.orchestra.run.smithi059.stdout:Updating Subscription Management repositories. 2023-12-11T17:23:45.215 INFO:teuthology.orchestra.run.smithi059.stdout:Unable to read consumer identity 2023-12-11T17:23:45.493 INFO:teuthology.orchestra.run.smithi059.stdout:Last metadata expiration check: 0:00:02 ago on Mon 11 Dec 2023 05:23:43 PM UTC. 2023-12-11T17:23:45.542 INFO:teuthology.orchestra.run.smithi059.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-12-11T17:23:45.567 INFO:teuthology.orchestra.run.smithi059.stderr:Error: No packages marked for reinstall. 2023-12-11T17:23:45.619 DEBUG:teuthology.orchestra.run.smithi059:> rpm -q kernel | sort -rV | head -n 1 2023-12-11T17:23:45.702 INFO:teuthology.orchestra.run.smithi059.stdout:kernel-4.18.0-372.9.1.el8.x86_64 2023-12-11T17:23:45.703 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-372.9.1.el8.x86_64 2023-12-11T17:23:45.703 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2023-12-11T17:23:45.703 DEBUG:teuthology.parallel:result is None 2023-12-11T17:23:45.703 INFO:teuthology.run_tasks:Running task internal.base... 2023-12-11T17:23:45.712 INFO:teuthology.task.internal:Creating test directory... 2023-12-11T17:23:45.713 DEBUG:teuthology.orchestra.run.smithi059:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-12-11T17:23:45.715 DEBUG:teuthology.orchestra.run.smithi115:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-12-11T17:23:45.717 DEBUG:teuthology.orchestra.run.smithi181:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-12-11T17:23:45.737 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-12-11T17:23:45.748 INFO:teuthology.run_tasks:Running task internal.archive... 2023-12-11T17:23:45.762 INFO:teuthology.task.internal:Creating archive directory... 2023-12-11T17:23:45.763 DEBUG:teuthology.orchestra.run.smithi059:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-12-11T17:23:45.771 DEBUG:teuthology.orchestra.run.smithi115:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-12-11T17:23:45.778 DEBUG:teuthology.orchestra.run.smithi181:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-12-11T17:23:45.819 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-12-11T17:23:45.826 INFO:teuthology.task.internal:Enabling coredump saving... 2023-12-11T17:23:45.826 DEBUG:teuthology.orchestra.run.smithi115:> test -f /run/.containerenv -o -f /.dockerenv 2023-12-11T17:23:45.866 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-11T17:23:45.866 DEBUG:teuthology.orchestra.run.smithi181:> test -f /run/.containerenv -o -f /.dockerenv 2023-12-11T17:23:45.885 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-11T17:23:45.886 DEBUG:teuthology.orchestra.run.smithi059:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-12-11T17:23:45.888 DEBUG:teuthology.orchestra.run.smithi115:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-12-11T17:23:45.911 DEBUG:teuthology.orchestra.run.smithi181:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-12-11T17:23:45.918 INFO:teuthology.orchestra.run.smithi059.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-12-11T17:23:45.936 INFO:teuthology.orchestra.run.smithi059.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-12-11T17:23:45.956 INFO:teuthology.orchestra.run.smithi115.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-12-11T17:23:45.966 INFO:teuthology.orchestra.run.smithi181.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-12-11T17:23:45.971 INFO:teuthology.orchestra.run.smithi115.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-12-11T17:23:45.985 INFO:teuthology.orchestra.run.smithi181.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-12-11T17:23:45.987 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-12-11T17:23:45.999 INFO:teuthology.task.internal:Configuring sudo... 2023-12-11T17:23:45.999 DEBUG:teuthology.orchestra.run.smithi059:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-12-11T17:23:46.001 DEBUG:teuthology.orchestra.run.smithi115:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-12-11T17:23:46.003 DEBUG:teuthology.orchestra.run.smithi181:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-12-11T17:23:46.057 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-12-11T17:23:46.070 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-12-11T17:23:46.071 DEBUG:teuthology.orchestra.run.smithi059:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-12-11T17:23:46.073 DEBUG:teuthology.orchestra.run.smithi115:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-12-11T17:23:46.075 DEBUG:teuthology.orchestra.run.smithi181:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-12-11T17:23:46.118 DEBUG:teuthology.orchestra.run.smithi059:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-12-11T17:23:46.171 DEBUG:teuthology.orchestra.run.smithi059:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-12-11T17:23:46.221 DEBUG:teuthology.orchestra.run.smithi059:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-12-11T17:23:46.304 DEBUG:teuthology.orchestra.run.smithi059:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-12-11T17:23:46.347 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2023-12-11T17:23:46.347 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-12-11T17:23:46.415 DEBUG:teuthology.orchestra.run.smithi115:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-12-11T17:23:46.460 DEBUG:teuthology.orchestra.run.smithi115:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-12-11T17:23:46.524 DEBUG:teuthology.orchestra.run.smithi115:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-12-11T17:23:46.568 DEBUG:teuthology.orchestra.run.smithi115:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-12-11T17:23:46.610 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-12-11T17:23:46.610 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-12-11T17:23:46.677 DEBUG:teuthology.orchestra.run.smithi181:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-12-11T17:23:46.723 DEBUG:teuthology.orchestra.run.smithi181:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-12-11T17:23:46.786 DEBUG:teuthology.orchestra.run.smithi181:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-12-11T17:23:46.832 DEBUG:teuthology.orchestra.run.smithi181:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-12-11T17:23:46.875 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-12-11T17:23:46.875 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-12-11T17:23:46.945 DEBUG:teuthology.orchestra.run.smithi059:> sudo service rsyslog restart 2023-12-11T17:23:46.947 DEBUG:teuthology.orchestra.run.smithi115:> sudo service rsyslog restart 2023-12-11T17:23:46.949 DEBUG:teuthology.orchestra.run.smithi181:> sudo service rsyslog restart 2023-12-11T17:23:47.008 INFO:teuthology.orchestra.run.smithi115.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-12-11T17:23:47.042 INFO:teuthology.orchestra.run.smithi181.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-12-11T17:23:47.063 INFO:teuthology.orchestra.run.smithi059.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-12-11T17:23:47.418 INFO:teuthology.run_tasks:Running task internal.timer... 2023-12-11T17:23:47.427 INFO:teuthology.task.internal:Starting timer... 2023-12-11T17:23:47.427 INFO:teuthology.run_tasks:Running task pcp... 2023-12-11T17:23:47.450 INFO:teuthology.run_tasks:Running task selinux... 2023-12-11T17:23:47.497 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:logrotate_t:s0']} 2023-12-11T17:23:47.498 DEBUG:teuthology.orchestra.run.smithi059:> sudo service auditd rotate 2023-12-11T17:23:47.568 INFO:teuthology.orchestra.run.smithi059.stdout:Rotating logs: 2023-12-11T17:23:47.569 DEBUG:teuthology.orchestra.run.smithi115:> sudo service auditd rotate 2023-12-11T17:23:47.627 INFO:teuthology.orchestra.run.smithi115.stdout:Rotating logs: 2023-12-11T17:23:47.628 DEBUG:teuthology.orchestra.run.smithi181:> sudo service auditd rotate 2023-12-11T17:23:47.688 INFO:teuthology.orchestra.run.smithi181.stdout:Rotating logs: 2023-12-11T17:23:47.690 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-12-11T17:23:47.691 DEBUG:teuthology.orchestra.run.smithi059:> /usr/sbin/getenforce 2023-12-11T17:23:47.717 INFO:teuthology.orchestra.run.smithi059.stdout:Permissive 2023-12-11T17:23:47.717 DEBUG:teuthology.orchestra.run.smithi115:> /usr/sbin/getenforce 2023-12-11T17:23:47.744 INFO:teuthology.orchestra.run.smithi115.stdout:Permissive 2023-12-11T17:23:47.745 DEBUG:teuthology.orchestra.run.smithi181:> /usr/sbin/getenforce 2023-12-11T17:23:47.765 INFO:teuthology.orchestra.run.smithi181.stdout:Permissive 2023-12-11T17:23:47.766 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi059.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi115.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi181.front.sepia.ceph.com': 'permissive'} 2023-12-11T17:23:47.766 DEBUG:teuthology.orchestra.run.smithi059:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-12-11T17:23:47.793 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-11T17:23:47.794 DEBUG:teuthology.orchestra.run.smithi115:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-12-11T17:23:47.821 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-11T17:23:47.822 DEBUG:teuthology.orchestra.run.smithi181:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-12-11T17:23:47.848 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-11T17:23:47.849 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-12-11T17:23:47.849 DEBUG:teuthology.orchestra.run.smithi059:> sudo /usr/sbin/setenforce permissive 2023-12-11T17:23:47.883 DEBUG:teuthology.orchestra.run.smithi115:> sudo /usr/sbin/setenforce permissive 2023-12-11T17:23:47.910 DEBUG:teuthology.orchestra.run.smithi181:> sudo /usr/sbin/setenforce permissive 2023-12-11T17:23:47.936 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-12-11T17:23:47.947 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2023-12-11T17:23:47.969 INFO:teuthology.repo_utils:Fetching main from origin 2023-12-11T17:23:48.092 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-12-11T17:23:48.109 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-12-11T17:23:48.110 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi059.front.sepia.ceph.com,smithi115.front.sepia.ceph.com,smithi181.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-12-11T17:33:20.834 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi059.front.sepia.ceph.com'), Remote(name='ubuntu@smithi115.front.sepia.ceph.com'), Remote(name='ubuntu@smithi181.front.sepia.ceph.com')] 2023-12-11T17:33:20.835 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi059.front.sepia.ceph.com' 2023-12-11T17:33:20.836 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi059.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-11T17:33:20.907 DEBUG:teuthology.orchestra.run.smithi059:> true 2023-12-11T17:33:20.981 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi059.front.sepia.ceph.com' 2023-12-11T17:33:20.981 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi115.front.sepia.ceph.com' 2023-12-11T17:33:20.982 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi115.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-11T17:33:21.051 DEBUG:teuthology.orchestra.run.smithi115:> true 2023-12-11T17:33:21.076 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi115.front.sepia.ceph.com' 2023-12-11T17:33:21.077 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi181.front.sepia.ceph.com' 2023-12-11T17:33:21.077 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-11T17:33:21.146 DEBUG:teuthology.orchestra.run.smithi181:> true 2023-12-11T17:33:21.217 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi181.front.sepia.ceph.com' 2023-12-11T17:33:21.217 INFO:teuthology.run_tasks:Running task clock... 2023-12-11T17:33:21.228 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-12-11T17:33:21.229 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-12-11T17:33:21.229 DEBUG:teuthology.orchestra.run.smithi059:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-12-11T17:33:21.231 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-12-11T17:33:21.231 DEBUG:teuthology.orchestra.run.smithi115:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-12-11T17:33:21.233 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-12-11T17:33:21.234 DEBUG:teuthology.orchestra.run.smithi181:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-12-11T17:33:21.265 INFO:teuthology.orchestra.run.smithi115.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-12-11T17:33:21.265 INFO:teuthology.orchestra.run.smithi059.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-12-11T17:33:21.280 INFO:teuthology.orchestra.run.smithi115.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-12-11T17:33:21.281 INFO:teuthology.orchestra.run.smithi059.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-12-11T17:33:21.292 INFO:teuthology.orchestra.run.smithi181.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-12-11T17:33:21.308 INFO:teuthology.orchestra.run.smithi181.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-12-11T17:33:21.349 INFO:teuthology.orchestra.run.smithi115.stderr:sudo: ntpd: command not found 2023-12-11T17:33:21.355 INFO:teuthology.orchestra.run.smithi059.stderr:sudo: ntpd: command not found 2023-12-11T17:33:21.360 INFO:teuthology.orchestra.run.smithi115.stdout:506 Cannot talk to daemon 2023-12-11T17:33:21.368 INFO:teuthology.orchestra.run.smithi059.stdout:506 Cannot talk to daemon 2023-12-11T17:33:21.375 INFO:teuthology.orchestra.run.smithi115.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-12-11T17:33:21.384 INFO:teuthology.orchestra.run.smithi059.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-12-11T17:33:21.388 INFO:teuthology.orchestra.run.smithi115.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-12-11T17:33:21.397 INFO:teuthology.orchestra.run.smithi181.stderr:sudo: ntpd: command not found 2023-12-11T17:33:21.399 INFO:teuthology.orchestra.run.smithi059.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-12-11T17:33:21.408 INFO:teuthology.orchestra.run.smithi181.stdout:506 Cannot talk to daemon 2023-12-11T17:33:21.423 INFO:teuthology.orchestra.run.smithi181.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-12-11T17:33:21.438 INFO:teuthology.orchestra.run.smithi181.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-12-11T17:33:21.467 INFO:teuthology.orchestra.run.smithi115.stderr:bash: ntpq: command not found 2023-12-11T17:33:21.470 INFO:teuthology.orchestra.run.smithi115.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-12-11T17:33:21.470 INFO:teuthology.orchestra.run.smithi115.stdout:=============================================================================== 2023-12-11T17:33:21.470 INFO:teuthology.orchestra.run.smithi115.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-11T17:33:21.470 INFO:teuthology.orchestra.run.smithi115.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-11T17:33:21.470 INFO:teuthology.orchestra.run.smithi115.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-11T17:33:21.470 INFO:teuthology.orchestra.run.smithi115.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-11T17:33:21.472 INFO:teuthology.orchestra.run.smithi059.stderr:bash: ntpq: command not found 2023-12-11T17:33:21.476 INFO:teuthology.orchestra.run.smithi059.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-12-11T17:33:21.476 INFO:teuthology.orchestra.run.smithi059.stdout:=============================================================================== 2023-12-11T17:33:21.476 INFO:teuthology.orchestra.run.smithi059.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-11T17:33:21.476 INFO:teuthology.orchestra.run.smithi059.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-11T17:33:21.476 INFO:teuthology.orchestra.run.smithi059.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-11T17:33:21.476 INFO:teuthology.orchestra.run.smithi059.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-11T17:33:21.509 INFO:teuthology.orchestra.run.smithi181.stderr:bash: ntpq: command not found 2023-12-11T17:33:21.513 INFO:teuthology.orchestra.run.smithi181.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-12-11T17:33:21.513 INFO:teuthology.orchestra.run.smithi181.stdout:=============================================================================== 2023-12-11T17:33:21.513 INFO:teuthology.orchestra.run.smithi181.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-11T17:33:21.513 INFO:teuthology.orchestra.run.smithi181.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-11T17:33:21.513 INFO:teuthology.orchestra.run.smithi181.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-11T17:33:21.513 INFO:teuthology.orchestra.run.smithi181.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-11T17:33:21.514 INFO:teuthology.run_tasks:Running task pexec... 2023-12-11T17:33:21.525 INFO:teuthology.task.pexec:Executing custom commands... 2023-12-11T17:33:21.525 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi059.front.sepia.ceph.com 2023-12-11T17:33:21.525 DEBUG:teuthology.orchestra.run.smithi059:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-12-11T17:33:21.526 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi115.front.sepia.ceph.com 2023-12-11T17:33:21.526 DEBUG:teuthology.orchestra.run.smithi115:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-12-11T17:33:21.526 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi181.front.sepia.ceph.com 2023-12-11T17:33:21.526 DEBUG:teuthology.orchestra.run.smithi181:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-12-11T17:33:21.895 INFO:teuthology.orchestra.run.smithi115.stdout:Updating Subscription Management repositories. 2023-12-11T17:33:21.918 INFO:teuthology.orchestra.run.smithi059.stdout:Updating Subscription Management repositories. 2023-12-11T17:33:21.926 INFO:teuthology.orchestra.run.smithi181.stdout:Updating Subscription Management repositories. 2023-12-11T17:33:23.836 INFO:teuthology.orchestra.run.smithi115.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 30 kB/s | 4.5 kB 00:00 2023-12-11T17:33:23.937 INFO:teuthology.orchestra.run.smithi059.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 35 kB/s | 4.1 kB 00:00 2023-12-11T17:33:23.978 INFO:teuthology.orchestra.run.smithi181.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 33 kB/s | 4.1 kB 00:00 2023-12-11T17:33:24.273 INFO:teuthology.orchestra.run.smithi115.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 25 kB/s | 4.1 kB 00:00 2023-12-11T17:33:24.291 INFO:teuthology.orchestra.run.smithi181.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 34 kB/s | 4.5 kB 00:00 2023-12-11T17:33:24.347 INFO:teuthology.orchestra.run.smithi059.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 23 kB/s | 4.5 kB 00:00 2023-12-11T17:33:26.184 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2023-12-11T17:33:26.185 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2023-12-11T17:33:26.185 INFO:teuthology.orchestra.run.smithi115.stdout: Package Architecture Version Repository Size 2023-12-11T17:33:26.185 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2023-12-11T17:33:26.185 INFO:teuthology.orchestra.run.smithi115.stdout:Resetting modules: 2023-12-11T17:33:26.185 INFO:teuthology.orchestra.run.smithi115.stdout: container-tools 2023-12-11T17:33:26.185 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-11T17:33:26.185 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2023-12-11T17:33:26.185 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2023-12-11T17:33:26.186 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-11T17:33:26.214 INFO:teuthology.orchestra.run.smithi181.stdout:Dependencies resolved. 2023-12-11T17:33:26.215 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2023-12-11T17:33:26.215 INFO:teuthology.orchestra.run.smithi181.stdout: Package Architecture Version Repository Size 2023-12-11T17:33:26.215 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2023-12-11T17:33:26.215 INFO:teuthology.orchestra.run.smithi181.stdout:Resetting modules: 2023-12-11T17:33:26.215 INFO:teuthology.orchestra.run.smithi181.stdout: container-tools 2023-12-11T17:33:26.215 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-12-11T17:33:26.216 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction Summary 2023-12-11T17:33:26.216 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2023-12-11T17:33:26.216 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-12-11T17:33:26.487 INFO:teuthology.orchestra.run.smithi059.stdout:Dependencies resolved. 2023-12-11T17:33:26.488 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2023-12-11T17:33:26.489 INFO:teuthology.orchestra.run.smithi059.stdout: Package Architecture Version Repository Size 2023-12-11T17:33:26.489 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2023-12-11T17:33:26.489 INFO:teuthology.orchestra.run.smithi059.stdout:Resetting modules: 2023-12-11T17:33:26.489 INFO:teuthology.orchestra.run.smithi059.stdout: container-tools 2023-12-11T17:33:26.489 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-12-11T17:33:26.489 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction Summary 2023-12-11T17:33:26.489 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2023-12-11T17:33:26.489 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-12-11T17:33:36.211 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2023-12-11T17:33:36.684 INFO:teuthology.orchestra.run.smithi115.stdout:Updating Subscription Management repositories. 2023-12-11T17:33:37.224 INFO:teuthology.orchestra.run.smithi181.stdout:Complete! 2023-12-11T17:33:37.688 INFO:teuthology.orchestra.run.smithi181.stdout:Updating Subscription Management repositories. 2023-12-11T17:33:37.868 INFO:teuthology.orchestra.run.smithi059.stdout:Complete! 2023-12-11T17:33:38.337 INFO:teuthology.orchestra.run.smithi059.stdout:Updating Subscription Management repositories. 2023-12-11T17:33:38.484 INFO:teuthology.orchestra.run.smithi115.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 29 kB/s | 4.5 kB 00:00 2023-12-11T17:33:38.825 INFO:teuthology.orchestra.run.smithi115.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 57 kB/s | 4.1 kB 00:00 2023-12-11T17:33:39.524 INFO:teuthology.orchestra.run.smithi181.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 33 kB/s | 4.1 kB 00:00 2023-12-11T17:33:39.830 INFO:teuthology.orchestra.run.smithi181.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 35 kB/s | 4.5 kB 00:00 2023-12-11T17:33:40.265 INFO:teuthology.orchestra.run.smithi059.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 26 kB/s | 4.1 kB 00:00 2023-12-11T17:33:40.565 INFO:teuthology.orchestra.run.smithi059.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 39 kB/s | 4.5 kB 00:00 2023-12-11T17:33:40.706 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2023-12-11T17:33:40.800 INFO:teuthology.orchestra.run.smithi115.stdout:============================================================================================================ 2023-12-11T17:33:40.801 INFO:teuthology.orchestra.run.smithi115.stdout: Package Arch Version Repository Size 2023-12-11T17:33:40.801 INFO:teuthology.orchestra.run.smithi115.stdout:============================================================================================================ 2023-12-11T17:33:40.801 INFO:teuthology.orchestra.run.smithi115.stdout:Installing group/module packages: 2023-12-11T17:33:40.801 INFO:teuthology.orchestra.run.smithi115.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-12-11T17:33:40.801 INFO:teuthology.orchestra.run.smithi115.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-12-11T17:33:40.801 INFO:teuthology.orchestra.run.smithi115.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-12-11T17:33:40.801 INFO:teuthology.orchestra.run.smithi115.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-12-11T17:33:40.801 INFO:teuthology.orchestra.run.smithi115.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-12-11T17:33:40.801 INFO:teuthology.orchestra.run.smithi115.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-12-11T17:33:40.801 INFO:teuthology.orchestra.run.smithi115.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-12-11T17:33:40.801 INFO:teuthology.orchestra.run.smithi115.stdout:Installing dependencies: 2023-12-11T17:33:40.801 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-12-11T17:33:40.801 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-12-11T17:33:40.801 INFO:teuthology.orchestra.run.smithi115.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-12-11T17:33:40.801 INFO:teuthology.orchestra.run.smithi115.stdout:Installing module profiles: 2023-12-11T17:33:40.802 INFO:teuthology.orchestra.run.smithi115.stdout: container-tools/common 2023-12-11T17:33:40.802 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-11T17:33:40.802 INFO:teuthology.orchestra.run.smithi115.stdout:Enabling module streams: 2023-12-11T17:33:40.802 INFO:teuthology.orchestra.run.smithi115.stdout: container-tools rhel8 2023-12-11T17:33:40.802 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-11T17:33:40.802 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2023-12-11T17:33:40.802 INFO:teuthology.orchestra.run.smithi115.stdout:============================================================================================================ 2023-12-11T17:33:40.802 INFO:teuthology.orchestra.run.smithi115.stdout:Install 10 Packages 2023-12-11T17:33:40.802 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-11T17:33:40.803 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 18 M 2023-12-11T17:33:40.803 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 65 M 2023-12-11T17:33:40.803 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2023-12-11T17:33:41.069 INFO:teuthology.orchestra.run.smithi115.stdout:(1/10): python3-pytoml-0.1.14-5.git7dea353.el8. 93 kB/s | 25 kB 00:00 2023-12-11T17:33:41.119 INFO:teuthology.orchestra.run.smithi115.stdout:(2/10): python3-pyxdg-0.25-16.el8.noarch.rpm 297 kB/s | 94 kB 00:00 2023-12-11T17:33:41.153 INFO:teuthology.orchestra.run.smithi115.stdout:(3/10): yajl-2.1.0-10.el8.x86_64.rpm 116 kB/s | 41 kB 00:00 2023-12-11T17:33:41.304 INFO:teuthology.orchestra.run.smithi115.stdout:(4/10): udica-0.2.6-4.module+el8.6.0+16771+28df 211 kB/s | 49 kB 00:00 2023-12-11T17:33:41.370 INFO:teuthology.orchestra.run.smithi115.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 2.0 MB/s | 524 kB 00:00 2023-12-11T17:33:41.537 INFO:teuthology.orchestra.run.smithi115.stdout:(6/10): buildah-1.26.2-1.module+el8.6.0+16771+2 21 MB/s | 8.0 MB 00:00 2023-12-11T17:33:41.637 INFO:teuthology.orchestra.run.smithi115.stdout:(7/10): python3-podman-4.0.1-1.module+el8.6.0+1 448 kB/s | 149 kB 00:00 2023-12-11T17:33:41.757 INFO:teuthology.orchestra.run.smithi181.stdout:Dependencies resolved. 2023-12-11T17:33:41.771 INFO:teuthology.orchestra.run.smithi115.stdout:(8/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 532 kB/s | 213 kB 00:00 2023-12-11T17:33:41.850 INFO:teuthology.orchestra.run.smithi181.stdout:============================================================================================================ 2023-12-11T17:33:41.850 INFO:teuthology.orchestra.run.smithi181.stdout: Package Arch Version Repository Size 2023-12-11T17:33:41.850 INFO:teuthology.orchestra.run.smithi181.stdout:============================================================================================================ 2023-12-11T17:33:41.850 INFO:teuthology.orchestra.run.smithi181.stdout:Installing group/module packages: 2023-12-11T17:33:41.850 INFO:teuthology.orchestra.run.smithi181.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-12-11T17:33:41.850 INFO:teuthology.orchestra.run.smithi181.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-12-11T17:33:41.850 INFO:teuthology.orchestra.run.smithi181.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-12-11T17:33:41.850 INFO:teuthology.orchestra.run.smithi181.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-12-11T17:33:41.850 INFO:teuthology.orchestra.run.smithi181.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-12-11T17:33:41.850 INFO:teuthology.orchestra.run.smithi181.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-12-11T17:33:41.850 INFO:teuthology.orchestra.run.smithi181.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-12-11T17:33:41.850 INFO:teuthology.orchestra.run.smithi181.stdout:Installing dependencies: 2023-12-11T17:33:41.850 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-12-11T17:33:41.851 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-12-11T17:33:41.851 INFO:teuthology.orchestra.run.smithi181.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-12-11T17:33:41.851 INFO:teuthology.orchestra.run.smithi181.stdout:Installing module profiles: 2023-12-11T17:33:41.851 INFO:teuthology.orchestra.run.smithi181.stdout: container-tools/common 2023-12-11T17:33:41.851 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-12-11T17:33:41.851 INFO:teuthology.orchestra.run.smithi181.stdout:Enabling module streams: 2023-12-11T17:33:41.851 INFO:teuthology.orchestra.run.smithi181.stdout: container-tools rhel8 2023-12-11T17:33:41.851 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-12-11T17:33:41.851 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction Summary 2023-12-11T17:33:41.851 INFO:teuthology.orchestra.run.smithi181.stdout:============================================================================================================ 2023-12-11T17:33:41.851 INFO:teuthology.orchestra.run.smithi181.stdout:Install 10 Packages 2023-12-11T17:33:41.851 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-12-11T17:33:41.852 INFO:teuthology.orchestra.run.smithi181.stdout:Total download size: 18 M 2023-12-11T17:33:41.852 INFO:teuthology.orchestra.run.smithi181.stdout:Installed size: 65 M 2023-12-11T17:33:41.852 INFO:teuthology.orchestra.run.smithi181.stdout:Downloading Packages: 2023-12-11T17:33:41.863 INFO:teuthology.orchestra.run.smithi115.stdout:(9/10): skopeo-1.9.1-1.module+el8.6.0+16771+28d 22 MB/s | 7.0 MB 00:00 2023-12-11T17:33:41.957 INFO:teuthology.orchestra.run.smithi181.stdout:(1/10): yajl-2.1.0-10.el8.x86_64.rpm 389 kB/s | 41 kB 00:00 2023-12-11T17:33:41.963 INFO:teuthology.orchestra.run.smithi115.stdout:(10/10): toolbox-0.0.99.3-0.6.module+el8.6.0+16 6.7 MB/s | 2.2 MB 00:00 2023-12-11T17:33:41.963 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2023-12-11T17:33:41.963 INFO:teuthology.orchestra.run.smithi115.stdout:Total 16 MB/s | 18 MB 00:01 2023-12-11T17:33:41.999 INFO:teuthology.orchestra.run.smithi181.stdout:(2/10): python3-pyxdg-0.25-16.el8.noarch.rpm 639 kB/s | 94 kB 00:00 2023-12-11T17:33:42.050 INFO:teuthology.orchestra.run.smithi181.stdout:(3/10): python3-pytoml-0.1.14-5.git7dea353.el8. 126 kB/s | 25 kB 00:00 2023-12-11T17:33:42.142 INFO:teuthology.orchestra.run.smithi181.stdout:(4/10): udica-0.2.6-4.module+el8.6.0+16771+28df 268 kB/s | 49 kB 00:00 2023-12-11T17:33:42.163 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2023-12-11T17:33:42.181 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2023-12-11T17:33:42.182 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2023-12-11T17:33:42.191 INFO:teuthology.orchestra.run.smithi181.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 2.7 MB/s | 524 kB 00:00 2023-12-11T17:33:42.334 INFO:teuthology.orchestra.run.smithi181.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 779 kB/s | 149 kB 00:00 2023-12-11T17:33:42.359 INFO:teuthology.orchestra.run.smithi181.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 1.2 MB/s | 213 kB 00:00 2023-12-11T17:33:42.362 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2023-12-11T17:33:42.366 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2023-12-11T17:33:42.460 INFO:teuthology.orchestra.run.smithi181.stdout:(8/10): buildah-1.26.2-1.module+el8.6.0+16771+2 19 MB/s | 8.0 MB 00:00 2023-12-11T17:33:42.610 INFO:teuthology.orchestra.run.smithi181.stdout:(9/10): skopeo-1.9.1-1.module+el8.6.0+16771+28d 25 MB/s | 7.0 MB 00:00 2023-12-11T17:33:42.668 INFO:teuthology.orchestra.run.smithi181.stdout:(10/10): toolbox-0.0.99.3-0.6.module+el8.6.0+16 7.1 MB/s | 2.2 MB 00:00 2023-12-11T17:33:42.669 INFO:teuthology.orchestra.run.smithi181.stdout:-------------------------------------------------------------------------------- 2023-12-11T17:33:42.669 INFO:teuthology.orchestra.run.smithi181.stdout:Total 22 MB/s | 18 MB 00:00 2023-12-11T17:33:42.699 INFO:teuthology.orchestra.run.smithi059.stdout:Dependencies resolved. 2023-12-11T17:33:42.800 INFO:teuthology.orchestra.run.smithi059.stdout:============================================================================================================ 2023-12-11T17:33:42.801 INFO:teuthology.orchestra.run.smithi059.stdout: Package Arch Version Repository Size 2023-12-11T17:33:42.801 INFO:teuthology.orchestra.run.smithi059.stdout:============================================================================================================ 2023-12-11T17:33:42.801 INFO:teuthology.orchestra.run.smithi059.stdout:Installing group/module packages: 2023-12-11T17:33:42.801 INFO:teuthology.orchestra.run.smithi059.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-12-11T17:33:42.801 INFO:teuthology.orchestra.run.smithi059.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-12-11T17:33:42.801 INFO:teuthology.orchestra.run.smithi059.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-12-11T17:33:42.801 INFO:teuthology.orchestra.run.smithi059.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-12-11T17:33:42.801 INFO:teuthology.orchestra.run.smithi059.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-12-11T17:33:42.801 INFO:teuthology.orchestra.run.smithi059.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-12-11T17:33:42.801 INFO:teuthology.orchestra.run.smithi059.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-12-11T17:33:42.801 INFO:teuthology.orchestra.run.smithi059.stdout:Installing dependencies: 2023-12-11T17:33:42.801 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-12-11T17:33:42.801 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-12-11T17:33:42.802 INFO:teuthology.orchestra.run.smithi059.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-12-11T17:33:42.802 INFO:teuthology.orchestra.run.smithi059.stdout:Installing module profiles: 2023-12-11T17:33:42.802 INFO:teuthology.orchestra.run.smithi059.stdout: container-tools/common 2023-12-11T17:33:42.802 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-12-11T17:33:42.802 INFO:teuthology.orchestra.run.smithi059.stdout:Enabling module streams: 2023-12-11T17:33:42.802 INFO:teuthology.orchestra.run.smithi059.stdout: container-tools rhel8 2023-12-11T17:33:42.802 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-12-11T17:33:42.802 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction Summary 2023-12-11T17:33:42.802 INFO:teuthology.orchestra.run.smithi059.stdout:============================================================================================================ 2023-12-11T17:33:42.802 INFO:teuthology.orchestra.run.smithi059.stdout:Install 10 Packages 2023-12-11T17:33:42.802 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-12-11T17:33:42.803 INFO:teuthology.orchestra.run.smithi059.stdout:Total download size: 18 M 2023-12-11T17:33:42.803 INFO:teuthology.orchestra.run.smithi059.stdout:Installed size: 65 M 2023-12-11T17:33:42.803 INFO:teuthology.orchestra.run.smithi059.stdout:Downloading Packages: 2023-12-11T17:33:42.869 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction check 2023-12-11T17:33:42.887 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction check succeeded. 2023-12-11T17:33:42.887 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction test 2023-12-11T17:33:42.888 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2023-12-11T17:33:42.997 INFO:teuthology.orchestra.run.smithi059.stdout:(1/10): yajl-2.1.0-10.el8.x86_64.rpm 210 kB/s | 41 kB 00:00 2023-12-11T17:33:43.014 INFO:teuthology.orchestra.run.smithi059.stdout:(2/10): python3-pytoml-0.1.14-5.git7dea353.el8. 118 kB/s | 25 kB 00:00 2023-12-11T17:33:43.040 INFO:teuthology.orchestra.run.smithi059.stdout:(3/10): python3-pyxdg-0.25-16.el8.noarch.rpm 397 kB/s | 94 kB 00:00 2023-12-11T17:33:43.059 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-12-11T17:33:43.068 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction test succeeded. 2023-12-11T17:33:43.072 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction 2023-12-11T17:33:43.173 INFO:teuthology.orchestra.run.smithi059.stdout:(4/10): udica-0.2.6-4.module+el8.6.0+16771+28df 282 kB/s | 49 kB 00:00 2023-12-11T17:33:43.207 INFO:teuthology.orchestra.run.smithi059.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 2.7 MB/s | 524 kB 00:00 2023-12-11T17:33:43.274 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-12-11T17:33:43.324 INFO:teuthology.orchestra.run.smithi059.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 996 kB/s | 149 kB 00:00 2023-12-11T17:33:43.449 INFO:teuthology.orchestra.run.smithi059.stdout:(7/10): buildah-1.26.2-1.module+el8.6.0+16771+2 19 MB/s | 8.0 MB 00:00 2023-12-11T17:33:43.459 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-12-11T17:33:43.474 INFO:teuthology.orchestra.run.smithi059.stdout:(8/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 797 kB/s | 213 kB 00:00 2023-12-11T17:33:43.596 INFO:teuthology.orchestra.run.smithi181.stdout: Preparing : 1/1 2023-12-11T17:33:43.607 INFO:teuthology.orchestra.run.smithi059.stdout:(9/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 14 MB/s | 2.2 MB 00:00 2023-12-11T17:33:43.678 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-12-11T17:33:43.801 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-12-11T17:33:43.816 INFO:teuthology.orchestra.run.smithi059.stdout:(10/10): skopeo-1.9.1-1.module+el8.6.0+16771+28 14 MB/s | 7.0 MB 00:00 2023-12-11T17:33:43.817 INFO:teuthology.orchestra.run.smithi059.stdout:-------------------------------------------------------------------------------- 2023-12-11T17:33:43.817 INFO:teuthology.orchestra.run.smithi059.stdout:Total 18 MB/s | 18 MB 00:01 2023-12-11T17:33:43.963 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-12-11T17:33:44.025 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction check 2023-12-11T17:33:44.035 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-12-11T17:33:44.046 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction check succeeded. 2023-12-11T17:33:44.046 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction test 2023-12-11T17:33:44.182 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-12-11T17:33:44.242 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction test succeeded. 2023-12-11T17:33:44.246 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction 2023-12-11T17:33:44.402 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-12-11T17:33:44.750 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-12-11T17:33:44.768 INFO:teuthology.orchestra.run.smithi059.stdout: Preparing : 1/1 2023-12-11T17:33:44.788 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-12-11T17:33:44.945 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-12-11T17:33:45.132 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-12-11T17:33:45.321 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-12-11T17:33:45.504 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-12-11T17:33:45.559 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-12-11T17:33:45.730 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-12-11T17:33:45.907 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-12-11T17:33:45.951 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-12-11T17:33:46.155 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-12-11T17:33:46.332 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-12-11T17:33:46.403 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-12-11T17:33:46.599 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-12-11T17:33:46.599 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-12-11T17:33:46.599 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-12-11T17:33:46.599 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-12-11T17:33:46.599 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-12-11T17:33:46.599 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-12-11T17:33:46.599 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-12-11T17:33:46.599 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-12-11T17:33:46.599 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-12-11T17:33:46.599 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-12-11T17:33:46.602 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-12-11T17:33:46.695 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-12-11T17:33:46.795 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-12-11T17:33:46.997 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-12-11T17:33:47.263 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-12-11T17:33:47.263 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-12-11T17:33:47.263 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-12-11T17:33:47.263 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-12-11T17:33:47.263 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-12-11T17:33:47.263 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-12-11T17:33:47.263 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-12-11T17:33:47.263 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-12-11T17:33:47.264 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-12-11T17:33:47.264 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-12-11T17:33:47.608 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-12-11T17:33:47.845 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-12-11T17:33:48.022 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-12-11T17:33:48.240 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-12-11T17:33:48.518 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-12-11T17:33:48.518 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-12-11T17:33:48.518 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-12-11T17:33:48.518 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-12-11T17:33:48.518 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-12-11T17:33:48.518 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-12-11T17:33:48.519 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-12-11T17:33:48.519 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-12-11T17:33:48.519 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-12-11T17:33:48.519 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-12-11T17:33:55.211 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-12-11T17:33:55.283 INFO:teuthology.orchestra.run.smithi181.stdout:Installed products updated. 2023-12-11T17:33:55.401 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-12-11T17:33:55.401 INFO:teuthology.orchestra.run.smithi181.stdout:Installed: 2023-12-11T17:33:55.401 INFO:teuthology.orchestra.run.smithi181.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-12-11T17:33:55.401 INFO:teuthology.orchestra.run.smithi181.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-12-11T17:33:55.401 INFO:teuthology.orchestra.run.smithi181.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-12-11T17:33:55.401 INFO:teuthology.orchestra.run.smithi181.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-12-11T17:33:55.401 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-12-11T17:33:55.401 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-12-11T17:33:55.401 INFO:teuthology.orchestra.run.smithi181.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-12-11T17:33:55.402 INFO:teuthology.orchestra.run.smithi181.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-12-11T17:33:55.402 INFO:teuthology.orchestra.run.smithi181.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-12-11T17:33:55.402 INFO:teuthology.orchestra.run.smithi181.stdout: yajl-2.1.0-10.el8.x86_64 2023-12-11T17:33:55.402 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-12-11T17:33:55.402 INFO:teuthology.orchestra.run.smithi181.stdout:Complete! 2023-12-11T17:33:55.538 DEBUG:teuthology.parallel:result is None 2023-12-11T17:33:55.553 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-12-11T17:33:55.553 INFO:teuthology.orchestra.run.smithi115.stdout:Installed products updated. 2023-12-11T17:33:55.752 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-11T17:33:55.752 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2023-12-11T17:33:55.752 INFO:teuthology.orchestra.run.smithi115.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-12-11T17:33:55.752 INFO:teuthology.orchestra.run.smithi115.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-12-11T17:33:55.752 INFO:teuthology.orchestra.run.smithi115.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-12-11T17:33:55.752 INFO:teuthology.orchestra.run.smithi115.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-12-11T17:33:55.752 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-12-11T17:33:55.752 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-12-11T17:33:55.753 INFO:teuthology.orchestra.run.smithi115.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-12-11T17:33:55.753 INFO:teuthology.orchestra.run.smithi115.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-12-11T17:33:55.753 INFO:teuthology.orchestra.run.smithi115.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-12-11T17:33:55.753 INFO:teuthology.orchestra.run.smithi115.stdout: yajl-2.1.0-10.el8.x86_64 2023-12-11T17:33:55.753 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-11T17:33:55.753 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2023-12-11T17:33:55.888 DEBUG:teuthology.parallel:result is None 2023-12-11T17:33:57.978 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-12-11T17:33:57.978 INFO:teuthology.orchestra.run.smithi059.stdout:Installed products updated. 2023-12-11T17:33:58.194 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-12-11T17:33:58.194 INFO:teuthology.orchestra.run.smithi059.stdout:Installed: 2023-12-11T17:33:58.194 INFO:teuthology.orchestra.run.smithi059.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-12-11T17:33:58.194 INFO:teuthology.orchestra.run.smithi059.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-12-11T17:33:58.194 INFO:teuthology.orchestra.run.smithi059.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-12-11T17:33:58.194 INFO:teuthology.orchestra.run.smithi059.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-12-11T17:33:58.194 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-12-11T17:33:58.194 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-12-11T17:33:58.194 INFO:teuthology.orchestra.run.smithi059.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-12-11T17:33:58.194 INFO:teuthology.orchestra.run.smithi059.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-12-11T17:33:58.195 INFO:teuthology.orchestra.run.smithi059.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-12-11T17:33:58.195 INFO:teuthology.orchestra.run.smithi059.stdout: yajl-2.1.0-10.el8.x86_64 2023-12-11T17:33:58.195 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-12-11T17:33:58.195 INFO:teuthology.orchestra.run.smithi059.stdout:Complete! 2023-12-11T17:33:58.361 DEBUG:teuthology.parallel:result is None 2023-12-11T17:33:58.362 INFO:teuthology.run_tasks:Running task install... 2023-12-11T17:33:58.371 DEBUG:teuthology.task.install:project ceph 2023-12-11T17:33:58.371 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'a62fda73efba5520d409bb5a21bd8373289dda5d'}} 2023-12-11T17:33:58.372 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'a62fda73efba5520d409bb5a21bd8373289dda5d'} 2023-12-11T17:33:58.372 INFO:teuthology.task.install:Using flavor: default 2023-12-11T17:33:58.378 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-12-11T17:33:58.378 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-12-11T17:33:58.378 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a62fda73efba5520d409bb5a21bd8373289dda5d', 'tag': None, 'wait_for_package': False} 2023-12-11T17:33:58.379 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=a62fda73efba5520d409bb5a21bd8373289dda5d 2023-12-11T17:33:58.381 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a62fda73efba5520d409bb5a21bd8373289dda5d', 'tag': None, 'wait_for_package': False} 2023-12-11T17:33:58.381 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=a62fda73efba5520d409bb5a21bd8373289dda5d 2023-12-11T17:33:58.382 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a62fda73efba5520d409bb5a21bd8373289dda5d', 'tag': None, 'wait_for_package': False} 2023-12-11T17:33:58.382 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=a62fda73efba5520d409bb5a21bd8373289dda5d 2023-12-11T17:33:58.531 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/rishabh-2023dec11/a62fda73efba5520d409bb5a21bd8373289dda5d/centos/8/flavors/default/ 2023-12-11T17:33:58.531 INFO:teuthology.task.install.rpm:Package version is 19.0.0-119.ga62fda73 2023-12-11T17:33:58.601 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/rishabh-2023dec11/a62fda73efba5520d409bb5a21bd8373289dda5d/centos/8/flavors/default/ 2023-12-11T17:33:58.601 INFO:teuthology.task.install.rpm:Package version is 19.0.0-119.ga62fda73 2023-12-11T17:33:58.609 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/rishabh-2023dec11/a62fda73efba5520d409bb5a21bd8373289dda5d/centos/8/flavors/default/ 2023-12-11T17:33:58.610 INFO:teuthology.task.install.rpm:Package version is 19.0.0-119.ga62fda73 2023-12-11T17:33:58.712 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/rishabh-2023dec11/a62fda73efba5520d409bb5a21bd8373289dda5d/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/rishabh-2023dec11/a62fda73efba5520d409bb5a21bd8373289dda5d/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/rishabh-2023dec11/a62fda73efba5520d409bb5a21bd8373289dda5d/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-12-11T17:33:58.713 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-12-11T17:33:58.713 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-12-11T17:33:58.715 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/rishabh-2023dec11/a62fda73efba5520d409bb5a21bd8373289dda5d/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/rishabh-2023dec11/a62fda73efba5520d409bb5a21bd8373289dda5d/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/rishabh-2023dec11/a62fda73efba5520d409bb5a21bd8373289dda5d/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-12-11T17:33:58.715 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-12-11T17:33:58.715 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-12-11T17:33:58.721 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/rishabh-2023dec11/a62fda73efba5520d409bb5a21bd8373289dda5d/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/rishabh-2023dec11/a62fda73efba5520d409bb5a21bd8373289dda5d/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/rishabh-2023dec11/a62fda73efba5520d409bb5a21bd8373289dda5d/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-12-11T17:33:58.721 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2023-12-11T17:33:58.721 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-12-11T17:33:58.747 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-12-11T17:33:58.747 DEBUG:teuthology.orchestra.run.smithi115:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/a62fda73efba5520d409bb5a21bd8373289dda5d/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-12-11T17:33:58.748 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-12-11T17:33:58.748 DEBUG:teuthology.orchestra.run.smithi181:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/a62fda73efba5520d409bb5a21bd8373289dda5d/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-12-11T17:33:58.751 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-12-11T17:33:58.751 DEBUG:teuthology.orchestra.run.smithi059:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/a62fda73efba5520d409bb5a21bd8373289dda5d/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-12-11T17:33:58.818 DEBUG:teuthology.orchestra.run.smithi115:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-12-11T17:33:58.820 DEBUG:teuthology.orchestra.run.smithi181:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-12-11T17:33:58.821 DEBUG:teuthology.orchestra.run.smithi059:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-12-11T17:33:58.898 DEBUG:teuthology.orchestra.run.smithi115:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-12-11T17:33:58.900 DEBUG:teuthology.orchestra.run.smithi059:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-12-11T17:33:58.901 DEBUG:teuthology.orchestra.run.smithi181:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-12-11T17:33:58.966 INFO:teuthology.orchestra.run.smithi115.stdout:check_obsoletes = 1 2023-12-11T17:33:58.967 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum clean all 2023-12-11T17:33:58.968 INFO:teuthology.orchestra.run.smithi059.stdout:check_obsoletes = 1 2023-12-11T17:33:58.969 DEBUG:teuthology.orchestra.run.smithi059:> sudo yum clean all 2023-12-11T17:33:58.970 INFO:teuthology.orchestra.run.smithi181.stdout:check_obsoletes = 1 2023-12-11T17:33:58.970 DEBUG:teuthology.orchestra.run.smithi181:> sudo yum clean all 2023-12-11T17:33:59.340 INFO:teuthology.orchestra.run.smithi115.stdout:Updating Subscription Management repositories. 2023-12-11T17:33:59.344 INFO:teuthology.orchestra.run.smithi181.stdout:Updating Subscription Management repositories. 2023-12-11T17:33:59.362 INFO:teuthology.orchestra.run.smithi059.stdout:Updating Subscription Management repositories. 2023-12-11T17:34:00.550 INFO:teuthology.orchestra.run.smithi115.stdout:44 files removed 2023-12-11T17:34:00.601 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-12-11T17:34:00.635 INFO:teuthology.orchestra.run.smithi059.stdout:44 files removed 2023-12-11T17:34:00.689 DEBUG:teuthology.orchestra.run.smithi059:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-12-11T17:34:00.723 INFO:teuthology.orchestra.run.smithi181.stdout:44 files removed 2023-12-11T17:34:00.776 DEBUG:teuthology.orchestra.run.smithi181:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-12-11T17:34:00.934 INFO:teuthology.orchestra.run.smithi115.stdout:Updating Subscription Management repositories. 2023-12-11T17:34:01.042 INFO:teuthology.orchestra.run.smithi059.stdout:Updating Subscription Management repositories. 2023-12-11T17:34:01.110 INFO:teuthology.orchestra.run.smithi181.stdout:Updating Subscription Management repositories. 2023-12-11T17:34:02.881 INFO:teuthology.orchestra.run.smithi115.stdout:ceph packages for x86_64 224 kB/s | 81 kB 00:00 2023-12-11T17:34:02.929 INFO:teuthology.orchestra.run.smithi181.stdout:ceph packages for x86_64 202 kB/s | 81 kB 00:00 2023-12-11T17:34:03.203 INFO:teuthology.orchestra.run.smithi059.stdout:ceph packages for x86_64 167 kB/s | 81 kB 00:00 2023-12-11T17:34:03.274 INFO:teuthology.orchestra.run.smithi115.stdout:ceph noarch packages 26 kB/s | 9.5 kB 00:00 2023-12-11T17:34:03.288 INFO:teuthology.orchestra.run.smithi181.stdout:ceph noarch packages 29 kB/s | 9.5 kB 00:00 2023-12-11T17:34:03.589 INFO:teuthology.orchestra.run.smithi181.stdout:ceph source packages 5.3 kB/s | 1.5 kB 00:00 2023-12-11T17:34:03.590 INFO:teuthology.orchestra.run.smithi115.stdout:ceph source packages 5.0 kB/s | 1.5 kB 00:00 2023-12-11T17:34:03.596 INFO:teuthology.orchestra.run.smithi059.stdout:ceph noarch packages 26 kB/s | 9.5 kB 00:00 2023-12-11T17:34:03.714 INFO:teuthology.orchestra.run.smithi181.stdout:Copr repo for python3-asyncssh owned by ceph 33 kB/s | 3.5 kB 00:00 2023-12-11T17:34:03.716 INFO:teuthology.orchestra.run.smithi115.stdout:Copr repo for python3-asyncssh owned by ceph 33 kB/s | 3.5 kB 00:00 2023-12-11T17:34:03.921 INFO:teuthology.orchestra.run.smithi059.stdout:ceph source packages 4.9 kB/s | 1.5 kB 00:00 2023-12-11T17:34:04.072 INFO:teuthology.orchestra.run.smithi059.stdout:Copr repo for python3-asyncssh owned by ceph 27 kB/s | 3.5 kB 00:00 2023-12-11T17:34:04.934 INFO:teuthology.orchestra.run.smithi181.stdout:Extra Packages for Enterprise Linux 13 MB/s | 16 MB 00:01 2023-12-11T17:34:05.658 INFO:teuthology.orchestra.run.smithi059.stdout:Extra Packages for Enterprise Linux 10 MB/s | 16 MB 00:01 2023-12-11T17:34:06.217 INFO:teuthology.orchestra.run.smithi115.stdout:Extra Packages for Enterprise Linux 6.4 MB/s | 16 MB 00:02 2023-12-11T17:34:08.482 INFO:teuthology.orchestra.run.smithi181.stdout:lab-extras 388 kB/s | 24 kB 00:00 2023-12-11T17:34:09.356 INFO:teuthology.orchestra.run.smithi059.stdout:lab-extras 415 kB/s | 24 kB 00:00 2023-12-11T17:34:09.692 INFO:teuthology.orchestra.run.smithi181.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 47 MB/s | 53 MB 00:01 2023-12-11T17:34:09.775 INFO:teuthology.orchestra.run.smithi115.stdout:lab-extras 331 kB/s | 24 kB 00:00 2023-12-11T17:34:10.566 INFO:teuthology.orchestra.run.smithi059.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 47 MB/s | 53 MB 00:01 2023-12-11T17:34:10.944 INFO:teuthology.orchestra.run.smithi115.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 43 MB/s | 47 MB 00:01 2023-12-11T17:34:18.118 INFO:teuthology.orchestra.run.smithi181.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 42 MB/s | 47 MB 00:01 2023-12-11T17:34:19.095 INFO:teuthology.orchestra.run.smithi115.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 49 MB/s | 53 MB 00:01 2023-12-11T17:34:19.459 INFO:teuthology.orchestra.run.smithi059.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 51 MB/s | 47 MB 00:00 2023-12-11T17:34:29.124 INFO:teuthology.orchestra.run.smithi181.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-12-11T17:34:29.125 INFO:teuthology.orchestra.run.smithi181.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-12-11T17:34:29.139 INFO:teuthology.orchestra.run.smithi181.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-12-11T17:34:29.140 INFO:teuthology.orchestra.run.smithi181.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-12-11T17:34:29.141 INFO:teuthology.orchestra.run.smithi181.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-12-11T17:34:29.142 INFO:teuthology.orchestra.run.smithi181.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-12-11T17:34:29.144 INFO:teuthology.orchestra.run.smithi181.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-12-11T17:34:29.147 INFO:teuthology.orchestra.run.smithi181.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-12-11T17:34:29.149 INFO:teuthology.orchestra.run.smithi181.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-12-11T17:34:29.287 INFO:teuthology.orchestra.run.smithi181.stdout:Dependencies resolved. 2023-12-11T17:34:29.300 INFO:teuthology.orchestra.run.smithi181.stdout:=========================================================================================================================== 2023-12-11T17:34:29.300 INFO:teuthology.orchestra.run.smithi181.stdout: Package Arch Version Repository Size 2023-12-11T17:34:29.300 INFO:teuthology.orchestra.run.smithi181.stdout:=========================================================================================================================== 2023-12-11T17:34:29.300 INFO:teuthology.orchestra.run.smithi181.stdout:Installing: 2023-12-11T17:34:29.300 INFO:teuthology.orchestra.run.smithi181.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-12-11T17:34:29.300 INFO:teuthology.orchestra.run.smithi181.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-12-11T17:34:29.300 INFO:teuthology.orchestra.run.smithi181.stdout: ceph x86_64 2:19.0.0-119.ga62fda73.el8 ceph 6.4 k 2023-12-11T17:34:29.300 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-base x86_64 2:19.0.0-119.ga62fda73.el8 ceph 5.4 M 2023-12-11T17:34:29.300 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-fuse x86_64 2:19.0.0-119.ga62fda73.el8 ceph 919 k 2023-12-11T17:34:29.301 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-119.ga62fda73.el8 ceph 171 k 2023-12-11T17:34:29.301 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr x86_64 2:19.0.0-119.ga62fda73.el8 ceph 1.6 M 2023-12-11T17:34:29.301 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-cephadm noarch 2:19.0.0-119.ga62fda73.el8 ceph-noarch 133 k 2023-12-11T17:34:29.301 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-dashboard noarch 2:19.0.0-119.ga62fda73.el8 ceph-noarch 1.8 M 2023-12-11T17:34:29.301 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-119.ga62fda73.el8 ceph-noarch 7.4 M 2023-12-11T17:34:29.301 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-rook noarch 2:19.0.0-119.ga62fda73.el8 ceph-noarch 50 k 2023-12-11T17:34:29.301 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-radosgw x86_64 2:19.0.0-119.ga62fda73.el8 ceph 14 M 2023-12-11T17:34:29.301 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-test x86_64 2:19.0.0-119.ga62fda73.el8 ceph 49 M 2023-12-11T17:34:29.301 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-volume noarch 2:19.0.0-119.ga62fda73.el8 ceph-noarch 262 k 2023-12-11T17:34:29.301 INFO:teuthology.orchestra.run.smithi181.stdout: cephadm noarch 2:19.0.0-119.ga62fda73.el8 ceph-noarch 415 k 2023-12-11T17:34:29.301 INFO:teuthology.orchestra.run.smithi181.stdout: cephfs-mirror x86_64 2:19.0.0-119.ga62fda73.el8 ceph 262 k 2023-12-11T17:34:29.301 INFO:teuthology.orchestra.run.smithi181.stdout: cephfs-top noarch 2:19.0.0-119.ga62fda73.el8 ceph-noarch 18 k 2023-12-11T17:34:29.301 INFO:teuthology.orchestra.run.smithi181.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-12-11T17:34:29.301 INFO:teuthology.orchestra.run.smithi181.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-12-11T17:34:29.302 INFO:teuthology.orchestra.run.smithi181.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-12-11T17:34:29.302 INFO:teuthology.orchestra.run.smithi181.stdout: libcephfs-devel x86_64 2:19.0.0-119.ga62fda73.el8 ceph 31 k 2023-12-11T17:34:29.302 INFO:teuthology.orchestra.run.smithi181.stdout: libcephfs2 x86_64 2:19.0.0-119.ga62fda73.el8 ceph 838 k 2023-12-11T17:34:29.302 INFO:teuthology.orchestra.run.smithi181.stdout: librados-devel x86_64 2:19.0.0-119.ga62fda73.el8 ceph 139 k 2023-12-11T17:34:29.302 INFO:teuthology.orchestra.run.smithi181.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-12-11T17:34:29.302 INFO:teuthology.orchestra.run.smithi181.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-12-11T17:34:29.302 INFO:teuthology.orchestra.run.smithi181.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-12-11T17:34:29.302 INFO:teuthology.orchestra.run.smithi181.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-12-11T17:34:29.302 INFO:teuthology.orchestra.run.smithi181.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-12-11T17:34:29.302 INFO:teuthology.orchestra.run.smithi181.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-12-11T17:34:29.302 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cephfs x86_64 2:19.0.0-119.ga62fda73.el8 ceph 197 k 2023-12-11T17:34:29.302 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rados x86_64 2:19.0.0-119.ga62fda73.el8 ceph 371 k 2023-12-11T17:34:29.303 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rbd x86_64 2:19.0.0-119.ga62fda73.el8 ceph 364 k 2023-12-11T17:34:29.303 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rgw x86_64 2:19.0.0-119.ga62fda73.el8 ceph 107 k 2023-12-11T17:34:29.303 INFO:teuthology.orchestra.run.smithi181.stdout: rbd-fuse x86_64 2:19.0.0-119.ga62fda73.el8 ceph 93 k 2023-12-11T17:34:29.303 INFO:teuthology.orchestra.run.smithi181.stdout: rbd-mirror x86_64 2:19.0.0-119.ga62fda73.el8 ceph 3.6 M 2023-12-11T17:34:29.303 INFO:teuthology.orchestra.run.smithi181.stdout: rbd-nbd x86_64 2:19.0.0-119.ga62fda73.el8 ceph 181 k 2023-12-11T17:34:29.303 INFO:teuthology.orchestra.run.smithi181.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-12-11T17:34:29.303 INFO:teuthology.orchestra.run.smithi181.stdout:Upgrading: 2023-12-11T17:34:29.303 INFO:teuthology.orchestra.run.smithi181.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-12-11T17:34:29.303 INFO:teuthology.orchestra.run.smithi181.stdout: librados2 x86_64 2:19.0.0-119.ga62fda73.el8 ceph 3.8 M 2023-12-11T17:34:29.303 INFO:teuthology.orchestra.run.smithi181.stdout: librbd1 x86_64 2:19.0.0-119.ga62fda73.el8 ceph 3.7 M 2023-12-11T17:34:29.303 INFO:teuthology.orchestra.run.smithi181.stdout:Installing dependencies: 2023-12-11T17:34:29.303 INFO:teuthology.orchestra.run.smithi181.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-12-11T17:34:29.303 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-common x86_64 2:19.0.0-119.ga62fda73.el8 ceph 23 M 2023-12-11T17:34:29.303 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-grafana-dashboards noarch 2:19.0.0-119.ga62fda73.el8 ceph-noarch 28 k 2023-12-11T17:34:29.303 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mds x86_64 2:19.0.0-119.ga62fda73.el8 ceph 2.3 M 2023-12-11T17:34:29.304 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-modules-core noarch 2:19.0.0-119.ga62fda73.el8 ceph-noarch 243 k 2023-12-11T17:34:29.304 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mon x86_64 2:19.0.0-119.ga62fda73.el8 ceph 4.6 M 2023-12-11T17:34:29.304 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-osd x86_64 2:19.0.0-119.ga62fda73.el8 ceph 18 M 2023-12-11T17:34:29.304 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-prometheus-alerts noarch 2:19.0.0-119.ga62fda73.el8 ceph-noarch 15 k 2023-12-11T17:34:29.304 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-selinux x86_64 2:19.0.0-119.ga62fda73.el8 ceph 24 k 2023-12-11T17:34:29.304 INFO:teuthology.orchestra.run.smithi181.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-12-11T17:34:29.304 INFO:teuthology.orchestra.run.smithi181.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-12-11T17:34:29.304 INFO:teuthology.orchestra.run.smithi181.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-12-11T17:34:29.304 INFO:teuthology.orchestra.run.smithi181.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-12-11T17:34:29.304 INFO:teuthology.orchestra.run.smithi181.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-12-11T17:34:29.304 INFO:teuthology.orchestra.run.smithi181.stdout: libcephsqlite x86_64 2:19.0.0-119.ga62fda73.el8 ceph 183 k 2023-12-11T17:34:29.304 INFO:teuthology.orchestra.run.smithi181.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-12-11T17:34:29.304 INFO:teuthology.orchestra.run.smithi181.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-12-11T17:34:29.304 INFO:teuthology.orchestra.run.smithi181.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-12-11T17:34:29.304 INFO:teuthology.orchestra.run.smithi181.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-12-11T17:34:29.305 INFO:teuthology.orchestra.run.smithi181.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-12-11T17:34:29.305 INFO:teuthology.orchestra.run.smithi181.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-12-11T17:34:29.305 INFO:teuthology.orchestra.run.smithi181.stdout: libradosstriper1 x86_64 2:19.0.0-119.ga62fda73.el8 ceph 517 k 2023-12-11T17:34:29.305 INFO:teuthology.orchestra.run.smithi181.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-12-11T17:34:29.305 INFO:teuthology.orchestra.run.smithi181.stdout: librgw2 x86_64 2:19.0.0-119.ga62fda73.el8 ceph 6.9 M 2023-12-11T17:34:29.305 INFO:teuthology.orchestra.run.smithi181.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-12-11T17:34:29.305 INFO:teuthology.orchestra.run.smithi181.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-12-11T17:34:29.305 INFO:teuthology.orchestra.run.smithi181.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-12-11T17:34:29.305 INFO:teuthology.orchestra.run.smithi181.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-12-11T17:34:29.305 INFO:teuthology.orchestra.run.smithi181.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-12-11T17:34:29.305 INFO:teuthology.orchestra.run.smithi181.stdout: luarocks noarch 3.5.0-1.el8 epel 167 k 2023-12-11T17:34:29.305 INFO:teuthology.orchestra.run.smithi181.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-12-11T17:34:29.305 INFO:teuthology.orchestra.run.smithi181.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-12-11T17:34:29.305 INFO:teuthology.orchestra.run.smithi181.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-12-11T17:34:29.306 INFO:teuthology.orchestra.run.smithi181.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-12-11T17:34:29.306 INFO:teuthology.orchestra.run.smithi181.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-12-11T17:34:29.306 INFO:teuthology.orchestra.run.smithi181.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-12-11T17:34:29.306 INFO:teuthology.orchestra.run.smithi181.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-12-11T17:34:29.306 INFO:teuthology.orchestra.run.smithi181.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-12-11T17:34:29.306 INFO:teuthology.orchestra.run.smithi181.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-12-11T17:34:29.306 INFO:teuthology.orchestra.run.smithi181.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-12-11T17:34:29.306 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-12-11T17:34:29.306 INFO:teuthology.orchestra.run.smithi181.stdout: python3-ceph-argparse x86_64 2:19.0.0-119.ga62fda73.el8 ceph 45 k 2023-12-11T17:34:29.306 INFO:teuthology.orchestra.run.smithi181.stdout: python3-ceph-common x86_64 2:19.0.0-119.ga62fda73.el8 ceph 129 k 2023-12-11T17:34:29.306 INFO:teuthology.orchestra.run.smithi181.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-12-11T17:34:29.306 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-12-11T17:34:29.306 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-12-11T17:34:29.306 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-12-11T17:34:29.307 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-12-11T17:34:29.307 INFO:teuthology.orchestra.run.smithi181.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-12-11T17:34:29.307 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-12-11T17:34:29.307 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-12-11T17:34:29.307 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-12-11T17:34:29.307 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-12-11T17:34:29.307 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-12-11T17:34:29.307 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-12-11T17:34:29.307 INFO:teuthology.orchestra.run.smithi181.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-12-11T17:34:29.307 INFO:teuthology.orchestra.run.smithi181.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-12-11T17:34:29.307 INFO:teuthology.orchestra.run.smithi181.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-12-11T17:34:29.307 INFO:teuthology.orchestra.run.smithi181.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-12-11T17:34:29.307 INFO:teuthology.orchestra.run.smithi181.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-12-11T17:34:29.307 INFO:teuthology.orchestra.run.smithi181.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-12-11T17:34:29.307 INFO:teuthology.orchestra.run.smithi181.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-12-11T17:34:29.308 INFO:teuthology.orchestra.run.smithi181.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-12-11T17:34:29.308 INFO:teuthology.orchestra.run.smithi181.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-12-11T17:34:29.308 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-12-11T17:34:29.308 INFO:teuthology.orchestra.run.smithi181.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-12-11T17:34:29.308 INFO:teuthology.orchestra.run.smithi181.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-12-11T17:34:29.308 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-12-11T17:34:29.308 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-12-11T17:34:29.308 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-12-11T17:34:29.308 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-12-11T17:34:29.308 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-12-11T17:34:29.308 INFO:teuthology.orchestra.run.smithi181.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-12-11T17:34:29.308 INFO:teuthology.orchestra.run.smithi181.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-12-11T17:34:29.308 INFO:teuthology.orchestra.run.smithi181.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-12-11T17:34:29.308 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-12-11T17:34:29.308 INFO:teuthology.orchestra.run.smithi181.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-12-11T17:34:29.309 INFO:teuthology.orchestra.run.smithi181.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-12-11T17:34:29.309 INFO:teuthology.orchestra.run.smithi181.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-12-11T17:34:29.309 INFO:teuthology.orchestra.run.smithi181.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-12-11T17:34:29.309 INFO:teuthology.orchestra.run.smithi181.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-12-11T17:34:29.309 INFO:teuthology.orchestra.run.smithi181.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-12-11T17:34:29.309 INFO:teuthology.orchestra.run.smithi181.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-12-11T17:34:29.309 INFO:teuthology.orchestra.run.smithi181.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-12-11T17:34:29.309 INFO:teuthology.orchestra.run.smithi181.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-12-11T17:34:29.309 INFO:teuthology.orchestra.run.smithi181.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-12-11T17:34:29.309 INFO:teuthology.orchestra.run.smithi181.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-12-11T17:34:29.309 INFO:teuthology.orchestra.run.smithi181.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-12-11T17:34:29.309 INFO:teuthology.orchestra.run.smithi181.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-12-11T17:34:29.309 INFO:teuthology.orchestra.run.smithi181.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-12-11T17:34:29.309 INFO:teuthology.orchestra.run.smithi181.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-12-11T17:34:29.310 INFO:teuthology.orchestra.run.smithi181.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-12-11T17:34:29.310 INFO:teuthology.orchestra.run.smithi181.stdout:Enabling module streams: 2023-12-11T17:34:29.310 INFO:teuthology.orchestra.run.smithi181.stdout: postgresql 10 2023-12-11T17:34:29.310 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-12-11T17:34:29.310 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction Summary 2023-12-11T17:34:29.310 INFO:teuthology.orchestra.run.smithi181.stdout:=========================================================================================================================== 2023-12-11T17:34:29.310 INFO:teuthology.orchestra.run.smithi181.stdout:Install 129 Packages 2023-12-11T17:34:29.310 INFO:teuthology.orchestra.run.smithi181.stdout:Upgrade 3 Packages 2023-12-11T17:34:29.310 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-12-11T17:34:29.311 INFO:teuthology.orchestra.run.smithi181.stdout:Total download size: 209 M 2023-12-11T17:34:29.311 INFO:teuthology.orchestra.run.smithi181.stdout:Downloading Packages: 2023-12-11T17:34:29.793 INFO:teuthology.orchestra.run.smithi181.stdout:(1/132): ceph-19.0.0-119.ga62fda73.el8.x86_64.r 53 kB/s | 6.4 kB 00:00 2023-12-11T17:34:30.002 INFO:teuthology.orchestra.run.smithi181.stdout:(2/132): ceph-fuse-19.0.0-119.ga62fda73.el8.x86 4.3 MB/s | 919 kB 00:00 2023-12-11T17:34:30.069 INFO:teuthology.orchestra.run.smithi181.stdout:(3/132): ceph-immutable-object-cache-19.0.0-119 2.5 MB/s | 171 kB 00:00 2023-12-11T17:34:30.368 INFO:teuthology.orchestra.run.smithi115.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-12-11T17:34:30.369 INFO:teuthology.orchestra.run.smithi115.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-12-11T17:34:30.385 INFO:teuthology.orchestra.run.smithi115.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-12-11T17:34:30.386 INFO:teuthology.orchestra.run.smithi115.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-12-11T17:34:30.386 INFO:teuthology.orchestra.run.smithi115.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-12-11T17:34:30.387 INFO:teuthology.orchestra.run.smithi115.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-12-11T17:34:30.389 INFO:teuthology.orchestra.run.smithi115.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-12-11T17:34:30.393 INFO:teuthology.orchestra.run.smithi115.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-12-11T17:34:30.394 INFO:teuthology.orchestra.run.smithi115.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-12-11T17:34:30.486 INFO:teuthology.orchestra.run.smithi181.stdout:(4/132): ceph-mds-19.0.0-119.ga62fda73.el8.x86_ 5.5 MB/s | 2.3 MB 00:00 2023-12-11T17:34:30.527 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2023-12-11T17:34:30.540 INFO:teuthology.orchestra.run.smithi115.stdout:=========================================================================================================================== 2023-12-11T17:34:30.540 INFO:teuthology.orchestra.run.smithi115.stdout: Package Arch Version Repository Size 2023-12-11T17:34:30.540 INFO:teuthology.orchestra.run.smithi115.stdout:=========================================================================================================================== 2023-12-11T17:34:30.540 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2023-12-11T17:34:30.540 INFO:teuthology.orchestra.run.smithi115.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-12-11T17:34:30.540 INFO:teuthology.orchestra.run.smithi115.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-12-11T17:34:30.541 INFO:teuthology.orchestra.run.smithi115.stdout: ceph x86_64 2:19.0.0-119.ga62fda73.el8 ceph 6.4 k 2023-12-11T17:34:30.541 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-base x86_64 2:19.0.0-119.ga62fda73.el8 ceph 5.4 M 2023-12-11T17:34:30.541 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-fuse x86_64 2:19.0.0-119.ga62fda73.el8 ceph 919 k 2023-12-11T17:34:30.541 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-119.ga62fda73.el8 ceph 171 k 2023-12-11T17:34:30.541 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr x86_64 2:19.0.0-119.ga62fda73.el8 ceph 1.6 M 2023-12-11T17:34:30.541 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-cephadm noarch 2:19.0.0-119.ga62fda73.el8 ceph-noarch 133 k 2023-12-11T17:34:30.542 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-dashboard noarch 2:19.0.0-119.ga62fda73.el8 ceph-noarch 1.8 M 2023-12-11T17:34:30.542 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-119.ga62fda73.el8 ceph-noarch 7.4 M 2023-12-11T17:34:30.542 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-rook noarch 2:19.0.0-119.ga62fda73.el8 ceph-noarch 50 k 2023-12-11T17:34:30.542 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-radosgw x86_64 2:19.0.0-119.ga62fda73.el8 ceph 14 M 2023-12-11T17:34:30.542 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-test x86_64 2:19.0.0-119.ga62fda73.el8 ceph 49 M 2023-12-11T17:34:30.542 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-volume noarch 2:19.0.0-119.ga62fda73.el8 ceph-noarch 262 k 2023-12-11T17:34:30.542 INFO:teuthology.orchestra.run.smithi115.stdout: cephadm noarch 2:19.0.0-119.ga62fda73.el8 ceph-noarch 415 k 2023-12-11T17:34:30.543 INFO:teuthology.orchestra.run.smithi115.stdout: cephfs-mirror x86_64 2:19.0.0-119.ga62fda73.el8 ceph 262 k 2023-12-11T17:34:30.543 INFO:teuthology.orchestra.run.smithi115.stdout: cephfs-top noarch 2:19.0.0-119.ga62fda73.el8 ceph-noarch 18 k 2023-12-11T17:34:30.543 INFO:teuthology.orchestra.run.smithi115.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-12-11T17:34:30.543 INFO:teuthology.orchestra.run.smithi115.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-12-11T17:34:30.543 INFO:teuthology.orchestra.run.smithi115.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-12-11T17:34:30.543 INFO:teuthology.orchestra.run.smithi115.stdout: libcephfs-devel x86_64 2:19.0.0-119.ga62fda73.el8 ceph 31 k 2023-12-11T17:34:30.543 INFO:teuthology.orchestra.run.smithi115.stdout: libcephfs2 x86_64 2:19.0.0-119.ga62fda73.el8 ceph 838 k 2023-12-11T17:34:30.543 INFO:teuthology.orchestra.run.smithi115.stdout: librados-devel x86_64 2:19.0.0-119.ga62fda73.el8 ceph 139 k 2023-12-11T17:34:30.544 INFO:teuthology.orchestra.run.smithi115.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-12-11T17:34:30.544 INFO:teuthology.orchestra.run.smithi115.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-12-11T17:34:30.544 INFO:teuthology.orchestra.run.smithi115.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-12-11T17:34:30.544 INFO:teuthology.orchestra.run.smithi115.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-12-11T17:34:30.544 INFO:teuthology.orchestra.run.smithi115.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-12-11T17:34:30.544 INFO:teuthology.orchestra.run.smithi115.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-12-11T17:34:30.544 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cephfs x86_64 2:19.0.0-119.ga62fda73.el8 ceph 197 k 2023-12-11T17:34:30.544 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rados x86_64 2:19.0.0-119.ga62fda73.el8 ceph 371 k 2023-12-11T17:34:30.545 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rbd x86_64 2:19.0.0-119.ga62fda73.el8 ceph 364 k 2023-12-11T17:34:30.545 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rgw x86_64 2:19.0.0-119.ga62fda73.el8 ceph 107 k 2023-12-11T17:34:30.545 INFO:teuthology.orchestra.run.smithi115.stdout: rbd-fuse x86_64 2:19.0.0-119.ga62fda73.el8 ceph 93 k 2023-12-11T17:34:30.545 INFO:teuthology.orchestra.run.smithi115.stdout: rbd-mirror x86_64 2:19.0.0-119.ga62fda73.el8 ceph 3.6 M 2023-12-11T17:34:30.545 INFO:teuthology.orchestra.run.smithi115.stdout: rbd-nbd x86_64 2:19.0.0-119.ga62fda73.el8 ceph 181 k 2023-12-11T17:34:30.545 INFO:teuthology.orchestra.run.smithi115.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-12-11T17:34:30.545 INFO:teuthology.orchestra.run.smithi115.stdout:Upgrading: 2023-12-11T17:34:30.545 INFO:teuthology.orchestra.run.smithi115.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-12-11T17:34:30.545 INFO:teuthology.orchestra.run.smithi115.stdout: librados2 x86_64 2:19.0.0-119.ga62fda73.el8 ceph 3.8 M 2023-12-11T17:34:30.545 INFO:teuthology.orchestra.run.smithi115.stdout: librbd1 x86_64 2:19.0.0-119.ga62fda73.el8 ceph 3.7 M 2023-12-11T17:34:30.545 INFO:teuthology.orchestra.run.smithi115.stdout:Installing dependencies: 2023-12-11T17:34:30.545 INFO:teuthology.orchestra.run.smithi115.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-12-11T17:34:30.545 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-common x86_64 2:19.0.0-119.ga62fda73.el8 ceph 23 M 2023-12-11T17:34:30.545 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-grafana-dashboards noarch 2:19.0.0-119.ga62fda73.el8 ceph-noarch 28 k 2023-12-11T17:34:30.546 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mds x86_64 2:19.0.0-119.ga62fda73.el8 ceph 2.3 M 2023-12-11T17:34:30.546 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-modules-core noarch 2:19.0.0-119.ga62fda73.el8 ceph-noarch 243 k 2023-12-11T17:34:30.546 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mon x86_64 2:19.0.0-119.ga62fda73.el8 ceph 4.6 M 2023-12-11T17:34:30.546 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-osd x86_64 2:19.0.0-119.ga62fda73.el8 ceph 18 M 2023-12-11T17:34:30.546 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-prometheus-alerts noarch 2:19.0.0-119.ga62fda73.el8 ceph-noarch 15 k 2023-12-11T17:34:30.546 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-selinux x86_64 2:19.0.0-119.ga62fda73.el8 ceph 24 k 2023-12-11T17:34:30.546 INFO:teuthology.orchestra.run.smithi115.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-12-11T17:34:30.546 INFO:teuthology.orchestra.run.smithi115.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-12-11T17:34:30.546 INFO:teuthology.orchestra.run.smithi115.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-12-11T17:34:30.546 INFO:teuthology.orchestra.run.smithi115.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-12-11T17:34:30.546 INFO:teuthology.orchestra.run.smithi115.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-12-11T17:34:30.546 INFO:teuthology.orchestra.run.smithi115.stdout: libcephsqlite x86_64 2:19.0.0-119.ga62fda73.el8 ceph 183 k 2023-12-11T17:34:30.546 INFO:teuthology.orchestra.run.smithi115.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-12-11T17:34:30.547 INFO:teuthology.orchestra.run.smithi115.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-12-11T17:34:30.547 INFO:teuthology.orchestra.run.smithi115.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-12-11T17:34:30.547 INFO:teuthology.orchestra.run.smithi115.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-12-11T17:34:30.547 INFO:teuthology.orchestra.run.smithi115.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-12-11T17:34:30.547 INFO:teuthology.orchestra.run.smithi115.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-12-11T17:34:30.547 INFO:teuthology.orchestra.run.smithi115.stdout: libradosstriper1 x86_64 2:19.0.0-119.ga62fda73.el8 ceph 517 k 2023-12-11T17:34:30.547 INFO:teuthology.orchestra.run.smithi115.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-12-11T17:34:30.547 INFO:teuthology.orchestra.run.smithi115.stdout: librgw2 x86_64 2:19.0.0-119.ga62fda73.el8 ceph 6.9 M 2023-12-11T17:34:30.547 INFO:teuthology.orchestra.run.smithi115.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-12-11T17:34:30.547 INFO:teuthology.orchestra.run.smithi115.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-12-11T17:34:30.547 INFO:teuthology.orchestra.run.smithi115.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-12-11T17:34:30.547 INFO:teuthology.orchestra.run.smithi115.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-12-11T17:34:30.547 INFO:teuthology.orchestra.run.smithi115.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-12-11T17:34:30.547 INFO:teuthology.orchestra.run.smithi115.stdout: luarocks noarch 3.5.0-1.el8 epel 167 k 2023-12-11T17:34:30.548 INFO:teuthology.orchestra.run.smithi115.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-12-11T17:34:30.548 INFO:teuthology.orchestra.run.smithi115.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-12-11T17:34:30.548 INFO:teuthology.orchestra.run.smithi115.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-12-11T17:34:30.548 INFO:teuthology.orchestra.run.smithi115.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-12-11T17:34:30.548 INFO:teuthology.orchestra.run.smithi115.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-12-11T17:34:30.548 INFO:teuthology.orchestra.run.smithi115.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-12-11T17:34:30.548 INFO:teuthology.orchestra.run.smithi115.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-12-11T17:34:30.548 INFO:teuthology.orchestra.run.smithi115.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-12-11T17:34:30.548 INFO:teuthology.orchestra.run.smithi115.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-12-11T17:34:30.548 INFO:teuthology.orchestra.run.smithi115.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-12-11T17:34:30.548 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-12-11T17:34:30.548 INFO:teuthology.orchestra.run.smithi115.stdout: python3-ceph-argparse x86_64 2:19.0.0-119.ga62fda73.el8 ceph 45 k 2023-12-11T17:34:30.548 INFO:teuthology.orchestra.run.smithi115.stdout: python3-ceph-common x86_64 2:19.0.0-119.ga62fda73.el8 ceph 129 k 2023-12-11T17:34:30.548 INFO:teuthology.orchestra.run.smithi115.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-12-11T17:34:30.549 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-12-11T17:34:30.549 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-12-11T17:34:30.549 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-12-11T17:34:30.549 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-12-11T17:34:30.549 INFO:teuthology.orchestra.run.smithi115.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-12-11T17:34:30.549 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-12-11T17:34:30.549 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-12-11T17:34:30.549 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-12-11T17:34:30.549 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-12-11T17:34:30.549 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-12-11T17:34:30.549 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-12-11T17:34:30.549 INFO:teuthology.orchestra.run.smithi115.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-12-11T17:34:30.549 INFO:teuthology.orchestra.run.smithi115.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-12-11T17:34:30.550 INFO:teuthology.orchestra.run.smithi115.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-12-11T17:34:30.550 INFO:teuthology.orchestra.run.smithi115.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-12-11T17:34:30.550 INFO:teuthology.orchestra.run.smithi115.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-12-11T17:34:30.550 INFO:teuthology.orchestra.run.smithi115.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-12-11T17:34:30.550 INFO:teuthology.orchestra.run.smithi115.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-12-11T17:34:30.550 INFO:teuthology.orchestra.run.smithi115.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-12-11T17:34:30.550 INFO:teuthology.orchestra.run.smithi115.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-12-11T17:34:30.550 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-12-11T17:34:30.550 INFO:teuthology.orchestra.run.smithi115.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-12-11T17:34:30.550 INFO:teuthology.orchestra.run.smithi115.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-12-11T17:34:30.550 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-12-11T17:34:30.550 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-12-11T17:34:30.550 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-12-11T17:34:30.550 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-12-11T17:34:30.551 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-12-11T17:34:30.551 INFO:teuthology.orchestra.run.smithi115.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-12-11T17:34:30.551 INFO:teuthology.orchestra.run.smithi115.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-12-11T17:34:30.551 INFO:teuthology.orchestra.run.smithi115.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-12-11T17:34:30.551 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-12-11T17:34:30.551 INFO:teuthology.orchestra.run.smithi115.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-12-11T17:34:30.551 INFO:teuthology.orchestra.run.smithi115.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-12-11T17:34:30.551 INFO:teuthology.orchestra.run.smithi115.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-12-11T17:34:30.551 INFO:teuthology.orchestra.run.smithi115.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-12-11T17:34:30.551 INFO:teuthology.orchestra.run.smithi115.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-12-11T17:34:30.551 INFO:teuthology.orchestra.run.smithi115.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-12-11T17:34:30.551 INFO:teuthology.orchestra.run.smithi115.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-12-11T17:34:30.551 INFO:teuthology.orchestra.run.smithi115.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-12-11T17:34:30.551 INFO:teuthology.orchestra.run.smithi115.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-12-11T17:34:30.551 INFO:teuthology.orchestra.run.smithi115.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-12-11T17:34:30.552 INFO:teuthology.orchestra.run.smithi115.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-12-11T17:34:30.552 INFO:teuthology.orchestra.run.smithi115.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-12-11T17:34:30.552 INFO:teuthology.orchestra.run.smithi115.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-12-11T17:34:30.552 INFO:teuthology.orchestra.run.smithi115.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-12-11T17:34:30.552 INFO:teuthology.orchestra.run.smithi115.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-12-11T17:34:30.552 INFO:teuthology.orchestra.run.smithi115.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-12-11T17:34:30.552 INFO:teuthology.orchestra.run.smithi115.stdout:Enabling module streams: 2023-12-11T17:34:30.552 INFO:teuthology.orchestra.run.smithi115.stdout: postgresql 10 2023-12-11T17:34:30.552 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-11T17:34:30.552 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2023-12-11T17:34:30.552 INFO:teuthology.orchestra.run.smithi115.stdout:=========================================================================================================================== 2023-12-11T17:34:30.552 INFO:teuthology.orchestra.run.smithi115.stdout:Install 129 Packages 2023-12-11T17:34:30.552 INFO:teuthology.orchestra.run.smithi115.stdout:Upgrade 3 Packages 2023-12-11T17:34:30.552 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-11T17:34:30.553 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 209 M 2023-12-11T17:34:30.553 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2023-12-11T17:34:30.701 INFO:teuthology.orchestra.run.smithi181.stdout:(5/132): ceph-mgr-19.0.0-119.ga62fda73.el8.x86_ 7.6 MB/s | 1.6 MB 00:00 2023-12-11T17:34:30.802 INFO:teuthology.orchestra.run.smithi115.stdout:(1/132): ceph-19.0.0-119.ga62fda73.el8.x86_64.r 30 kB/s | 6.4 kB 00:00 2023-12-11T17:34:30.928 INFO:teuthology.orchestra.run.smithi181.stdout:(6/132): ceph-base-19.0.0-119.ga62fda73.el8.x86 4.3 MB/s | 5.4 MB 00:01 2023-12-11T17:34:31.145 INFO:teuthology.orchestra.run.smithi181.stdout:(7/132): ceph-mon-19.0.0-119.ga62fda73.el8.x86_ 10 MB/s | 4.6 MB 00:00 2023-12-11T17:34:31.160 INFO:teuthology.orchestra.run.smithi059.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-12-11T17:34:31.160 INFO:teuthology.orchestra.run.smithi059.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-12-11T17:34:31.176 INFO:teuthology.orchestra.run.smithi059.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-12-11T17:34:31.177 INFO:teuthology.orchestra.run.smithi059.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-12-11T17:34:31.178 INFO:teuthology.orchestra.run.smithi059.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-12-11T17:34:31.179 INFO:teuthology.orchestra.run.smithi059.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-12-11T17:34:31.181 INFO:teuthology.orchestra.run.smithi059.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-12-11T17:34:31.185 INFO:teuthology.orchestra.run.smithi059.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-12-11T17:34:31.186 INFO:teuthology.orchestra.run.smithi059.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-12-11T17:34:31.327 INFO:teuthology.orchestra.run.smithi059.stdout:Dependencies resolved. 2023-12-11T17:34:31.340 INFO:teuthology.orchestra.run.smithi059.stdout:=========================================================================================================================== 2023-12-11T17:34:31.341 INFO:teuthology.orchestra.run.smithi059.stdout: Package Arch Version Repository Size 2023-12-11T17:34:31.341 INFO:teuthology.orchestra.run.smithi059.stdout:=========================================================================================================================== 2023-12-11T17:34:31.341 INFO:teuthology.orchestra.run.smithi059.stdout:Installing: 2023-12-11T17:34:31.341 INFO:teuthology.orchestra.run.smithi059.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-12-11T17:34:31.341 INFO:teuthology.orchestra.run.smithi059.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-12-11T17:34:31.341 INFO:teuthology.orchestra.run.smithi059.stdout: ceph x86_64 2:19.0.0-119.ga62fda73.el8 ceph 6.4 k 2023-12-11T17:34:31.341 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-base x86_64 2:19.0.0-119.ga62fda73.el8 ceph 5.4 M 2023-12-11T17:34:31.341 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-fuse x86_64 2:19.0.0-119.ga62fda73.el8 ceph 919 k 2023-12-11T17:34:31.341 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-119.ga62fda73.el8 ceph 171 k 2023-12-11T17:34:31.341 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr x86_64 2:19.0.0-119.ga62fda73.el8 ceph 1.6 M 2023-12-11T17:34:31.341 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-cephadm noarch 2:19.0.0-119.ga62fda73.el8 ceph-noarch 133 k 2023-12-11T17:34:31.341 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-dashboard noarch 2:19.0.0-119.ga62fda73.el8 ceph-noarch 1.8 M 2023-12-11T17:34:31.341 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-119.ga62fda73.el8 ceph-noarch 7.4 M 2023-12-11T17:34:31.342 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-rook noarch 2:19.0.0-119.ga62fda73.el8 ceph-noarch 50 k 2023-12-11T17:34:31.342 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-radosgw x86_64 2:19.0.0-119.ga62fda73.el8 ceph 14 M 2023-12-11T17:34:31.342 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-test x86_64 2:19.0.0-119.ga62fda73.el8 ceph 49 M 2023-12-11T17:34:31.342 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-volume noarch 2:19.0.0-119.ga62fda73.el8 ceph-noarch 262 k 2023-12-11T17:34:31.342 INFO:teuthology.orchestra.run.smithi059.stdout: cephadm noarch 2:19.0.0-119.ga62fda73.el8 ceph-noarch 415 k 2023-12-11T17:34:31.342 INFO:teuthology.orchestra.run.smithi059.stdout: cephfs-mirror x86_64 2:19.0.0-119.ga62fda73.el8 ceph 262 k 2023-12-11T17:34:31.342 INFO:teuthology.orchestra.run.smithi059.stdout: cephfs-top noarch 2:19.0.0-119.ga62fda73.el8 ceph-noarch 18 k 2023-12-11T17:34:31.342 INFO:teuthology.orchestra.run.smithi059.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-12-11T17:34:31.342 INFO:teuthology.orchestra.run.smithi059.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-12-11T17:34:31.342 INFO:teuthology.orchestra.run.smithi059.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-12-11T17:34:31.342 INFO:teuthology.orchestra.run.smithi059.stdout: libcephfs-devel x86_64 2:19.0.0-119.ga62fda73.el8 ceph 31 k 2023-12-11T17:34:31.342 INFO:teuthology.orchestra.run.smithi059.stdout: libcephfs2 x86_64 2:19.0.0-119.ga62fda73.el8 ceph 838 k 2023-12-11T17:34:31.342 INFO:teuthology.orchestra.run.smithi059.stdout: librados-devel x86_64 2:19.0.0-119.ga62fda73.el8 ceph 139 k 2023-12-11T17:34:31.343 INFO:teuthology.orchestra.run.smithi059.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-12-11T17:34:31.343 INFO:teuthology.orchestra.run.smithi059.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-12-11T17:34:31.343 INFO:teuthology.orchestra.run.smithi059.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-12-11T17:34:31.343 INFO:teuthology.orchestra.run.smithi059.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-12-11T17:34:31.343 INFO:teuthology.orchestra.run.smithi059.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-12-11T17:34:31.343 INFO:teuthology.orchestra.run.smithi059.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-12-11T17:34:31.343 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cephfs x86_64 2:19.0.0-119.ga62fda73.el8 ceph 197 k 2023-12-11T17:34:31.343 INFO:teuthology.orchestra.run.smithi059.stdout: python3-rados x86_64 2:19.0.0-119.ga62fda73.el8 ceph 371 k 2023-12-11T17:34:31.343 INFO:teuthology.orchestra.run.smithi059.stdout: python3-rbd x86_64 2:19.0.0-119.ga62fda73.el8 ceph 364 k 2023-12-11T17:34:31.343 INFO:teuthology.orchestra.run.smithi059.stdout: python3-rgw x86_64 2:19.0.0-119.ga62fda73.el8 ceph 107 k 2023-12-11T17:34:31.343 INFO:teuthology.orchestra.run.smithi059.stdout: rbd-fuse x86_64 2:19.0.0-119.ga62fda73.el8 ceph 93 k 2023-12-11T17:34:31.343 INFO:teuthology.orchestra.run.smithi059.stdout: rbd-mirror x86_64 2:19.0.0-119.ga62fda73.el8 ceph 3.6 M 2023-12-11T17:34:31.343 INFO:teuthology.orchestra.run.smithi059.stdout: rbd-nbd x86_64 2:19.0.0-119.ga62fda73.el8 ceph 181 k 2023-12-11T17:34:31.343 INFO:teuthology.orchestra.run.smithi059.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-12-11T17:34:31.343 INFO:teuthology.orchestra.run.smithi059.stdout:Upgrading: 2023-12-11T17:34:31.344 INFO:teuthology.orchestra.run.smithi059.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-12-11T17:34:31.344 INFO:teuthology.orchestra.run.smithi059.stdout: librados2 x86_64 2:19.0.0-119.ga62fda73.el8 ceph 3.8 M 2023-12-11T17:34:31.344 INFO:teuthology.orchestra.run.smithi059.stdout: librbd1 x86_64 2:19.0.0-119.ga62fda73.el8 ceph 3.7 M 2023-12-11T17:34:31.344 INFO:teuthology.orchestra.run.smithi059.stdout:Installing dependencies: 2023-12-11T17:34:31.344 INFO:teuthology.orchestra.run.smithi059.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-12-11T17:34:31.344 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-common x86_64 2:19.0.0-119.ga62fda73.el8 ceph 23 M 2023-12-11T17:34:31.344 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-grafana-dashboards noarch 2:19.0.0-119.ga62fda73.el8 ceph-noarch 28 k 2023-12-11T17:34:31.344 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mds x86_64 2:19.0.0-119.ga62fda73.el8 ceph 2.3 M 2023-12-11T17:34:31.344 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-modules-core noarch 2:19.0.0-119.ga62fda73.el8 ceph-noarch 243 k 2023-12-11T17:34:31.344 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mon x86_64 2:19.0.0-119.ga62fda73.el8 ceph 4.6 M 2023-12-11T17:34:31.344 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-osd x86_64 2:19.0.0-119.ga62fda73.el8 ceph 18 M 2023-12-11T17:34:31.344 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-prometheus-alerts noarch 2:19.0.0-119.ga62fda73.el8 ceph-noarch 15 k 2023-12-11T17:34:31.344 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-selinux x86_64 2:19.0.0-119.ga62fda73.el8 ceph 24 k 2023-12-11T17:34:31.344 INFO:teuthology.orchestra.run.smithi059.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-12-11T17:34:31.345 INFO:teuthology.orchestra.run.smithi059.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-12-11T17:34:31.345 INFO:teuthology.orchestra.run.smithi059.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-12-11T17:34:31.345 INFO:teuthology.orchestra.run.smithi059.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-12-11T17:34:31.345 INFO:teuthology.orchestra.run.smithi059.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-12-11T17:34:31.345 INFO:teuthology.orchestra.run.smithi059.stdout: libcephsqlite x86_64 2:19.0.0-119.ga62fda73.el8 ceph 183 k 2023-12-11T17:34:31.345 INFO:teuthology.orchestra.run.smithi059.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-12-11T17:34:31.345 INFO:teuthology.orchestra.run.smithi059.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-12-11T17:34:31.345 INFO:teuthology.orchestra.run.smithi059.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-12-11T17:34:31.345 INFO:teuthology.orchestra.run.smithi059.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-12-11T17:34:31.345 INFO:teuthology.orchestra.run.smithi059.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-12-11T17:34:31.345 INFO:teuthology.orchestra.run.smithi059.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-12-11T17:34:31.345 INFO:teuthology.orchestra.run.smithi059.stdout: libradosstriper1 x86_64 2:19.0.0-119.ga62fda73.el8 ceph 517 k 2023-12-11T17:34:31.345 INFO:teuthology.orchestra.run.smithi059.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-12-11T17:34:31.346 INFO:teuthology.orchestra.run.smithi059.stdout: librgw2 x86_64 2:19.0.0-119.ga62fda73.el8 ceph 6.9 M 2023-12-11T17:34:31.346 INFO:teuthology.orchestra.run.smithi059.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-12-11T17:34:31.346 INFO:teuthology.orchestra.run.smithi059.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-12-11T17:34:31.346 INFO:teuthology.orchestra.run.smithi059.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-12-11T17:34:31.346 INFO:teuthology.orchestra.run.smithi059.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-12-11T17:34:31.346 INFO:teuthology.orchestra.run.smithi059.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-12-11T17:34:31.346 INFO:teuthology.orchestra.run.smithi059.stdout: luarocks noarch 3.5.0-1.el8 epel 167 k 2023-12-11T17:34:31.346 INFO:teuthology.orchestra.run.smithi059.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-12-11T17:34:31.346 INFO:teuthology.orchestra.run.smithi059.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-12-11T17:34:31.346 INFO:teuthology.orchestra.run.smithi059.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-12-11T17:34:31.346 INFO:teuthology.orchestra.run.smithi059.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-12-11T17:34:31.346 INFO:teuthology.orchestra.run.smithi059.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-12-11T17:34:31.346 INFO:teuthology.orchestra.run.smithi059.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-12-11T17:34:31.346 INFO:teuthology.orchestra.run.smithi059.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-12-11T17:34:31.347 INFO:teuthology.orchestra.run.smithi059.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-12-11T17:34:31.347 INFO:teuthology.orchestra.run.smithi059.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-12-11T17:34:31.347 INFO:teuthology.orchestra.run.smithi059.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-12-11T17:34:31.347 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-12-11T17:34:31.347 INFO:teuthology.orchestra.run.smithi059.stdout: python3-ceph-argparse x86_64 2:19.0.0-119.ga62fda73.el8 ceph 45 k 2023-12-11T17:34:31.347 INFO:teuthology.orchestra.run.smithi059.stdout: python3-ceph-common x86_64 2:19.0.0-119.ga62fda73.el8 ceph 129 k 2023-12-11T17:34:31.347 INFO:teuthology.orchestra.run.smithi059.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-12-11T17:34:31.347 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-12-11T17:34:31.347 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-12-11T17:34:31.347 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-12-11T17:34:31.347 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-12-11T17:34:31.347 INFO:teuthology.orchestra.run.smithi059.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-12-11T17:34:31.347 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-12-11T17:34:31.347 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-12-11T17:34:31.348 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-12-11T17:34:31.348 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-12-11T17:34:31.348 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-12-11T17:34:31.348 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-12-11T17:34:31.348 INFO:teuthology.orchestra.run.smithi059.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-12-11T17:34:31.348 INFO:teuthology.orchestra.run.smithi059.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-12-11T17:34:31.348 INFO:teuthology.orchestra.run.smithi059.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-12-11T17:34:31.348 INFO:teuthology.orchestra.run.smithi059.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-12-11T17:34:31.348 INFO:teuthology.orchestra.run.smithi059.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-12-11T17:34:31.348 INFO:teuthology.orchestra.run.smithi059.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-12-11T17:34:31.348 INFO:teuthology.orchestra.run.smithi059.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-12-11T17:34:31.349 INFO:teuthology.orchestra.run.smithi059.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-12-11T17:34:31.349 INFO:teuthology.orchestra.run.smithi059.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-12-11T17:34:31.349 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-12-11T17:34:31.349 INFO:teuthology.orchestra.run.smithi059.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-12-11T17:34:31.349 INFO:teuthology.orchestra.run.smithi059.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-12-11T17:34:31.349 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-12-11T17:34:31.349 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-12-11T17:34:31.349 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-12-11T17:34:31.349 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-12-11T17:34:31.349 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-12-11T17:34:31.349 INFO:teuthology.orchestra.run.smithi059.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-12-11T17:34:31.349 INFO:teuthology.orchestra.run.smithi059.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-12-11T17:34:31.349 INFO:teuthology.orchestra.run.smithi059.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-12-11T17:34:31.349 INFO:teuthology.orchestra.run.smithi059.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-12-11T17:34:31.350 INFO:teuthology.orchestra.run.smithi059.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-12-11T17:34:31.350 INFO:teuthology.orchestra.run.smithi059.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-12-11T17:34:31.350 INFO:teuthology.orchestra.run.smithi059.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-12-11T17:34:31.350 INFO:teuthology.orchestra.run.smithi059.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-12-11T17:34:31.350 INFO:teuthology.orchestra.run.smithi059.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-12-11T17:34:31.350 INFO:teuthology.orchestra.run.smithi059.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-12-11T17:34:31.350 INFO:teuthology.orchestra.run.smithi059.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-12-11T17:34:31.350 INFO:teuthology.orchestra.run.smithi059.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-12-11T17:34:31.350 INFO:teuthology.orchestra.run.smithi059.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-12-11T17:34:31.350 INFO:teuthology.orchestra.run.smithi059.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-12-11T17:34:31.350 INFO:teuthology.orchestra.run.smithi059.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-12-11T17:34:31.350 INFO:teuthology.orchestra.run.smithi059.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-12-11T17:34:31.350 INFO:teuthology.orchestra.run.smithi059.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-12-11T17:34:31.350 INFO:teuthology.orchestra.run.smithi059.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-12-11T17:34:31.351 INFO:teuthology.orchestra.run.smithi059.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-12-11T17:34:31.351 INFO:teuthology.orchestra.run.smithi059.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-12-11T17:34:31.351 INFO:teuthology.orchestra.run.smithi059.stdout:Enabling module streams: 2023-12-11T17:34:31.351 INFO:teuthology.orchestra.run.smithi059.stdout: postgresql 10 2023-12-11T17:34:31.351 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-12-11T17:34:31.351 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction Summary 2023-12-11T17:34:31.351 INFO:teuthology.orchestra.run.smithi059.stdout:=========================================================================================================================== 2023-12-11T17:34:31.351 INFO:teuthology.orchestra.run.smithi059.stdout:Install 129 Packages 2023-12-11T17:34:31.351 INFO:teuthology.orchestra.run.smithi059.stdout:Upgrade 3 Packages 2023-12-11T17:34:31.351 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-12-11T17:34:31.352 INFO:teuthology.orchestra.run.smithi059.stdout:Total download size: 209 M 2023-12-11T17:34:31.352 INFO:teuthology.orchestra.run.smithi059.stdout:Downloading Packages: 2023-12-11T17:34:31.386 INFO:teuthology.orchestra.run.smithi115.stdout:(2/132): ceph-fuse-19.0.0-119.ga62fda73.el8.x86 1.5 MB/s | 919 kB 00:00 2023-12-11T17:34:31.578 INFO:teuthology.orchestra.run.smithi115.stdout:(3/132): ceph-immutable-object-cache-19.0.0-119 891 kB/s | 171 kB 00:00 2023-12-11T17:34:31.777 INFO:teuthology.orchestra.run.smithi059.stdout:(1/132): ceph-19.0.0-119.ga62fda73.el8.x86_64.r 16 kB/s | 6.4 kB 00:00 2023-12-11T17:34:32.518 INFO:teuthology.orchestra.run.smithi059.stdout:(2/132): ceph-fuse-19.0.0-119.ga62fda73.el8.x86 1.2 MB/s | 919 kB 00:00 2023-12-11T17:34:32.619 INFO:teuthology.orchestra.run.smithi059.stdout:(3/132): ceph-immutable-object-cache-19.0.0-119 1.7 MB/s | 171 kB 00:00 2023-12-11T17:34:32.811 INFO:teuthology.orchestra.run.smithi115.stdout:(4/132): ceph-mds-19.0.0-119.ga62fda73.el8.x86_ 1.9 MB/s | 2.3 MB 00:01 2023-12-11T17:34:32.937 INFO:teuthology.orchestra.run.smithi181.stdout:(8/132): ceph-radosgw-19.0.0-119.ga62fda73.el8. 7.7 MB/s | 14 MB 00:01 2023-12-11T17:34:33.037 INFO:teuthology.orchestra.run.smithi181.stdout:(9/132): ceph-selinux-19.0.0-119.ga62fda73.el8. 244 kB/s | 24 kB 00:00 2023-12-11T17:34:33.427 INFO:teuthology.orchestra.run.smithi059.stdout:(4/132): ceph-mds-19.0.0-119.ga62fda73.el8.x86_ 2.9 MB/s | 2.3 MB 00:00 2023-12-11T17:34:33.495 INFO:teuthology.orchestra.run.smithi115.stdout:(5/132): ceph-base-19.0.0-119.ga62fda73.el8.x86 1.8 MB/s | 5.4 MB 00:02 2023-12-11T17:34:33.612 INFO:teuthology.orchestra.run.smithi115.stdout:(6/132): ceph-mgr-19.0.0-119.ga62fda73.el8.x86_ 2.0 MB/s | 1.6 MB 00:00 2023-12-11T17:34:33.886 INFO:teuthology.orchestra.run.smithi059.stdout:(5/132): ceph-base-19.0.0-119.ga62fda73.el8.x86 2.1 MB/s | 5.4 MB 00:02 2023-12-11T17:34:33.928 INFO:teuthology.orchestra.run.smithi059.stdout:(6/132): ceph-mgr-19.0.0-119.ga62fda73.el8.x86_ 3.3 MB/s | 1.6 MB 00:00 2023-12-11T17:34:34.057 INFO:teuthology.orchestra.run.smithi181.stdout:(10/132): ceph-osd-19.0.0-119.ga62fda73.el8.x86 5.8 MB/s | 18 MB 00:03 2023-12-11T17:34:34.313 INFO:teuthology.orchestra.run.smithi181.stdout:(11/132): cephfs-mirror-19.0.0-119.ga62fda73.el 1.0 MB/s | 262 kB 00:00 2023-12-11T17:34:34.421 INFO:teuthology.orchestra.run.smithi181.stdout:(12/132): libcephfs-devel-19.0.0-119.ga62fda73. 287 kB/s | 31 kB 00:00 2023-12-11T17:34:34.462 INFO:teuthology.orchestra.run.smithi115.stdout:(7/132): ceph-mon-19.0.0-119.ga62fda73.el8.x86_ 4.8 MB/s | 4.6 MB 00:00 2023-12-11T17:34:35.080 INFO:teuthology.orchestra.run.smithi181.stdout:(13/132): libcephfs2-19.0.0-119.ga62fda73.el8.x 1.2 MB/s | 838 kB 00:00 2023-12-11T17:34:35.305 INFO:teuthology.orchestra.run.smithi181.stdout:(14/132): ceph-common-19.0.0-119.ga62fda73.el8. 4.0 MB/s | 23 MB 00:05 2023-12-11T17:34:35.330 INFO:teuthology.orchestra.run.smithi181.stdout:(15/132): libcephsqlite-19.0.0-119.ga62fda73.el 731 kB/s | 183 kB 00:00 2023-12-11T17:34:35.430 INFO:teuthology.orchestra.run.smithi181.stdout:(16/132): librados-devel-19.0.0-119.ga62fda73.e 1.1 MB/s | 139 kB 00:00 2023-12-11T17:34:35.689 INFO:teuthology.orchestra.run.smithi181.stdout:(17/132): libradosstriper1-19.0.0-119.ga62fda73 1.4 MB/s | 517 kB 00:00 2023-12-11T17:34:35.778 INFO:teuthology.orchestra.run.smithi059.stdout:(7/132): ceph-mon-19.0.0-119.ga62fda73.el8.x86_ 2.5 MB/s | 4.6 MB 00:01 2023-12-11T17:34:35.823 INFO:teuthology.orchestra.run.smithi181.stdout:(18/132): python3-ceph-argparse-19.0.0-119.ga62 335 kB/s | 45 kB 00:00 2023-12-11T17:34:35.996 INFO:teuthology.orchestra.run.smithi181.stdout:(19/132): python3-ceph-common-19.0.0-119.ga62fd 742 kB/s | 129 kB 00:00 2023-12-11T17:34:36.354 INFO:teuthology.orchestra.run.smithi115.stdout:(8/132): ceph-radosgw-19.0.0-119.ga62fda73.el8. 7.3 MB/s | 14 MB 00:01 2023-12-11T17:34:36.429 INFO:teuthology.orchestra.run.smithi115.stdout:(9/132): ceph-selinux-19.0.0-119.ga62fda73.el8. 326 kB/s | 24 kB 00:00 2023-12-11T17:34:36.440 INFO:teuthology.orchestra.run.smithi181.stdout:(20/132): python3-cephfs-19.0.0-119.ga62fda73.e 444 kB/s | 197 kB 00:00 2023-12-11T17:34:36.782 INFO:teuthology.orchestra.run.smithi181.stdout:(21/132): python3-rados-19.0.0-119.ga62fda73.el 1.1 MB/s | 371 kB 00:00 2023-12-11T17:34:37.057 INFO:teuthology.orchestra.run.smithi181.stdout:(22/132): python3-rbd-19.0.0-119.ga62fda73.el8. 1.3 MB/s | 364 kB 00:00 2023-12-11T17:34:37.157 INFO:teuthology.orchestra.run.smithi181.stdout:(23/132): librgw2-19.0.0-119.ga62fda73.el8.x86_ 4.0 MB/s | 6.9 MB 00:01 2023-12-11T17:34:37.183 INFO:teuthology.orchestra.run.smithi181.stdout:(24/132): python3-rgw-19.0.0-119.ga62fda73.el8. 854 kB/s | 107 kB 00:00 2023-12-11T17:34:37.250 INFO:teuthology.orchestra.run.smithi181.stdout:(25/132): rbd-fuse-19.0.0-119.ga62fda73.el8.x86 1.0 MB/s | 93 kB 00:00 2023-12-11T17:34:37.350 INFO:teuthology.orchestra.run.smithi181.stdout:(26/132): rbd-nbd-19.0.0-119.ga62fda73.el8.x86_ 1.8 MB/s | 181 kB 00:00 2023-12-11T17:34:37.450 INFO:teuthology.orchestra.run.smithi181.stdout:(27/132): ceph-grafana-dashboards-19.0.0-119.ga 280 kB/s | 28 kB 00:00 2023-12-11T17:34:37.551 INFO:teuthology.orchestra.run.smithi181.stdout:(28/132): ceph-mgr-cephadm-19.0.0-119.ga62fda73 1.3 MB/s | 133 kB 00:00 2023-12-11T17:34:38.009 INFO:teuthology.orchestra.run.smithi181.stdout:(29/132): ceph-mgr-dashboard-19.0.0-119.ga62fda 3.8 MB/s | 1.8 MB 00:00 2023-12-11T17:34:39.195 INFO:teuthology.orchestra.run.smithi059.stdout:(8/132): ceph-osd-19.0.0-119.ga62fda73.el8.x86_ 3.4 MB/s | 18 MB 00:05 2023-12-11T17:34:39.303 INFO:teuthology.orchestra.run.smithi059.stdout:(9/132): ceph-selinux-19.0.0-119.ga62fda73.el8. 225 kB/s | 24 kB 00:00 2023-12-11T17:34:39.356 INFO:teuthology.orchestra.run.smithi181.stdout:(30/132): rbd-mirror-19.0.0-119.ga62fda73.el8.x 1.7 MB/s | 3.6 MB 00:02 2023-12-11T17:34:39.560 INFO:teuthology.orchestra.run.smithi181.stdout:(31/132): ceph-mgr-modules-core-19.0.0-119.ga62 1.2 MB/s | 243 kB 00:00 2023-12-11T17:34:39.677 INFO:teuthology.orchestra.run.smithi181.stdout:(32/132): ceph-mgr-rook-19.0.0-119.ga62fda73.el 430 kB/s | 50 kB 00:00 2023-12-11T17:34:39.842 INFO:teuthology.orchestra.run.smithi181.stdout:(33/132): ceph-mgr-diskprediction-local-19.0.0- 4.0 MB/s | 7.4 MB 00:01 2023-12-11T17:34:39.869 INFO:teuthology.orchestra.run.smithi181.stdout:(34/132): ceph-prometheus-alerts-19.0.0-119.ga6 78 kB/s | 15 kB 00:00 2023-12-11T17:34:40.030 INFO:teuthology.orchestra.run.smithi181.stdout:(35/132): ceph-volume-19.0.0-119.ga62fda73.el8. 1.4 MB/s | 262 kB 00:00 2023-12-11T17:34:40.053 INFO:teuthology.orchestra.run.smithi181.stdout:(36/132): cephadm-19.0.0-119.ga62fda73.el8.noar 2.2 MB/s | 415 kB 00:00 2023-12-11T17:34:40.120 INFO:teuthology.orchestra.run.smithi181.stdout:(37/132): cephfs-top-19.0.0-119.ga62fda73.el8.n 197 kB/s | 18 kB 00:00 2023-12-11T17:34:40.362 INFO:teuthology.orchestra.run.smithi181.stdout:(38/132): liboath-2.6.2-3.el8.x86_64.rpm 243 kB/s | 59 kB 00:00 2023-12-11T17:34:40.487 INFO:teuthology.orchestra.run.smithi181.stdout:(39/132): gperftools-libs-2.7-9.el8.x86_64.rpm 706 kB/s | 306 kB 00:00 2023-12-11T17:34:40.512 INFO:teuthology.orchestra.run.smithi181.stdout:(40/132): libunwind-1.3.1-3.el8.x86_64.rpm 500 kB/s | 75 kB 00:00 2023-12-11T17:34:40.629 INFO:teuthology.orchestra.run.smithi181.stdout:(41/132): luarocks-3.5.0-1.el8.noarch.rpm 1.2 MB/s | 167 kB 00:00 2023-12-11T17:34:40.721 INFO:teuthology.orchestra.run.smithi181.stdout:(42/132): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 476 kB/s | 44 kB 00:00 2023-12-11T17:34:40.746 INFO:teuthology.orchestra.run.smithi181.stdout:(43/132): python3-asyncssh-2.7.0-2.el8.noarch.r 2.0 MB/s | 476 kB 00:00 2023-12-11T17:34:40.847 INFO:teuthology.orchestra.run.smithi181.stdout:(44/132): python3-cachetools-3.1.1-4.el8.noarch 330 kB/s | 33 kB 00:00 2023-12-11T17:34:40.872 INFO:teuthology.orchestra.run.smithi181.stdout:(45/132): python3-beautifulsoup4-4.6.3-2.el8.1. 1.2 MB/s | 185 kB 00:00 2023-12-11T17:34:40.922 INFO:teuthology.orchestra.run.smithi181.stdout:(46/132): python3-certifi-2018.10.15-7.el8.noar 217 kB/s | 16 kB 00:00 2023-12-11T17:34:41.106 INFO:teuthology.orchestra.run.smithi181.stdout:(47/132): python3-cherrypy-18.4.0-1.el8.noarch. 2.1 MB/s | 384 kB 00:00 2023-12-11T17:34:41.137 INFO:teuthology.orchestra.run.smithi059.stdout:(10/132): ceph-common-19.0.0-119.ga62fda73.el8. 2.3 MB/s | 23 MB 00:09 2023-12-11T17:34:41.173 INFO:teuthology.orchestra.run.smithi181.stdout:(48/132): python3-cheroot-8.5.2-1.el8.noarch.rp 574 kB/s | 173 kB 00:00 2023-12-11T17:34:41.198 INFO:teuthology.orchestra.run.smithi181.stdout:(49/132): python3-google-auth-1.1.1-10.el8.noar 884 kB/s | 81 kB 00:00 2023-12-11T17:34:41.265 INFO:teuthology.orchestra.run.smithi181.stdout:(50/132): python3-jaraco-6.2-6.el8.noarch.rpm 121 kB/s | 11 kB 00:00 2023-12-11T17:34:41.316 INFO:teuthology.orchestra.run.smithi181.stdout:(51/132): python3-jaraco-functools-2.0-4.el8.no 150 kB/s | 18 kB 00:00 2023-12-11T17:34:41.320 INFO:teuthology.orchestra.run.smithi059.stdout:(11/132): cephfs-mirror-19.0.0-119.ga62fda73.el 1.4 MB/s | 262 kB 00:00 2023-12-11T17:34:41.379 INFO:teuthology.orchestra.run.smithi059.stdout:(12/132): libcephfs-devel-19.0.0-119.ga62fda73. 532 kB/s | 31 kB 00:00 2023-12-11T17:34:41.399 INFO:teuthology.orchestra.run.smithi181.stdout:(52/132): python3-logutils-0.3.5-11.el8.noarch. 584 kB/s | 49 kB 00:00 2023-12-11T17:34:41.491 INFO:teuthology.orchestra.run.smithi181.stdout:(53/132): python3-more-itertools-7.2.0-3.el8.no 641 kB/s | 59 kB 00:00 2023-12-11T17:34:41.566 INFO:teuthology.orchestra.run.smithi181.stdout:(54/132): python3-natsort-7.1.1-2.el8.noarch.rp 806 kB/s | 60 kB 00:00 2023-12-11T17:34:41.675 INFO:teuthology.orchestra.run.smithi181.stdout:(55/132): python3-pecan-1.3.2-9.el8.noarch.rpm 2.6 MB/s | 283 kB 00:00 2023-12-11T17:34:41.738 INFO:teuthology.orchestra.run.smithi059.stdout:(13/132): ceph-radosgw-19.0.0-119.ga62fda73.el8 2.3 MB/s | 14 MB 00:05 2023-12-11T17:34:41.750 INFO:teuthology.orchestra.run.smithi181.stdout:(56/132): python3-portend-2.6-1.el8.noarch.rpm 207 kB/s | 16 kB 00:00 2023-12-11T17:34:41.771 INFO:teuthology.orchestra.run.smithi059.stdout:(14/132): libcephfs2-19.0.0-119.ga62fda73.el8.x 2.1 MB/s | 838 kB 00:00 2023-12-11T17:34:41.847 INFO:teuthology.orchestra.run.smithi059.stdout:(15/132): librados-devel-19.0.0-119.ga62fda73.e 1.8 MB/s | 139 kB 00:00 2023-12-11T17:34:41.872 INFO:teuthology.orchestra.run.smithi059.stdout:(16/132): libcephsqlite-19.0.0-119.ga62fda73.el 1.3 MB/s | 183 kB 00:00 2023-12-11T17:34:42.014 INFO:teuthology.orchestra.run.smithi059.stdout:(17/132): libradosstriper1-19.0.0-119.ga62fda73 3.0 MB/s | 517 kB 00:00 2023-12-11T17:34:42.038 INFO:teuthology.orchestra.run.smithi181.stdout:(57/132): python3-repoze-lru-0.7-6.el8.noarch.r 117 kB/s | 34 kB 00:00 2023-12-11T17:34:42.056 INFO:teuthology.orchestra.run.smithi059.stdout:(18/132): python3-ceph-argparse-19.0.0-119.ga62 1.0 MB/s | 45 kB 00:00 2023-12-11T17:34:42.109 INFO:teuthology.orchestra.run.smithi181.stdout:(58/132): python3-kubernetes-11.0.0-6.el8.noarc 1.4 MB/s | 1.2 MB 00:00 2023-12-11T17:34:42.126 INFO:teuthology.orchestra.run.smithi059.stdout:(19/132): python3-ceph-common-19.0.0-119.ga62fd 1.8 MB/s | 129 kB 00:00 2023-12-11T17:34:42.176 INFO:teuthology.orchestra.run.smithi181.stdout:(59/132): python3-routes-2.4.1-12.el8.noarch.rp 1.4 MB/s | 196 kB 00:00 2023-12-11T17:34:42.207 INFO:teuthology.orchestra.run.smithi059.stdout:(20/132): python3-cephfs-19.0.0-119.ga62fda73.e 2.4 MB/s | 197 kB 00:00 2023-12-11T17:34:42.210 INFO:teuthology.orchestra.run.smithi181.stdout:(60/132): python3-rsa-4.9-2.el8.noarch.rpm 640 kB/s | 64 kB 00:00 2023-12-11T17:34:42.277 INFO:teuthology.orchestra.run.smithi181.stdout:(61/132): python3-simplegeneric-0.8.1-17.el8.no 186 kB/s | 19 kB 00:00 2023-12-11T17:34:42.294 INFO:teuthology.orchestra.run.smithi181.stdout:(62/132): python3-singledispatch-3.4.0.3-18.el8 286 kB/s | 24 kB 00:00 2023-12-11T17:34:42.306 INFO:teuthology.orchestra.run.smithi059.stdout:(21/132): python3-rados-19.0.0-119.ga62fda73.el 3.6 MB/s | 371 kB 00:00 2023-12-11T17:34:42.390 INFO:teuthology.orchestra.run.smithi059.stdout:(22/132): python3-rbd-19.0.0-119.ga62fda73.el8. 4.3 MB/s | 364 kB 00:00 2023-12-11T17:34:42.437 INFO:teuthology.orchestra.run.smithi115.stdout:(10/132): ceph-test-19.0.0-119.ga62fda73.el8.x8 8.2 MB/s | 49 MB 00:06 2023-12-11T17:34:42.440 INFO:teuthology.orchestra.run.smithi059.stdout:(23/132): python3-rgw-19.0.0-119.ga62fda73.el8. 2.1 MB/s | 107 kB 00:00 2023-12-11T17:34:42.504 INFO:teuthology.orchestra.run.smithi115.stdout:(11/132): cephfs-mirror-19.0.0-119.ga62fda73.el 3.9 MB/s | 262 kB 00:00 2023-12-11T17:34:42.528 INFO:teuthology.orchestra.run.smithi059.stdout:(24/132): rbd-fuse-19.0.0-119.ga62fda73.el8.x86 1.0 MB/s | 93 kB 00:00 2023-12-11T17:34:42.742 INFO:teuthology.orchestra.run.smithi181.stdout:(63/132): ceph-test-19.0.0-119.ga62fda73.el8.x8 5.1 MB/s | 49 MB 00:09 2023-12-11T17:34:42.761 INFO:teuthology.orchestra.run.smithi181.stdout:(64/132): python3-tempora-1.14.1-5.el8.noarch.r 60 kB/s | 29 kB 00:00 2023-12-11T17:34:42.778 INFO:teuthology.orchestra.run.smithi181.stdout:(65/132): python3-trustme-0.6.0-4.el8.noarch.rp 57 kB/s | 27 kB 00:00 2023-12-11T17:34:42.784 INFO:teuthology.orchestra.run.smithi115.stdout:(12/132): ceph-common-19.0.0-119.ga62fda73.el8. 1.8 MB/s | 23 MB 00:12 2023-12-11T17:34:42.804 INFO:teuthology.orchestra.run.smithi115.stdout:(13/132): libcephfs-devel-19.0.0-119.ga62fda73. 103 kB/s | 31 kB 00:00 2023-12-11T17:34:42.862 INFO:teuthology.orchestra.run.smithi181.stdout:(66/132): python3-websocket-client-0.56.0-5.el8 737 kB/s | 61 kB 00:00 2023-12-11T17:34:42.879 INFO:teuthology.orchestra.run.smithi115.stdout:(14/132): libcephsqlite-19.0.0-119.ga62fda73.el 2.4 MB/s | 183 kB 00:00 2023-12-11T17:34:42.937 INFO:teuthology.orchestra.run.smithi181.stdout:(67/132): python3-webtest-2.0.33-1.el8.noarch.r 1.1 MB/s | 85 kB 00:00 2023-12-11T17:34:42.954 INFO:teuthology.orchestra.run.smithi115.stdout:(15/132): librados-devel-19.0.0-119.ga62fda73.e 1.8 MB/s | 139 kB 00:00 2023-12-11T17:34:43.004 INFO:teuthology.orchestra.run.smithi181.stdout:(68/132): python3-webob-1.8.5-1.el8.1.noarch.rp 1.1 MB/s | 251 kB 00:00 2023-12-11T17:34:43.021 INFO:teuthology.orchestra.run.smithi181.stdout:(69/132): python3-zc-lockfile-2.0-2.el8.noarch. 275 kB/s | 23 kB 00:00 2023-12-11T17:34:43.024 INFO:teuthology.orchestra.run.smithi059.stdout:(25/132): librgw2-19.0.0-119.ga62fda73.el8.x86_ 6.0 MB/s | 6.9 MB 00:01 2023-12-11T17:34:43.046 INFO:teuthology.orchestra.run.smithi181.stdout:(70/132): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.7 MB/s | 70 kB 00:00 2023-12-11T17:34:43.055 INFO:teuthology.orchestra.run.smithi115.stdout:(16/132): libcephfs2-19.0.0-119.ga62fda73.el8.x 3.0 MB/s | 838 kB 00:00 2023-12-11T17:34:43.071 INFO:teuthology.orchestra.run.smithi181.stdout:(71/132): python3-waitress-1.4.3-1.el8.noarch.r 775 kB/s | 255 kB 00:00 2023-12-11T17:34:43.080 INFO:teuthology.orchestra.run.smithi115.stdout:(17/132): libradosstriper1-19.0.0-119.ga62fda73 4.0 MB/s | 517 kB 00:00 2023-12-11T17:34:43.108 INFO:teuthology.orchestra.run.smithi059.stdout:(26/132): rbd-mirror-19.0.0-119.ga62fda73.el8.x 6.2 MB/s | 3.6 MB 00:00 2023-12-11T17:34:43.130 INFO:teuthology.orchestra.run.smithi181.stdout:(72/132): librabbitmq-0.9.0-3.el8.x86_64.rpm 561 kB/s | 47 kB 00:00 2023-12-11T17:34:43.133 INFO:teuthology.orchestra.run.smithi059.stdout:(27/132): rbd-nbd-19.0.0-119.ga62fda73.el8.x86_ 1.6 MB/s | 181 kB 00:00 2023-12-11T17:34:43.155 INFO:teuthology.orchestra.run.smithi181.stdout:(73/132): libselinux-devel-2.9-5.el8.x86_64.rpm 2.5 MB/s | 200 kB 00:00 2023-12-11T17:34:43.159 INFO:teuthology.orchestra.run.smithi059.stdout:(28/132): ceph-grafana-dashboards-19.0.0-119.ga 557 kB/s | 28 kB 00:00 2023-12-11T17:34:43.180 INFO:teuthology.orchestra.run.smithi181.stdout:(74/132): python3-jwt-1.6.1-2.el8.noarch.rpm 848 kB/s | 43 kB 00:00 2023-12-11T17:34:43.184 INFO:teuthology.orchestra.run.smithi059.stdout:(29/132): ceph-mgr-cephadm-19.0.0-119.ga62fda73 2.6 MB/s | 133 kB 00:00 2023-12-11T17:34:43.236 INFO:teuthology.orchestra.run.smithi181.stdout:(75/132): pcre2-utf32-10.32-2.el8.x86_64.rpm 2.7 MB/s | 220 kB 00:00 2023-12-11T17:34:43.264 INFO:teuthology.orchestra.run.smithi181.stdout:(76/132): keyutils-libs-devel-1.5.10-9.el8.x86_ 576 kB/s | 48 kB 00:00 2023-12-11T17:34:43.272 INFO:teuthology.orchestra.run.smithi115.stdout:(18/132): ceph-osd-19.0.0-119.ga62fda73.el8.x86 1.9 MB/s | 18 MB 00:09 2023-12-11T17:34:43.290 INFO:teuthology.orchestra.run.smithi181.stdout:(77/132): libsepol-devel-2.9-3.el8.x86_64.rpm 1.6 MB/s | 87 kB 00:00 2023-12-11T17:34:43.297 INFO:teuthology.orchestra.run.smithi115.stdout:(19/132): python3-ceph-argparse-19.0.0-119.ga62 206 kB/s | 45 kB 00:00 2023-12-11T17:34:43.315 INFO:teuthology.orchestra.run.smithi181.stdout:(78/132): python3-requests-oauthlib-1.0.0-1.el8 849 kB/s | 43 kB 00:00 2023-12-11T17:34:43.339 INFO:teuthology.orchestra.run.smithi115.stdout:(20/132): python3-ceph-common-19.0.0-119.ga62fd 1.9 MB/s | 129 kB 00:00 2023-12-11T17:34:43.340 INFO:teuthology.orchestra.run.smithi181.stdout:(79/132): python3-pycparser-2.14-14.el8.noarch. 2.1 MB/s | 109 kB 00:00 2023-12-11T17:34:43.365 INFO:teuthology.orchestra.run.smithi181.stdout:(80/132): pcre2-utf16-10.32-2.el8.x86_64.rpm 4.5 MB/s | 229 kB 00:00 2023-12-11T17:34:43.373 INFO:teuthology.orchestra.run.smithi115.stdout:(21/132): python3-cephfs-19.0.0-119.ga62fda73.e 2.6 MB/s | 197 kB 00:00 2023-12-11T17:34:43.399 INFO:teuthology.orchestra.run.smithi181.stdout:(81/132): krb5-devel-1.18.2-14.el8.x86_64.rpm 9.3 MB/s | 560 kB 00:00 2023-12-11T17:34:43.424 INFO:teuthology.orchestra.run.smithi181.stdout:(82/132): libaio-devel-0.3.112-1.el8.x86_64.rpm 319 kB/s | 19 kB 00:00 2023-12-11T17:34:43.442 INFO:teuthology.orchestra.run.smithi181.stdout:(83/132): python3-oauthlib-2.1.0-1.el8.noarch.r 3.6 MB/s | 155 kB 00:00 2023-12-11T17:34:43.459 INFO:teuthology.orchestra.run.smithi059.stdout:(30/132): ceph-mgr-dashboard-19.0.0-119.ga62fda 5.9 MB/s | 1.8 MB 00:00 2023-12-11T17:34:43.465 INFO:teuthology.orchestra.run.smithi115.stdout:(22/132): python3-rbd-19.0.0-119.ga62fda73.el8. 3.9 MB/s | 364 kB 00:00 2023-12-11T17:34:43.475 INFO:teuthology.orchestra.run.smithi181.stdout:(84/132): python3-cryptography-3.2.1-5.el8.x86_ 11 MB/s | 559 kB 00:00 2023-12-11T17:34:43.500 INFO:teuthology.orchestra.run.smithi181.stdout:(85/132): libverto-devel-0.3.0-5.el8.x86_64.rpm 306 kB/s | 18 kB 00:00 2023-12-11T17:34:43.526 INFO:teuthology.orchestra.run.smithi059.stdout:(31/132): ceph-mgr-modules-core-19.0.0-119.ga62 3.6 MB/s | 243 kB 00:00 2023-12-11T17:34:43.532 INFO:teuthology.orchestra.run.smithi115.stdout:(23/132): python3-rados-19.0.0-119.ga62fda73.el 1.9 MB/s | 371 kB 00:00 2023-12-11T17:34:43.542 INFO:teuthology.orchestra.run.smithi181.stdout:(86/132): pcre2-devel-10.32-2.el8.x86_64.rpm 8.8 MB/s | 605 kB 00:00 2023-12-11T17:34:43.549 INFO:teuthology.orchestra.run.smithi115.stdout:(24/132): python3-rgw-19.0.0-119.ga62fda73.el8. 1.2 MB/s | 107 kB 00:00 2023-12-11T17:34:43.567 INFO:teuthology.orchestra.run.smithi181.stdout:(87/132): python3-cffi-1.11.5-5.el8.x86_64.rpm 3.5 MB/s | 238 kB 00:00 2023-12-11T17:34:43.576 INFO:teuthology.orchestra.run.smithi059.stdout:(32/132): ceph-mgr-rook-19.0.0-119.ga62fda73.el 1.0 MB/s | 50 kB 00:00 2023-12-11T17:34:43.607 INFO:teuthology.orchestra.run.smithi115.stdout:(25/132): rbd-fuse-19.0.0-119.ga62fda73.el8.x86 1.2 MB/s | 93 kB 00:00 2023-12-11T17:34:43.635 INFO:teuthology.orchestra.run.smithi181.stdout:(88/132): openssl-devel-1.1.1k-7.el8_6.x86_64.r 25 MB/s | 2.3 MB 00:00 2023-12-11T17:34:43.635 INFO:teuthology.orchestra.run.smithi059.stdout:(33/132): ceph-prometheus-alerts-19.0.0-119.ga6 257 kB/s | 15 kB 00:00 2023-12-11T17:34:43.660 INFO:teuthology.orchestra.run.smithi181.stdout:(89/132): xfsprogs-devel-5.0.0-10.el8.x86_64.rp 787 kB/s | 72 kB 00:00 2023-12-11T17:34:43.710 INFO:teuthology.orchestra.run.smithi181.stdout:(90/132): fuse-2.9.7-15.el8.x86_64.rpm 1.1 MB/s | 83 kB 00:00 2023-12-11T17:34:43.744 INFO:teuthology.orchestra.run.smithi181.stdout:(91/132): elfutils-libelf-devel-0.186-1.el8.x86 714 kB/s | 60 kB 00:00 2023-12-11T17:34:43.749 INFO:teuthology.orchestra.run.smithi115.stdout:(26/132): rbd-nbd-19.0.0-119.ga62fda73.el8.x86_ 1.3 MB/s | 181 kB 00:00 2023-12-11T17:34:43.760 INFO:teuthology.orchestra.run.smithi059.stdout:(34/132): ceph-volume-19.0.0-119.ga62fda73.el8. 2.0 MB/s | 262 kB 00:00 2023-12-11T17:34:43.769 INFO:teuthology.orchestra.run.smithi181.stdout:(92/132): libuuid-devel-2.32.1-35.el8.x86_64.rp 1.6 MB/s | 98 kB 00:00 2023-12-11T17:34:43.794 INFO:teuthology.orchestra.run.smithi181.stdout:(93/132): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2023-12-11T17:34:43.816 INFO:teuthology.orchestra.run.smithi115.stdout:(27/132): ceph-grafana-dashboards-19.0.0-119.ga 421 kB/s | 28 kB 00:00 2023-12-11T17:34:43.820 INFO:teuthology.orchestra.run.smithi181.stdout:(94/132): libcom_err-devel-1.45.6-4.el8.x86_64. 770 kB/s | 39 kB 00:00 2023-12-11T17:34:43.853 INFO:teuthology.orchestra.run.smithi181.stdout:(95/132): libgfortran-8.5.0-10.1.el8_6.x86_64.r 11 MB/s | 644 kB 00:00 2023-12-11T17:34:43.878 INFO:teuthology.orchestra.run.smithi181.stdout:(96/132): libquadmath-8.5.0-10.1.el8_6.x86_64.r 2.9 MB/s | 171 kB 00:00 2023-12-11T17:34:43.904 INFO:teuthology.orchestra.run.smithi181.stdout:(97/132): NetworkManager-initscripts-updown-1.3 2.7 MB/s | 138 kB 00:00 2023-12-11T17:34:43.908 INFO:teuthology.orchestra.run.smithi115.stdout:(28/132): ceph-mgr-cephadm-19.0.0-119.ga62fda73 1.4 MB/s | 133 kB 00:00 2023-12-11T17:34:43.911 INFO:teuthology.orchestra.run.smithi059.stdout:(35/132): cephadm-19.0.0-119.ga62fda73.el8.noar 2.7 MB/s | 415 kB 00:00 2023-12-11T17:34:43.962 INFO:teuthology.orchestra.run.smithi181.stdout:(98/132): thrift-0.13.0-2.el8.x86_64.rpm 1.8 MB/s | 1.7 MB 00:00 2023-12-11T17:34:43.969 INFO:teuthology.orchestra.run.smithi059.stdout:(36/132): cephfs-top-19.0.0-119.ga62fda73.el8.n 312 kB/s | 18 kB 00:00 2023-12-11T17:34:43.975 INFO:teuthology.orchestra.run.smithi115.stdout:(29/132): rbd-mirror-19.0.0-119.ga62fda73.el8.x 8.4 MB/s | 3.6 MB 00:00 2023-12-11T17:34:44.121 INFO:teuthology.orchestra.run.smithi181.stdout:(99/132): python3-pyasn1-modules-0.3.7-6.el8.no 508 kB/s | 110 kB 00:00 2023-12-11T17:34:44.196 INFO:teuthology.orchestra.run.smithi181.stdout:(100/132): python3-pytz-2017.2-9.el8.noarch.rpm 232 kB/s | 54 kB 00:00 2023-12-11T17:34:44.288 INFO:teuthology.orchestra.run.smithi181.stdout:(101/132): lttng-ust-2.8.1-11.el8.x86_64.rpm 1.5 MB/s | 259 kB 00:00 2023-12-11T17:34:44.314 INFO:teuthology.orchestra.run.smithi181.stdout:(102/132): socat-1.7.4.1-1.el8.x86_64.rpm 2.7 MB/s | 323 kB 00:00 2023-12-11T17:34:44.372 INFO:teuthology.orchestra.run.smithi181.stdout:(103/132): python3-markupsafe-0.23-19.el8.x86_6 465 kB/s | 39 kB 00:00 2023-12-11T17:34:44.406 INFO:teuthology.orchestra.run.smithi181.stdout:(104/132): bison-3.0.4-10.el8.x86_64.rpm 7.3 MB/s | 688 kB 00:00 2023-12-11T17:34:44.439 INFO:teuthology.orchestra.run.smithi181.stdout:(105/132): python3-jinja2-2.10.1-3.el8.noarch.r 7.9 MB/s | 538 kB 00:00 2023-12-11T17:34:44.465 INFO:teuthology.orchestra.run.smithi181.stdout:(106/132): python3-mako-1.0.6-13.el8.noarch.rpm 2.6 MB/s | 157 kB 00:00 2023-12-11T17:34:44.490 INFO:teuthology.orchestra.run.smithi181.stdout:(107/132): oniguruma-6.8.2-2.el8.x86_64.rpm 3.6 MB/s | 187 kB 00:00 2023-12-11T17:34:44.492 INFO:teuthology.orchestra.run.smithi115.stdout:(30/132): ceph-mgr-dashboard-19.0.0-119.ga62fda 3.0 MB/s | 1.8 MB 00:00 2023-12-11T17:34:44.494 INFO:teuthology.orchestra.run.smithi059.stdout:(37/132): ceph-mgr-diskprediction-local-19.0.0- 5.6 MB/s | 7.4 MB 00:01 2023-12-11T17:34:44.524 INFO:teuthology.orchestra.run.smithi181.stdout:(108/132): automake-1.16.1-7.el8.noarch.rpm 12 MB/s | 713 kB 00:00 2023-12-11T17:34:44.617 INFO:teuthology.orchestra.run.smithi115.stdout:(31/132): ceph-mgr-modules-core-19.0.0-119.ga62 1.9 MB/s | 243 kB 00:00 2023-12-11T17:34:44.620 INFO:teuthology.orchestra.run.smithi059.stdout:(38/132): gperftools-libs-2.7-9.el8.x86_64.rpm 471 kB/s | 306 kB 00:00 2023-12-11T17:34:44.682 INFO:teuthology.orchestra.run.smithi181.stdout:(109/132): python3-scipy-1.0.0-21.module+el8.5. 17 MB/s | 14 MB 00:00 2023-12-11T17:34:44.708 INFO:teuthology.orchestra.run.smithi181.stdout:(110/132): python3-jsonpatch-1.21-2.el8.noarch. 144 kB/s | 27 kB 00:00 2023-12-11T17:34:44.737 INFO:teuthology.orchestra.run.smithi059.stdout:(39/132): libunwind-1.3.1-3.el8.x86_64.rpm 643 kB/s | 75 kB 00:00 2023-12-11T17:34:44.750 INFO:teuthology.orchestra.run.smithi181.stdout:(111/132): flex-2.6.1-9.el8.x86_64.rpm 4.7 MB/s | 320 kB 00:00 2023-12-11T17:34:44.751 INFO:teuthology.orchestra.run.smithi115.stdout:(32/132): ceph-mgr-diskprediction-local-19.0.0- 9.5 MB/s | 7.4 MB 00:00 2023-12-11T17:34:44.775 INFO:teuthology.orchestra.run.smithi181.stdout:(112/132): python3-prettytable-0.7.2-14.el8.noa 660 kB/s | 44 kB 00:00 2023-12-11T17:34:44.784 INFO:teuthology.orchestra.run.smithi115.stdout:(33/132): ceph-mgr-rook-19.0.0-119.ga62fda73.el 300 kB/s | 50 kB 00:00 2023-12-11T17:34:44.810 INFO:teuthology.orchestra.run.smithi115.stdout:(34/132): ceph-prometheus-alerts-19.0.0-119.ga6 255 kB/s | 15 kB 00:00 2023-12-11T17:34:44.833 INFO:teuthology.orchestra.run.smithi181.stdout:(113/132): python3-werkzeug-0.12.2-4.el8.noarch 7.7 MB/s | 457 kB 00:00 2023-12-11T17:34:44.871 INFO:teuthology.orchestra.run.smithi059.stdout:(40/132): luarocks-3.5.0-1.el8.noarch.rpm 1.2 MB/s | 167 kB 00:00 2023-12-11T17:34:44.877 INFO:teuthology.orchestra.run.smithi115.stdout:(35/132): cephadm-19.0.0-119.ga62fda73.el8.noar 6.1 MB/s | 415 kB 00:00 2023-12-11T17:34:44.896 INFO:teuthology.orchestra.run.smithi059.stdout:(41/132): liboath-2.6.2-3.el8.x86_64.rpm 147 kB/s | 59 kB 00:00 2023-12-11T17:34:44.902 INFO:teuthology.orchestra.run.smithi115.stdout:(36/132): ceph-volume-19.0.0-119.ga62fda73.el8. 2.2 MB/s | 262 kB 00:00 2023-12-11T17:34:44.908 INFO:teuthology.orchestra.run.smithi181.stdout:(114/132): python3-numpy-1.14.3-10.el8.x86_64.r 24 MB/s | 3.7 MB 00:00 2023-12-11T17:34:44.934 INFO:teuthology.orchestra.run.smithi181.stdout:(115/132): python3-numpy-f2py-1.14.3-10.el8.x86 2.2 MB/s | 225 kB 00:00 2023-12-11T17:34:44.959 INFO:teuthology.orchestra.run.smithi181.stdout:(116/132): uuid-1.6.2-43.el8.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2023-12-11T17:34:44.961 INFO:teuthology.orchestra.run.smithi115.stdout:(37/132): cephfs-top-19.0.0-119.ga62fda73.el8.n 217 kB/s | 18 kB 00:00 2023-12-11T17:34:45.026 INFO:teuthology.orchestra.run.smithi181.stdout:(117/132): python3-babel-2.5.1-7.el8.noarch.rpm 8.9 MB/s | 4.8 MB 00:00 2023-12-11T17:34:45.029 INFO:teuthology.orchestra.run.smithi059.stdout:(42/132): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 327 kB/s | 44 kB 00:00 2023-12-11T17:34:45.045 INFO:teuthology.orchestra.run.smithi115.stdout:(38/132): librgw2-19.0.0-119.ga62fda73.el8.x86_ 3.5 MB/s | 6.9 MB 00:01 2023-12-11T17:34:45.051 INFO:teuthology.orchestra.run.smithi181.stdout:(118/132): python3-jsonpointer-1.10-11.el8.noar 217 kB/s | 20 kB 00:00 2023-12-11T17:34:45.076 INFO:teuthology.orchestra.run.smithi181.stdout:(119/132): python3-pyasn1-0.3.7-6.el8.noarch.rp 887 kB/s | 126 kB 00:00 2023-12-11T17:34:45.102 INFO:teuthology.orchestra.run.smithi181.stdout:(120/132): python3-pyOpenSSL-19.0.0-1.el8.noarc 1.3 MB/s | 103 kB 00:00 2023-12-11T17:34:45.105 INFO:teuthology.orchestra.run.smithi059.stdout:(43/132): python3-asyncssh-2.7.0-2.el8.noarch.r 2.0 MB/s | 476 kB 00:00 2023-12-11T17:34:45.136 INFO:teuthology.orchestra.run.smithi181.stdout:(121/132): libtool-2.4.6-25.el8.x86_64.rpm 8.2 MB/s | 709 kB 00:00 2023-12-11T17:34:45.170 INFO:teuthology.orchestra.run.smithi115.stdout:(39/132): gperftools-libs-2.7-9.el8.x86_64.rpm 1.1 MB/s | 306 kB 00:00 2023-12-11T17:34:45.187 INFO:teuthology.orchestra.run.smithi115.stdout:(40/132): liboath-2.6.2-3.el8.x86_64.rpm 261 kB/s | 59 kB 00:00 2023-12-11T17:34:45.202 INFO:teuthology.orchestra.run.smithi181.stdout:(122/132): libpq-13.5-1.el8.x86_64.rpm 1.9 MB/s | 198 kB 00:00 2023-12-11T17:34:45.212 INFO:teuthology.orchestra.run.smithi115.stdout:(41/132): libunwind-1.3.1-3.el8.x86_64.rpm 449 kB/s | 75 kB 00:00 2023-12-11T17:34:45.213 INFO:teuthology.orchestra.run.smithi059.stdout:(44/132): python3-cachetools-3.1.1-4.el8.noarch 305 kB/s | 33 kB 00:00 2023-12-11T17:34:45.245 INFO:teuthology.orchestra.run.smithi115.stdout:(42/132): luarocks-3.5.0-1.el8.noarch.rpm 2.2 MB/s | 167 kB 00:00 2023-12-11T17:34:45.253 INFO:teuthology.orchestra.run.smithi181.stdout:(123/132): jq-1.6-3.el8.x86_64.rpm 4.0 MB/s | 202 kB 00:00 2023-12-11T17:34:45.255 INFO:teuthology.orchestra.run.smithi059.stdout:(45/132): python3-beautifulsoup4-4.6.3-2.el8.1. 821 kB/s | 185 kB 00:00 2023-12-11T17:34:45.271 INFO:teuthology.orchestra.run.smithi115.stdout:(43/132): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 744 kB/s | 44 kB 00:00 2023-12-11T17:34:45.303 INFO:teuthology.orchestra.run.smithi181.stdout:(124/132): librdkafka-0.11.4-3.el8.x86_64.rpm 7.0 MB/s | 354 kB 00:00 2023-12-11T17:34:45.338 INFO:teuthology.orchestra.run.smithi115.stdout:(44/132): python3-beautifulsoup4-4.6.3-2.el8.1. 2.0 MB/s | 185 kB 00:00 2023-12-11T17:34:45.340 INFO:teuthology.orchestra.run.smithi059.stdout:(46/132): python3-certifi-2018.10.15-7.el8.noar 129 kB/s | 16 kB 00:00 2023-12-11T17:34:45.357 INFO:teuthology.orchestra.run.smithi115.stdout:(45/132): python3-cachetools-3.1.1-4.el8.noarch 395 kB/s | 33 kB 00:00 2023-12-11T17:34:45.370 INFO:teuthology.orchestra.run.smithi181.stdout:(125/132): openblas-threads-0.3.15-3.el8.x86_64 18 MB/s | 4.9 MB 00:00 2023-12-11T17:34:45.380 INFO:teuthology.orchestra.run.smithi115.stdout:(46/132): python3-asyncssh-2.7.0-2.el8.noarch.r 2.4 MB/s | 476 kB 00:00 2023-12-11T17:34:45.398 INFO:teuthology.orchestra.run.smithi059.stdout:(47/132): python3-cheroot-8.5.2-1.el8.noarch.rp 1.2 MB/s | 173 kB 00:00 2023-12-11T17:34:45.405 INFO:teuthology.orchestra.run.smithi115.stdout:(47/132): python3-certifi-2018.10.15-7.el8.noar 242 kB/s | 16 kB 00:00 2023-12-11T17:34:45.453 INFO:teuthology.orchestra.run.smithi181.stdout:(126/132): openblas-0.3.15-3.el8.x86_64.rpm 15 MB/s | 4.7 MB 00:00 2023-12-11T17:34:45.456 INFO:teuthology.orchestra.run.smithi115.stdout:(48/132): python3-cherrypy-18.4.0-1.el8.noarch. 5.0 MB/s | 384 kB 00:00 2023-12-11T17:34:45.481 INFO:teuthology.orchestra.run.smithi115.stdout:(49/132): python3-cheroot-8.5.2-1.el8.noarch.rp 1.3 MB/s | 173 kB 00:00 2023-12-11T17:34:45.487 INFO:teuthology.orchestra.run.smithi181.stdout:(127/132): postgresql-contrib-10.21-2.module+el 6.8 MB/s | 808 kB 00:00 2023-12-11T17:34:45.498 INFO:teuthology.orchestra.run.smithi115.stdout:(50/132): python3-google-auth-1.1.1-10.el8.noar 883 kB/s | 81 kB 00:00 2023-12-11T17:34:45.506 INFO:teuthology.orchestra.run.smithi059.stdout:(48/132): python3-google-auth-1.1.1-10.el8.noar 751 kB/s | 81 kB 00:00 2023-12-11T17:34:45.523 INFO:teuthology.orchestra.run.smithi115.stdout:(51/132): python3-jaraco-6.2-6.el8.noarch.rpm 166 kB/s | 11 kB 00:00 2023-12-11T17:34:45.546 INFO:teuthology.orchestra.run.smithi181.stdout:(128/132): postgresql-10.21-2.module+el8.6.0+15 16 MB/s | 1.5 MB 00:00 2023-12-11T17:34:45.548 INFO:teuthology.orchestra.run.smithi059.stdout:(49/132): python3-cherrypy-18.4.0-1.el8.noarch. 1.8 MB/s | 384 kB 00:00 2023-12-11T17:34:45.548 INFO:teuthology.orchestra.run.smithi115.stdout:(52/132): python3-jaraco-functools-2.0-4.el8.no 262 kB/s | 18 kB 00:00 2023-12-11T17:34:45.582 INFO:teuthology.orchestra.run.smithi115.stdout:(53/132): python3-logutils-0.3.5-11.el8.noarch. 829 kB/s | 49 kB 00:00 2023-12-11T17:34:45.608 INFO:teuthology.orchestra.run.smithi115.stdout:(54/132): python3-more-itertools-7.2.0-3.el8.no 1.0 MB/s | 59 kB 00:00 2023-12-11T17:34:45.621 INFO:teuthology.orchestra.run.smithi181.stdout:(129/132): postgresql-server-10.21-2.module+el8 16 MB/s | 5.1 MB 00:00 2023-12-11T17:34:45.623 INFO:teuthology.orchestra.run.smithi059.stdout:(50/132): python3-jaraco-6.2-6.el8.noarch.rpm 95 kB/s | 11 kB 00:00 2023-12-11T17:34:45.641 INFO:teuthology.orchestra.run.smithi115.stdout:(55/132): python3-natsort-7.1.1-2.el8.noarch.rp 1.0 MB/s | 60 kB 00:00 2023-12-11T17:34:45.657 INFO:teuthology.orchestra.run.smithi059.stdout:(51/132): python3-jaraco-functools-2.0-4.el8.no 162 kB/s | 18 kB 00:00 2023-12-11T17:34:45.680 INFO:teuthology.orchestra.run.smithi181.stdout:(130/132): iproute-5.15.0-4.el8_6.1.x86_64.rpm 13 MB/s | 798 kB 00:00 2023-12-11T17:34:45.699 INFO:teuthology.orchestra.run.smithi115.stdout:(56/132): python3-kubernetes-11.0.0-6.el8.noarc 5.8 MB/s | 1.2 MB 00:00 2023-12-11T17:34:45.725 INFO:teuthology.orchestra.run.smithi115.stdout:(57/132): python3-portend-2.6-1.el8.noarch.rpm 186 kB/s | 16 kB 00:00 2023-12-11T17:34:45.750 INFO:teuthology.orchestra.run.smithi115.stdout:(58/132): python3-pecan-1.3.2-9.el8.noarch.rpm 1.9 MB/s | 283 kB 00:00 2023-12-11T17:34:45.775 INFO:teuthology.orchestra.run.smithi115.stdout:(59/132): python3-repoze-lru-0.7-6.el8.noarch.r 445 kB/s | 34 kB 00:00 2023-12-11T17:34:45.782 INFO:teuthology.orchestra.run.smithi059.stdout:(52/132): python3-logutils-0.3.5-11.el8.noarch. 390 kB/s | 49 kB 00:00 2023-12-11T17:34:45.801 INFO:teuthology.orchestra.run.smithi115.stdout:(60/132): python3-routes-2.4.1-12.el8.noarch.rp 2.5 MB/s | 196 kB 00:00 2023-12-11T17:34:45.826 INFO:teuthology.orchestra.run.smithi115.stdout:(61/132): python3-rsa-4.9-2.el8.noarch.rpm 850 kB/s | 64 kB 00:00 2023-12-11T17:34:45.860 INFO:teuthology.orchestra.run.smithi115.stdout:(62/132): python3-simplegeneric-0.8.1-17.el8.no 222 kB/s | 19 kB 00:00 2023-12-11T17:34:45.885 INFO:teuthology.orchestra.run.smithi115.stdout:(63/132): python3-singledispatch-3.4.0.3-18.el8 285 kB/s | 24 kB 00:00 2023-12-11T17:34:45.910 INFO:teuthology.orchestra.run.smithi115.stdout:(64/132): python3-tempora-1.14.1-5.el8.noarch.r 348 kB/s | 29 kB 00:00 2023-12-11T17:34:45.933 INFO:teuthology.orchestra.run.smithi059.stdout:(53/132): python3-more-itertools-7.2.0-3.el8.no 391 kB/s | 59 kB 00:00 2023-12-11T17:34:45.944 INFO:teuthology.orchestra.run.smithi115.stdout:(65/132): python3-trustme-0.6.0-4.el8.noarch.rp 330 kB/s | 27 kB 00:00 2023-12-11T17:34:45.971 INFO:teuthology.orchestra.run.smithi181.stdout:(131/132): librados2-19.0.0-119.ga62fda73.el8.x 7.8 MB/s | 3.8 MB 00:00 2023-12-11T17:34:45.977 INFO:teuthology.orchestra.run.smithi115.stdout:(66/132): python3-waitress-1.4.3-1.el8.noarch.r 3.7 MB/s | 255 kB 00:00 2023-12-11T17:34:46.003 INFO:teuthology.orchestra.run.smithi115.stdout:(67/132): python3-websocket-client-0.56.0-5.el8 1.0 MB/s | 61 kB 00:00 2023-12-11T17:34:46.050 INFO:teuthology.orchestra.run.smithi059.stdout:(54/132): python3-kubernetes-11.0.0-6.el8.noarc 2.7 MB/s | 1.2 MB 00:00 2023-12-11T17:34:46.053 INFO:teuthology.orchestra.run.smithi115.stdout:(68/132): python3-webob-1.8.5-1.el8.1.noarch.rp 1.7 MB/s | 251 kB 00:00 2023-12-11T17:34:46.067 INFO:teuthology.orchestra.run.smithi059.stdout:(55/132): python3-natsort-7.1.1-2.el8.noarch.rp 451 kB/s | 60 kB 00:00 2023-12-11T17:34:46.078 INFO:teuthology.orchestra.run.smithi115.stdout:(69/132): python3-zc-lockfile-2.0-2.el8.noarch. 305 kB/s | 23 kB 00:00 2023-12-11T17:34:46.103 INFO:teuthology.orchestra.run.smithi115.stdout:(70/132): python3-webtest-2.0.33-1.el8.noarch.r 680 kB/s | 85 kB 00:00 2023-12-11T17:34:46.129 INFO:teuthology.orchestra.run.smithi115.stdout:(71/132): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.4 MB/s | 70 kB 00:00 2023-12-11T17:34:46.179 INFO:teuthology.orchestra.run.smithi115.stdout:(72/132): python3-pyasn1-modules-0.3.7-6.el8.no 2.4 MB/s | 110 kB 00:00 2023-12-11T17:34:46.184 INFO:teuthology.orchestra.run.smithi059.stdout:(56/132): python3-pecan-1.3.2-9.el8.noarch.rpm 2.1 MB/s | 283 kB 00:00 2023-12-11T17:34:46.201 INFO:teuthology.orchestra.run.smithi059.stdout:(57/132): python3-portend-2.6-1.el8.noarch.rpm 117 kB/s | 16 kB 00:00 2023-12-11T17:34:46.213 INFO:teuthology.orchestra.run.smithi115.stdout:(73/132): python3-pytz-2017.2-9.el8.noarch.rpm 1.6 MB/s | 54 kB 00:00 2023-12-11T17:34:46.255 INFO:teuthology.orchestra.run.smithi115.stdout:(74/132): thrift-0.13.0-2.el8.x86_64.rpm 8.6 MB/s | 1.7 MB 00:00 2023-12-11T17:34:46.288 INFO:teuthology.orchestra.run.smithi115.stdout:(75/132): lttng-ust-2.8.1-11.el8.x86_64.rpm 3.4 MB/s | 259 kB 00:00 2023-12-11T17:34:46.292 INFO:teuthology.orchestra.run.smithi059.stdout:(58/132): python3-repoze-lru-0.7-6.el8.noarch.r 310 kB/s | 34 kB 00:00 2023-12-11T17:34:46.322 INFO:teuthology.orchestra.run.smithi115.stdout:(76/132): python3-markupsafe-0.23-19.el8.x86_64 1.4 MB/s | 39 kB 00:00 2023-12-11T17:34:46.335 INFO:teuthology.orchestra.run.smithi059.stdout:(59/132): python3-routes-2.4.1-12.el8.noarch.rp 1.4 MB/s | 196 kB 00:00 2023-12-11T17:34:46.355 INFO:teuthology.orchestra.run.smithi115.stdout:(77/132): socat-1.7.4.1-1.el8.x86_64.rpm 3.1 MB/s | 323 kB 00:00 2023-12-11T17:34:46.414 INFO:teuthology.orchestra.run.smithi115.stdout:(78/132): bison-3.0.4-10.el8.x86_64.rpm 7.3 MB/s | 688 kB 00:00 2023-12-11T17:34:46.418 INFO:teuthology.orchestra.run.smithi059.stdout:(60/132): python3-rsa-4.9-2.el8.noarch.rpm 512 kB/s | 64 kB 00:00 2023-12-11T17:34:46.444 INFO:teuthology.orchestra.run.smithi059.stdout:(61/132): python3-simplegeneric-0.8.1-17.el8.no 172 kB/s | 19 kB 00:00 2023-12-11T17:34:46.560 INFO:teuthology.orchestra.run.smithi059.stdout:(62/132): python3-singledispatch-3.4.0.3-18.el8 169 kB/s | 24 kB 00:00 2023-12-11T17:34:46.586 INFO:teuthology.orchestra.run.smithi059.stdout:(63/132): python3-tempora-1.14.1-5.el8.noarch.r 206 kB/s | 29 kB 00:00 2023-12-11T17:34:46.686 INFO:teuthology.orchestra.run.smithi059.stdout:(64/132): python3-trustme-0.6.0-4.el8.noarch.rp 220 kB/s | 27 kB 00:00 2023-12-11T17:34:46.720 INFO:teuthology.orchestra.run.smithi059.stdout:(65/132): python3-waitress-1.4.3-1.el8.noarch.r 1.9 MB/s | 255 kB 00:00 2023-12-11T17:34:46.730 INFO:teuthology.orchestra.run.smithi181.stdout:(132/132): librbd1-19.0.0-119.ga62fda73.el8.x86 3.1 MB/s | 3.7 MB 00:01 2023-12-11T17:34:46.731 INFO:teuthology.orchestra.run.smithi115.stdout:(79/132): python3-jinja2-2.10.1-3.el8.noarch.rp 1.4 MB/s | 538 kB 00:00 2023-12-11T17:34:46.733 INFO:teuthology.orchestra.run.smithi181.stdout:-------------------------------------------------------------------------------- 2023-12-11T17:34:46.733 INFO:teuthology.orchestra.run.smithi181.stdout:Total 12 MB/s | 209 MB 00:17 2023-12-11T17:34:46.782 INFO:teuthology.orchestra.run.smithi115.stdout:(80/132): python3-mako-1.0.6-13.el8.noarch.rpm 428 kB/s | 157 kB 00:00 2023-12-11T17:34:46.807 INFO:teuthology.orchestra.run.smithi115.stdout:(81/132): oniguruma-6.8.2-2.el8.x86_64.rpm 2.4 MB/s | 187 kB 00:00 2023-12-11T17:34:46.845 INFO:teuthology.orchestra.run.smithi059.stdout:(66/132): python3-webob-1.8.5-1.el8.1.noarch.rp 1.5 MB/s | 251 kB 00:00 2023-12-11T17:34:46.848 INFO:teuthology.orchestra.run.smithi115.stdout:(82/132): automake-1.16.1-7.el8.noarch.rpm 10 MB/s | 713 kB 00:00 2023-12-11T17:34:46.871 INFO:teuthology.orchestra.run.smithi059.stdout:(67/132): python3-websocket-client-0.56.0-5.el8 407 kB/s | 61 kB 00:00 2023-12-11T17:34:46.874 INFO:teuthology.orchestra.run.smithi115.stdout:(83/132): python3-jsonpatch-1.21-2.el8.noarch.r 1.0 MB/s | 27 kB 00:00 2023-12-11T17:34:46.916 INFO:teuthology.orchestra.run.smithi115.stdout:(84/132): flex-2.6.1-9.el8.x86_64.rpm 7.5 MB/s | 320 kB 00:00 2023-12-11T17:34:46.950 INFO:teuthology.orchestra.run.smithi115.stdout:(85/132): python3-prettytable-0.7.2-14.el8.noar 1.3 MB/s | 44 kB 00:00 2023-12-11T17:34:46.971 INFO:teuthology.orchestra.run.smithi059.stdout:(68/132): python3-webtest-2.0.33-1.el8.noarch.r 682 kB/s | 85 kB 00:00 2023-12-11T17:34:46.998 INFO:teuthology.orchestra.run.smithi059.stdout:(69/132): python3-zc-lockfile-2.0-2.el8.noarch. 183 kB/s | 23 kB 00:00 2023-12-11T17:34:47.013 INFO:teuthology.orchestra.run.smithi059.stdout:(70/132): xmlstarlet-1.6.1-20.el8.x86_64.rpm 4.1 MB/s | 70 kB 00:00 2023-12-11T17:34:47.042 INFO:teuthology.orchestra.run.smithi115.stdout:(86/132): python3-babel-2.5.1-7.el8.noarch.rpm 20 MB/s | 4.8 MB 00:00 2023-12-11T17:34:47.072 INFO:teuthology.orchestra.run.smithi059.stdout:(71/132): librabbitmq-0.9.0-3.el8.x86_64.rpm 803 kB/s | 47 kB 00:00 2023-12-11T17:34:47.092 INFO:teuthology.orchestra.run.smithi115.stdout:(87/132): python3-werkzeug-0.12.2-4.el8.noarch. 9.0 MB/s | 457 kB 00:00 2023-12-11T17:34:47.114 INFO:teuthology.orchestra.run.smithi059.stdout:(72/132): libselinux-devel-2.9-5.el8.x86_64.rpm 4.7 MB/s | 200 kB 00:00 2023-12-11T17:34:47.148 INFO:teuthology.orchestra.run.smithi059.stdout:(73/132): python3-jwt-1.6.1-2.el8.noarch.rpm 1.2 MB/s | 43 kB 00:00 2023-12-11T17:34:47.189 INFO:teuthology.orchestra.run.smithi059.stdout:(74/132): pcre2-utf32-10.32-2.el8.x86_64.rpm 5.2 MB/s | 220 kB 00:00 2023-12-11T17:34:47.231 INFO:teuthology.orchestra.run.smithi059.stdout:(75/132): keyutils-libs-devel-1.5.10-9.el8.x86_ 1.1 MB/s | 48 kB 00:00 2023-12-11T17:34:47.250 INFO:teuthology.orchestra.run.smithi115.stdout:(88/132): python3-scipy-1.0.0-21.module+el8.5.0 12 MB/s | 14 MB 00:01 2023-12-11T17:34:47.276 INFO:teuthology.orchestra.run.smithi115.stdout:(89/132): python3-numpy-f2py-1.14.3-10.el8.x86_ 1.2 MB/s | 225 kB 00:00 2023-12-11T17:34:47.301 INFO:teuthology.orchestra.run.smithi115.stdout:(90/132): uuid-1.6.2-43.el8.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2023-12-11T17:34:47.327 INFO:teuthology.orchestra.run.smithi115.stdout:(91/132): python3-pyasn1-0.3.7-6.el8.noarch.rpm 2.4 MB/s | 126 kB 00:00 2023-12-11T17:34:47.377 INFO:teuthology.orchestra.run.smithi115.stdout:(92/132): python3-numpy-1.14.3-10.el8.x86_64.rp 8.7 MB/s | 3.7 MB 00:00 2023-12-11T17:34:47.402 INFO:teuthology.orchestra.run.smithi115.stdout:(93/132): python3-jsonpointer-1.10-11.el8.noarc 198 kB/s | 20 kB 00:00 2023-12-11T17:34:47.427 INFO:teuthology.orchestra.run.smithi115.stdout:(94/132): python3-pyOpenSSL-19.0.0-1.el8.noarch 1.0 MB/s | 103 kB 00:00 2023-12-11T17:34:47.491 INFO:teuthology.orchestra.run.smithi115.stdout:(95/132): libtool-2.4.6-25.el8.x86_64.rpm 6.1 MB/s | 709 kB 00:00 2023-12-11T17:34:47.520 INFO:teuthology.orchestra.run.smithi115.stdout:(96/132): libpq-13.5-1.el8.x86_64.rpm 2.1 MB/s | 198 kB 00:00 2023-12-11T17:34:47.562 INFO:teuthology.orchestra.run.smithi115.stdout:(97/132): jq-1.6-3.el8.x86_64.rpm 4.7 MB/s | 202 kB 00:00 2023-12-11T17:34:47.570 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction check 2023-12-11T17:34:47.604 INFO:teuthology.orchestra.run.smithi115.stdout:(98/132): librdkafka-0.11.4-3.el8.x86_64.rpm 8.3 MB/s | 354 kB 00:00 2023-12-11T17:34:47.673 INFO:teuthology.orchestra.run.smithi059.stdout:(76/132): ceph-test-19.0.0-119.ga62fda73.el8.x8 5.9 MB/s | 49 MB 00:08 2023-12-11T17:34:47.693 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction check succeeded. 2023-12-11T17:34:47.693 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction test 2023-12-11T17:34:47.695 INFO:teuthology.orchestra.run.smithi115.stdout:(99/132): openblas-threads-0.3.15-3.el8.x86_64. 17 MB/s | 4.9 MB 00:00 2023-12-11T17:34:47.698 INFO:teuthology.orchestra.run.smithi059.stdout:(77/132): libsepol-devel-2.9-3.el8.x86_64.rpm 186 kB/s | 87 kB 00:00 2023-12-11T17:34:47.771 INFO:teuthology.orchestra.run.smithi115.stdout:(100/132): openblas-0.3.15-3.el8.x86_64.rpm 17 MB/s | 4.7 MB 00:00 2023-12-11T17:34:47.774 INFO:teuthology.orchestra.run.smithi059.stdout:(78/132): thrift-0.13.0-2.el8.x86_64.rpm 2.2 MB/s | 1.7 MB 00:00 2023-12-11T17:34:47.799 INFO:teuthology.orchestra.run.smithi059.stdout:(79/132): python3-requests-oauthlib-1.0.0-1.el8 341 kB/s | 43 kB 00:00 2023-12-11T17:34:47.812 INFO:teuthology.orchestra.run.smithi115.stdout:(101/132): postgresql-contrib-10.21-2.module+el 6.8 MB/s | 808 kB 00:00 2023-12-11T17:34:47.833 INFO:teuthology.orchestra.run.smithi059.stdout:(80/132): python3-pycparser-2.14-14.el8.noarch. 869 kB/s | 109 kB 00:00 2023-12-11T17:34:47.866 INFO:teuthology.orchestra.run.smithi059.stdout:(81/132): pcre2-utf16-10.32-2.el8.x86_64.rpm 2.4 MB/s | 229 kB 00:00 2023-12-11T17:34:47.879 INFO:teuthology.orchestra.run.smithi115.stdout:(102/132): librabbitmq-0.9.0-3.el8.x86_64.rpm 703 kB/s | 47 kB 00:00 2023-12-11T17:34:47.891 INFO:teuthology.orchestra.run.smithi059.stdout:(82/132): libaio-devel-0.3.112-1.el8.x86_64.rpm 320 kB/s | 19 kB 00:00 2023-12-11T17:34:47.917 INFO:teuthology.orchestra.run.smithi059.stdout:(83/132): python3-oauthlib-2.1.0-1.el8.noarch.r 3.0 MB/s | 155 kB 00:00 2023-12-11T17:34:47.921 INFO:teuthology.orchestra.run.smithi115.stdout:(103/132): postgresql-10.21-2.module+el8.6.0+15 10 MB/s | 1.5 MB 00:00 2023-12-11T17:34:47.947 INFO:teuthology.orchestra.run.smithi115.stdout:(104/132): libselinux-devel-2.9-5.el8.x86_64.rp 2.9 MB/s | 200 kB 00:00 2023-12-11T17:34:47.950 INFO:teuthology.orchestra.run.smithi059.stdout:(84/132): krb5-devel-1.18.2-14.el8.x86_64.rpm 3.8 MB/s | 560 kB 00:00 2023-12-11T17:34:47.972 INFO:teuthology.orchestra.run.smithi115.stdout:(105/132): python3-jwt-1.6.1-2.el8.noarch.rpm 849 kB/s | 43 kB 00:00 2023-12-11T17:34:47.976 INFO:teuthology.orchestra.run.smithi059.stdout:(85/132): libverto-devel-0.3.0-5.el8.x86_64.rpm 305 kB/s | 18 kB 00:00 2023-12-11T17:34:47.997 INFO:teuthology.orchestra.run.smithi115.stdout:(106/132): pcre2-utf32-10.32-2.el8.x86_64.rpm 4.3 MB/s | 220 kB 00:00 2023-12-11T17:34:48.009 INFO:teuthology.orchestra.run.smithi059.stdout:(86/132): python3-cryptography-3.2.1-5.el8.x86_ 4.7 MB/s | 559 kB 00:00 2023-12-11T17:34:48.023 INFO:teuthology.orchestra.run.smithi115.stdout:(107/132): keyutils-libs-devel-1.5.10-9.el8.x86 956 kB/s | 48 kB 00:00 2023-12-11T17:34:48.043 INFO:teuthology.orchestra.run.smithi059.stdout:(87/132): pcre2-devel-10.32-2.el8.x86_64.rpm 6.4 MB/s | 605 kB 00:00 2023-12-11T17:34:48.048 INFO:teuthology.orchestra.run.smithi115.stdout:(108/132): libsepol-devel-2.9-3.el8.x86_64.rpm 1.7 MB/s | 87 kB 00:00 2023-12-11T17:34:48.068 INFO:teuthology.orchestra.run.smithi059.stdout:(88/132): python3-cffi-1.11.5-5.el8.x86_64.rpm 2.5 MB/s | 238 kB 00:00 2023-12-11T17:34:48.073 INFO:teuthology.orchestra.run.smithi115.stdout:(109/132): python3-requests-oauthlib-1.0.0-1.el 852 kB/s | 43 kB 00:00 2023-12-11T17:34:48.094 INFO:teuthology.orchestra.run.smithi059.stdout:(89/132): xfsprogs-devel-5.0.0-10.el8.x86_64.rp 1.4 MB/s | 72 kB 00:00 2023-12-11T17:34:48.098 INFO:teuthology.orchestra.run.smithi115.stdout:(110/132): python3-pycparser-2.14-14.el8.noarch 2.1 MB/s | 109 kB 00:00 2023-12-11T17:34:48.119 INFO:teuthology.orchestra.run.smithi059.stdout:(90/132): fuse-2.9.7-15.el8.x86_64.rpm 1.6 MB/s | 83 kB 00:00 2023-12-11T17:34:48.124 INFO:teuthology.orchestra.run.smithi115.stdout:(111/132): pcre2-utf16-10.32-2.el8.x86_64.rpm 4.4 MB/s | 229 kB 00:00 2023-12-11T17:34:48.161 INFO:teuthology.orchestra.run.smithi059.stdout:(91/132): elfutils-libelf-devel-0.186-1.el8.x86 897 kB/s | 60 kB 00:00 2023-12-11T17:34:48.186 INFO:teuthology.orchestra.run.smithi059.stdout:(92/132): libuuid-devel-2.32.1-35.el8.x86_64.rp 1.4 MB/s | 98 kB 00:00 2023-12-11T17:34:48.203 INFO:teuthology.orchestra.run.smithi059.stdout:(93/132): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 1.3 MB/s | 58 kB 00:00 2023-12-11T17:34:48.215 INFO:teuthology.orchestra.run.smithi115.stdout:(112/132): postgresql-server-10.21-2.module+el8 8.3 MB/s | 5.1 MB 00:00 2023-12-11T17:34:48.220 INFO:teuthology.orchestra.run.smithi059.stdout:(94/132): libcom_err-devel-1.45.6-4.el8.x86_64. 1.1 MB/s | 39 kB 00:00 2023-12-11T17:34:48.241 INFO:teuthology.orchestra.run.smithi115.stdout:(113/132): libaio-devel-0.3.112-1.el8.x86_64.rp 160 kB/s | 19 kB 00:00 2023-12-11T17:34:48.262 INFO:teuthology.orchestra.run.smithi059.stdout:(95/132): libgfortran-8.5.0-10.1.el8_6.x86_64.r 11 MB/s | 644 kB 00:00 2023-12-11T17:34:48.266 INFO:teuthology.orchestra.run.smithi115.stdout:(114/132): python3-oauthlib-2.1.0-1.el8.noarch. 3.0 MB/s | 155 kB 00:00 2023-12-11T17:34:48.287 INFO:teuthology.orchestra.run.smithi059.stdout:(96/132): libquadmath-8.5.0-10.1.el8_6.x86_64.r 2.5 MB/s | 171 kB 00:00 2023-12-11T17:34:48.300 INFO:teuthology.orchestra.run.smithi115.stdout:(115/132): krb5-devel-1.18.2-14.el8.x86_64.rpm 2.7 MB/s | 560 kB 00:00 2023-12-11T17:34:48.313 INFO:teuthology.orchestra.run.smithi059.stdout:(97/132): NetworkManager-initscripts-updown-1.3 2.7 MB/s | 138 kB 00:00 2023-12-11T17:34:48.317 INFO:teuthology.orchestra.run.smithi115.stdout:(116/132): libverto-devel-0.3.0-5.el8.x86_64.rp 356 kB/s | 18 kB 00:00 2023-12-11T17:34:48.350 INFO:teuthology.orchestra.run.smithi115.stdout:(117/132): python3-cryptography-3.2.1-5.el8.x86 5.0 MB/s | 559 kB 00:00 2023-12-11T17:34:48.371 INFO:teuthology.orchestra.run.smithi059.stdout:(98/132): openssl-devel-1.1.1k-7.el8_6.x86_64.r 6.4 MB/s | 2.3 MB 00:00 2023-12-11T17:34:48.376 INFO:teuthology.orchestra.run.smithi115.stdout:(118/132): python3-cffi-1.11.5-5.el8.x86_64.rpm 4.0 MB/s | 238 kB 00:00 2023-12-11T17:34:48.396 INFO:teuthology.orchestra.run.smithi059.stdout:(99/132): python3-pyasn1-modules-0.3.7-6.el8.no 1.3 MB/s | 110 kB 00:00 2023-12-11T17:34:48.409 INFO:teuthology.orchestra.run.smithi115.stdout:(119/132): pcre2-devel-10.32-2.el8.x86_64.rpm 5.4 MB/s | 605 kB 00:00 2023-12-11T17:34:48.430 INFO:teuthology.orchestra.run.smithi059.stdout:(100/132): python3-pytz-2017.2-9.el8.noarch.rpm 1.6 MB/s | 54 kB 00:00 2023-12-11T17:34:48.435 INFO:teuthology.orchestra.run.smithi115.stdout:(120/132): xfsprogs-devel-5.0.0-10.el8.x86_64.r 1.2 MB/s | 72 kB 00:00 2023-12-11T17:34:48.455 INFO:teuthology.orchestra.run.smithi059.stdout:(101/132): lttng-ust-2.8.1-11.el8.x86_64.rpm 4.3 MB/s | 259 kB 00:00 2023-12-11T17:34:48.460 INFO:teuthology.orchestra.run.smithi115.stdout:(121/132): fuse-2.9.7-15.el8.x86_64.rpm 1.6 MB/s | 83 kB 00:00 2023-12-11T17:34:48.486 INFO:teuthology.orchestra.run.smithi115.stdout:(122/132): elfutils-libelf-devel-0.186-1.el8.x8 1.2 MB/s | 60 kB 00:00 2023-12-11T17:34:48.489 INFO:teuthology.orchestra.run.smithi059.stdout:(102/132): socat-1.7.4.1-1.el8.x86_64.rpm 5.4 MB/s | 323 kB 00:00 2023-12-11T17:34:48.511 INFO:teuthology.orchestra.run.smithi115.stdout:(123/132): libuuid-devel-2.32.1-35.el8.x86_64.r 1.9 MB/s | 98 kB 00:00 2023-12-11T17:34:48.522 INFO:teuthology.orchestra.run.smithi059.stdout:(103/132): python3-markupsafe-0.23-19.el8.x86_6 581 kB/s | 39 kB 00:00 2023-12-11T17:34:48.536 INFO:teuthology.orchestra.run.smithi115.stdout:(124/132): zlib-devel-1.2.11-18.el8_5.x86_64.rp 1.1 MB/s | 58 kB 00:00 2023-12-11T17:34:48.585 INFO:teuthology.orchestra.run.smithi115.stdout:(125/132): libcom_err-devel-1.45.6-4.el8.x86_64 518 kB/s | 39 kB 00:00 2023-12-11T17:34:48.589 INFO:teuthology.orchestra.run.smithi059.stdout:(104/132): bison-3.0.4-10.el8.x86_64.rpm 6.7 MB/s | 688 kB 00:00 2023-12-11T17:34:48.623 INFO:teuthology.orchestra.run.smithi059.stdout:(105/132): python3-jinja2-2.10.1-3.el8.noarch.r 5.2 MB/s | 538 kB 00:00 2023-12-11T17:34:48.628 INFO:teuthology.orchestra.run.smithi115.stdout:(126/132): libgfortran-8.5.0-10.1.el8_6.x86_64. 6.9 MB/s | 644 kB 00:00 2023-12-11T17:34:48.648 INFO:teuthology.orchestra.run.smithi059.stdout:(106/132): python3-mako-1.0.6-13.el8.noarch.rpm 2.6 MB/s | 157 kB 00:00 2023-12-11T17:34:48.653 INFO:teuthology.orchestra.run.smithi115.stdout:(127/132): libquadmath-8.5.0-10.1.el8_6.x86_64. 2.5 MB/s | 171 kB 00:00 2023-12-11T17:34:48.674 INFO:teuthology.orchestra.run.smithi059.stdout:(107/132): oniguruma-6.8.2-2.el8.x86_64.rpm 3.7 MB/s | 187 kB 00:00 2023-12-11T17:34:48.679 INFO:teuthology.orchestra.run.smithi115.stdout:(128/132): NetworkManager-initscripts-updown-1. 2.7 MB/s | 138 kB 00:00 2023-12-11T17:34:48.716 INFO:teuthology.orchestra.run.smithi059.stdout:(108/132): automake-1.16.1-7.el8.noarch.rpm 10 MB/s | 713 kB 00:00 2023-12-11T17:34:48.729 INFO:teuthology.orchestra.run.smithi115.stdout:(129/132): openssl-devel-1.1.1k-7.el8_6.x86_64. 6.1 MB/s | 2.3 MB 00:00 2023-12-11T17:34:48.758 INFO:teuthology.orchestra.run.smithi059.stdout:(109/132): python3-jsonpatch-1.21-2.el8.noarch. 637 kB/s | 27 kB 00:00 2023-12-11T17:34:48.788 INFO:teuthology.orchestra.run.smithi115.stdout:(130/132): iproute-5.15.0-4.el8_6.1.x86_64.rpm 13 MB/s | 798 kB 00:00 2023-12-11T17:34:48.799 INFO:teuthology.orchestra.run.smithi059.stdout:(110/132): flex-2.6.1-9.el8.x86_64.rpm 7.5 MB/s | 320 kB 00:00 2023-12-11T17:34:48.825 INFO:teuthology.orchestra.run.smithi059.stdout:(111/132): python3-prettytable-0.7.2-14.el8.noa 1.7 MB/s | 44 kB 00:00 2023-12-11T17:34:48.830 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction test succeeded. 2023-12-11T17:34:48.834 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction 2023-12-11T17:34:48.956 INFO:teuthology.orchestra.run.smithi059.stdout:(112/132): python3-babel-2.5.1-7.el8.noarch.rpm 17 MB/s | 4.8 MB 00:00 2023-12-11T17:34:49.009 INFO:teuthology.orchestra.run.smithi059.stdout:(113/132): python3-werkzeug-0.12.2-4.el8.noarch 8.5 MB/s | 457 kB 00:00 2023-12-11T17:34:49.051 INFO:teuthology.orchestra.run.smithi059.stdout:(114/132): python3-numpy-f2py-1.14.3-10.el8.x86 5.3 MB/s | 225 kB 00:00 2023-12-11T17:34:49.109 INFO:teuthology.orchestra.run.smithi059.stdout:(115/132): python3-numpy-1.14.3-10.el8.x86_64.r 13 MB/s | 3.7 MB 00:00 2023-12-11T17:34:49.135 INFO:teuthology.orchestra.run.smithi059.stdout:(116/132): uuid-1.6.2-43.el8.x86_64.rpm 767 kB/s | 64 kB 00:00 2023-12-11T17:34:49.160 INFO:teuthology.orchestra.run.smithi059.stdout:(117/132): python3-pyasn1-0.3.7-6.el8.noarch.rp 2.5 MB/s | 126 kB 00:00 2023-12-11T17:34:49.185 INFO:teuthology.orchestra.run.smithi059.stdout:(118/132): python3-jsonpointer-1.10-11.el8.noar 398 kB/s | 20 kB 00:00 2023-12-11T17:34:49.210 INFO:teuthology.orchestra.run.smithi059.stdout:(119/132): python3-pyOpenSSL-19.0.0-1.el8.noarc 2.0 MB/s | 103 kB 00:00 2023-12-11T17:34:49.244 INFO:teuthology.orchestra.run.smithi059.stdout:(120/132): libtool-2.4.6-25.el8.x86_64.rpm 12 MB/s | 709 kB 00:00 2023-12-11T17:34:49.355 INFO:teuthology.orchestra.run.smithi115.stdout:(131/132): librbd1-19.0.0-119.ga62fda73.el8.x86 5.5 MB/s | 3.7 MB 00:00 2023-12-11T17:34:49.411 INFO:teuthology.orchestra.run.smithi059.stdout:(121/132): python3-scipy-1.0.0-21.module+el8.5. 12 MB/s | 14 MB 00:01 2023-12-11T17:34:49.436 INFO:teuthology.orchestra.run.smithi059.stdout:(122/132): libpq-13.5-1.el8.x86_64.rpm 1.0 MB/s | 198 kB 00:00 2023-12-11T17:34:49.470 INFO:teuthology.orchestra.run.smithi059.stdout:(123/132): jq-1.6-3.el8.x86_64.rpm 6.0 MB/s | 202 kB 00:00 2023-12-11T17:34:49.521 INFO:teuthology.orchestra.run.smithi059.stdout:(124/132): librdkafka-0.11.4-3.el8.x86_64.rpm 6.9 MB/s | 354 kB 00:00 2023-12-11T17:34:49.621 INFO:teuthology.orchestra.run.smithi059.stdout:(125/132): openblas-threads-0.3.15-3.el8.x86_64 12 MB/s | 4.9 MB 00:00 2023-12-11T17:34:49.696 INFO:teuthology.orchestra.run.smithi059.stdout:(126/132): openblas-0.3.15-3.el8.x86_64.rpm 17 MB/s | 4.7 MB 00:00 2023-12-11T17:34:49.738 INFO:teuthology.orchestra.run.smithi059.stdout:(127/132): postgresql-contrib-10.21-2.module+el 6.8 MB/s | 808 kB 00:00 2023-12-11T17:34:49.822 INFO:teuthology.orchestra.run.smithi059.stdout:(128/132): postgresql-10.21-2.module+el8.6.0+15 12 MB/s | 1.5 MB 00:00 2023-12-11T17:34:49.871 INFO:teuthology.orchestra.run.smithi115.stdout:(132/132): librados2-19.0.0-119.ga62fda73.el8.x 3.1 MB/s | 3.8 MB 00:01 2023-12-11T17:34:49.874 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2023-12-11T17:34:49.874 INFO:teuthology.orchestra.run.smithi115.stdout:Total 11 MB/s | 209 MB 00:19 2023-12-11T17:34:49.897 INFO:teuthology.orchestra.run.smithi059.stdout:(129/132): postgresql-server-10.21-2.module+el8 13 MB/s | 5.1 MB 00:00 2023-12-11T17:34:49.947 INFO:teuthology.orchestra.run.smithi059.stdout:(130/132): iproute-5.15.0-4.el8_6.1.x86_64.rpm 16 MB/s | 798 kB 00:00 2023-12-11T17:34:50.356 INFO:teuthology.orchestra.run.smithi059.stdout:(131/132): librbd1-19.0.0-119.ga62fda73.el8.x86 6.9 MB/s | 3.7 MB 00:00 2023-12-11T17:34:50.481 INFO:teuthology.orchestra.run.smithi059.stdout:(132/132): librados2-19.0.0-119.ga62fda73.el8.x 5.1 MB/s | 3.8 MB 00:00 2023-12-11T17:34:50.484 INFO:teuthology.orchestra.run.smithi059.stdout:-------------------------------------------------------------------------------- 2023-12-11T17:34:50.484 INFO:teuthology.orchestra.run.smithi059.stdout:Total 11 MB/s | 209 MB 00:19 2023-12-11T17:34:50.702 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2023-12-11T17:34:50.824 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2023-12-11T17:34:50.824 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2023-12-11T17:34:51.326 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction check 2023-12-11T17:34:51.460 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction check succeeded. 2023-12-11T17:34:51.460 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction test 2023-12-11T17:34:51.971 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2023-12-11T17:34:51.975 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2023-12-11T17:34:52.664 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction test succeeded. 2023-12-11T17:34:52.668 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction 2023-12-11T17:34:53.781 INFO:teuthology.orchestra.run.smithi181.stdout: Preparing : 1/1 2023-12-11T17:34:53.979 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/135 2023-12-11T17:34:54.162 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/135 2023-12-11T17:34:54.348 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 3/135 2023-12-11T17:34:54.454 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 3/135 2023-12-11T17:34:54.664 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/135 2023-12-11T17:34:55.073 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/135 2023-12-11T17:34:55.264 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : librados2-2:19.0.0-119.ga62fda73.el8.x86_64 5/135 2023-12-11T17:34:55.682 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: librados2-2:19.0.0-119.ga62fda73.el8.x86_64 5/135 2023-12-11T17:34:55.839 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : librbd1-2:19.0.0-119.ga62fda73.el8.x86_64 6/135 2023-12-11T17:34:55.992 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: librbd1-2:19.0.0-119.ga62fda73.el8.x86_64 6/135 2023-12-11T17:34:56.181 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libcephfs2-2:19.0.0-119.ga62fda73.el8.x86_64 7/135 2023-12-11T17:34:56.259 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: libcephfs2-2:19.0.0-119.ga62fda73.el8.x86_64 7/135 2023-12-11T17:34:56.453 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-rados-2:19.0.0-119.ga62fda73.el8.x86_64 8/135 2023-12-11T17:34:57.043 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2023-12-11T17:34:57.204 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 9/135 2023-12-11T17:34:57.389 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : librgw2-2:19.0.0-119.ga62fda73.el8.x86_64 10/135 2023-12-11T17:34:57.464 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: librgw2-2:19.0.0-119.ga62fda73.el8.x86_64 10/135 2023-12-11T17:34:57.484 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/135 2023-12-11T17:34:57.642 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/135 2023-12-11T17:34:57.701 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libpq-13.5-1.el8.x86_64 11/135 2023-12-11T17:34:57.707 INFO:teuthology.orchestra.run.smithi059.stdout: Preparing : 1/1 2023-12-11T17:34:57.877 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 3/135 2023-12-11T17:34:57.906 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/135 2023-12-11T17:34:57.925 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 12/135 2023-12-11T17:34:58.094 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 4/135 2023-12-11T17:34:58.099 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/135 2023-12-11T17:34:58.191 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 4/135 2023-12-11T17:34:58.275 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 3/135 2023-12-11T17:34:58.298 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 13/135 2023-12-11T17:34:58.394 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 3/135 2023-12-11T17:34:58.494 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 5/135 2023-12-11T17:34:58.642 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/135 2023-12-11T17:34:58.755 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 14/135 2023-12-11T17:34:58.902 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 5/135 2023-12-11T17:34:58.930 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libcephsqlite-2:19.0.0-119.ga62fda73.el8.x86_64 15/135 2023-12-11T17:34:59.046 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: libcephsqlite-2:19.0.0-119.ga62fda73.el8.x86_64 15/135 2023-12-11T17:34:59.058 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/135 2023-12-11T17:34:59.121 INFO:teuthology.orchestra.run.smithi115.stdout: Upgrading : librados2-2:19.0.0-119.ga62fda73.el8.x86_64 6/135 2023-12-11T17:34:59.214 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libradosstriper1-2:19.0.0-119.ga62fda73.el8.x86_ 16/135 2023-12-11T17:34:59.249 INFO:teuthology.orchestra.run.smithi059.stdout: Upgrading : librados2-2:19.0.0-119.ga62fda73.el8.x86_64 5/135 2023-12-11T17:34:59.366 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: libradosstriper1-2:19.0.0-119.ga62fda73.el8.x86_ 16/135 2023-12-11T17:34:59.534 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: librados2-2:19.0.0-119.ga62fda73.el8.x86_64 6/135 2023-12-11T17:34:59.577 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 17/135 2023-12-11T17:34:59.674 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: librados2-2:19.0.0-119.ga62fda73.el8.x86_64 5/135 2023-12-11T17:34:59.710 INFO:teuthology.orchestra.run.smithi115.stdout: Upgrading : librbd1-2:19.0.0-119.ga62fda73.el8.x86_64 7/135 2023-12-11T17:34:59.781 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 18/135 2023-12-11T17:34:59.844 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: librbd1-2:19.0.0-119.ga62fda73.el8.x86_64 7/135 2023-12-11T17:34:59.874 INFO:teuthology.orchestra.run.smithi059.stdout: Upgrading : librbd1-2:19.0.0-119.ga62fda73.el8.x86_64 6/135 2023-12-11T17:34:59.981 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 19/135 2023-12-11T17:35:00.006 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: librbd1-2:19.0.0-119.ga62fda73.el8.x86_64 6/135 2023-12-11T17:35:00.028 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libcephfs2-2:19.0.0-119.ga62fda73.el8.x86_64 8/135 2023-12-11T17:35:00.126 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: libcephfs2-2:19.0.0-119.ga62fda73.el8.x86_64 8/135 2023-12-11T17:35:00.147 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 20/135 2023-12-11T17:35:00.208 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libcephfs2-2:19.0.0-119.ga62fda73.el8.x86_64 7/135 2023-12-11T17:35:00.282 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 20/135 2023-12-11T17:35:00.313 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: libcephfs2-2:19.0.0-119.ga62fda73.el8.x86_64 7/135 2023-12-11T17:35:00.430 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 21/135 2023-12-11T17:35:00.454 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-rados-2:19.0.0-119.ga62fda73.el8.x86_64 8/135 2023-12-11T17:35:00.959 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-rados-2:19.0.0-119.ga62fda73.el8.x86_64 9/135 2023-12-11T17:35:01.034 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 21/135 2023-12-11T17:35:01.119 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : librgw2-2:19.0.0-119.ga62fda73.el8.x86_64 10/135 2023-12-11T17:35:01.180 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 22/135 2023-12-11T17:35:01.192 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: librgw2-2:19.0.0-119.ga62fda73.el8.x86_64 10/135 2023-12-11T17:35:01.247 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 9/135 2023-12-11T17:35:01.391 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : librgw2-2:19.0.0-119.ga62fda73.el8.x86_64 10/135 2023-12-11T17:35:01.465 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libpq-13.5-1.el8.x86_64 11/135 2023-12-11T17:35:01.495 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: librgw2-2:19.0.0-119.ga62fda73.el8.x86_64 10/135 2023-12-11T17:35:01.705 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 12/135 2023-12-11T17:35:01.706 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libpq-13.5-1.el8.x86_64 11/135 2023-12-11T17:35:01.800 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 22/135 2023-12-11T17:35:01.919 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 12/135 2023-12-11T17:35:01.947 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : openblas-0.3.15-3.el8.x86_64 23/135 2023-12-11T17:35:02.101 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 13/135 2023-12-11T17:35:02.315 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 13/135 2023-12-11T17:35:02.325 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 14/135 2023-12-11T17:35:02.501 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libcephsqlite-2:19.0.0-119.ga62fda73.el8.x86_64 15/135 2023-12-11T17:35:02.541 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 14/135 2023-12-11T17:35:02.610 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: libcephsqlite-2:19.0.0-119.ga62fda73.el8.x86_64 15/135 2023-12-11T17:35:02.635 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 23/135 2023-12-11T17:35:02.732 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libcephsqlite-2:19.0.0-119.ga62fda73.el8.x86_64 15/135 2023-12-11T17:35:02.801 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libradosstriper1-2:19.0.0-119.ga62fda73.el8.x86_ 16/135 2023-12-11T17:35:02.840 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: libcephsqlite-2:19.0.0-119.ga62fda73.el8.x86_64 15/135 2023-12-11T17:35:02.903 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 24/135 2023-12-11T17:35:02.990 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libradosstriper1-2:19.0.0-119.ga62fda73.el8.x86_ 16/135 2023-12-11T17:35:03.086 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 25/135 2023-12-11T17:35:03.091 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: libradosstriper1-2:19.0.0-119.ga62fda73.el8.x86_ 16/135 2023-12-11T17:35:03.159 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: libradosstriper1-2:19.0.0-119.ga62fda73.el8.x86_ 16/135 2023-12-11T17:35:03.251 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 17/135 2023-12-11T17:35:03.341 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 26/135 2023-12-11T17:35:03.378 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 17/135 2023-12-11T17:35:03.399 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 17/135 2023-12-11T17:35:03.503 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-ceph-common-2:19.0.0-119.ga62fda73.el8.x 27/135 2023-12-11T17:35:03.603 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 18/135 2023-12-11T17:35:03.609 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 18/135 2023-12-11T17:35:03.633 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-ceph-argparse-2:19.0.0-119.ga62fda73.el8 28/135 2023-12-11T17:35:03.762 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 19/135 2023-12-11T17:35:03.884 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-cephfs-2:19.0.0-119.ga62fda73.el8.x86_64 29/135 2023-12-11T17:35:03.949 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 20/135 2023-12-11T17:35:04.063 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 30/135 2023-12-11T17:35:04.096 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 20/135 2023-12-11T17:35:04.213 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 18/135 2023-12-11T17:35:04.290 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 21/135 2023-12-11T17:35:04.401 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 19/135 2023-12-11T17:35:04.955 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 21/135 2023-12-11T17:35:04.978 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 19/135 2023-12-11T17:35:05.167 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : openblas-0.3.15-3.el8.x86_64 20/135 2023-12-11T17:35:05.174 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 22/135 2023-12-11T17:35:05.810 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 22/135 2023-12-11T17:35:05.821 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 30/135 2023-12-11T17:35:05.869 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 20/135 2023-12-11T17:35:06.015 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : openblas-0.3.15-3.el8.x86_64 23/135 2023-12-11T17:35:06.166 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 21/135 2023-12-11T17:35:06.394 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 22/135 2023-12-11T17:35:06.613 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 23/135 2023-12-11T17:35:06.752 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 23/135 2023-12-11T17:35:06.815 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 24/135 2023-12-11T17:35:06.868 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 31/135 2023-12-11T17:35:06.989 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 24/135 2023-12-11T17:35:07.014 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 25/135 2023-12-11T17:35:07.201 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 25/135 2023-12-11T17:35:07.210 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 32/135 2023-12-11T17:35:07.261 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 26/135 2023-12-11T17:35:07.432 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-ceph-common-2:19.0.0-119.ga62fda73.el8.x 27/135 2023-12-11T17:35:07.446 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 26/135 2023-12-11T17:35:07.455 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 33/135 2023-12-11T17:35:07.595 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-ceph-argparse-2:19.0.0-119.ga62fda73.el8 28/135 2023-12-11T17:35:07.647 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-ceph-common-2:19.0.0-119.ga62fda73.el8.x 27/135 2023-12-11T17:35:07.707 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 34/135 2023-12-11T17:35:07.777 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-ceph-argparse-2:19.0.0-119.ga62fda73.el8 28/135 2023-12-11T17:35:07.915 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-rsa-4.9-2.el8.noarch 35/135 2023-12-11T17:35:08.044 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-cephfs-2:19.0.0-119.ga62fda73.el8.x86_64 29/135 2023-12-11T17:35:08.107 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 36/135 2023-12-11T17:35:08.240 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 30/135 2023-12-11T17:35:08.306 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-rgw-2:19.0.0-119.ga62fda73.el8.x86_64 37/135 2023-12-11T17:35:08.476 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-rbd-2:19.0.0-119.ga62fda73.el8.x86_64 38/135 2023-12-11T17:35:08.497 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-cephfs-2:19.0.0-119.ga62fda73.el8.x86_64 29/135 2023-12-11T17:35:08.668 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : librados-devel-2:19.0.0-119.ga62fda73.el8.x86_64 39/135 2023-12-11T17:35:08.819 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 40/135 2023-12-11T17:35:08.849 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 30/135 2023-12-11T17:35:08.963 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 41/135 2023-12-11T17:35:09.153 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 31/135 2023-12-11T17:35:09.179 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : uuid-1.6.2-43.el8.x86_64 42/135 2023-12-11T17:35:09.235 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 42/135 2023-12-11T17:35:09.433 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 43/135 2023-12-11T17:35:09.445 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 32/135 2023-12-11T17:35:09.665 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : automake-1.16.1-7.el8.noarch 44/135 2023-12-11T17:35:09.733 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 33/135 2023-12-11T17:35:09.812 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 45/135 2023-12-11T17:35:09.888 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 45/135 2023-12-11T17:35:10.117 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 30/135 2023-12-11T17:35:10.141 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : jq-1.6-3.el8.x86_64 46/135 2023-12-11T17:35:10.299 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 47/135 2023-12-11T17:35:10.460 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 48/135 2023-12-11T17:35:10.662 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 49/135 2023-12-11T17:35:10.839 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : fuse-2.9.7-15.el8.x86_64 50/135 2023-12-11T17:35:11.015 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 51/135 2023-12-11T17:35:11.175 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 31/135 2023-12-11T17:35:11.216 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 52/135 2023-12-11T17:35:11.389 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 53/135 2023-12-11T17:35:11.479 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 32/135 2023-12-11T17:35:11.506 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 33/135 2023-12-11T17:35:11.772 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 33/135 2023-12-11T17:35:11.834 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 34/135 2023-12-11T17:35:11.836 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 54/135 2023-12-11T17:35:12.018 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 34/135 2023-12-11T17:35:12.028 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-rsa-4.9-2.el8.noarch 35/135 2023-12-11T17:35:12.146 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 55/135 2023-12-11T17:35:12.182 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 36/135 2023-12-11T17:35:12.229 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-rsa-4.9-2.el8.noarch 35/135 2023-12-11T17:35:12.372 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 56/135 2023-12-11T17:35:12.409 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-rgw-2:19.0.0-119.ga62fda73.el8.x86_64 37/135 2023-12-11T17:35:12.427 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 36/135 2023-12-11T17:35:12.598 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 57/135 2023-12-11T17:35:12.610 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-rgw-2:19.0.0-119.ga62fda73.el8.x86_64 37/135 2023-12-11T17:35:12.621 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-rbd-2:19.0.0-119.ga62fda73.el8.x86_64 38/135 2023-12-11T17:35:12.769 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : librados-devel-2:19.0.0-119.ga62fda73.el8.x86_64 39/135 2023-12-11T17:35:12.824 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 58/135 2023-12-11T17:35:12.829 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-rbd-2:19.0.0-119.ga62fda73.el8.x86_64 38/135 2023-12-11T17:35:12.956 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 40/135 2023-12-11T17:35:12.997 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : librados-devel-2:19.0.0-119.ga62fda73.el8.x86_64 39/135 2023-12-11T17:35:13.131 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 40/135 2023-12-11T17:35:13.142 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 41/135 2023-12-11T17:35:13.195 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 59/135 2023-12-11T17:35:13.288 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 41/135 2023-12-11T17:35:13.360 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : fuse-2.9.7-15.el8.x86_64 42/135 2023-12-11T17:35:13.419 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 60/135 2023-12-11T17:35:13.456 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : uuid-1.6.2-43.el8.x86_64 42/135 2023-12-11T17:35:13.514 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 42/135 2023-12-11T17:35:13.520 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 43/135 2023-12-11T17:35:13.629 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 61/135 2023-12-11T17:35:13.757 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 44/135 2023-12-11T17:35:13.758 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 43/135 2023-12-11T17:35:13.849 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 62/135 2023-12-11T17:35:13.964 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 45/135 2023-12-11T17:35:14.011 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : automake-1.16.1-7.el8.noarch 44/135 2023-12-11T17:35:14.031 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 63/135 2023-12-11T17:35:14.173 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 45/135 2023-12-11T17:35:14.251 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 45/135 2023-12-11T17:35:14.355 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 46/135 2023-12-11T17:35:14.507 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : jq-1.6-3.el8.x86_64 46/135 2023-12-11T17:35:14.531 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 64/135 2023-12-11T17:35:14.660 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 47/135 2023-12-11T17:35:14.669 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 47/135 2023-12-11T17:35:14.843 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 65/135 2023-12-11T17:35:14.848 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 48/135 2023-12-11T17:35:14.917 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 48/135 2023-12-11T17:35:15.009 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 49/135 2023-12-11T17:35:15.100 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 66/135 2023-12-11T17:35:15.186 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : fuse-2.9.7-15.el8.x86_64 50/135 2023-12-11T17:35:15.196 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 49/135 2023-12-11T17:35:15.345 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 51/135 2023-12-11T17:35:15.349 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 67/135 2023-12-11T17:35:15.429 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 50/135 2023-12-11T17:35:15.549 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 52/135 2023-12-11T17:35:15.579 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 68/135 2023-12-11T17:35:15.727 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 69/135 2023-12-11T17:35:15.782 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 51/135 2023-12-11T17:35:15.783 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 53/135 2023-12-11T17:35:15.930 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 70/135 2023-12-11T17:35:16.006 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 52/135 2023-12-11T17:35:16.169 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 71/135 2023-12-11T17:35:16.212 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 54/135 2023-12-11T17:35:16.265 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 53/135 2023-12-11T17:35:16.409 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 72/135 2023-12-11T17:35:16.495 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 54/135 2023-12-11T17:35:16.565 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 73/135 2023-12-11T17:35:16.568 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 55/135 2023-12-11T17:35:16.686 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 55/135 2023-12-11T17:35:16.713 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 74/135 2023-12-11T17:35:16.770 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 56/135 2023-12-11T17:35:16.843 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 56/135 2023-12-11T17:35:16.968 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 75/135 2023-12-11T17:35:17.014 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 57/135 2023-12-11T17:35:17.188 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 76/135 2023-12-11T17:35:17.203 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 57/135 2023-12-11T17:35:17.265 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 58/135 2023-12-11T17:35:17.404 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 77/135 2023-12-11T17:35:17.496 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 58/135 2023-12-11T17:35:17.581 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 59/135 2023-12-11T17:35:17.589 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 78/135 2023-12-11T17:35:17.745 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 59/135 2023-12-11T17:35:17.755 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 60/135 2023-12-11T17:35:17.783 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 79/135 2023-12-11T17:35:17.960 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 61/135 2023-12-11T17:35:17.972 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 60/135 2023-12-11T17:35:17.986 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 80/135 2023-12-11T17:35:18.180 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 61/135 2023-12-11T17:35:18.198 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 81/135 2023-12-11T17:35:18.241 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 62/135 2023-12-11T17:35:18.350 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 62/135 2023-12-11T17:35:18.416 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 82/135 2023-12-11T17:35:18.463 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 63/135 2023-12-11T17:35:18.539 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : uuid-1.6.2-43.el8.x86_64 63/135 2023-12-11T17:35:18.613 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 63/135 2023-12-11T17:35:18.627 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 64/135 2023-12-11T17:35:18.658 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-portend-2.6-1.el8.noarch 83/135 2023-12-11T17:35:18.835 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 64/135 2023-12-11T17:35:18.877 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 84/135 2023-12-11T17:35:18.971 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 65/135 2023-12-11T17:35:19.034 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 85/135 2023-12-11T17:35:19.043 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : automake-1.16.1-7.el8.noarch 65/135 2023-12-11T17:35:19.231 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 66/135 2023-12-11T17:35:19.246 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 66/135 2023-12-11T17:35:19.312 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 86/135 2023-12-11T17:35:19.315 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 66/135 2023-12-11T17:35:19.497 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 67/135 2023-12-11T17:35:19.544 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : jq-1.6-3.el8.x86_64 67/135 2023-12-11T17:35:19.724 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 68/135 2023-12-11T17:35:19.740 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 68/135 2023-12-11T17:35:19.905 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 69/135 2023-12-11T17:35:19.923 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 69/135 2023-12-11T17:35:20.111 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 70/135 2023-12-11T17:35:20.135 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 70/135 2023-12-11T17:35:20.381 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 71/135 2023-12-11T17:35:20.390 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 71/135 2023-12-11T17:35:20.402 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 87/135 2023-12-11T17:35:20.572 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 72/135 2023-12-11T17:35:20.638 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 72/135 2023-12-11T17:35:20.673 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 88/135 2023-12-11T17:35:20.761 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 73/135 2023-12-11T17:35:20.785 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 73/135 2023-12-11T17:35:20.910 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 74/135 2023-12-11T17:35:20.927 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 89/135 2023-12-11T17:35:20.976 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 74/135 2023-12-11T17:35:21.142 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 75/135 2023-12-11T17:35:21.176 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 75/135 2023-12-11T17:35:21.220 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 90/135 2023-12-11T17:35:21.352 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 76/135 2023-12-11T17:35:21.395 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 76/135 2023-12-11T17:35:21.547 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 91/135 2023-12-11T17:35:21.568 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 77/135 2023-12-11T17:35:21.576 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 77/135 2023-12-11T17:35:21.758 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 78/135 2023-12-11T17:35:21.784 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 78/135 2023-12-11T17:35:21.807 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : luarocks-3.5.0-1.el8.noarch 92/135 2023-12-11T17:35:21.920 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 79/135 2023-12-11T17:35:21.937 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 79/135 2023-12-11T17:35:22.010 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 93/135 2023-12-11T17:35:22.120 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 80/135 2023-12-11T17:35:22.187 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 80/135 2023-12-11T17:35:22.210 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 94/135 2023-12-11T17:35:22.352 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 81/135 2023-12-11T17:35:22.402 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 81/135 2023-12-11T17:35:22.586 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 82/135 2023-12-11T17:35:22.611 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 82/135 2023-12-11T17:35:22.844 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-portend-2.6-1.el8.noarch 83/135 2023-12-11T17:35:22.884 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-portend-2.6-1.el8.noarch 83/135 2023-12-11T17:35:23.072 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 84/135 2023-12-11T17:35:23.130 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 84/135 2023-12-11T17:35:23.280 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 85/135 2023-12-11T17:35:23.305 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 85/135 2023-12-11T17:35:23.549 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 86/135 2023-12-11T17:35:23.559 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 86/135 2023-12-11T17:35:24.727 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 87/135 2023-12-11T17:35:24.754 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 87/135 2023-12-11T17:35:25.001 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 88/135 2023-12-11T17:35:25.052 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 88/135 2023-12-11T17:35:25.263 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 89/135 2023-12-11T17:35:25.297 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 89/135 2023-12-11T17:35:25.607 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 90/135 2023-12-11T17:35:25.629 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 90/135 2023-12-11T17:35:25.970 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 91/135 2023-12-11T17:35:25.974 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 91/135 2023-12-11T17:35:26.017 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-common-2:19.0.0-119.ga62fda73.el8.x86_64 95/135 2023-12-11T17:35:26.171 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : luarocks-3.5.0-1.el8.noarch 92/135 2023-12-11T17:35:26.234 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : luarocks-3.5.0-1.el8.noarch 92/135 2023-12-11T17:35:26.269 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-common-2:19.0.0-119.ga62fda73.el8.x86_64 95/135 2023-12-11T17:35:26.395 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 93/135 2023-12-11T17:35:26.453 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 93/135 2023-12-11T17:35:26.571 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 94/135 2023-12-11T17:35:26.688 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 94/135 2023-12-11T17:35:26.868 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-common-2:19.0.0-119.ga62fda73.el8.x86_64 95/135 2023-12-11T17:35:27.170 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-base-2:19.0.0-119.ga62fda73.el8.x86_64 96/135 2023-12-11T17:35:27.253 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-base-2:19.0.0-119.ga62fda73.el8.x86_64 96/135 2023-12-11T17:35:27.385 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-selinux-2:19.0.0-119.ga62fda73.el8.x86_64 97/135 2023-12-11T17:35:30.428 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-common-2:19.0.0-119.ga62fda73.el8.x86_64 95/135 2023-12-11T17:35:30.688 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-common-2:19.0.0-119.ga62fda73.el8.x86_64 95/135 2023-12-11T17:35:30.944 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-common-2:19.0.0-119.ga62fda73.el8.x86_64 95/135 2023-12-11T17:35:31.146 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-common-2:19.0.0-119.ga62fda73.el8.x86_64 95/135 2023-12-11T17:35:31.276 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-common-2:19.0.0-119.ga62fda73.el8.x86_64 95/135 2023-12-11T17:35:31.543 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-base-2:19.0.0-119.ga62fda73.el8.x86_64 96/135 2023-12-11T17:35:31.659 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-base-2:19.0.0-119.ga62fda73.el8.x86_64 96/135 2023-12-11T17:35:31.782 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-common-2:19.0.0-119.ga62fda73.el8.x86_64 95/135 2023-12-11T17:35:31.804 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-selinux-2:19.0.0-119.ga62fda73.el8.x86_64 97/135 2023-12-11T17:35:32.063 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-base-2:19.0.0-119.ga62fda73.el8.x86_64 96/135 2023-12-11T17:35:32.150 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-base-2:19.0.0-119.ga62fda73.el8.x86_64 96/135 2023-12-11T17:35:32.287 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-selinux-2:19.0.0-119.ga62fda73.el8.x86_64 97/135 2023-12-11T17:35:46.259 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-selinux-2:19.0.0-119.ga62fda73.el8.x86_64 97/135 2023-12-11T17:35:46.259 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /sys 2023-12-11T17:35:46.259 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /proc 2023-12-11T17:35:46.260 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /dev 2023-12-11T17:35:46.260 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /run 2023-12-11T17:35:46.260 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /mnt 2023-12-11T17:35:46.260 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /var/tmp 2023-12-11T17:35:46.260 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /home 2023-12-11T17:35:46.260 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /tmp 2023-12-11T17:35:46.260 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /dev 2023-12-11T17:35:46.260 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-12-11T17:35:46.649 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-volume-2:19.0.0-119.ga62fda73.el8.noarch 98/135 2023-12-11T17:35:48.258 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-volume-2:19.0.0-119.ga62fda73.el8.noarch 98/135 2023-12-11T17:35:48.475 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-osd-2:19.0.0-119.ga62fda73.el8.x86_64 99/135 2023-12-11T17:35:48.700 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-osd-2:19.0.0-119.ga62fda73.el8.x86_64 99/135 2023-12-11T17:35:48.898 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mds-2:19.0.0-119.ga62fda73.el8.x86_64 100/135 2023-12-11T17:35:49.354 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-mds-2:19.0.0-119.ga62fda73.el8.x86_64 100/135 2023-12-11T17:35:49.498 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mon-2:19.0.0-119.ga62fda73.el8.x86_64 101/135 2023-12-11T17:35:49.514 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-mon-2:19.0.0-119.ga62fda73.el8.x86_64 101/135 2023-12-11T17:35:50.544 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-selinux-2:19.0.0-119.ga62fda73.el8.x86_64 97/135 2023-12-11T17:35:50.544 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /sys 2023-12-11T17:35:50.544 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /proc 2023-12-11T17:35:50.544 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /dev 2023-12-11T17:35:50.544 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /run 2023-12-11T17:35:50.544 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /mnt 2023-12-11T17:35:50.544 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /var/tmp 2023-12-11T17:35:50.544 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /home 2023-12-11T17:35:50.544 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /tmp 2023-12-11T17:35:50.545 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /dev 2023-12-11T17:35:50.545 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-11T17:35:50.900 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-volume-2:19.0.0-119.ga62fda73.el8.noarch 98/135 2023-12-11T17:35:51.210 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: cephadm-2:19.0.0-119.ga62fda73.el8.noarch 102/135 2023-12-11T17:35:51.427 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : cephadm-2:19.0.0-119.ga62fda73.el8.noarch 102/135 2023-12-11T17:35:51.546 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-119.ga62fda73.el 103/135 2023-12-11T17:35:51.772 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-119.ga62fda73.e 104/135 2023-12-11T17:35:51.981 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-119.ga62fda73.el8.noar 105/135 2023-12-11T17:35:52.204 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-119.ga62fda73.el8.noar 105/135 2023-12-11T17:35:52.320 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-selinux-2:19.0.0-119.ga62fda73.el8.x86_64 97/135 2023-12-11T17:35:52.320 INFO:teuthology.orchestra.run.smithi059.stdout:skipping the directory /sys 2023-12-11T17:35:52.320 INFO:teuthology.orchestra.run.smithi059.stdout:skipping the directory /proc 2023-12-11T17:35:52.321 INFO:teuthology.orchestra.run.smithi059.stdout:skipping the directory /dev 2023-12-11T17:35:52.321 INFO:teuthology.orchestra.run.smithi059.stdout:skipping the directory /run 2023-12-11T17:35:52.321 INFO:teuthology.orchestra.run.smithi059.stdout:skipping the directory /mnt 2023-12-11T17:35:52.321 INFO:teuthology.orchestra.run.smithi059.stdout:skipping the directory /var/tmp 2023-12-11T17:35:52.321 INFO:teuthology.orchestra.run.smithi059.stdout:skipping the directory /home 2023-12-11T17:35:52.321 INFO:teuthology.orchestra.run.smithi059.stdout:skipping the directory /tmp 2023-12-11T17:35:52.321 INFO:teuthology.orchestra.run.smithi059.stdout:skipping the directory /dev 2023-12-11T17:35:52.321 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-12-11T17:35:52.474 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-119.ga62fda73.el8.no 106/135 2023-12-11T17:35:52.515 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-volume-2:19.0.0-119.ga62fda73.el8.noarch 98/135 2023-12-11T17:35:52.685 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-volume-2:19.0.0-119.ga62fda73.el8.noarch 98/135 2023-12-11T17:35:52.688 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-osd-2:19.0.0-119.ga62fda73.el8.x86_64 99/135 2023-12-11T17:35:52.915 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-osd-2:19.0.0-119.ga62fda73.el8.x86_64 99/135 2023-12-11T17:35:53.099 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mds-2:19.0.0-119.ga62fda73.el8.x86_64 100/135 2023-12-11T17:35:53.435 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-119.ga62fda73.el8.no 106/135 2023-12-11T17:35:53.555 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-mds-2:19.0.0-119.ga62fda73.el8.x86_64 100/135 2023-12-11T17:35:53.605 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-119.ga62f 107/135 2023-12-11T17:35:53.696 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-119.ga62f 107/135 2023-12-11T17:35:53.715 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mon-2:19.0.0-119.ga62fda73.el8.x86_64 101/135 2023-12-11T17:35:53.732 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-mon-2:19.0.0-119.ga62fda73.el8.x86_64 101/135 2023-12-11T17:35:54.083 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-119.ga62fda73.el8 108/135 2023-12-11T17:35:54.346 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mgr-2:19.0.0-119.ga62fda73.el8.x86_64 109/135 2023-12-11T17:35:54.384 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-mgr-2:19.0.0-119.ga62fda73.el8.x86_64 109/135 2023-12-11T17:35:54.397 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-volume-2:19.0.0-119.ga62fda73.el8.noarch 98/135 2023-12-11T17:35:54.555 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mgr-rook-2:19.0.0-119.ga62fda73.el8.noarch 110/135 2023-12-11T17:35:54.566 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-119.ga62fda73.el8.noarch 110/135 2023-12-11T17:35:54.568 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-osd-2:19.0.0-119.ga62fda73.el8.x86_64 99/135 2023-12-11T17:35:54.808 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-osd-2:19.0.0-119.ga62fda73.el8.x86_64 99/135 2023-12-11T17:35:54.976 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-mds-2:19.0.0-119.ga62fda73.el8.x86_64 100/135 2023-12-11T17:35:55.461 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-mds-2:19.0.0-119.ga62fda73.el8.x86_64 100/135 2023-12-11T17:35:55.501 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: cephadm-2:19.0.0-119.ga62fda73.el8.noarch 102/135 2023-12-11T17:35:55.642 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-mon-2:19.0.0-119.ga62fda73.el8.x86_64 101/135 2023-12-11T17:35:55.660 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-mon-2:19.0.0-119.ga62fda73.el8.x86_64 101/135 2023-12-11T17:35:55.707 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : cephadm-2:19.0.0-119.ga62fda73.el8.noarch 102/135 2023-12-11T17:35:55.822 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-119.ga62fda73.el 103/135 2023-12-11T17:35:55.881 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-2:19.0.0-119.ga62fda73.el8.x86_64 111/135 2023-12-11T17:35:56.039 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-119.ga62fda73.e 104/135 2023-12-11T17:35:56.099 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-radosgw-2:19.0.0-119.ga62fda73.el8.x86_64 112/135 2023-12-11T17:35:56.133 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-119.ga62fda73.el8.x86_64 112/135 2023-12-11T17:35:56.256 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-119.ga62fda73.el8.noar 105/135 2023-12-11T17:35:56.347 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-119.ga62fda 113/135 2023-12-11T17:35:56.389 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-119.ga62fda 113/135 2023-12-11T17:35:56.479 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-119.ga62fda73.el8.noar 105/135 2023-12-11T17:35:56.555 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : cephfs-mirror-2:19.0.0-119.ga62fda73.el8.x86_64 114/135 2023-12-11T17:35:56.704 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-119.ga62fda73.el8.no 106/135 2023-12-11T17:35:56.916 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-119.ga62fda73.el8.x86_64 114/135 2023-12-11T17:35:57.071 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : rbd-mirror-2:19.0.0-119.ga62fda73.el8.x86_64 115/135 2023-12-11T17:35:57.574 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: cephadm-2:19.0.0-119.ga62fda73.el8.noarch 102/135 2023-12-11T17:35:57.664 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-119.ga62fda73.el8.no 106/135 2023-12-11T17:35:57.787 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : cephadm-2:19.0.0-119.ga62fda73.el8.noarch 102/135 2023-12-11T17:35:57.900 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-119.ga62fda73.el 103/135 2023-12-11T17:35:57.914 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-119.ga62f 107/135 2023-12-11T17:35:58.007 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-119.ga62f 107/135 2023-12-11T17:35:58.134 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-119.ga62fda73.e 104/135 2023-12-11T17:35:58.317 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-119.ga62fda73.el8.noar 105/135 2023-12-11T17:35:58.404 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-119.ga62fda73.el8 108/135 2023-12-11T17:35:58.548 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-119.ga62fda73.el8.noar 105/135 2023-12-11T17:35:58.671 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mgr-2:19.0.0-119.ga62fda73.el8.x86_64 109/135 2023-12-11T17:35:58.708 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-mgr-2:19.0.0-119.ga62fda73.el8.x86_64 109/135 2023-12-11T17:35:58.759 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-119.ga62fda73.el8.no 106/135 2023-12-11T17:35:58.881 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mgr-rook-2:19.0.0-119.ga62fda73.el8.noarch 110/135 2023-12-11T17:35:58.892 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-119.ga62fda73.el8.noarch 110/135 2023-12-11T17:35:59.779 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-119.ga62fda73.el8.no 106/135 2023-12-11T17:35:59.975 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-119.ga62f 107/135 2023-12-11T17:36:00.070 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-119.ga62f 107/135 2023-12-11T17:36:00.242 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-2:19.0.0-119.ga62fda73.el8.x86_64 111/135 2023-12-11T17:36:00.419 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-119.ga62fda73.el8 108/135 2023-12-11T17:36:00.497 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-radosgw-2:19.0.0-119.ga62fda73.el8.x86_64 112/135 2023-12-11T17:36:00.532 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-119.ga62fda73.el8.x86_64 112/135 2023-12-11T17:36:00.660 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-mgr-2:19.0.0-119.ga62fda73.el8.x86_64 109/135 2023-12-11T17:36:00.689 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-119.ga62fda 113/135 2023-12-11T17:36:00.698 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-mgr-2:19.0.0-119.ga62fda73.el8.x86_64 109/135 2023-12-11T17:36:00.731 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-119.ga62fda 113/135 2023-12-11T17:36:00.883 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-mgr-rook-2:19.0.0-119.ga62fda73.el8.noarch 110/135 2023-12-11T17:36:00.895 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-119.ga62fda73.el8.noarch 110/135 2023-12-11T17:36:00.931 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : cephfs-mirror-2:19.0.0-119.ga62fda73.el8.x86_64 114/135 2023-12-11T17:36:01.289 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-119.ga62fda73.el8.x86_64 114/135 2023-12-11T17:36:01.438 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : rbd-mirror-2:19.0.0-119.ga62fda73.el8.x86_64 115/135 2023-12-11T17:36:01.749 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: rbd-mirror-2:19.0.0-119.ga62fda73.el8.x86_64 115/135 2023-12-11T17:36:02.280 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-2:19.0.0-119.ga62fda73.el8.x86_64 111/135 2023-12-11T17:36:02.488 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-radosgw-2:19.0.0-119.ga62fda73.el8.x86_64 112/135 2023-12-11T17:36:02.523 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-119.ga62fda73.el8.x86_64 112/135 2023-12-11T17:36:02.656 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-test-2:19.0.0-119.ga62fda73.el8.x86_64 116/135 2023-12-11T17:36:02.666 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-119.ga62fda 113/135 2023-12-11T17:36:02.711 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-119.ga62fda 113/135 2023-12-11T17:36:02.892 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : cephfs-mirror-2:19.0.0-119.ga62fda73.el8.x86_64 114/135 2023-12-11T17:36:03.007 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 117/135 2023-12-11T17:36:03.215 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-fuse-2:19.0.0-119.ga62fda73.el8.x86_64 118/135 2023-12-11T17:36:03.273 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-119.ga62fda73.el8.x86_64 114/135 2023-12-11T17:36:03.433 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : rbd-mirror-2:19.0.0-119.ga62fda73.el8.x86_64 115/135 2023-12-11T17:36:03.488 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 119/135 2023-12-11T17:36:03.654 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libtool-2.4.6-25.el8.x86_64 120/135 2023-12-11T17:36:03.844 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 120/135 2023-12-11T17:36:04.100 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 121/135 2023-12-11T17:36:04.266 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libcephfs-devel-2:19.0.0-119.ga62fda73.el8.x86_6 122/135 2023-12-11T17:36:04.457 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 123/135 2023-12-11T17:36:06.130 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: rbd-mirror-2:19.0.0-119.ga62fda73.el8.x86_64 115/135 2023-12-11T17:36:06.465 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-test-2:19.0.0-119.ga62fda73.el8.x86_64 116/135 2023-12-11T17:36:06.654 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libtool-2.4.6-25.el8.x86_64 117/135 2023-12-11T17:36:06.662 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 124/135 2023-12-11T17:36:06.855 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 117/135 2023-12-11T17:36:06.937 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 124/135 2023-12-11T17:36:06.956 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 124/135 2023-12-11T17:36:07.104 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : cephfs-top-2:19.0.0-119.ga62fda73.el8.noarch 125/135 2023-12-11T17:36:07.478 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : rbd-fuse-2:19.0.0-119.ga62fda73.el8.x86_64 126/135 2023-12-11T17:36:07.786 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : rbd-nbd-2:19.0.0-119.ga62fda73.el8.x86_64 127/135 2023-12-11T17:36:07.794 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 118/135 2023-12-11T17:36:08.159 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 128/135 2023-12-11T17:36:08.165 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 119/135 2023-12-11T17:36:08.344 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : flex-2.6.1-9.el8.x86_64 129/135 2023-12-11T17:36:08.364 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-fuse-2:19.0.0-119.ga62fda73.el8.x86_64 120/135 2023-12-11T17:36:08.421 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: rbd-mirror-2:19.0.0-119.ga62fda73.el8.x86_64 115/135 2023-12-11T17:36:08.427 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 129/135 2023-12-11T17:36:08.566 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 121/135 2023-12-11T17:36:08.620 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : bison-3.0.4-10.el8.x86_64 130/135 2023-12-11T17:36:08.632 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 130/135 2023-12-11T17:36:08.799 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libcephfs-devel-2:19.0.0-119.ga62fda73.el8.x86_6 122/135 2023-12-11T17:36:08.803 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 131/135 2023-12-11T17:36:08.811 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 131/135 2023-12-11T17:36:08.930 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 132/135 2023-12-11T17:36:08.930 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 133/135 2023-12-11T17:36:08.956 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 123/135 2023-12-11T17:36:09.131 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 133/135 2023-12-11T17:36:09.131 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 134/135 2023-12-11T17:36:09.377 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 134/135 2023-12-11T17:36:09.380 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-test-2:19.0.0-119.ga62fda73.el8.x86_64 116/135 2023-12-11T17:36:09.585 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 135/135 2023-12-11T17:36:09.707 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 117/135 2023-12-11T17:36:09.886 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-fuse-2:19.0.0-119.ga62fda73.el8.x86_64 118/135 2023-12-11T17:36:10.112 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 119/135 2023-12-11T17:36:10.113 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 135/135 2023-12-11T17:36:10.113 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-2:19.0.0-119.ga62fda73.el8.x86_64 1/135 2023-12-11T17:36:10.113 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-base-2:19.0.0-119.ga62fda73.el8.x86_64 2/135 2023-12-11T17:36:10.114 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-common-2:19.0.0-119.ga62fda73.el8.x86_64 3/135 2023-12-11T17:36:10.114 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-fuse-2:19.0.0-119.ga62fda73.el8.x86_64 4/135 2023-12-11T17:36:10.114 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-119.ga62fda 5/135 2023-12-11T17:36:10.114 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mds-2:19.0.0-119.ga62fda73.el8.x86_64 6/135 2023-12-11T17:36:10.114 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mgr-2:19.0.0-119.ga62fda73.el8.x86_64 7/135 2023-12-11T17:36:10.114 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mon-2:19.0.0-119.ga62fda73.el8.x86_64 8/135 2023-12-11T17:36:10.114 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-osd-2:19.0.0-119.ga62fda73.el8.x86_64 9/135 2023-12-11T17:36:10.114 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-radosgw-2:19.0.0-119.ga62fda73.el8.x86_64 10/135 2023-12-11T17:36:10.114 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-selinux-2:19.0.0-119.ga62fda73.el8.x86_64 11/135 2023-12-11T17:36:10.114 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-test-2:19.0.0-119.ga62fda73.el8.x86_64 12/135 2023-12-11T17:36:10.114 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : cephfs-mirror-2:19.0.0-119.ga62fda73.el8.x86_64 13/135 2023-12-11T17:36:10.114 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libcephfs-devel-2:19.0.0-119.ga62fda73.el8.x86_6 14/135 2023-12-11T17:36:10.114 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libcephfs2-2:19.0.0-119.ga62fda73.el8.x86_64 15/135 2023-12-11T17:36:10.114 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libcephsqlite-2:19.0.0-119.ga62fda73.el8.x86_64 16/135 2023-12-11T17:36:10.115 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librados-devel-2:19.0.0-119.ga62fda73.el8.x86_64 17/135 2023-12-11T17:36:10.115 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libradosstriper1-2:19.0.0-119.ga62fda73.el8.x86_ 18/135 2023-12-11T17:36:10.115 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librgw2-2:19.0.0-119.ga62fda73.el8.x86_64 19/135 2023-12-11T17:36:10.115 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-ceph-argparse-2:19.0.0-119.ga62fda73.el8 20/135 2023-12-11T17:36:10.115 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-ceph-common-2:19.0.0-119.ga62fda73.el8.x 21/135 2023-12-11T17:36:10.115 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-cephfs-2:19.0.0-119.ga62fda73.el8.x86_64 22/135 2023-12-11T17:36:10.115 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-rados-2:19.0.0-119.ga62fda73.el8.x86_64 23/135 2023-12-11T17:36:10.115 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-rbd-2:19.0.0-119.ga62fda73.el8.x86_64 24/135 2023-12-11T17:36:10.116 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-rgw-2:19.0.0-119.ga62fda73.el8.x86_64 25/135 2023-12-11T17:36:10.116 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : rbd-fuse-2:19.0.0-119.ga62fda73.el8.x86_64 26/135 2023-12-11T17:36:10.116 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : rbd-mirror-2:19.0.0-119.ga62fda73.el8.x86_64 27/135 2023-12-11T17:36:10.116 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : rbd-nbd-2:19.0.0-119.ga62fda73.el8.x86_64 28/135 2023-12-11T17:36:10.116 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-119.ga62fda73.e 29/135 2023-12-11T17:36:10.116 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-119.ga62fda73.el8.noar 30/135 2023-12-11T17:36:10.116 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-119.ga62fda73.el8.no 31/135 2023-12-11T17:36:10.116 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-119.ga62f 32/135 2023-12-11T17:36:10.116 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-119.ga62fda73.el8 33/135 2023-12-11T17:36:10.116 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mgr-rook-2:19.0.0-119.ga62fda73.el8.noarch 34/135 2023-12-11T17:36:10.116 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-119.ga62fda73.el 35/135 2023-12-11T17:36:10.116 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-volume-2:19.0.0-119.ga62fda73.el8.noarch 36/135 2023-12-11T17:36:10.117 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : cephadm-2:19.0.0-119.ga62fda73.el8.noarch 37/135 2023-12-11T17:36:10.117 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : cephfs-top-2:19.0.0-119.ga62fda73.el8.noarch 38/135 2023-12-11T17:36:10.117 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/135 2023-12-11T17:36:10.117 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/135 2023-12-11T17:36:10.117 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/135 2023-12-11T17:36:10.117 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : luarocks-3.5.0-1.el8.noarch 42/135 2023-12-11T17:36:10.117 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 43/135 2023-12-11T17:36:10.117 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 44/135 2023-12-11T17:36:10.117 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 45/135 2023-12-11T17:36:10.117 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 46/135 2023-12-11T17:36:10.117 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 47/135 2023-12-11T17:36:10.117 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 48/135 2023-12-11T17:36:10.117 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 49/135 2023-12-11T17:36:10.118 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 50/135 2023-12-11T17:36:10.118 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 51/135 2023-12-11T17:36:10.118 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 52/135 2023-12-11T17:36:10.118 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 53/135 2023-12-11T17:36:10.118 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 54/135 2023-12-11T17:36:10.118 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 55/135 2023-12-11T17:36:10.118 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 56/135 2023-12-11T17:36:10.118 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 57/135 2023-12-11T17:36:10.118 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-portend-2.6-1.el8.noarch 58/135 2023-12-11T17:36:10.118 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 59/135 2023-12-11T17:36:10.118 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 60/135 2023-12-11T17:36:10.118 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 61/135 2023-12-11T17:36:10.118 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 62/135 2023-12-11T17:36:10.118 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 63/135 2023-12-11T17:36:10.119 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 64/135 2023-12-11T17:36:10.119 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 65/135 2023-12-11T17:36:10.119 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 66/135 2023-12-11T17:36:10.119 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 67/135 2023-12-11T17:36:10.119 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 68/135 2023-12-11T17:36:10.119 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 69/135 2023-12-11T17:36:10.119 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 70/135 2023-12-11T17:36:10.119 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 71/135 2023-12-11T17:36:10.119 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 72/135 2023-12-11T17:36:10.119 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 73/135 2023-12-11T17:36:10.119 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 74/135 2023-12-11T17:36:10.119 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 75/135 2023-12-11T17:36:10.119 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 76/135 2023-12-11T17:36:10.120 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 77/135 2023-12-11T17:36:10.120 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 78/135 2023-12-11T17:36:10.120 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 79/135 2023-12-11T17:36:10.120 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 80/135 2023-12-11T17:36:10.120 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 81/135 2023-12-11T17:36:10.120 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 82/135 2023-12-11T17:36:10.120 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 83/135 2023-12-11T17:36:10.120 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 84/135 2023-12-11T17:36:10.120 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 85/135 2023-12-11T17:36:10.120 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 86/135 2023-12-11T17:36:10.120 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 87/135 2023-12-11T17:36:10.120 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 88/135 2023-12-11T17:36:10.120 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 89/135 2023-12-11T17:36:10.120 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 90/135 2023-12-11T17:36:10.121 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 91/135 2023-12-11T17:36:10.121 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 92/135 2023-12-11T17:36:10.121 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 93/135 2023-12-11T17:36:10.121 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 94/135 2023-12-11T17:36:10.121 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 95/135 2023-12-11T17:36:10.121 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 96/135 2023-12-11T17:36:10.121 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 97/135 2023-12-11T17:36:10.121 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 98/135 2023-12-11T17:36:10.121 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 99/135 2023-12-11T17:36:10.121 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 100/135 2023-12-11T17:36:10.121 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 101/135 2023-12-11T17:36:10.121 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 102/135 2023-12-11T17:36:10.121 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 103/135 2023-12-11T17:36:10.122 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 104/135 2023-12-11T17:36:10.122 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : bison-3.0.4-10.el8.x86_64 105/135 2023-12-11T17:36:10.122 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 106/135 2023-12-11T17:36:10.122 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 107/135 2023-12-11T17:36:10.122 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 108/135 2023-12-11T17:36:10.122 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : automake-1.16.1-7.el8.noarch 109/135 2023-12-11T17:36:10.122 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 110/135 2023-12-11T17:36:10.122 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 111/135 2023-12-11T17:36:10.122 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : flex-2.6.1-9.el8.x86_64 112/135 2023-12-11T17:36:10.122 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 113/135 2023-12-11T17:36:10.122 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 114/135 2023-12-11T17:36:10.122 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 115/135 2023-12-11T17:36:10.122 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 116/135 2023-12-11T17:36:10.122 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 117/135 2023-12-11T17:36:10.123 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 118/135 2023-12-11T17:36:10.123 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 119/135 2023-12-11T17:36:10.123 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 120/135 2023-12-11T17:36:10.123 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 121/135 2023-12-11T17:36:10.123 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 122/135 2023-12-11T17:36:10.123 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libpq-13.5-1.el8.x86_64 123/135 2023-12-11T17:36:10.123 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 124/135 2023-12-11T17:36:10.123 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : jq-1.6-3.el8.x86_64 125/135 2023-12-11T17:36:10.123 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 126/135 2023-12-11T17:36:10.123 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 127/135 2023-12-11T17:36:10.123 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 128/135 2023-12-11T17:36:10.123 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 129/135 2023-12-11T17:36:10.123 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librados2-2:19.0.0-119.ga62fda73.el8.x86_64 130/135 2023-12-11T17:36:10.123 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 131/135 2023-12-11T17:36:10.123 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librbd1-2:19.0.0-119.ga62fda73.el8.x86_64 132/135 2023-12-11T17:36:10.124 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 133/135 2023-12-11T17:36:10.124 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 134/135 2023-12-11T17:36:10.265 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libtool-2.4.6-25.el8.x86_64 120/135 2023-12-11T17:36:10.482 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 120/135 2023-12-11T17:36:10.720 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 121/135 2023-12-11T17:36:10.928 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libcephfs-devel-2:19.0.0-119.ga62fda73.el8.x86_6 122/135 2023-12-11T17:36:11.077 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 123/135 2023-12-11T17:36:11.232 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 124/135 2023-12-11T17:36:11.536 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 124/135 2023-12-11T17:36:11.555 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 124/135 2023-12-11T17:36:11.833 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : cephfs-top-2:19.0.0-119.ga62fda73.el8.noarch 125/135 2023-12-11T17:36:12.102 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : rbd-fuse-2:19.0.0-119.ga62fda73.el8.x86_64 126/135 2023-12-11T17:36:12.403 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : rbd-nbd-2:19.0.0-119.ga62fda73.el8.x86_64 127/135 2023-12-11T17:36:12.788 INFO:teuthology.orchestra.run.smithi115.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 128/135 2023-12-11T17:36:12.944 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 129/135 2023-12-11T17:36:12.953 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 129/135 2023-12-11T17:36:13.141 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 130/135 2023-12-11T17:36:13.327 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : flex-2.6.1-9.el8.x86_64 131/135 2023-12-11T17:36:13.416 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 131/135 2023-12-11T17:36:13.432 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 124/135 2023-12-11T17:36:13.644 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : bison-3.0.4-10.el8.x86_64 132/135 2023-12-11T17:36:13.658 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 132/135 2023-12-11T17:36:13.658 INFO:teuthology.orchestra.run.smithi115.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 133/135 2023-12-11T17:36:13.682 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 124/135 2023-12-11T17:36:13.700 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 124/135 2023-12-11T17:36:13.899 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : cephfs-top-2:19.0.0-119.ga62fda73.el8.noarch 125/135 2023-12-11T17:36:13.905 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 133/135 2023-12-11T17:36:13.905 INFO:teuthology.orchestra.run.smithi115.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 134/135 2023-12-11T17:36:14.141 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 134/135 2023-12-11T17:36:14.224 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : rbd-fuse-2:19.0.0-119.ga62fda73.el8.x86_64 126/135 2023-12-11T17:36:14.360 INFO:teuthology.orchestra.run.smithi115.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 135/135 2023-12-11T17:36:14.527 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : rbd-nbd-2:19.0.0-119.ga62fda73.el8.x86_64 127/135 2023-12-11T17:36:14.854 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 135/135 2023-12-11T17:36:14.854 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-2:19.0.0-119.ga62fda73.el8.x86_64 1/135 2023-12-11T17:36:14.855 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-base-2:19.0.0-119.ga62fda73.el8.x86_64 2/135 2023-12-11T17:36:14.855 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-common-2:19.0.0-119.ga62fda73.el8.x86_64 3/135 2023-12-11T17:36:14.855 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-fuse-2:19.0.0-119.ga62fda73.el8.x86_64 4/135 2023-12-11T17:36:14.855 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-119.ga62fda 5/135 2023-12-11T17:36:14.855 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mds-2:19.0.0-119.ga62fda73.el8.x86_64 6/135 2023-12-11T17:36:14.855 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mgr-2:19.0.0-119.ga62fda73.el8.x86_64 7/135 2023-12-11T17:36:14.855 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mon-2:19.0.0-119.ga62fda73.el8.x86_64 8/135 2023-12-11T17:36:14.855 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-osd-2:19.0.0-119.ga62fda73.el8.x86_64 9/135 2023-12-11T17:36:14.855 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-radosgw-2:19.0.0-119.ga62fda73.el8.x86_64 10/135 2023-12-11T17:36:14.855 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-selinux-2:19.0.0-119.ga62fda73.el8.x86_64 11/135 2023-12-11T17:36:14.855 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-test-2:19.0.0-119.ga62fda73.el8.x86_64 12/135 2023-12-11T17:36:14.855 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : cephfs-mirror-2:19.0.0-119.ga62fda73.el8.x86_64 13/135 2023-12-11T17:36:14.855 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libcephfs-devel-2:19.0.0-119.ga62fda73.el8.x86_6 14/135 2023-12-11T17:36:14.855 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libcephfs2-2:19.0.0-119.ga62fda73.el8.x86_64 15/135 2023-12-11T17:36:14.855 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libcephsqlite-2:19.0.0-119.ga62fda73.el8.x86_64 16/135 2023-12-11T17:36:14.855 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librados-devel-2:19.0.0-119.ga62fda73.el8.x86_64 17/135 2023-12-11T17:36:14.856 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libradosstriper1-2:19.0.0-119.ga62fda73.el8.x86_ 18/135 2023-12-11T17:36:14.856 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librgw2-2:19.0.0-119.ga62fda73.el8.x86_64 19/135 2023-12-11T17:36:14.856 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-ceph-argparse-2:19.0.0-119.ga62fda73.el8 20/135 2023-12-11T17:36:14.856 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-ceph-common-2:19.0.0-119.ga62fda73.el8.x 21/135 2023-12-11T17:36:14.856 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-cephfs-2:19.0.0-119.ga62fda73.el8.x86_64 22/135 2023-12-11T17:36:14.856 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-rados-2:19.0.0-119.ga62fda73.el8.x86_64 23/135 2023-12-11T17:36:14.856 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-rbd-2:19.0.0-119.ga62fda73.el8.x86_64 24/135 2023-12-11T17:36:14.856 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-rgw-2:19.0.0-119.ga62fda73.el8.x86_64 25/135 2023-12-11T17:36:14.856 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : rbd-fuse-2:19.0.0-119.ga62fda73.el8.x86_64 26/135 2023-12-11T17:36:14.856 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : rbd-mirror-2:19.0.0-119.ga62fda73.el8.x86_64 27/135 2023-12-11T17:36:14.856 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : rbd-nbd-2:19.0.0-119.ga62fda73.el8.x86_64 28/135 2023-12-11T17:36:14.866 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-119.ga62fda73.e 29/135 2023-12-11T17:36:14.866 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-119.ga62fda73.el8.noar 30/135 2023-12-11T17:36:14.866 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-119.ga62fda73.el8.no 31/135 2023-12-11T17:36:14.866 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-119.ga62f 32/135 2023-12-11T17:36:14.866 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-119.ga62fda73.el8 33/135 2023-12-11T17:36:14.866 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mgr-rook-2:19.0.0-119.ga62fda73.el8.noarch 34/135 2023-12-11T17:36:14.866 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-119.ga62fda73.el 35/135 2023-12-11T17:36:14.866 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-volume-2:19.0.0-119.ga62fda73.el8.noarch 36/135 2023-12-11T17:36:14.866 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : cephadm-2:19.0.0-119.ga62fda73.el8.noarch 37/135 2023-12-11T17:36:14.866 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : cephfs-top-2:19.0.0-119.ga62fda73.el8.noarch 38/135 2023-12-11T17:36:14.866 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/135 2023-12-11T17:36:14.867 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/135 2023-12-11T17:36:14.867 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/135 2023-12-11T17:36:14.867 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : luarocks-3.5.0-1.el8.noarch 42/135 2023-12-11T17:36:14.867 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 43/135 2023-12-11T17:36:14.867 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 44/135 2023-12-11T17:36:14.867 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 45/135 2023-12-11T17:36:14.867 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 46/135 2023-12-11T17:36:14.867 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 47/135 2023-12-11T17:36:14.867 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 48/135 2023-12-11T17:36:14.867 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 49/135 2023-12-11T17:36:14.867 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 50/135 2023-12-11T17:36:14.867 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 51/135 2023-12-11T17:36:14.867 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 52/135 2023-12-11T17:36:14.867 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 53/135 2023-12-11T17:36:14.868 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 54/135 2023-12-11T17:36:14.868 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 55/135 2023-12-11T17:36:14.868 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 56/135 2023-12-11T17:36:14.868 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 57/135 2023-12-11T17:36:14.868 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-portend-2.6-1.el8.noarch 58/135 2023-12-11T17:36:14.868 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 59/135 2023-12-11T17:36:14.868 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 60/135 2023-12-11T17:36:14.868 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 61/135 2023-12-11T17:36:14.868 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 62/135 2023-12-11T17:36:14.868 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 63/135 2023-12-11T17:36:14.868 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 64/135 2023-12-11T17:36:14.868 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 65/135 2023-12-11T17:36:14.868 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 66/135 2023-12-11T17:36:14.868 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 67/135 2023-12-11T17:36:14.869 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 68/135 2023-12-11T17:36:14.869 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 69/135 2023-12-11T17:36:14.869 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 70/135 2023-12-11T17:36:14.869 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 71/135 2023-12-11T17:36:14.869 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 72/135 2023-12-11T17:36:14.869 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 73/135 2023-12-11T17:36:14.869 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 74/135 2023-12-11T17:36:14.869 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 75/135 2023-12-11T17:36:14.869 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 76/135 2023-12-11T17:36:14.869 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 77/135 2023-12-11T17:36:14.869 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 78/135 2023-12-11T17:36:14.869 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : bison-3.0.4-10.el8.x86_64 79/135 2023-12-11T17:36:14.869 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 80/135 2023-12-11T17:36:14.870 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 81/135 2023-12-11T17:36:14.870 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 82/135 2023-12-11T17:36:14.870 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : automake-1.16.1-7.el8.noarch 83/135 2023-12-11T17:36:14.870 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 84/135 2023-12-11T17:36:14.870 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 85/135 2023-12-11T17:36:14.870 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : flex-2.6.1-9.el8.x86_64 86/135 2023-12-11T17:36:14.870 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 87/135 2023-12-11T17:36:14.870 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 88/135 2023-12-11T17:36:14.870 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 89/135 2023-12-11T17:36:14.870 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 90/135 2023-12-11T17:36:14.870 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 91/135 2023-12-11T17:36:14.870 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 92/135 2023-12-11T17:36:14.870 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 93/135 2023-12-11T17:36:14.870 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 94/135 2023-12-11T17:36:14.871 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 95/135 2023-12-11T17:36:14.871 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 96/135 2023-12-11T17:36:14.871 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libpq-13.5-1.el8.x86_64 97/135 2023-12-11T17:36:14.871 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 98/135 2023-12-11T17:36:14.871 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : jq-1.6-3.el8.x86_64 99/135 2023-12-11T17:36:14.871 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 100/135 2023-12-11T17:36:14.871 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 101/135 2023-12-11T17:36:14.871 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 102/135 2023-12-11T17:36:14.871 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 103/135 2023-12-11T17:36:14.871 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 104/135 2023-12-11T17:36:14.871 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 105/135 2023-12-11T17:36:14.871 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 106/135 2023-12-11T17:36:14.871 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 107/135 2023-12-11T17:36:14.871 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 108/135 2023-12-11T17:36:14.871 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 109/135 2023-12-11T17:36:14.872 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 110/135 2023-12-11T17:36:14.872 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 111/135 2023-12-11T17:36:14.872 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 112/135 2023-12-11T17:36:14.872 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 113/135 2023-12-11T17:36:14.872 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 114/135 2023-12-11T17:36:14.872 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 115/135 2023-12-11T17:36:14.872 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 116/135 2023-12-11T17:36:14.872 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 117/135 2023-12-11T17:36:14.872 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 118/135 2023-12-11T17:36:14.872 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 119/135 2023-12-11T17:36:14.872 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 120/135 2023-12-11T17:36:14.872 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 121/135 2023-12-11T17:36:14.872 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 122/135 2023-12-11T17:36:14.872 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 123/135 2023-12-11T17:36:14.873 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 124/135 2023-12-11T17:36:14.873 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 125/135 2023-12-11T17:36:14.873 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 126/135 2023-12-11T17:36:14.873 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 127/135 2023-12-11T17:36:14.873 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 128/135 2023-12-11T17:36:14.873 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 129/135 2023-12-11T17:36:14.873 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librados2-2:19.0.0-119.ga62fda73.el8.x86_64 130/135 2023-12-11T17:36:14.873 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 131/135 2023-12-11T17:36:14.873 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librbd1-2:19.0.0-119.ga62fda73.el8.x86_64 132/135 2023-12-11T17:36:14.873 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 133/135 2023-12-11T17:36:14.873 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 134/135 2023-12-11T17:36:14.874 INFO:teuthology.orchestra.run.smithi059.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 128/135 2023-12-11T17:36:15.081 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : flex-2.6.1-9.el8.x86_64 129/135 2023-12-11T17:36:15.173 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 129/135 2023-12-11T17:36:15.381 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : bison-3.0.4-10.el8.x86_64 130/135 2023-12-11T17:36:15.397 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 130/135 2023-12-11T17:36:15.565 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 131/135 2023-12-11T17:36:15.573 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 131/135 2023-12-11T17:36:15.576 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 135/135 2023-12-11T17:36:15.576 INFO:teuthology.orchestra.run.smithi181.stdout:Installed products updated. 2023-12-11T17:36:15.684 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 132/135 2023-12-11T17:36:15.684 INFO:teuthology.orchestra.run.smithi059.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 133/135 2023-12-11T17:36:15.789 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-12-11T17:36:15.789 INFO:teuthology.orchestra.run.smithi181.stdout:Upgraded: 2023-12-11T17:36:15.789 INFO:teuthology.orchestra.run.smithi181.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-12-11T17:36:15.789 INFO:teuthology.orchestra.run.smithi181.stdout: librados2-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T17:36:15.790 INFO:teuthology.orchestra.run.smithi181.stdout: librbd1-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T17:36:15.790 INFO:teuthology.orchestra.run.smithi181.stdout:Installed: 2023-12-11T17:36:15.790 INFO:teuthology.orchestra.run.smithi181.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-12-11T17:36:15.790 INFO:teuthology.orchestra.run.smithi181.stdout: automake-1.16.1-7.el8.noarch 2023-12-11T17:36:15.790 INFO:teuthology.orchestra.run.smithi181.stdout: bison-3.0.4-10.el8.x86_64 2023-12-11T17:36:15.790 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T17:36:15.790 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-base-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T17:36:15.790 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-common-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T17:36:15.790 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-fuse-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T17:36:15.790 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-grafana-dashboards-2:19.0.0-119.ga62fda73.el8.noarch 2023-12-11T17:36:15.790 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-immutable-object-cache-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T17:36:15.790 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mds-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T17:36:15.791 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T17:36:15.791 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-cephadm-2:19.0.0-119.ga62fda73.el8.noarch 2023-12-11T17:36:15.791 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-dashboard-2:19.0.0-119.ga62fda73.el8.noarch 2023-12-11T17:36:15.791 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-diskprediction-local-2:19.0.0-119.ga62fda73.el8.noarch 2023-12-11T17:36:15.791 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-modules-core-2:19.0.0-119.ga62fda73.el8.noarch 2023-12-11T17:36:15.791 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-rook-2:19.0.0-119.ga62fda73.el8.noarch 2023-12-11T17:36:15.791 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mon-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T17:36:15.791 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-osd-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T17:36:15.791 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-prometheus-alerts-2:19.0.0-119.ga62fda73.el8.noarch 2023-12-11T17:36:15.791 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-radosgw-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T17:36:15.791 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-selinux-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T17:36:15.791 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-test-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T17:36:15.791 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-volume-2:19.0.0-119.ga62fda73.el8.noarch 2023-12-11T17:36:15.791 INFO:teuthology.orchestra.run.smithi181.stdout: cephadm-2:19.0.0-119.ga62fda73.el8.noarch 2023-12-11T17:36:15.792 INFO:teuthology.orchestra.run.smithi181.stdout: cephfs-mirror-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T17:36:15.792 INFO:teuthology.orchestra.run.smithi181.stdout: cephfs-top-2:19.0.0-119.ga62fda73.el8.noarch 2023-12-11T17:36:15.792 INFO:teuthology.orchestra.run.smithi181.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-12-11T17:36:15.792 INFO:teuthology.orchestra.run.smithi181.stdout: flex-2.6.1-9.el8.x86_64 2023-12-11T17:36:15.792 INFO:teuthology.orchestra.run.smithi181.stdout: fuse-2.9.7-15.el8.x86_64 2023-12-11T17:36:15.792 INFO:teuthology.orchestra.run.smithi181.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-12-11T17:36:15.792 INFO:teuthology.orchestra.run.smithi181.stdout: jq-1.6-3.el8.x86_64 2023-12-11T17:36:15.792 INFO:teuthology.orchestra.run.smithi181.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-12-11T17:36:15.792 INFO:teuthology.orchestra.run.smithi181.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-12-11T17:36:15.792 INFO:teuthology.orchestra.run.smithi181.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-12-11T17:36:15.792 INFO:teuthology.orchestra.run.smithi181.stdout: libcephfs-devel-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T17:36:15.792 INFO:teuthology.orchestra.run.smithi181.stdout: libcephfs2-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T17:36:15.792 INFO:teuthology.orchestra.run.smithi181.stdout: libcephsqlite-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T17:36:15.792 INFO:teuthology.orchestra.run.smithi181.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-12-11T17:36:15.793 INFO:teuthology.orchestra.run.smithi181.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-12-11T17:36:15.793 INFO:teuthology.orchestra.run.smithi181.stdout: liboath-2.6.2-3.el8.x86_64 2023-12-11T17:36:15.793 INFO:teuthology.orchestra.run.smithi181.stdout: libpq-13.5-1.el8.x86_64 2023-12-11T17:36:15.793 INFO:teuthology.orchestra.run.smithi181.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-12-11T17:36:15.793 INFO:teuthology.orchestra.run.smithi181.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-12-11T17:36:15.793 INFO:teuthology.orchestra.run.smithi181.stdout: librados-devel-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T17:36:15.793 INFO:teuthology.orchestra.run.smithi181.stdout: libradosstriper1-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T17:36:15.793 INFO:teuthology.orchestra.run.smithi181.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-12-11T17:36:15.793 INFO:teuthology.orchestra.run.smithi181.stdout: librgw2-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T17:36:15.793 INFO:teuthology.orchestra.run.smithi181.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-12-11T17:36:15.793 INFO:teuthology.orchestra.run.smithi181.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-12-11T17:36:15.793 INFO:teuthology.orchestra.run.smithi181.stdout: libtool-2.4.6-25.el8.x86_64 2023-12-11T17:36:15.794 INFO:teuthology.orchestra.run.smithi181.stdout: libunwind-1.3.1-3.el8.x86_64 2023-12-11T17:36:15.794 INFO:teuthology.orchestra.run.smithi181.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-12-11T17:36:15.794 INFO:teuthology.orchestra.run.smithi181.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-12-11T17:36:15.794 INFO:teuthology.orchestra.run.smithi181.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-12-11T17:36:15.794 INFO:teuthology.orchestra.run.smithi181.stdout: luarocks-3.5.0-1.el8.noarch 2023-12-11T17:36:15.794 INFO:teuthology.orchestra.run.smithi181.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-12-11T17:36:15.794 INFO:teuthology.orchestra.run.smithi181.stdout: openblas-0.3.15-3.el8.x86_64 2023-12-11T17:36:15.794 INFO:teuthology.orchestra.run.smithi181.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-12-11T17:36:15.794 INFO:teuthology.orchestra.run.smithi181.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-12-11T17:36:15.794 INFO:teuthology.orchestra.run.smithi181.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-12-11T17:36:15.794 INFO:teuthology.orchestra.run.smithi181.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-12-11T17:36:15.794 INFO:teuthology.orchestra.run.smithi181.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-12-11T17:36:15.794 INFO:teuthology.orchestra.run.smithi181.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-12-11T17:36:15.794 INFO:teuthology.orchestra.run.smithi181.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-12-11T17:36:15.794 INFO:teuthology.orchestra.run.smithi181.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-12-11T17:36:15.795 INFO:teuthology.orchestra.run.smithi181.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-12-11T17:36:15.795 INFO:teuthology.orchestra.run.smithi181.stdout: python3-babel-2.5.1-7.el8.noarch 2023-12-11T17:36:15.795 INFO:teuthology.orchestra.run.smithi181.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-12-11T17:36:15.795 INFO:teuthology.orchestra.run.smithi181.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-12-11T17:36:15.795 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-12-11T17:36:15.795 INFO:teuthology.orchestra.run.smithi181.stdout: python3-ceph-argparse-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T17:36:15.795 INFO:teuthology.orchestra.run.smithi181.stdout: python3-ceph-common-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T17:36:15.795 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cephfs-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T17:36:15.795 INFO:teuthology.orchestra.run.smithi181.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-12-11T17:36:15.795 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-12-11T17:36:15.795 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-12-11T17:36:15.795 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-12-11T17:36:15.795 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-12-11T17:36:15.796 INFO:teuthology.orchestra.run.smithi181.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-12-11T17:36:15.796 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jaraco-6.2-6.el8.noarch 2023-12-11T17:36:15.796 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-12-11T17:36:15.796 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-12-11T17:36:15.796 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-12-11T17:36:15.796 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-12-11T17:36:15.796 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-12-11T17:36:15.796 INFO:teuthology.orchestra.run.smithi181.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-12-11T17:36:15.796 INFO:teuthology.orchestra.run.smithi181.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-12-11T17:36:15.796 INFO:teuthology.orchestra.run.smithi181.stdout: python3-mako-1.0.6-13.el8.noarch 2023-12-11T17:36:15.796 INFO:teuthology.orchestra.run.smithi181.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-12-11T17:36:15.796 INFO:teuthology.orchestra.run.smithi181.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-12-11T17:36:15.796 INFO:teuthology.orchestra.run.smithi181.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-12-11T17:36:15.796 INFO:teuthology.orchestra.run.smithi181.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-12-11T17:36:15.797 INFO:teuthology.orchestra.run.smithi181.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-12-11T17:36:15.797 INFO:teuthology.orchestra.run.smithi181.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-12-11T17:36:15.797 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-12-11T17:36:15.797 INFO:teuthology.orchestra.run.smithi181.stdout: python3-portend-2.6-1.el8.noarch 2023-12-11T17:36:15.797 INFO:teuthology.orchestra.run.smithi181.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-12-11T17:36:15.797 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-12-11T17:36:15.797 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-12-11T17:36:15.797 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-12-11T17:36:15.797 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pycparser-2.14-14.el8.noarch 2023-12-11T17:36:15.797 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pytz-2017.2-9.el8.noarch 2023-12-11T17:36:15.797 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rados-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T17:36:15.798 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rbd-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T17:36:15.798 INFO:teuthology.orchestra.run.smithi181.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-12-11T17:36:15.798 INFO:teuthology.orchestra.run.smithi181.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-12-11T17:36:15.798 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rgw-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T17:36:15.798 INFO:teuthology.orchestra.run.smithi181.stdout: python3-routes-2.4.1-12.el8.noarch 2023-12-11T17:36:15.798 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rsa-4.9-2.el8.noarch 2023-12-11T17:36:15.798 INFO:teuthology.orchestra.run.smithi181.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-12-11T17:36:15.798 INFO:teuthology.orchestra.run.smithi181.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-12-11T17:36:15.798 INFO:teuthology.orchestra.run.smithi181.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-12-11T17:36:15.798 INFO:teuthology.orchestra.run.smithi181.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-12-11T17:36:15.798 INFO:teuthology.orchestra.run.smithi181.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-12-11T17:36:15.799 INFO:teuthology.orchestra.run.smithi181.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-12-11T17:36:15.799 INFO:teuthology.orchestra.run.smithi181.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-12-11T17:36:15.799 INFO:teuthology.orchestra.run.smithi181.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-12-11T17:36:15.799 INFO:teuthology.orchestra.run.smithi181.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-12-11T17:36:15.799 INFO:teuthology.orchestra.run.smithi181.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-12-11T17:36:15.799 INFO:teuthology.orchestra.run.smithi181.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-12-11T17:36:15.799 INFO:teuthology.orchestra.run.smithi181.stdout: rbd-fuse-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T17:36:15.799 INFO:teuthology.orchestra.run.smithi181.stdout: rbd-mirror-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T17:36:15.799 INFO:teuthology.orchestra.run.smithi181.stdout: rbd-nbd-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T17:36:15.799 INFO:teuthology.orchestra.run.smithi181.stdout: socat-1.7.4.1-1.el8.x86_64 2023-12-11T17:36:15.799 INFO:teuthology.orchestra.run.smithi181.stdout: thrift-0.13.0-2.el8.x86_64 2023-12-11T17:36:15.799 INFO:teuthology.orchestra.run.smithi181.stdout: uuid-1.6.2-43.el8.x86_64 2023-12-11T17:36:15.799 INFO:teuthology.orchestra.run.smithi181.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-12-11T17:36:15.799 INFO:teuthology.orchestra.run.smithi181.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-12-11T17:36:15.800 INFO:teuthology.orchestra.run.smithi181.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-12-11T17:36:15.800 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-12-11T17:36:15.800 INFO:teuthology.orchestra.run.smithi181.stdout:Complete! 2023-12-11T17:36:15.902 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 133/135 2023-12-11T17:36:15.902 INFO:teuthology.orchestra.run.smithi059.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 134/135 2023-12-11T17:36:15.990 DEBUG:teuthology.parallel:result is None 2023-12-11T17:36:16.116 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 134/135 2023-12-11T17:36:16.339 INFO:teuthology.orchestra.run.smithi059.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 135/135 2023-12-11T17:36:16.940 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 135/135 2023-12-11T17:36:16.940 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-2:19.0.0-119.ga62fda73.el8.x86_64 1/135 2023-12-11T17:36:16.940 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-base-2:19.0.0-119.ga62fda73.el8.x86_64 2/135 2023-12-11T17:36:16.940 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-common-2:19.0.0-119.ga62fda73.el8.x86_64 3/135 2023-12-11T17:36:16.940 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-fuse-2:19.0.0-119.ga62fda73.el8.x86_64 4/135 2023-12-11T17:36:16.940 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-119.ga62fda 5/135 2023-12-11T17:36:16.941 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-mds-2:19.0.0-119.ga62fda73.el8.x86_64 6/135 2023-12-11T17:36:16.941 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-mgr-2:19.0.0-119.ga62fda73.el8.x86_64 7/135 2023-12-11T17:36:16.941 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-mon-2:19.0.0-119.ga62fda73.el8.x86_64 8/135 2023-12-11T17:36:16.941 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-osd-2:19.0.0-119.ga62fda73.el8.x86_64 9/135 2023-12-11T17:36:16.941 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-radosgw-2:19.0.0-119.ga62fda73.el8.x86_64 10/135 2023-12-11T17:36:16.941 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-selinux-2:19.0.0-119.ga62fda73.el8.x86_64 11/135 2023-12-11T17:36:16.941 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-test-2:19.0.0-119.ga62fda73.el8.x86_64 12/135 2023-12-11T17:36:16.941 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : cephfs-mirror-2:19.0.0-119.ga62fda73.el8.x86_64 13/135 2023-12-11T17:36:16.941 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libcephfs-devel-2:19.0.0-119.ga62fda73.el8.x86_6 14/135 2023-12-11T17:36:16.941 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libcephfs2-2:19.0.0-119.ga62fda73.el8.x86_64 15/135 2023-12-11T17:36:16.941 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libcephsqlite-2:19.0.0-119.ga62fda73.el8.x86_64 16/135 2023-12-11T17:36:16.942 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : librados-devel-2:19.0.0-119.ga62fda73.el8.x86_64 17/135 2023-12-11T17:36:16.942 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libradosstriper1-2:19.0.0-119.ga62fda73.el8.x86_ 18/135 2023-12-11T17:36:16.942 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : librgw2-2:19.0.0-119.ga62fda73.el8.x86_64 19/135 2023-12-11T17:36:16.942 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-ceph-argparse-2:19.0.0-119.ga62fda73.el8 20/135 2023-12-11T17:36:16.942 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-ceph-common-2:19.0.0-119.ga62fda73.el8.x 21/135 2023-12-11T17:36:16.942 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-cephfs-2:19.0.0-119.ga62fda73.el8.x86_64 22/135 2023-12-11T17:36:16.942 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-rados-2:19.0.0-119.ga62fda73.el8.x86_64 23/135 2023-12-11T17:36:16.942 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-rbd-2:19.0.0-119.ga62fda73.el8.x86_64 24/135 2023-12-11T17:36:16.942 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-rgw-2:19.0.0-119.ga62fda73.el8.x86_64 25/135 2023-12-11T17:36:16.942 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : rbd-fuse-2:19.0.0-119.ga62fda73.el8.x86_64 26/135 2023-12-11T17:36:16.942 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : rbd-mirror-2:19.0.0-119.ga62fda73.el8.x86_64 27/135 2023-12-11T17:36:16.943 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : rbd-nbd-2:19.0.0-119.ga62fda73.el8.x86_64 28/135 2023-12-11T17:36:16.943 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-119.ga62fda73.e 29/135 2023-12-11T17:36:16.943 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-119.ga62fda73.el8.noar 30/135 2023-12-11T17:36:16.943 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-119.ga62fda73.el8.no 31/135 2023-12-11T17:36:16.943 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-119.ga62f 32/135 2023-12-11T17:36:16.943 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-119.ga62fda73.el8 33/135 2023-12-11T17:36:16.943 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-mgr-rook-2:19.0.0-119.ga62fda73.el8.noarch 34/135 2023-12-11T17:36:16.944 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-119.ga62fda73.el 35/135 2023-12-11T17:36:16.944 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-volume-2:19.0.0-119.ga62fda73.el8.noarch 36/135 2023-12-11T17:36:16.944 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : cephadm-2:19.0.0-119.ga62fda73.el8.noarch 37/135 2023-12-11T17:36:16.944 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : cephfs-top-2:19.0.0-119.ga62fda73.el8.noarch 38/135 2023-12-11T17:36:16.944 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/135 2023-12-11T17:36:16.944 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/135 2023-12-11T17:36:16.944 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/135 2023-12-11T17:36:16.944 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : luarocks-3.5.0-1.el8.noarch 42/135 2023-12-11T17:36:16.944 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 43/135 2023-12-11T17:36:16.944 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 44/135 2023-12-11T17:36:16.944 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 45/135 2023-12-11T17:36:16.944 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 46/135 2023-12-11T17:36:16.945 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 47/135 2023-12-11T17:36:16.945 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 48/135 2023-12-11T17:36:16.945 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 49/135 2023-12-11T17:36:16.945 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 50/135 2023-12-11T17:36:16.945 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 51/135 2023-12-11T17:36:16.945 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 52/135 2023-12-11T17:36:16.945 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 53/135 2023-12-11T17:36:16.945 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 54/135 2023-12-11T17:36:16.945 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 55/135 2023-12-11T17:36:16.946 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 56/135 2023-12-11T17:36:16.946 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 57/135 2023-12-11T17:36:16.946 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-portend-2.6-1.el8.noarch 58/135 2023-12-11T17:36:16.946 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 59/135 2023-12-11T17:36:16.946 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 60/135 2023-12-11T17:36:16.946 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 61/135 2023-12-11T17:36:16.946 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 62/135 2023-12-11T17:36:16.946 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 63/135 2023-12-11T17:36:16.946 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 64/135 2023-12-11T17:36:16.946 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 65/135 2023-12-11T17:36:16.946 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 66/135 2023-12-11T17:36:16.946 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 67/135 2023-12-11T17:36:16.946 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 68/135 2023-12-11T17:36:16.947 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 69/135 2023-12-11T17:36:16.947 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 70/135 2023-12-11T17:36:16.947 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 71/135 2023-12-11T17:36:16.947 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 72/135 2023-12-11T17:36:16.947 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 73/135 2023-12-11T17:36:16.947 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 74/135 2023-12-11T17:36:16.947 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 75/135 2023-12-11T17:36:16.947 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 76/135 2023-12-11T17:36:16.947 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 77/135 2023-12-11T17:36:16.947 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 78/135 2023-12-11T17:36:16.947 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 79/135 2023-12-11T17:36:16.948 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 80/135 2023-12-11T17:36:16.948 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 81/135 2023-12-11T17:36:16.948 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 82/135 2023-12-11T17:36:16.948 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 83/135 2023-12-11T17:36:16.948 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 84/135 2023-12-11T17:36:16.948 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 85/135 2023-12-11T17:36:16.948 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 86/135 2023-12-11T17:36:16.948 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 87/135 2023-12-11T17:36:16.948 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 88/135 2023-12-11T17:36:16.948 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 89/135 2023-12-11T17:36:16.948 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 90/135 2023-12-11T17:36:16.948 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 91/135 2023-12-11T17:36:16.948 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 92/135 2023-12-11T17:36:16.948 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 93/135 2023-12-11T17:36:16.949 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 94/135 2023-12-11T17:36:16.949 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 95/135 2023-12-11T17:36:16.949 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 96/135 2023-12-11T17:36:16.949 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 97/135 2023-12-11T17:36:16.949 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 98/135 2023-12-11T17:36:16.949 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 99/135 2023-12-11T17:36:16.949 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 100/135 2023-12-11T17:36:16.949 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 101/135 2023-12-11T17:36:16.949 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 102/135 2023-12-11T17:36:16.949 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 103/135 2023-12-11T17:36:16.949 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 104/135 2023-12-11T17:36:16.949 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : bison-3.0.4-10.el8.x86_64 105/135 2023-12-11T17:36:16.949 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 106/135 2023-12-11T17:36:16.949 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 107/135 2023-12-11T17:36:16.949 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 108/135 2023-12-11T17:36:16.950 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : automake-1.16.1-7.el8.noarch 109/135 2023-12-11T17:36:16.950 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 110/135 2023-12-11T17:36:16.950 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 111/135 2023-12-11T17:36:16.950 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : flex-2.6.1-9.el8.x86_64 112/135 2023-12-11T17:36:16.950 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 113/135 2023-12-11T17:36:16.950 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 114/135 2023-12-11T17:36:16.950 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 115/135 2023-12-11T17:36:16.950 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 116/135 2023-12-11T17:36:16.950 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 117/135 2023-12-11T17:36:16.950 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 118/135 2023-12-11T17:36:16.950 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 119/135 2023-12-11T17:36:16.950 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 120/135 2023-12-11T17:36:16.951 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 121/135 2023-12-11T17:36:16.951 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 122/135 2023-12-11T17:36:16.951 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libpq-13.5-1.el8.x86_64 123/135 2023-12-11T17:36:16.951 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 124/135 2023-12-11T17:36:16.951 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : jq-1.6-3.el8.x86_64 125/135 2023-12-11T17:36:16.951 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 126/135 2023-12-11T17:36:16.951 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 127/135 2023-12-11T17:36:16.951 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 128/135 2023-12-11T17:36:16.951 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 129/135 2023-12-11T17:36:16.951 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : librados2-2:19.0.0-119.ga62fda73.el8.x86_64 130/135 2023-12-11T17:36:16.951 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 131/135 2023-12-11T17:36:16.951 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : librbd1-2:19.0.0-119.ga62fda73.el8.x86_64 132/135 2023-12-11T17:36:16.951 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 133/135 2023-12-11T17:36:16.952 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 134/135 2023-12-11T17:36:25.239 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 135/135 2023-12-11T17:36:25.240 INFO:teuthology.orchestra.run.smithi115.stdout:Installed products updated. 2023-12-11T17:36:25.454 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-11T17:36:25.454 INFO:teuthology.orchestra.run.smithi115.stdout:Upgraded: 2023-12-11T17:36:25.454 INFO:teuthology.orchestra.run.smithi115.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-12-11T17:36:25.454 INFO:teuthology.orchestra.run.smithi115.stdout: librados2-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T17:36:25.454 INFO:teuthology.orchestra.run.smithi115.stdout: librbd1-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T17:36:25.454 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2023-12-11T17:36:25.454 INFO:teuthology.orchestra.run.smithi115.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-12-11T17:36:25.454 INFO:teuthology.orchestra.run.smithi115.stdout: automake-1.16.1-7.el8.noarch 2023-12-11T17:36:25.454 INFO:teuthology.orchestra.run.smithi115.stdout: bison-3.0.4-10.el8.x86_64 2023-12-11T17:36:25.454 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T17:36:25.454 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-base-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T17:36:25.455 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-common-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T17:36:25.455 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-fuse-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T17:36:25.455 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-grafana-dashboards-2:19.0.0-119.ga62fda73.el8.noarch 2023-12-11T17:36:25.455 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-immutable-object-cache-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T17:36:25.455 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mds-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T17:36:25.455 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T17:36:25.455 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-cephadm-2:19.0.0-119.ga62fda73.el8.noarch 2023-12-11T17:36:25.455 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-dashboard-2:19.0.0-119.ga62fda73.el8.noarch 2023-12-11T17:36:25.455 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-diskprediction-local-2:19.0.0-119.ga62fda73.el8.noarch 2023-12-11T17:36:25.455 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-modules-core-2:19.0.0-119.ga62fda73.el8.noarch 2023-12-11T17:36:25.455 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-rook-2:19.0.0-119.ga62fda73.el8.noarch 2023-12-11T17:36:25.455 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mon-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T17:36:25.455 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-osd-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T17:36:25.455 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-prometheus-alerts-2:19.0.0-119.ga62fda73.el8.noarch 2023-12-11T17:36:25.456 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-radosgw-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T17:36:25.456 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-selinux-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T17:36:25.456 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-test-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T17:36:25.456 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-volume-2:19.0.0-119.ga62fda73.el8.noarch 2023-12-11T17:36:25.456 INFO:teuthology.orchestra.run.smithi115.stdout: cephadm-2:19.0.0-119.ga62fda73.el8.noarch 2023-12-11T17:36:25.456 INFO:teuthology.orchestra.run.smithi115.stdout: cephfs-mirror-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T17:36:25.456 INFO:teuthology.orchestra.run.smithi115.stdout: cephfs-top-2:19.0.0-119.ga62fda73.el8.noarch 2023-12-11T17:36:25.456 INFO:teuthology.orchestra.run.smithi115.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-12-11T17:36:25.456 INFO:teuthology.orchestra.run.smithi115.stdout: flex-2.6.1-9.el8.x86_64 2023-12-11T17:36:25.456 INFO:teuthology.orchestra.run.smithi115.stdout: fuse-2.9.7-15.el8.x86_64 2023-12-11T17:36:25.456 INFO:teuthology.orchestra.run.smithi115.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-12-11T17:36:25.456 INFO:teuthology.orchestra.run.smithi115.stdout: jq-1.6-3.el8.x86_64 2023-12-11T17:36:25.456 INFO:teuthology.orchestra.run.smithi115.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-12-11T17:36:25.456 INFO:teuthology.orchestra.run.smithi115.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-12-11T17:36:25.456 INFO:teuthology.orchestra.run.smithi115.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-12-11T17:36:25.457 INFO:teuthology.orchestra.run.smithi115.stdout: libcephfs-devel-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T17:36:25.457 INFO:teuthology.orchestra.run.smithi115.stdout: libcephfs2-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T17:36:25.457 INFO:teuthology.orchestra.run.smithi115.stdout: libcephsqlite-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T17:36:25.457 INFO:teuthology.orchestra.run.smithi115.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-12-11T17:36:25.457 INFO:teuthology.orchestra.run.smithi115.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-12-11T17:36:25.457 INFO:teuthology.orchestra.run.smithi115.stdout: liboath-2.6.2-3.el8.x86_64 2023-12-11T17:36:25.457 INFO:teuthology.orchestra.run.smithi115.stdout: libpq-13.5-1.el8.x86_64 2023-12-11T17:36:25.457 INFO:teuthology.orchestra.run.smithi115.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-12-11T17:36:25.457 INFO:teuthology.orchestra.run.smithi115.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-12-11T17:36:25.457 INFO:teuthology.orchestra.run.smithi115.stdout: librados-devel-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T17:36:25.457 INFO:teuthology.orchestra.run.smithi115.stdout: libradosstriper1-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T17:36:25.457 INFO:teuthology.orchestra.run.smithi115.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-12-11T17:36:25.457 INFO:teuthology.orchestra.run.smithi115.stdout: librgw2-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T17:36:25.457 INFO:teuthology.orchestra.run.smithi115.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-12-11T17:36:25.458 INFO:teuthology.orchestra.run.smithi115.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-12-11T17:36:25.458 INFO:teuthology.orchestra.run.smithi115.stdout: libtool-2.4.6-25.el8.x86_64 2023-12-11T17:36:25.458 INFO:teuthology.orchestra.run.smithi115.stdout: libunwind-1.3.1-3.el8.x86_64 2023-12-11T17:36:25.458 INFO:teuthology.orchestra.run.smithi115.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-12-11T17:36:25.458 INFO:teuthology.orchestra.run.smithi115.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-12-11T17:36:25.458 INFO:teuthology.orchestra.run.smithi115.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-12-11T17:36:25.458 INFO:teuthology.orchestra.run.smithi115.stdout: luarocks-3.5.0-1.el8.noarch 2023-12-11T17:36:25.458 INFO:teuthology.orchestra.run.smithi115.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-12-11T17:36:25.458 INFO:teuthology.orchestra.run.smithi115.stdout: openblas-0.3.15-3.el8.x86_64 2023-12-11T17:36:25.458 INFO:teuthology.orchestra.run.smithi115.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-12-11T17:36:25.458 INFO:teuthology.orchestra.run.smithi115.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-12-11T17:36:25.458 INFO:teuthology.orchestra.run.smithi115.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-12-11T17:36:25.458 INFO:teuthology.orchestra.run.smithi115.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-12-11T17:36:25.458 INFO:teuthology.orchestra.run.smithi115.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-12-11T17:36:25.459 INFO:teuthology.orchestra.run.smithi115.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-12-11T17:36:25.459 INFO:teuthology.orchestra.run.smithi115.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-12-11T17:36:25.459 INFO:teuthology.orchestra.run.smithi115.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-12-11T17:36:25.459 INFO:teuthology.orchestra.run.smithi115.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-12-11T17:36:25.459 INFO:teuthology.orchestra.run.smithi115.stdout: python3-babel-2.5.1-7.el8.noarch 2023-12-11T17:36:25.459 INFO:teuthology.orchestra.run.smithi115.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-12-11T17:36:25.459 INFO:teuthology.orchestra.run.smithi115.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-12-11T17:36:25.459 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-12-11T17:36:25.459 INFO:teuthology.orchestra.run.smithi115.stdout: python3-ceph-argparse-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T17:36:25.459 INFO:teuthology.orchestra.run.smithi115.stdout: python3-ceph-common-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T17:36:25.459 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cephfs-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T17:36:25.459 INFO:teuthology.orchestra.run.smithi115.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-12-11T17:36:25.459 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-12-11T17:36:25.459 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-12-11T17:36:25.459 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-12-11T17:36:25.460 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-12-11T17:36:25.460 INFO:teuthology.orchestra.run.smithi115.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-12-11T17:36:25.460 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-6.2-6.el8.noarch 2023-12-11T17:36:25.460 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-12-11T17:36:25.460 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-12-11T17:36:25.460 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-12-11T17:36:25.460 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-12-11T17:36:25.460 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-12-11T17:36:25.460 INFO:teuthology.orchestra.run.smithi115.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-12-11T17:36:25.460 INFO:teuthology.orchestra.run.smithi115.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-12-11T17:36:25.460 INFO:teuthology.orchestra.run.smithi115.stdout: python3-mako-1.0.6-13.el8.noarch 2023-12-11T17:36:25.460 INFO:teuthology.orchestra.run.smithi115.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-12-11T17:36:25.460 INFO:teuthology.orchestra.run.smithi115.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-12-11T17:36:25.460 INFO:teuthology.orchestra.run.smithi115.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-12-11T17:36:25.461 INFO:teuthology.orchestra.run.smithi115.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-12-11T17:36:25.461 INFO:teuthology.orchestra.run.smithi115.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-12-11T17:36:25.461 INFO:teuthology.orchestra.run.smithi115.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-12-11T17:36:25.461 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-12-11T17:36:25.461 INFO:teuthology.orchestra.run.smithi115.stdout: python3-portend-2.6-1.el8.noarch 2023-12-11T17:36:25.461 INFO:teuthology.orchestra.run.smithi115.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-12-11T17:36:25.461 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-12-11T17:36:25.461 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-12-11T17:36:25.461 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-12-11T17:36:25.461 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pycparser-2.14-14.el8.noarch 2023-12-11T17:36:25.461 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pytz-2017.2-9.el8.noarch 2023-12-11T17:36:25.461 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rados-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T17:36:25.461 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rbd-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T17:36:25.461 INFO:teuthology.orchestra.run.smithi115.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-12-11T17:36:25.461 INFO:teuthology.orchestra.run.smithi115.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-12-11T17:36:25.462 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rgw-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T17:36:25.462 INFO:teuthology.orchestra.run.smithi115.stdout: python3-routes-2.4.1-12.el8.noarch 2023-12-11T17:36:25.462 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rsa-4.9-2.el8.noarch 2023-12-11T17:36:25.462 INFO:teuthology.orchestra.run.smithi115.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-12-11T17:36:25.462 INFO:teuthology.orchestra.run.smithi115.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-12-11T17:36:25.462 INFO:teuthology.orchestra.run.smithi115.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-12-11T17:36:25.462 INFO:teuthology.orchestra.run.smithi115.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-12-11T17:36:25.462 INFO:teuthology.orchestra.run.smithi115.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-12-11T17:36:25.465 INFO:teuthology.orchestra.run.smithi115.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-12-11T17:36:25.465 INFO:teuthology.orchestra.run.smithi115.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-12-11T17:36:25.465 INFO:teuthology.orchestra.run.smithi115.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-12-11T17:36:25.465 INFO:teuthology.orchestra.run.smithi115.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-12-11T17:36:25.465 INFO:teuthology.orchestra.run.smithi115.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-12-11T17:36:25.465 INFO:teuthology.orchestra.run.smithi115.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-12-11T17:36:25.465 INFO:teuthology.orchestra.run.smithi115.stdout: rbd-fuse-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T17:36:25.465 INFO:teuthology.orchestra.run.smithi115.stdout: rbd-mirror-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T17:36:25.465 INFO:teuthology.orchestra.run.smithi115.stdout: rbd-nbd-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T17:36:25.465 INFO:teuthology.orchestra.run.smithi115.stdout: socat-1.7.4.1-1.el8.x86_64 2023-12-11T17:36:25.465 INFO:teuthology.orchestra.run.smithi115.stdout: thrift-0.13.0-2.el8.x86_64 2023-12-11T17:36:25.466 INFO:teuthology.orchestra.run.smithi115.stdout: uuid-1.6.2-43.el8.x86_64 2023-12-11T17:36:25.466 INFO:teuthology.orchestra.run.smithi115.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-12-11T17:36:25.466 INFO:teuthology.orchestra.run.smithi115.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-12-11T17:36:25.467 INFO:teuthology.orchestra.run.smithi115.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-12-11T17:36:25.467 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-11T17:36:25.467 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2023-12-11T17:36:26.139 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 135/135 2023-12-11T17:36:26.139 INFO:teuthology.orchestra.run.smithi059.stdout:Installed products updated. 2023-12-11T17:36:26.371 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-12-11T17:36:26.371 INFO:teuthology.orchestra.run.smithi059.stdout:Upgraded: 2023-12-11T17:36:26.371 INFO:teuthology.orchestra.run.smithi059.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-12-11T17:36:26.371 INFO:teuthology.orchestra.run.smithi059.stdout: librados2-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T17:36:26.371 INFO:teuthology.orchestra.run.smithi059.stdout: librbd1-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T17:36:26.371 INFO:teuthology.orchestra.run.smithi059.stdout:Installed: 2023-12-11T17:36:26.371 INFO:teuthology.orchestra.run.smithi059.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-12-11T17:36:26.371 INFO:teuthology.orchestra.run.smithi059.stdout: automake-1.16.1-7.el8.noarch 2023-12-11T17:36:26.371 INFO:teuthology.orchestra.run.smithi059.stdout: bison-3.0.4-10.el8.x86_64 2023-12-11T17:36:26.372 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T17:36:26.372 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-base-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T17:36:26.372 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-common-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T17:36:26.372 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-fuse-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T17:36:26.372 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-grafana-dashboards-2:19.0.0-119.ga62fda73.el8.noarch 2023-12-11T17:36:26.372 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-immutable-object-cache-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T17:36:26.372 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mds-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T17:36:26.372 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T17:36:26.372 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-cephadm-2:19.0.0-119.ga62fda73.el8.noarch 2023-12-11T17:36:26.372 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-dashboard-2:19.0.0-119.ga62fda73.el8.noarch 2023-12-11T17:36:26.372 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-diskprediction-local-2:19.0.0-119.ga62fda73.el8.noarch 2023-12-11T17:36:26.372 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-modules-core-2:19.0.0-119.ga62fda73.el8.noarch 2023-12-11T17:36:26.372 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-rook-2:19.0.0-119.ga62fda73.el8.noarch 2023-12-11T17:36:26.372 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mon-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T17:36:26.373 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-osd-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T17:36:26.373 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-prometheus-alerts-2:19.0.0-119.ga62fda73.el8.noarch 2023-12-11T17:36:26.373 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-radosgw-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T17:36:26.373 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-selinux-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T17:36:26.373 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-test-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T17:36:26.373 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-volume-2:19.0.0-119.ga62fda73.el8.noarch 2023-12-11T17:36:26.373 INFO:teuthology.orchestra.run.smithi059.stdout: cephadm-2:19.0.0-119.ga62fda73.el8.noarch 2023-12-11T17:36:26.373 INFO:teuthology.orchestra.run.smithi059.stdout: cephfs-mirror-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T17:36:26.373 INFO:teuthology.orchestra.run.smithi059.stdout: cephfs-top-2:19.0.0-119.ga62fda73.el8.noarch 2023-12-11T17:36:26.373 INFO:teuthology.orchestra.run.smithi059.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-12-11T17:36:26.373 INFO:teuthology.orchestra.run.smithi059.stdout: flex-2.6.1-9.el8.x86_64 2023-12-11T17:36:26.373 INFO:teuthology.orchestra.run.smithi059.stdout: fuse-2.9.7-15.el8.x86_64 2023-12-11T17:36:26.373 INFO:teuthology.orchestra.run.smithi059.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-12-11T17:36:26.373 INFO:teuthology.orchestra.run.smithi059.stdout: jq-1.6-3.el8.x86_64 2023-12-11T17:36:26.374 INFO:teuthology.orchestra.run.smithi059.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-12-11T17:36:26.374 INFO:teuthology.orchestra.run.smithi059.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-12-11T17:36:26.374 INFO:teuthology.orchestra.run.smithi059.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-12-11T17:36:26.374 INFO:teuthology.orchestra.run.smithi059.stdout: libcephfs-devel-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T17:36:26.374 INFO:teuthology.orchestra.run.smithi059.stdout: libcephfs2-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T17:36:26.374 INFO:teuthology.orchestra.run.smithi059.stdout: libcephsqlite-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T17:36:26.374 INFO:teuthology.orchestra.run.smithi059.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-12-11T17:36:26.374 INFO:teuthology.orchestra.run.smithi059.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-12-11T17:36:26.374 INFO:teuthology.orchestra.run.smithi059.stdout: liboath-2.6.2-3.el8.x86_64 2023-12-11T17:36:26.374 INFO:teuthology.orchestra.run.smithi059.stdout: libpq-13.5-1.el8.x86_64 2023-12-11T17:36:26.374 INFO:teuthology.orchestra.run.smithi059.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-12-11T17:36:26.374 INFO:teuthology.orchestra.run.smithi059.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-12-11T17:36:26.374 INFO:teuthology.orchestra.run.smithi059.stdout: librados-devel-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T17:36:26.375 INFO:teuthology.orchestra.run.smithi059.stdout: libradosstriper1-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T17:36:26.375 INFO:teuthology.orchestra.run.smithi059.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-12-11T17:36:26.375 INFO:teuthology.orchestra.run.smithi059.stdout: librgw2-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T17:36:26.375 INFO:teuthology.orchestra.run.smithi059.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-12-11T17:36:26.375 INFO:teuthology.orchestra.run.smithi059.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-12-11T17:36:26.375 INFO:teuthology.orchestra.run.smithi059.stdout: libtool-2.4.6-25.el8.x86_64 2023-12-11T17:36:26.375 INFO:teuthology.orchestra.run.smithi059.stdout: libunwind-1.3.1-3.el8.x86_64 2023-12-11T17:36:26.375 INFO:teuthology.orchestra.run.smithi059.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-12-11T17:36:26.375 INFO:teuthology.orchestra.run.smithi059.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-12-11T17:36:26.375 INFO:teuthology.orchestra.run.smithi059.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-12-11T17:36:26.375 INFO:teuthology.orchestra.run.smithi059.stdout: luarocks-3.5.0-1.el8.noarch 2023-12-11T17:36:26.376 INFO:teuthology.orchestra.run.smithi059.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-12-11T17:36:26.376 INFO:teuthology.orchestra.run.smithi059.stdout: openblas-0.3.15-3.el8.x86_64 2023-12-11T17:36:26.376 INFO:teuthology.orchestra.run.smithi059.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-12-11T17:36:26.376 INFO:teuthology.orchestra.run.smithi059.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-12-11T17:36:26.376 INFO:teuthology.orchestra.run.smithi059.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-12-11T17:36:26.376 INFO:teuthology.orchestra.run.smithi059.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-12-11T17:36:26.376 INFO:teuthology.orchestra.run.smithi059.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-12-11T17:36:26.376 INFO:teuthology.orchestra.run.smithi059.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-12-11T17:36:26.376 INFO:teuthology.orchestra.run.smithi059.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-12-11T17:36:26.376 INFO:teuthology.orchestra.run.smithi059.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-12-11T17:36:26.376 INFO:teuthology.orchestra.run.smithi059.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-12-11T17:36:26.376 INFO:teuthology.orchestra.run.smithi059.stdout: python3-babel-2.5.1-7.el8.noarch 2023-12-11T17:36:26.376 INFO:teuthology.orchestra.run.smithi059.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-12-11T17:36:26.377 INFO:teuthology.orchestra.run.smithi059.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-12-11T17:36:26.377 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-12-11T17:36:26.377 INFO:teuthology.orchestra.run.smithi059.stdout: python3-ceph-argparse-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T17:36:26.377 INFO:teuthology.orchestra.run.smithi059.stdout: python3-ceph-common-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T17:36:26.377 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cephfs-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T17:36:26.377 INFO:teuthology.orchestra.run.smithi059.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-12-11T17:36:26.377 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-12-11T17:36:26.377 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-12-11T17:36:26.377 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-12-11T17:36:26.377 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-12-11T17:36:26.377 INFO:teuthology.orchestra.run.smithi059.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-12-11T17:36:26.377 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jaraco-6.2-6.el8.noarch 2023-12-11T17:36:26.377 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-12-11T17:36:26.377 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-12-11T17:36:26.377 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-12-11T17:36:26.378 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-12-11T17:36:26.378 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-12-11T17:36:26.378 INFO:teuthology.orchestra.run.smithi059.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-12-11T17:36:26.378 INFO:teuthology.orchestra.run.smithi059.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-12-11T17:36:26.378 INFO:teuthology.orchestra.run.smithi059.stdout: python3-mako-1.0.6-13.el8.noarch 2023-12-11T17:36:26.378 INFO:teuthology.orchestra.run.smithi059.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-12-11T17:36:26.378 INFO:teuthology.orchestra.run.smithi059.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-12-11T17:36:26.378 INFO:teuthology.orchestra.run.smithi059.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-12-11T17:36:26.378 INFO:teuthology.orchestra.run.smithi059.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-12-11T17:36:26.378 INFO:teuthology.orchestra.run.smithi059.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-12-11T17:36:26.378 INFO:teuthology.orchestra.run.smithi059.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-12-11T17:36:26.378 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-12-11T17:36:26.378 INFO:teuthology.orchestra.run.smithi059.stdout: python3-portend-2.6-1.el8.noarch 2023-12-11T17:36:26.379 INFO:teuthology.orchestra.run.smithi059.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-12-11T17:36:26.379 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-12-11T17:36:26.379 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-12-11T17:36:26.379 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-12-11T17:36:26.379 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pycparser-2.14-14.el8.noarch 2023-12-11T17:36:26.379 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pytz-2017.2-9.el8.noarch 2023-12-11T17:36:26.379 INFO:teuthology.orchestra.run.smithi059.stdout: python3-rados-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T17:36:26.379 INFO:teuthology.orchestra.run.smithi059.stdout: python3-rbd-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T17:36:26.379 INFO:teuthology.orchestra.run.smithi059.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-12-11T17:36:26.379 INFO:teuthology.orchestra.run.smithi059.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-12-11T17:36:26.379 INFO:teuthology.orchestra.run.smithi059.stdout: python3-rgw-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T17:36:26.380 INFO:teuthology.orchestra.run.smithi059.stdout: python3-routes-2.4.1-12.el8.noarch 2023-12-11T17:36:26.380 INFO:teuthology.orchestra.run.smithi059.stdout: python3-rsa-4.9-2.el8.noarch 2023-12-11T17:36:26.380 INFO:teuthology.orchestra.run.smithi059.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-12-11T17:36:26.380 INFO:teuthology.orchestra.run.smithi059.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-12-11T17:36:26.380 INFO:teuthology.orchestra.run.smithi059.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-12-11T17:36:26.380 INFO:teuthology.orchestra.run.smithi059.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-12-11T17:36:26.380 INFO:teuthology.orchestra.run.smithi059.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-12-11T17:36:26.380 INFO:teuthology.orchestra.run.smithi059.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-12-11T17:36:26.380 INFO:teuthology.orchestra.run.smithi059.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-12-11T17:36:26.380 INFO:teuthology.orchestra.run.smithi059.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-12-11T17:36:26.380 INFO:teuthology.orchestra.run.smithi059.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-12-11T17:36:26.380 INFO:teuthology.orchestra.run.smithi059.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-12-11T17:36:26.380 INFO:teuthology.orchestra.run.smithi059.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-12-11T17:36:26.381 INFO:teuthology.orchestra.run.smithi059.stdout: rbd-fuse-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T17:36:26.381 INFO:teuthology.orchestra.run.smithi059.stdout: rbd-mirror-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T17:36:26.381 INFO:teuthology.orchestra.run.smithi059.stdout: rbd-nbd-2:19.0.0-119.ga62fda73.el8.x86_64 2023-12-11T17:36:26.381 INFO:teuthology.orchestra.run.smithi059.stdout: socat-1.7.4.1-1.el8.x86_64 2023-12-11T17:36:26.381 INFO:teuthology.orchestra.run.smithi059.stdout: thrift-0.13.0-2.el8.x86_64 2023-12-11T17:36:26.381 INFO:teuthology.orchestra.run.smithi059.stdout: uuid-1.6.2-43.el8.x86_64 2023-12-11T17:36:26.381 INFO:teuthology.orchestra.run.smithi059.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-12-11T17:36:26.381 INFO:teuthology.orchestra.run.smithi059.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-12-11T17:36:26.382 INFO:teuthology.orchestra.run.smithi059.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-12-11T17:36:26.382 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-12-11T17:36:26.382 INFO:teuthology.orchestra.run.smithi059.stdout:Complete! 2023-12-11T17:36:26.562 DEBUG:teuthology.parallel:result is None 2023-12-11T17:36:26.716 DEBUG:teuthology.parallel:result is None 2023-12-11T17:36:26.717 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=a62fda73efba5520d409bb5a21bd8373289dda5d 2023-12-11T17:36:26.985 DEBUG:teuthology.orchestra.run.smithi059:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-12-11T17:36:27.011 INFO:teuthology.orchestra.run.smithi059.stdout:19.0.0-119.ga62fda73.el8 2023-12-11T17:36:27.011 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-119.ga62fda73.el8 2023-12-11T17:36:27.011 INFO:teuthology.task.install:The correct ceph version 19.0.0-119.ga62fda73 is installed. 2023-12-11T17:36:27.013 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=a62fda73efba5520d409bb5a21bd8373289dda5d 2023-12-11T17:36:27.171 DEBUG:teuthology.orchestra.run.smithi115:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-12-11T17:36:27.197 INFO:teuthology.orchestra.run.smithi115.stdout:19.0.0-119.ga62fda73.el8 2023-12-11T17:36:27.197 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-119.ga62fda73.el8 2023-12-11T17:36:27.197 INFO:teuthology.task.install:The correct ceph version 19.0.0-119.ga62fda73 is installed. 2023-12-11T17:36:27.199 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=a62fda73efba5520d409bb5a21bd8373289dda5d 2023-12-11T17:36:27.343 DEBUG:teuthology.orchestra.run.smithi181:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-12-11T17:36:27.371 INFO:teuthology.orchestra.run.smithi181.stdout:19.0.0-119.ga62fda73.el8 2023-12-11T17:36:27.371 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-119.ga62fda73.el8 2023-12-11T17:36:27.371 INFO:teuthology.task.install:The correct ceph version 19.0.0-119.ga62fda73 is installed. 2023-12-11T17:36:27.373 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-12-11T17:36:27.373 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2023-12-11T17:36:27.373 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-12-11T17:36:27.423 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-12-11T17:36:27.424 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-12-11T17:36:27.469 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-12-11T17:36:27.469 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-12-11T17:36:27.517 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-12-11T17:36:27.518 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2023-12-11T17:36:27.518 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd of=/usr/bin/daemon-helper 2023-12-11T17:36:27.550 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-12-11T17:36:27.623 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-12-11T17:36:27.623 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/usr/bin/daemon-helper 2023-12-11T17:36:27.655 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-12-11T17:36:27.724 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-12-11T17:36:27.724 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/usr/bin/daemon-helper 2023-12-11T17:36:27.756 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-12-11T17:36:27.827 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-12-11T17:36:27.827 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2023-12-11T17:36:27.827 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd of=/usr/bin/adjust-ulimits 2023-12-11T17:36:27.860 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-12-11T17:36:27.932 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-12-11T17:36:27.932 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/usr/bin/adjust-ulimits 2023-12-11T17:36:27.962 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-12-11T17:36:28.032 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-12-11T17:36:28.032 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/usr/bin/adjust-ulimits 2023-12-11T17:36:28.063 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-12-11T17:36:28.133 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-12-11T17:36:28.133 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2023-12-11T17:36:28.133 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd of=/usr/bin/stdin-killer 2023-12-11T17:36:28.166 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-12-11T17:36:28.239 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-12-11T17:36:28.239 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/usr/bin/stdin-killer 2023-12-11T17:36:28.268 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-12-11T17:36:28.337 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-12-11T17:36:28.337 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/usr/bin/stdin-killer 2023-12-11T17:36:28.369 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-12-11T17:36:28.439 INFO:teuthology.run_tasks:Running task cephadm... 2023-12-11T17:36:28.520 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'sha1': 'a62fda73efba5520d409bb5a21bd8373289dda5d', 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2023-12-11T17:36:28.520 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d 2023-12-11T17:36:28.521 INFO:tasks.cephadm:Cluster fsid is d0a07eaa-984b-11ee-95a3-87774f69a715 2023-12-11T17:36:28.521 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2023-12-11T17:36:28.521 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.59', 'mon.b': '172.21.15.115', 'mon.c': '172.21.15.181'} 2023-12-11T17:36:28.521 INFO:tasks.cephadm:First mon is mon.a on smithi059 2023-12-11T17:36:28.521 INFO:tasks.cephadm:First mgr is x 2023-12-11T17:36:28.521 INFO:tasks.cephadm:Normalizing hostnames... 2023-12-11T17:36:28.522 DEBUG:teuthology.orchestra.run.smithi059:> sudo hostname $(hostname -s) 2023-12-11T17:36:28.556 DEBUG:teuthology.orchestra.run.smithi115:> sudo hostname $(hostname -s) 2023-12-11T17:36:28.587 DEBUG:teuthology.orchestra.run.smithi181:> sudo hostname $(hostname -s) 2023-12-11T17:36:28.620 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2023-12-11T17:36:28.621 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=a62fda73efba5520d409bb5a21bd8373289dda5d 2023-12-11T17:36:28.765 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'a62fda73efba5520d409bb5a21bd8373289dda5d', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=arm64,AVAILABLE_ARCH=arm64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic/74860/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-119-ga62fda73', 'node_name': '172.21.4.63+confusa01', 'job_name': 'ceph-dev-new-build/ARCH=arm64,AVAILABLE_ARCH=arm64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-119.ga62fda73'}, 'url': 'https://3.chacra.ceph.com/r/ceph/rishabh-2023dec11/a62fda73efba5520d409bb5a21bd8373289dda5d/centos/8/flavors/default/', 'distro_codename': None, 'modified': '2023-12-11 13:31:01.711792', 'distro_version': '8', 'project': 'ceph', 'flavor': 'default', 'ref': 'rishabh-2023dec11', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/rishabh-2023dec11/a62fda73efba5520d409bb5a21bd8373289dda5d/centos/8/flavors/default/', 'archs': ['x86_64', 'source', 'arm64'], 'distro': 'centos'}, {'status': 'ready', 'sha1': 'a62fda73efba5520d409bb5a21bd8373289dda5d', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic/74860/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-119-ga62fda73', 'node_name': '172.21.2.8+braggi08', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-119.ga62fda73'}, 'url': 'https://4.chacra.ceph.com/r/ceph/rishabh-2023dec11/a62fda73efba5520d409bb5a21bd8373289dda5d/centos/8/flavors/default/', 'distro_codename': None, 'modified': '2023-12-11 12:43:31.974707', 'distro_version': '8', 'project': 'ceph', 'flavor': 'default', 'ref': 'rishabh-2023dec11', 'chacra_url': 'https://4.chacra.ceph.com/repos/ceph/rishabh-2023dec11/a62fda73efba5520d409bb5a21bd8373289dda5d/centos/8/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2023-12-11T17:36:28.979 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref rishabh-2023dec11, sha1 a62fda73efba5520d409bb5a21bd8373289dda5d from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F8%2Fx86_64&flavor=default&sha1=a62fda73efba5520d409bb5a21bd8373289dda5d 2023-12-11T17:36:28.980 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/rishabh-2023dec11/a62fda73efba5520d409bb5a21bd8373289dda5d/centos/8/x86_64/flavors/default/cephadm 2023-12-11T17:36:28.981 DEBUG:teuthology.orchestra.run.smithi059:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/rishabh-2023dec11/a62fda73efba5520d409bb5a21bd8373289dda5d/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-12-11T17:36:29.468 INFO:teuthology.orchestra.run.smithi059.stdout:-rw-rw-r--. 1 ubuntu ubuntu 2272057 Dec 11 17:36 /home/ubuntu/cephtest/cephadm 2023-12-11T17:36:29.469 DEBUG:teuthology.orchestra.run.smithi115:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/rishabh-2023dec11/a62fda73efba5520d409bb5a21bd8373289dda5d/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-12-11T17:36:29.977 INFO:teuthology.orchestra.run.smithi115.stdout:-rw-rw-r--. 1 ubuntu ubuntu 2272057 Dec 11 17:36 /home/ubuntu/cephtest/cephadm 2023-12-11T17:36:29.978 DEBUG:teuthology.orchestra.run.smithi181:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/rishabh-2023dec11/a62fda73efba5520d409bb5a21bd8373289dda5d/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-12-11T17:36:30.539 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-rw-r--. 1 ubuntu ubuntu 2272057 Dec 11 17:36 /home/ubuntu/cephtest/cephadm 2023-12-11T17:36:30.539 DEBUG:teuthology.orchestra.run.smithi059:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-12-11T17:36:30.562 DEBUG:teuthology.orchestra.run.smithi115:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-12-11T17:36:30.584 DEBUG:teuthology.orchestra.run.smithi181:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-12-11T17:36:30.623 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d on all hosts... 2023-12-11T17:36:30.623 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d pull 2023-12-11T17:36:30.626 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d pull 2023-12-11T17:36:30.629 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d pull 2023-12-11T17:36:33.232 INFO:teuthology.orchestra.run.smithi181.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d... 2023-12-11T17:36:35.493 INFO:teuthology.orchestra.run.smithi059.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d... 2023-12-11T17:36:35.802 INFO:teuthology.orchestra.run.smithi115.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d... 2023-12-11T17:36:59.507 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2023-12-11T17:36:59.507 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_version": "ceph version 19.0.0-119-ga62fda73 (a62fda73efba5520d409bb5a21bd8373289dda5d) reef (dev)", 2023-12-11T17:36:59.507 INFO:teuthology.orchestra.run.smithi181.stdout: "image_id": "0b0a5e4a17a719ce4a079eef54b8d4e11d1472aa5fe9b5fa219d86551499aa8a", 2023-12-11T17:36:59.507 INFO:teuthology.orchestra.run.smithi181.stdout: "repo_digests": [ 2023-12-11T17:36:59.507 INFO:teuthology.orchestra.run.smithi181.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:84f4c788adfc97059464515a751d874ddacbeb7bf7cdb46b264d7a0bd4fc9d40" 2023-12-11T17:36:59.507 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2023-12-11T17:36:59.507 INFO:teuthology.orchestra.run.smithi181.stdout:} 2023-12-11T17:37:02.180 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2023-12-11T17:37:02.181 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_version": "ceph version 19.0.0-119-ga62fda73 (a62fda73efba5520d409bb5a21bd8373289dda5d) reef (dev)", 2023-12-11T17:37:02.181 INFO:teuthology.orchestra.run.smithi059.stdout: "image_id": "0b0a5e4a17a719ce4a079eef54b8d4e11d1472aa5fe9b5fa219d86551499aa8a", 2023-12-11T17:37:02.181 INFO:teuthology.orchestra.run.smithi059.stdout: "repo_digests": [ 2023-12-11T17:37:02.181 INFO:teuthology.orchestra.run.smithi059.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:84f4c788adfc97059464515a751d874ddacbeb7bf7cdb46b264d7a0bd4fc9d40" 2023-12-11T17:37:02.181 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2023-12-11T17:37:02.181 INFO:teuthology.orchestra.run.smithi059.stdout:} 2023-12-11T17:37:04.802 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2023-12-11T17:37:04.802 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_version": "ceph version 19.0.0-119-ga62fda73 (a62fda73efba5520d409bb5a21bd8373289dda5d) reef (dev)", 2023-12-11T17:37:04.802 INFO:teuthology.orchestra.run.smithi115.stdout: "image_id": "0b0a5e4a17a719ce4a079eef54b8d4e11d1472aa5fe9b5fa219d86551499aa8a", 2023-12-11T17:37:04.802 INFO:teuthology.orchestra.run.smithi115.stdout: "repo_digests": [ 2023-12-11T17:37:04.802 INFO:teuthology.orchestra.run.smithi115.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:84f4c788adfc97059464515a751d874ddacbeb7bf7cdb46b264d7a0bd4fc9d40" 2023-12-11T17:37:04.802 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2023-12-11T17:37:04.802 INFO:teuthology.orchestra.run.smithi115.stdout:} 2023-12-11T17:37:04.828 DEBUG:teuthology.orchestra.run.smithi059:> sudo mkdir -p /etc/ceph 2023-12-11T17:37:04.866 DEBUG:teuthology.orchestra.run.smithi115:> sudo mkdir -p /etc/ceph 2023-12-11T17:37:04.896 DEBUG:teuthology.orchestra.run.smithi181:> sudo mkdir -p /etc/ceph 2023-12-11T17:37:04.932 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod 777 /etc/ceph 2023-12-11T17:37:04.964 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod 777 /etc/ceph 2023-12-11T17:37:04.990 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod 777 /etc/ceph 2023-12-11T17:37:05.021 INFO:tasks.cephadm:Writing seed config... 2023-12-11T17:37:05.022 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2023-12-11T17:37:05.022 INFO:tasks.cephadm: override: [client] debug client = 20 2023-12-11T17:37:05.022 INFO:tasks.cephadm: override: [client] debug ms = 1 2023-12-11T17:37:05.022 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2023-12-11T17:37:05.023 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2023-12-11T17:37:05.023 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2023-12-11T17:37:05.023 INFO:tasks.cephadm: override: [mds] debug mds = 20 2023-12-11T17:37:05.023 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2023-12-11T17:37:05.023 INFO:tasks.cephadm: override: [mds] debug ms = 1 2023-12-11T17:37:05.023 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2023-12-11T17:37:05.023 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2023-12-11T17:37:05.023 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2023-12-11T17:37:05.023 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2023-12-11T17:37:05.023 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2023-12-11T17:37:05.023 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2023-12-11T17:37:05.023 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2023-12-11T17:37:05.023 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2023-12-11T17:37:05.023 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2023-12-11T17:37:05.024 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2023-12-11T17:37:05.024 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2023-12-11T17:37:05.024 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2023-12-11T17:37:05.024 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2023-12-11T17:37:05.024 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2023-12-11T17:37:05.024 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2023-12-11T17:37:05.024 INFO:tasks.cephadm: override: [mon] debug mon = 20 2023-12-11T17:37:05.024 INFO:tasks.cephadm: override: [mon] debug ms = 1 2023-12-11T17:37:05.024 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2023-12-11T17:37:05.024 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2023-12-11T17:37:05.024 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2023-12-11T17:37:05.024 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2023-12-11T17:37:05.024 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2023-12-11T17:37:05.024 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2023-12-11T17:37:05.024 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2023-12-11T17:37:05.025 INFO:tasks.cephadm: override: [osd] debug ms = 1 2023-12-11T17:37:05.025 INFO:tasks.cephadm: override: [osd] debug osd = 20 2023-12-11T17:37:05.025 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2023-12-11T17:37:05.025 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2023-12-11T17:37:05.025 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2023-12-11T17:37:05.025 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2023-12-11T17:37:05.025 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2023-12-11T17:37:05.025 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2023-12-11T17:37:05.025 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2023-12-11T17:37:05.025 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2023-12-11T17:37:05.025 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2023-12-11T17:37:05.026 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2023-12-11T17:37:05.026 DEBUG:teuthology.orchestra.run.smithi059:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2023-12-11T17:37:05.046 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = d0a07eaa-984b-11ee-95a3-87774f69a715 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2023-12-11T17:37:05.046 DEBUG:teuthology.orchestra.run.smithi059:mon.a> sudo journalctl -f -n 0 -u ceph-d0a07eaa-984b-11ee-95a3-87774f69a715@mon.a.service 2023-12-11T17:37:05.089 DEBUG:teuthology.orchestra.run.smithi059:mgr.x> sudo journalctl -f -n 0 -u ceph-d0a07eaa-984b-11ee-95a3-87774f69a715@mgr.x.service 2023-12-11T17:37:05.119 INFO:journalctl@ceph.mon.a.smithi059.stdout:-- Logs begin at Mon 2023-12-11 17:22:26 UTC. -- 2023-12-11T17:37:05.119 INFO:tasks.cephadm:Bootstrapping... 2023-12-11T17:37:05.120 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d -v bootstrap --fsid d0a07eaa-984b-11ee-95a3-87774f69a715 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.59 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2023-12-11T17:37:05.148 INFO:journalctl@ceph.mgr.x.smithi059.stdout:-- Logs begin at Mon 2023-12-11 17:22:26 UTC. -- 2023-12-11T17:37:05.334 INFO:teuthology.orchestra.run.smithi059.stdout:-------------------------------------------------------------------------------- 2023-12-11T17:37:05.334 INFO:teuthology.orchestra.run.smithi059.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d', '-v', 'bootstrap', '--fsid', 'd0a07eaa-984b-11ee-95a3-87774f69a715', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.59', '--skip-admin-label'] 2023-12-11T17:37:05.388 INFO:teuthology.orchestra.run.smithi059.stdout:/bin/podman: stdout 4.1.1 2023-12-11T17:37:05.389 INFO:teuthology.orchestra.run.smithi059.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2023-12-11T17:37:05.389 INFO:teuthology.orchestra.run.smithi059.stdout:Verifying podman|docker is present... 2023-12-11T17:37:05.535 INFO:teuthology.orchestra.run.smithi059.stdout:/bin/podman: stdout 4.1.1 2023-12-11T17:37:05.535 INFO:teuthology.orchestra.run.smithi059.stdout:Verifying lvm2 is present... 2023-12-11T17:37:05.535 INFO:teuthology.orchestra.run.smithi059.stdout:Verifying time synchronization is in place... 2023-12-11T17:37:05.543 INFO:teuthology.orchestra.run.smithi059.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2023-12-11T17:37:05.543 INFO:teuthology.orchestra.run.smithi059.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2023-12-11T17:37:05.549 INFO:teuthology.orchestra.run.smithi059.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2023-12-11T17:37:05.549 INFO:teuthology.orchestra.run.smithi059.stdout:systemctl: stdout inactive 2023-12-11T17:37:05.555 INFO:teuthology.orchestra.run.smithi059.stdout:systemctl: stdout enabled 2023-12-11T17:37:05.561 INFO:teuthology.orchestra.run.smithi059.stdout:systemctl: stdout active 2023-12-11T17:37:05.561 INFO:teuthology.orchestra.run.smithi059.stdout:Unit chronyd.service is enabled and running 2023-12-11T17:37:05.561 INFO:teuthology.orchestra.run.smithi059.stdout:Repeating the final host check... 2023-12-11T17:37:05.706 INFO:teuthology.orchestra.run.smithi059.stdout:/bin/podman: stdout 4.1.1 2023-12-11T17:37:05.707 INFO:teuthology.orchestra.run.smithi059.stdout:podman (/bin/podman) version 4.1.1 is present 2023-12-11T17:37:05.707 INFO:teuthology.orchestra.run.smithi059.stdout:systemctl is present 2023-12-11T17:37:05.707 INFO:teuthology.orchestra.run.smithi059.stdout:lvcreate is present 2023-12-11T17:37:05.713 INFO:teuthology.orchestra.run.smithi059.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2023-12-11T17:37:05.713 INFO:teuthology.orchestra.run.smithi059.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2023-12-11T17:37:05.719 INFO:teuthology.orchestra.run.smithi059.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2023-12-11T17:37:05.719 INFO:teuthology.orchestra.run.smithi059.stdout:systemctl: stdout inactive 2023-12-11T17:37:05.725 INFO:teuthology.orchestra.run.smithi059.stdout:systemctl: stdout enabled 2023-12-11T17:37:05.731 INFO:teuthology.orchestra.run.smithi059.stdout:systemctl: stdout active 2023-12-11T17:37:05.731 INFO:teuthology.orchestra.run.smithi059.stdout:Unit chronyd.service is enabled and running 2023-12-11T17:37:05.731 INFO:teuthology.orchestra.run.smithi059.stdout:Host looks OK 2023-12-11T17:37:05.731 INFO:teuthology.orchestra.run.smithi059.stdout:Cluster fsid: d0a07eaa-984b-11ee-95a3-87774f69a715 2023-12-11T17:37:05.731 INFO:teuthology.orchestra.run.smithi059.stdout:Acquiring lock 140036233672072 on /run/cephadm/d0a07eaa-984b-11ee-95a3-87774f69a715.lock 2023-12-11T17:37:05.731 INFO:teuthology.orchestra.run.smithi059.stdout:Lock 140036233672072 acquired on /run/cephadm/d0a07eaa-984b-11ee-95a3-87774f69a715.lock 2023-12-11T17:37:05.732 INFO:teuthology.orchestra.run.smithi059.stdout:Verifying IP 172.21.15.59 port 3300 ... 2023-12-11T17:37:05.732 INFO:teuthology.orchestra.run.smithi059.stdout:Verifying IP 172.21.15.59 port 6789 ... 2023-12-11T17:37:05.732 INFO:teuthology.orchestra.run.smithi059.stdout:Base mon IP(s) is [172.21.15.59:3300, 172.21.15.59:6789], mon addrv is [v2:172.21.15.59:3300,v1:172.21.15.59:6789] 2023-12-11T17:37:05.736 INFO:teuthology.orchestra.run.smithi059.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp metric 100 2023-12-11T17:37:05.736 INFO:teuthology.orchestra.run.smithi059.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.59 metric 100 2023-12-11T17:37:05.738 INFO:teuthology.orchestra.run.smithi059.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2023-12-11T17:37:05.739 INFO:teuthology.orchestra.run.smithi059.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 256 pref medium 2023-12-11T17:37:05.739 INFO:teuthology.orchestra.run.smithi059.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 1024 expires 1790sec hoplimit 64 pref medium 2023-12-11T17:37:05.741 INFO:teuthology.orchestra.run.smithi059.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2023-12-11T17:37:05.741 INFO:teuthology.orchestra.run.smithi059.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2023-12-11T17:37:05.742 INFO:teuthology.orchestra.run.smithi059.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2023-12-11T17:37:05.742 INFO:teuthology.orchestra.run.smithi059.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2023-12-11T17:37:05.742 INFO:teuthology.orchestra.run.smithi059.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:192a/64 scope link 2023-12-11T17:37:05.742 INFO:teuthology.orchestra.run.smithi059.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2023-12-11T17:37:05.742 INFO:teuthology.orchestra.run.smithi059.stdout:Mon IP `172.21.15.59` is in CIDR network `172.21.0.0/20` 2023-12-11T17:37:05.742 INFO:teuthology.orchestra.run.smithi059.stdout:Mon IP `172.21.15.59` is in CIDR network `172.21.0.0/20` 2023-12-11T17:37:05.742 INFO:teuthology.orchestra.run.smithi059.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2023-12-11T17:37:05.743 INFO:teuthology.orchestra.run.smithi059.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2023-12-11T17:37:05.744 INFO:teuthology.orchestra.run.smithi059.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d... 2023-12-11T17:37:06.187 INFO:teuthology.orchestra.run.smithi059.stdout:/bin/podman: stdout 0b0a5e4a17a719ce4a079eef54b8d4e11d1472aa5fe9b5fa219d86551499aa8a 2023-12-11T17:37:06.187 INFO:teuthology.orchestra.run.smithi059.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d... 2023-12-11T17:37:06.187 INFO:teuthology.orchestra.run.smithi059.stdout:/bin/podman: stderr Getting image source signatures 2023-12-11T17:37:06.187 INFO:teuthology.orchestra.run.smithi059.stdout:/bin/podman: stderr Copying blob sha256:933954f0af8f706d6a6186e87e0ea462c846b5abc6ef29ee9ef20dac46eff0f7 2023-12-11T17:37:06.187 INFO:teuthology.orchestra.run.smithi059.stdout:/bin/podman: stderr Copying blob sha256:d3e436ac93e4b5a6d94c7ac8024df331e66e67275836276615dc044fd3cbf83a 2023-12-11T17:37:06.187 INFO:teuthology.orchestra.run.smithi059.stdout:/bin/podman: stderr Copying config sha256:0b0a5e4a17a719ce4a079eef54b8d4e11d1472aa5fe9b5fa219d86551499aa8a 2023-12-11T17:37:06.187 INFO:teuthology.orchestra.run.smithi059.stdout:/bin/podman: stderr Writing manifest to image destination 2023-12-11T17:37:06.187 INFO:teuthology.orchestra.run.smithi059.stdout:/bin/podman: stderr Storing signatures 2023-12-11T17:37:07.038 INFO:teuthology.orchestra.run.smithi059.stdout:ceph: stdout ceph version 19.0.0-119-ga62fda73 (a62fda73efba5520d409bb5a21bd8373289dda5d) reef (dev) 2023-12-11T17:37:07.038 INFO:teuthology.orchestra.run.smithi059.stdout:Ceph version: ceph version 19.0.0-119-ga62fda73 (a62fda73efba5520d409bb5a21bd8373289dda5d) reef (dev) 2023-12-11T17:37:07.038 INFO:teuthology.orchestra.run.smithi059.stdout:Extracting ceph user uid/gid from container image... 2023-12-11T17:37:07.689 INFO:teuthology.orchestra.run.smithi059.stdout:stat: stdout 167 167 2023-12-11T17:37:07.689 INFO:teuthology.orchestra.run.smithi059.stdout:Creating initial keys... 2023-12-11T17:37:08.444 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph-authtool: stdout AQDESHdl6k7bBBAA5iLrPkiGWyr5nAc8JoXSwA== 2023-12-11T17:37:09.194 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph-authtool: stdout AQDESHdlnmpbLxAAgd1J2jgBe98VQhkrK89qRw== 2023-12-11T17:37:09.943 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph-authtool: stdout AQDFSHdlnh6sIRAA1kraCsrj9H/SeH9ceyB/wQ== 2023-12-11T17:37:09.944 INFO:teuthology.orchestra.run.smithi059.stdout:Creating initial monmap... 2023-12-11T17:37:10.700 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2023-12-11T17:37:10.701 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = pacific 2023-12-11T17:37:10.701 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to d0a07eaa-984b-11ee-95a3-87774f69a715 2023-12-11T17:37:10.701 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2023-12-11T17:37:10.701 INFO:teuthology.orchestra.run.smithi059.stdout:monmaptool for a [v2:172.21.15.59:3300,v1:172.21.15.59:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2023-12-11T17:37:10.701 INFO:teuthology.orchestra.run.smithi059.stdout:setting min_mon_release = pacific 2023-12-11T17:37:10.701 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/monmaptool: set fsid to d0a07eaa-984b-11ee-95a3-87774f69a715 2023-12-11T17:37:10.701 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2023-12-11T17:37:10.701 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-12-11T17:37:10.701 INFO:teuthology.orchestra.run.smithi059.stdout:Creating mon... 2023-12-11T17:37:12.158 INFO:teuthology.orchestra.run.smithi059.stdout:create mon.a on 2023-12-11T17:37:12.704 INFO:teuthology.orchestra.run.smithi059.stdout:systemctl: stderr Removed /etc/systemd/system/multi-user.target.wants/ceph.target. 2023-12-11T17:37:12.799 INFO:teuthology.orchestra.run.smithi059.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2023-12-11T17:37:13.095 INFO:teuthology.orchestra.run.smithi059.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-d0a07eaa-984b-11ee-95a3-87774f69a715.target → /etc/systemd/system/ceph-d0a07eaa-984b-11ee-95a3-87774f69a715.target. 2023-12-11T17:37:13.096 INFO:teuthology.orchestra.run.smithi059.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-d0a07eaa-984b-11ee-95a3-87774f69a715.target → /etc/systemd/system/ceph-d0a07eaa-984b-11ee-95a3-87774f69a715.target. 2023-12-11T17:37:13.620 INFO:teuthology.orchestra.run.smithi059.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-d0a07eaa-984b-11ee-95a3-87774f69a715@mon.a 2023-12-11T17:37:13.621 INFO:teuthology.orchestra.run.smithi059.stdout:systemctl: stderr Failed to reset failed state of unit ceph-d0a07eaa-984b-11ee-95a3-87774f69a715@mon.a.service: Unit ceph-d0a07eaa-984b-11ee-95a3-87774f69a715@mon.a.service not loaded. 2023-12-11T17:37:13.712 INFO:teuthology.orchestra.run.smithi059.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-d0a07eaa-984b-11ee-95a3-87774f69a715.target.wants/ceph-d0a07eaa-984b-11ee-95a3-87774f69a715@mon.a.service → /etc/systemd/system/ceph-d0a07eaa-984b-11ee-95a3-87774f69a715@.service. 2023-12-11T17:37:13.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:13 smithi059 systemd[1]: Starting Ceph mon.a for d0a07eaa-984b-11ee-95a3-87774f69a715... 2023-12-11T17:37:14.511 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:14 smithi059 podman[98306]: 2023-12-11T17:37:14.511 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:14 smithi059 bash[98306]: 159be1e3dc77ca690f12170624ababaccaaab88d86604ccb2c88b957c7e48797 2023-12-11T17:37:14.522 INFO:teuthology.orchestra.run.smithi059.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-12-11T17:37:14.522 INFO:teuthology.orchestra.run.smithi059.stdout:systemctl: stdout disabled 2023-12-11T17:37:14.530 INFO:teuthology.orchestra.run.smithi059.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-12-11T17:37:14.530 INFO:teuthology.orchestra.run.smithi059.stdout:systemctl: stdout inactive 2023-12-11T17:37:14.530 INFO:teuthology.orchestra.run.smithi059.stdout:firewalld.service is not enabled 2023-12-11T17:37:14.530 INFO:teuthology.orchestra.run.smithi059.stdout:Not possible to enable service . firewalld.service is not available 2023-12-11T17:37:14.531 INFO:teuthology.orchestra.run.smithi059.stdout:Waiting for mon to start... 2023-12-11T17:37:14.531 INFO:teuthology.orchestra.run.smithi059.stdout:Waiting for mon... 2023-12-11T17:37:14.774 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:14 smithi059 systemd[1]: Started Ceph mon.a for d0a07eaa-984b-11ee-95a3-87774f69a715. 2023-12-11T17:37:14.775 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:14 smithi059 ceph-mon[98346]: mkfs d0a07eaa-984b-11ee-95a3-87774f69a715 2023-12-11T17:37:14.775 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:14 smithi059 ceph-mon[98346]: mon.a is new leader, mons a in quorum (ranks 0) 2023-12-11T17:37:15.483 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout cluster: 2023-12-11T17:37:15.483 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout id: d0a07eaa-984b-11ee-95a3-87774f69a715 2023-12-11T17:37:15.483 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2023-12-11T17:37:15.483 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout 2023-12-11T17:37:15.483 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout services: 2023-12-11T17:37:15.483 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.652634s) 2023-12-11T17:37:15.483 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2023-12-11T17:37:15.483 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2023-12-11T17:37:15.484 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout 2023-12-11T17:37:15.484 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout data: 2023-12-11T17:37:15.484 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2023-12-11T17:37:15.484 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2023-12-11T17:37:15.484 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2023-12-11T17:37:15.484 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout pgs: 2023-12-11T17:37:15.484 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout 2023-12-11T17:37:15.484 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:15.023+0000 7fa660295700 1 Processor -- start 2023-12-11T17:37:15.484 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:15.023+0000 7fa660295700 1 -- start start 2023-12-11T17:37:15.484 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:15.023+0000 7fa660295700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa658105c20 0x7fa658106040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:37:15.484 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:15.023+0000 7fa660295700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa658106610 con 0x7fa658105c20 2023-12-11T17:37:15.484 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:15.024+0000 7fa65e031700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa658105c20 0x7fa658106040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:37:15.484 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:15.024+0000 7fa65e031700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa658105c20 0x7fa658106040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59744/0 (socket says 172.21.15.59:59744) 2023-12-11T17:37:15.484 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:15.024+0000 7fa65e031700 1 -- 172.21.15.59:0/1047786577 learned_addr learned my addr 172.21.15.59:0/1047786577 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:37:15.484 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:15.024+0000 7fa65e031700 1 -- 172.21.15.59:0/1047786577 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa658106e20 con 0x7fa658105c20 2023-12-11T17:37:15.485 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:15.024+0000 7fa65e031700 1 --2- 172.21.15.59:0/1047786577 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa658105c20 0x7fa658106040 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fa6540098a0 tx=0x7fa654009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8902b5867daea476 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:37:15.485 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:15.025+0000 7fa65d02f700 1 -- 172.21.15.59:0/1047786577 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa654004160 con 0x7fa658105c20 2023-12-11T17:37:15.485 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:15.025+0000 7fa65d02f700 1 -- 172.21.15.59:0/1047786577 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6540042c0 con 0x7fa658105c20 2023-12-11T17:37:15.485 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:15.025+0000 7fa65d02f700 1 -- 172.21.15.59:0/1047786577 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa6540045a0 con 0x7fa658105c20 2023-12-11T17:37:15.485 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:15.025+0000 7fa660295700 1 -- 172.21.15.59:0/1047786577 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa658105c20 msgr2=0x7fa658106040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:37:15.485 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:15.025+0000 7fa660295700 1 --2- 172.21.15.59:0/1047786577 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa658105c20 0x7fa658106040 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fa6540098a0 tx=0x7fa654009bb0 comp rx=0 tx=0).stop 2023-12-11T17:37:15.485 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:15.025+0000 7fa660295700 1 -- 172.21.15.59:0/1047786577 shutdown_connections 2023-12-11T17:37:15.485 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:15.025+0000 7fa660295700 1 --2- 172.21.15.59:0/1047786577 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa658105c20 0x7fa658106040 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:37:15.485 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:15.025+0000 7fa660295700 1 -- 172.21.15.59:0/1047786577 >> 172.21.15.59:0/1047786577 conn(0x7fa6581011c0 msgr2=0x7fa658103600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:37:15.485 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:15.025+0000 7fa660295700 1 -- 172.21.15.59:0/1047786577 shutdown_connections 2023-12-11T17:37:15.485 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:15.025+0000 7fa660295700 1 -- 172.21.15.59:0/1047786577 wait complete. 2023-12-11T17:37:15.485 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:15.026+0000 7fa660295700 1 Processor -- start 2023-12-11T17:37:15.485 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:15.026+0000 7fa660295700 1 -- start start 2023-12-11T17:37:15.485 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:15.026+0000 7fa660295700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa658105c20 0x7fa65818e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:37:15.486 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:15.026+0000 7fa660295700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa65818cf10 con 0x7fa658105c20 2023-12-11T17:37:15.486 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:15.026+0000 7fa65e031700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa658105c20 0x7fa65818e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:37:15.486 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:15.026+0000 7fa65e031700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa658105c20 0x7fa65818e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59746/0 (socket says 172.21.15.59:59746) 2023-12-11T17:37:15.486 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:15.026+0000 7fa65e031700 1 -- 172.21.15.59:0/1705225386 learned_addr learned my addr 172.21.15.59:0/1705225386 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:37:15.486 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:15.027+0000 7fa65e031700 1 -- 172.21.15.59:0/1705225386 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa654009550 con 0x7fa658105c20 2023-12-11T17:37:15.486 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:15.027+0000 7fa65e031700 1 --2- 172.21.15.59:0/1705225386 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa658105c20 0x7fa65818e970 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fa654004000 tx=0x7fa654004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:37:15.486 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:15.027+0000 7fa64f7fe700 1 -- 172.21.15.59:0/1705225386 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa654011630 con 0x7fa658105c20 2023-12-11T17:37:15.486 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:15.027+0000 7fa64f7fe700 1 -- 172.21.15.59:0/1705225386 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa654011790 con 0x7fa658105c20 2023-12-11T17:37:15.486 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:15.027+0000 7fa660295700 1 -- 172.21.15.59:0/1705225386 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa65818d110 con 0x7fa658105c20 2023-12-11T17:37:15.486 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:15.027+0000 7fa64f7fe700 1 -- 172.21.15.59:0/1705225386 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa654011a30 con 0x7fa658105c20 2023-12-11T17:37:15.486 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:15.027+0000 7fa660295700 1 -- 172.21.15.59:0/1705225386 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa65818d5b0 con 0x7fa658105c20 2023-12-11T17:37:15.486 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:15.028+0000 7fa64f7fe700 1 -- 172.21.15.59:0/1705225386 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fa654021070 con 0x7fa658105c20 2023-12-11T17:37:15.486 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:15.028+0000 7fa64f7fe700 1 -- 172.21.15.59:0/1705225386 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fa65401b780 con 0x7fa658105c20 2023-12-11T17:37:15.487 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:15.028+0000 7fa660295700 1 -- 172.21.15.59:0/1705225386 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa658193ea0 con 0x7fa658105c20 2023-12-11T17:37:15.487 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:15.030+0000 7fa64f7fe700 1 -- 172.21.15.59:0/1705225386 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7fa65401acc0 con 0x7fa658105c20 2023-12-11T17:37:15.487 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:15.163+0000 7fa660295700 1 -- 172.21.15.59:0/1705225386 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7fa658107190 con 0x7fa658105c20 2023-12-11T17:37:15.487 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:15.164+0000 7fa64f7fe700 1 -- 172.21.15.59:0/1705225386 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+317 (secure 0 0 0) 0x7fa65401ae60 con 0x7fa658105c20 2023-12-11T17:37:15.487 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:15.164+0000 7fa660295700 1 -- 172.21.15.59:0/1705225386 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa658105c20 msgr2=0x7fa65818e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:37:15.487 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:15.164+0000 7fa660295700 1 --2- 172.21.15.59:0/1705225386 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa658105c20 0x7fa65818e970 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fa654004000 tx=0x7fa654004750 comp rx=0 tx=0).stop 2023-12-11T17:37:15.487 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:15.164+0000 7fa660295700 1 -- 172.21.15.59:0/1705225386 shutdown_connections 2023-12-11T17:37:15.487 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:15.164+0000 7fa660295700 1 --2- 172.21.15.59:0/1705225386 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa658105c20 0x7fa65818e970 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:37:15.487 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:15.164+0000 7fa660295700 1 -- 172.21.15.59:0/1705225386 >> 172.21.15.59:0/1705225386 conn(0x7fa6581011c0 msgr2=0x7fa658190fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:37:15.487 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:15.165+0000 7fa660295700 1 -- 172.21.15.59:0/1705225386 shutdown_connections 2023-12-11T17:37:15.487 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:15.165+0000 7fa660295700 1 -- 172.21.15.59:0/1705225386 wait complete. 2023-12-11T17:37:15.487 INFO:teuthology.orchestra.run.smithi059.stdout:mon is available 2023-12-11T17:37:15.487 INFO:teuthology.orchestra.run.smithi059.stdout:Assimilating anything we can from ceph.conf... 2023-12-11T17:37:15.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:15 smithi059 ceph-mon[98346]: mon.a is new leader, mons a in quorum (ranks 0) 2023-12-11T17:37:15.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:15 smithi059 ceph-mon[98346]: monmap e1: 1 mons at {a=[v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0]} removed_ranks: {} disallowed_leaders: {} 2023-12-11T17:37:15.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:15 smithi059 ceph-mon[98346]: fsmap 2023-12-11T17:37:15.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:15 smithi059 ceph-mon[98346]: osdmap e1: 0 total, 0 up, 0 in 2023-12-11T17:37:15.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:15 smithi059 ceph-mon[98346]: mgrmap e1: no daemons active 2023-12-11T17:37:15.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:15 smithi059 ceph-mon[98346]: from='client.? 172.21.15.59:0/1705225386' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2023-12-11T17:37:16.523 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout 2023-12-11T17:37:16.529 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout [global] 2023-12-11T17:37:16.530 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout fsid = d0a07eaa-984b-11ee-95a3-87774f69a715 2023-12-11T17:37:16.530 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.59:3300,v1:172.21.15.59:6789] 2023-12-11T17:37:16.530 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2023-12-11T17:37:16.530 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2023-12-11T17:37:16.530 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2023-12-11T17:37:16.530 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2023-12-11T17:37:16.530 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout 2023-12-11T17:37:16.530 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout [mgr] 2023-12-11T17:37:16.530 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2023-12-11T17:37:16.530 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout 2023-12-11T17:37:16.530 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout [osd] 2023-12-11T17:37:16.530 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2023-12-11T17:37:16.530 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2023-12-11T17:37:16.531 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2023-12-11T17:37:16.531 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2023-12-11T17:37:16.531 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2023-12-11T17:37:16.531 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:15.966+0000 7f5326b76700 1 Processor -- start 2023-12-11T17:37:16.531 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:15.966+0000 7f5326b76700 1 -- start start 2023-12-11T17:37:16.531 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:15.966+0000 7f5326b76700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5320105be0 0x7f5320106000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:37:16.531 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:15.966+0000 7f5326b76700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53201065d0 con 0x7f5320105be0 2023-12-11T17:37:16.531 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:15.967+0000 7f5324912700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5320105be0 0x7f5320106000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:37:16.531 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:15.967+0000 7f5324912700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5320105be0 0x7f5320106000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59748/0 (socket says 172.21.15.59:59748) 2023-12-11T17:37:16.531 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:15.967+0000 7f5324912700 1 -- 172.21.15.59:0/857527017 learned_addr learned my addr 172.21.15.59:0/857527017 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:37:16.531 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:15.967+0000 7f5324912700 1 -- 172.21.15.59:0/857527017 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5320106de0 con 0x7f5320105be0 2023-12-11T17:37:16.531 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:15.967+0000 7f5324912700 1 --2- 172.21.15.59:0/857527017 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5320105be0 0x7f5320106000 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f53140098a0 tx=0x7f5314009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18a6aab3e987cef8 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:37:16.531 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:15.967+0000 7f531f7fe700 1 -- 172.21.15.59:0/857527017 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5314004160 con 0x7f5320105be0 2023-12-11T17:37:16.531 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:15.967+0000 7f531f7fe700 1 -- 172.21.15.59:0/857527017 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53140042c0 con 0x7f5320105be0 2023-12-11T17:37:16.531 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:15.967+0000 7f531f7fe700 1 -- 172.21.15.59:0/857527017 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f53140045a0 con 0x7f5320105be0 2023-12-11T17:37:16.532 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:15.968+0000 7f5326b76700 1 -- 172.21.15.59:0/857527017 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5320105be0 msgr2=0x7f5320106000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:37:16.532 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:15.968+0000 7f5326b76700 1 --2- 172.21.15.59:0/857527017 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5320105be0 0x7f5320106000 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f53140098a0 tx=0x7f5314009bb0 comp rx=0 tx=0).stop 2023-12-11T17:37:16.532 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:15.968+0000 7f5326b76700 1 -- 172.21.15.59:0/857527017 shutdown_connections 2023-12-11T17:37:16.532 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:15.968+0000 7f5326b76700 1 --2- 172.21.15.59:0/857527017 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5320105be0 0x7f5320106000 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:37:16.532 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:15.968+0000 7f5326b76700 1 -- 172.21.15.59:0/857527017 >> 172.21.15.59:0/857527017 conn(0x7f5320101160 msgr2=0x7f53201035c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:37:16.532 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:15.968+0000 7f5326b76700 1 -- 172.21.15.59:0/857527017 shutdown_connections 2023-12-11T17:37:16.532 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:15.968+0000 7f5326b76700 1 -- 172.21.15.59:0/857527017 wait complete. 2023-12-11T17:37:16.532 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:15.969+0000 7f5326b76700 1 Processor -- start 2023-12-11T17:37:16.532 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:15.969+0000 7f5326b76700 1 -- start start 2023-12-11T17:37:16.532 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:15.969+0000 7f5326b76700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5320105be0 0x7f532018e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:37:16.532 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:15.969+0000 7f5326b76700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f532018ced0 con 0x7f5320105be0 2023-12-11T17:37:16.532 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:15.969+0000 7f5324912700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5320105be0 0x7f532018e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:37:16.532 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:15.969+0000 7f5324912700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5320105be0 0x7f532018e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59750/0 (socket says 172.21.15.59:59750) 2023-12-11T17:37:16.532 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:15.969+0000 7f5324912700 1 -- 172.21.15.59:0/847497877 learned_addr learned my addr 172.21.15.59:0/847497877 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:37:16.532 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:15.969+0000 7f5324912700 1 -- 172.21.15.59:0/847497877 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5314009550 con 0x7f5320105be0 2023-12-11T17:37:16.533 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:15.969+0000 7f5324912700 1 --2- 172.21.15.59:0/847497877 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5320105be0 0x7f532018e930 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f5314004000 tx=0x7f5314004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:37:16.533 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:15.970+0000 7f531dffb700 1 -- 172.21.15.59:0/847497877 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5314011630 con 0x7f5320105be0 2023-12-11T17:37:16.533 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:15.970+0000 7f531dffb700 1 -- 172.21.15.59:0/847497877 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5314011790 con 0x7f5320105be0 2023-12-11T17:37:16.533 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:15.970+0000 7f5326b76700 1 -- 172.21.15.59:0/847497877 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f532018d0d0 con 0x7f5320105be0 2023-12-11T17:37:16.533 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:15.970+0000 7f531dffb700 1 -- 172.21.15.59:0/847497877 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5314011a30 con 0x7f5320105be0 2023-12-11T17:37:16.533 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:15.970+0000 7f5326b76700 1 -- 172.21.15.59:0/847497877 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f532018d570 con 0x7f5320105be0 2023-12-11T17:37:16.533 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:15.970+0000 7f531dffb700 1 -- 172.21.15.59:0/847497877 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f5314022070 con 0x7f5320105be0 2023-12-11T17:37:16.533 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:15.970+0000 7f531dffb700 1 -- 172.21.15.59:0/847497877 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f531401c450 con 0x7f5320105be0 2023-12-11T17:37:16.533 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:15.971+0000 7f5326b76700 1 -- 172.21.15.59:0/847497877 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5320193ec0 con 0x7f5320105be0 2023-12-11T17:37:16.533 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:15.973+0000 7f531dffb700 1 -- 172.21.15.59:0/847497877 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f531401b450 con 0x7f5320105be0 2023-12-11T17:37:16.533 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:16.099+0000 7f5326b76700 1 -- 172.21.15.59:0/847497877 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f5320067940 con 0x7f5320105be0 2023-12-11T17:37:16.533 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:16.102+0000 7f531dffb700 1 -- 172.21.15.59:0/847497877 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+429 (secure 0 0 0) 0x7f531401b5f0 con 0x7f5320105be0 2023-12-11T17:37:16.533 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:16.103+0000 7f531dffb700 1 -- 172.21.15.59:0/847497877 <== mon.0 v2:172.21.15.59:3300/0 8 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f531401b790 con 0x7f5320105be0 2023-12-11T17:37:16.533 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:16.104+0000 7f5326b76700 1 -- 172.21.15.59:0/847497877 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5320105be0 msgr2=0x7f532018e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:37:16.533 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:16.104+0000 7f5326b76700 1 --2- 172.21.15.59:0/847497877 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5320105be0 0x7f532018e930 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f5314004000 tx=0x7f5314004750 comp rx=0 tx=0).stop 2023-12-11T17:37:16.534 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:16.104+0000 7f5326b76700 1 -- 172.21.15.59:0/847497877 shutdown_connections 2023-12-11T17:37:16.534 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:16.104+0000 7f5326b76700 1 --2- 172.21.15.59:0/847497877 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5320105be0 0x7f532018e930 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:37:16.534 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:16.104+0000 7f5326b76700 1 -- 172.21.15.59:0/847497877 >> 172.21.15.59:0/847497877 conn(0x7f5320101160 msgr2=0x7f5320191000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:37:16.534 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:16.104+0000 7f5326b76700 1 -- 172.21.15.59:0/847497877 shutdown_connections 2023-12-11T17:37:16.534 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:16.104+0000 7f5326b76700 1 -- 172.21.15.59:0/847497877 wait complete. 2023-12-11T17:37:16.534 INFO:teuthology.orchestra.run.smithi059.stdout:Generating new minimal ceph.conf... 2023-12-11T17:37:16.730 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:16 smithi059 ceph-mon[98346]: from='client.? 172.21.15.59:0/847497877' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2023-12-11T17:37:16.730 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:16 smithi059 ceph-mon[98346]: from='client.? 172.21.15.59:0/847497877' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2023-12-11T17:37:17.574 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:17.008+0000 7ff95264a700 1 Processor -- start 2023-12-11T17:37:17.574 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:17.008+0000 7ff95264a700 1 -- start start 2023-12-11T17:37:17.574 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:17.008+0000 7ff95264a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff94c106d10 0x7ff94c107130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:37:17.574 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:17.008+0000 7ff95264a700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff94c107700 con 0x7ff94c106d10 2023-12-11T17:37:17.575 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:17.008+0000 7ff94bfff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff94c106d10 0x7ff94c107130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:37:17.575 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:17.008+0000 7ff94bfff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff94c106d10 0x7ff94c107130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59752/0 (socket says 172.21.15.59:59752) 2023-12-11T17:37:17.575 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:17.008+0000 7ff94bfff700 1 -- 172.21.15.59:0/691452694 learned_addr learned my addr 172.21.15.59:0/691452694 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:37:17.575 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:17.008+0000 7ff94bfff700 1 -- 172.21.15.59:0/691452694 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff94c107ec0 con 0x7ff94c106d10 2023-12-11T17:37:17.575 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:17.009+0000 7ff94bfff700 1 --2- 172.21.15.59:0/691452694 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff94c106d10 0x7ff94c107130 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ff93c0098a0 tx=0x7ff93c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=671b71f5594101d3 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:37:17.575 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:17.009+0000 7ff94affd700 1 -- 172.21.15.59:0/691452694 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff93c004160 con 0x7ff94c106d10 2023-12-11T17:37:17.575 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:17.009+0000 7ff94affd700 1 -- 172.21.15.59:0/691452694 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff93c0042c0 con 0x7ff94c106d10 2023-12-11T17:37:17.575 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:17.009+0000 7ff94affd700 1 -- 172.21.15.59:0/691452694 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff93c011a00 con 0x7ff94c106d10 2023-12-11T17:37:17.575 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:17.009+0000 7ff95264a700 1 -- 172.21.15.59:0/691452694 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff94c106d10 msgr2=0x7ff94c107130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:37:17.575 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:17.009+0000 7ff95264a700 1 --2- 172.21.15.59:0/691452694 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff94c106d10 0x7ff94c107130 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ff93c0098a0 tx=0x7ff93c009bb0 comp rx=0 tx=0).stop 2023-12-11T17:37:17.575 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:17.010+0000 7ff95264a700 1 -- 172.21.15.59:0/691452694 shutdown_connections 2023-12-11T17:37:17.575 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:17.010+0000 7ff95264a700 1 --2- 172.21.15.59:0/691452694 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff94c106d10 0x7ff94c107130 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:37:17.575 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:17.010+0000 7ff95264a700 1 -- 172.21.15.59:0/691452694 >> 172.21.15.59:0/691452694 conn(0x7ff94c102330 msgr2=0x7ff94c104750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:37:17.575 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:17.010+0000 7ff95264a700 1 -- 172.21.15.59:0/691452694 shutdown_connections 2023-12-11T17:37:17.576 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:17.010+0000 7ff95264a700 1 -- 172.21.15.59:0/691452694 wait complete. 2023-12-11T17:37:17.576 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:17.010+0000 7ff95264a700 1 Processor -- start 2023-12-11T17:37:17.576 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:17.010+0000 7ff95264a700 1 -- start start 2023-12-11T17:37:17.576 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:17.011+0000 7ff95264a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff94c106d10 0x7ff94c19ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:37:17.576 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:17.011+0000 7ff95264a700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff94c19b080 con 0x7ff94c106d10 2023-12-11T17:37:17.576 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:17.011+0000 7ff94bfff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff94c106d10 0x7ff94c19ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:37:17.576 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:17.011+0000 7ff94bfff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff94c106d10 0x7ff94c19ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59754/0 (socket says 172.21.15.59:59754) 2023-12-11T17:37:17.576 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:17.011+0000 7ff94bfff700 1 -- 172.21.15.59:0/421553132 learned_addr learned my addr 172.21.15.59:0/421553132 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:37:17.576 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:17.011+0000 7ff94bfff700 1 -- 172.21.15.59:0/421553132 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff93c009550 con 0x7ff94c106d10 2023-12-11T17:37:17.576 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:17.011+0000 7ff94bfff700 1 --2- 172.21.15.59:0/421553132 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff94c106d10 0x7ff94c19ab40 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff93c011710 tx=0x7ff93c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:37:17.576 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:17.011+0000 7ff9497fa700 1 -- 172.21.15.59:0/421553132 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff93c003aa0 con 0x7ff94c106d10 2023-12-11T17:37:17.576 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:17.011+0000 7ff9497fa700 1 -- 172.21.15.59:0/421553132 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff93c003c00 con 0x7ff94c106d10 2023-12-11T17:37:17.576 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:17.011+0000 7ff95264a700 1 -- 172.21.15.59:0/421553132 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff94c19b280 con 0x7ff94c106d10 2023-12-11T17:37:17.576 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:17.011+0000 7ff9497fa700 1 -- 172.21.15.59:0/421553132 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff93c024800 con 0x7ff94c106d10 2023-12-11T17:37:17.577 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:17.011+0000 7ff95264a700 1 -- 172.21.15.59:0/421553132 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff94c19bde0 con 0x7ff94c106d10 2023-12-11T17:37:17.577 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:17.013+0000 7ff9497fa700 1 -- 172.21.15.59:0/421553132 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7ff93c024a30 con 0x7ff94c106d10 2023-12-11T17:37:17.577 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:17.013+0000 7ff9497fa700 1 -- 172.21.15.59:0/421553132 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7ff93c02dab0 con 0x7ff94c106d10 2023-12-11T17:37:17.577 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:17.013+0000 7ff95264a700 1 -- 172.21.15.59:0/421553132 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff94c19b720 con 0x7ff94c106d10 2023-12-11T17:37:17.577 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:17.015+0000 7ff9497fa700 1 -- 172.21.15.59:0/421553132 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7ff94c19b720 con 0x7ff94c106d10 2023-12-11T17:37:17.577 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:17.142+0000 7ff95264a700 1 -- 172.21.15.59:0/421553132 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7ff94c062c00 con 0x7ff94c106d10 2023-12-11T17:37:17.577 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:17.143+0000 7ff9497fa700 1 -- 172.21.15.59:0/421553132 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+175 (secure 0 0 0) 0x7ff93c01a8d0 con 0x7ff94c106d10 2023-12-11T17:37:17.577 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:17.143+0000 7ff95264a700 1 -- 172.21.15.59:0/421553132 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff94c106d10 msgr2=0x7ff94c19ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:37:17.577 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:17.143+0000 7ff95264a700 1 --2- 172.21.15.59:0/421553132 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff94c106d10 0x7ff94c19ab40 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff93c011710 tx=0x7ff93c011ad0 comp rx=0 tx=0).stop 2023-12-11T17:37:17.577 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:17.143+0000 7ff95264a700 1 -- 172.21.15.59:0/421553132 shutdown_connections 2023-12-11T17:37:17.577 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:17.144+0000 7ff95264a700 1 --2- 172.21.15.59:0/421553132 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff94c106d10 0x7ff94c19ab40 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:37:17.577 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:17.144+0000 7ff95264a700 1 -- 172.21.15.59:0/421553132 >> 172.21.15.59:0/421553132 conn(0x7ff94c102330 msgr2=0x7ff94c102ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:37:17.577 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:17.144+0000 7ff95264a700 1 -- 172.21.15.59:0/421553132 shutdown_connections 2023-12-11T17:37:17.577 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:17.144+0000 7ff95264a700 1 -- 172.21.15.59:0/421553132 wait complete. 2023-12-11T17:37:17.577 INFO:teuthology.orchestra.run.smithi059.stdout:Restarting the monitor... 2023-12-11T17:37:17.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:17 smithi059 ceph-mon[98346]: from='client.? 172.21.15.59:0/421553132' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:37:17.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:17 smithi059 systemd[1]: Stopping Ceph mon.a for d0a07eaa-984b-11ee-95a3-87774f69a715... 2023-12-11T17:37:18.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:17 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mon-a[98322]: 2023-12-11T17:37:17.905+0000 7fa59f633700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-12-11T17:37:18.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:17 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mon-a[98322]: 2023-12-11T17:37:17.905+0000 7fa59f633700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2023-12-11T17:37:18.539 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:18 smithi059 bash[98679]: ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mon-a 2023-12-11T17:37:18.848 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:18 smithi059 systemd[1]: ceph-d0a07eaa-984b-11ee-95a3-87774f69a715@mon.a.service: Succeeded. 2023-12-11T17:37:18.849 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:18 smithi059 systemd[1]: Stopped Ceph mon.a for d0a07eaa-984b-11ee-95a3-87774f69a715. 2023-12-11T17:37:18.849 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:18 smithi059 systemd[1]: Starting Ceph mon.a for d0a07eaa-984b-11ee-95a3-87774f69a715... 2023-12-11T17:37:19.029 INFO:teuthology.orchestra.run.smithi059.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2023-12-11T17:37:19.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:18 smithi059 podman[98807]: 2023-12-11T17:37:19.155 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 bash[98807]: 05668a55c7a9a926e856c0f0bc7aa6ea39800ffc1f133b3bd7da929574880ca6 2023-12-11T17:37:19.155 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: set uid:gid to 167:167 (ceph:ceph) 2023-12-11T17:37:19.155 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: ceph version 19.0.0-119-ga62fda73 (a62fda73efba5520d409bb5a21bd8373289dda5d) reef (dev), process ceph-mon, pid 7 2023-12-11T17:37:19.155 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: pidfile_write: ignore empty --pid-file 2023-12-11T17:37:19.155 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: load: jerasure load: lrc 2023-12-11T17:37:19.155 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: RocksDB version: 7.9.2 2023-12-11T17:37:19.155 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Git sha 0 2023-12-11T17:37:19.155 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Compile date 2023-12-11 11:29:13 2023-12-11T17:37:19.155 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: DB SUMMARY 2023-12-11T17:37:19.155 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: DB Session ID: I6Q1S3JTF9C1TCLOFQ3A 2023-12-11T17:37:19.155 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: CURRENT file: CURRENT 2023-12-11T17:37:19.155 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: IDENTITY file: IDENTITY 2023-12-11T17:37:19.155 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2023-12-11T17:37:19.156 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2023-12-11T17:37:19.156 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 98829 ; 2023-12-11T17:37:19.156 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.error_if_exists: 0 2023-12-11T17:37:19.156 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.create_if_missing: 0 2023-12-11T17:37:19.156 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.paranoid_checks: 1 2023-12-11T17:37:19.156 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.flush_verify_memtable_count: 1 2023-12-11T17:37:19.156 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-12-11T17:37:19.156 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-12-11T17:37:19.156 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.env: 0x55a099017a60 2023-12-11T17:37:19.156 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.fs: PosixFileSystem 2023-12-11T17:37:19.156 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.info_log: 0x55a099a4b740 2023-12-11T17:37:19.156 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.max_file_opening_threads: 16 2023-12-11T17:37:19.156 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.statistics: (nil) 2023-12-11T17:37:19.156 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.use_fsync: 0 2023-12-11T17:37:19.157 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.max_log_file_size: 0 2023-12-11T17:37:19.157 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-12-11T17:37:19.157 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.log_file_time_to_roll: 0 2023-12-11T17:37:19.157 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.keep_log_file_num: 1000 2023-12-11T17:37:19.157 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.recycle_log_file_num: 0 2023-12-11T17:37:19.157 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.allow_fallocate: 1 2023-12-11T17:37:19.157 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.allow_mmap_reads: 0 2023-12-11T17:37:19.157 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.allow_mmap_writes: 0 2023-12-11T17:37:19.157 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.use_direct_reads: 0 2023-12-11T17:37:19.157 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-12-11T17:37:19.157 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.create_missing_column_families: 0 2023-12-11T17:37:19.157 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.db_log_dir: 2023-12-11T17:37:19.159 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.wal_dir: 2023-12-11T17:37:19.159 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.table_cache_numshardbits: 6 2023-12-11T17:37:19.159 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.WAL_ttl_seconds: 0 2023-12-11T17:37:19.159 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.WAL_size_limit_MB: 0 2023-12-11T17:37:19.159 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-12-11T17:37:19.159 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-12-11T17:37:19.159 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.is_fd_close_on_exec: 1 2023-12-11T17:37:19.159 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.advise_random_on_open: 1 2023-12-11T17:37:19.159 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.db_write_buffer_size: 0 2023-12-11T17:37:19.159 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.write_buffer_manager: 0x55a099a01360 2023-12-11T17:37:19.159 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-12-11T17:37:19.159 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-12-11T17:37:19.159 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.use_adaptive_mutex: 0 2023-12-11T17:37:19.159 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.rate_limiter: (nil) 2023-12-11T17:37:19.160 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-12-11T17:37:19.160 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.wal_recovery_mode: 2 2023-12-11T17:37:19.160 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.enable_thread_tracking: 0 2023-12-11T17:37:19.160 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.enable_pipelined_write: 0 2023-12-11T17:37:19.160 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.unordered_write: 0 2023-12-11T17:37:19.160 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-12-11T17:37:19.160 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-12-11T17:37:19.160 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-12-11T17:37:19.160 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-12-11T17:37:19.160 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.row_cache: None 2023-12-11T17:37:19.160 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.wal_filter: None 2023-12-11T17:37:19.160 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-12-11T17:37:19.160 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.allow_ingest_behind: 0 2023-12-11T17:37:19.160 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.two_write_queues: 0 2023-12-11T17:37:19.160 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.manual_wal_flush: 0 2023-12-11T17:37:19.161 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.wal_compression: 0 2023-12-11T17:37:19.161 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.atomic_flush: 0 2023-12-11T17:37:19.161 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-12-11T17:37:19.161 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.persist_stats_to_disk: 0 2023-12-11T17:37:19.161 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.write_dbid_to_manifest: 0 2023-12-11T17:37:19.161 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.log_readahead_size: 0 2023-12-11T17:37:19.161 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-12-11T17:37:19.161 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.best_efforts_recovery: 0 2023-12-11T17:37:19.161 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-12-11T17:37:19.161 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-12-11T17:37:19.161 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.allow_data_in_errors: 0 2023-12-11T17:37:19.161 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.db_host_id: __hostname__ 2023-12-11T17:37:19.161 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.enforce_single_del_contracts: true 2023-12-11T17:37:19.161 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.max_background_jobs: 2 2023-12-11T17:37:19.161 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.max_background_compactions: -1 2023-12-11T17:37:19.162 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.max_subcompactions: 1 2023-12-11T17:37:19.162 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-12-11T17:37:19.162 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-12-11T17:37:19.162 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.delayed_write_rate : 16777216 2023-12-11T17:37:19.162 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.max_total_wal_size: 0 2023-12-11T17:37:19.162 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-12-11T17:37:19.162 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.stats_dump_period_sec: 600 2023-12-11T17:37:19.162 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.stats_persist_period_sec: 600 2023-12-11T17:37:19.162 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-12-11T17:37:19.162 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.max_open_files: -1 2023-12-11T17:37:19.162 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.bytes_per_sync: 0 2023-12-11T17:37:19.162 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.wal_bytes_per_sync: 0 2023-12-11T17:37:19.162 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.strict_bytes_per_sync: 0 2023-12-11T17:37:19.163 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.compaction_readahead_size: 0 2023-12-11T17:37:19.163 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.max_background_flushes: -1 2023-12-11T17:37:19.163 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Compression algorithms supported: 2023-12-11T17:37:19.163 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-12-11T17:37:19.163 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: kZSTD supported: 0 2023-12-11T17:37:19.163 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: kXpressCompression supported: 0 2023-12-11T17:37:19.163 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: kLZ4HCCompression supported: 1 2023-12-11T17:37:19.163 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: kZlibCompression supported: 1 2023-12-11T17:37:19.163 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: kSnappyCompression supported: 1 2023-12-11T17:37:19.163 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: kLZ4Compression supported: 1 2023-12-11T17:37:19.163 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: kBZip2Compression supported: 0 2023-12-11T17:37:19.163 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Fast CRC32 supported: Supported on x86 2023-12-11T17:37:19.163 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: DMutex implementation: pthread_mutex_t 2023-12-11T17:37:19.163 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2023-12-11T17:37:19.163 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-12-11T17:37:19.163 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-12-11T17:37:19.163 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.merge_operator: 2023-12-11T17:37:19.164 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.compaction_filter: None 2023-12-11T17:37:19.164 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.compaction_filter_factory: None 2023-12-11T17:37:19.164 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.sst_partitioner_factory: None 2023-12-11T17:37:19.164 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.memtable_factory: SkipListFactory 2023-12-11T17:37:19.164 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.table_factory: BlockBasedTable 2023-12-11T17:37:19.164 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55a099a4b860) 2023-12-11T17:37:19.164 INFO:journalctl@ceph.mon.a.smithi059.stdout: cache_index_and_filter_blocks: 1 2023-12-11T17:37:19.164 INFO:journalctl@ceph.mon.a.smithi059.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-12-11T17:37:19.164 INFO:journalctl@ceph.mon.a.smithi059.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-12-11T17:37:19.164 INFO:journalctl@ceph.mon.a.smithi059.stdout: pin_top_level_index_and_filter: 1 2023-12-11T17:37:19.164 INFO:journalctl@ceph.mon.a.smithi059.stdout: index_type: 0 2023-12-11T17:37:19.164 INFO:journalctl@ceph.mon.a.smithi059.stdout: data_block_index_type: 0 2023-12-11T17:37:19.164 INFO:journalctl@ceph.mon.a.smithi059.stdout: index_shortening: 1 2023-12-11T17:37:19.164 INFO:journalctl@ceph.mon.a.smithi059.stdout: data_block_hash_table_util_ratio: 0.750000 2023-12-11T17:37:19.164 INFO:journalctl@ceph.mon.a.smithi059.stdout: checksum: 4 2023-12-11T17:37:19.165 INFO:journalctl@ceph.mon.a.smithi059.stdout: no_block_cache: 0 2023-12-11T17:37:19.165 INFO:journalctl@ceph.mon.a.smithi059.stdout: block_cache: 0x55a099aa1090 2023-12-11T17:37:19.165 INFO:journalctl@ceph.mon.a.smithi059.stdout: block_cache_name: BinnedLRUCache 2023-12-11T17:37:19.165 INFO:journalctl@ceph.mon.a.smithi059.stdout: block_cache_options: 2023-12-11T17:37:19.165 INFO:journalctl@ceph.mon.a.smithi059.stdout: capacity : 536870912 2023-12-11T17:37:19.165 INFO:journalctl@ceph.mon.a.smithi059.stdout: num_shard_bits : 4 2023-12-11T17:37:19.165 INFO:journalctl@ceph.mon.a.smithi059.stdout: strict_capacity_limit : 0 2023-12-11T17:37:19.165 INFO:journalctl@ceph.mon.a.smithi059.stdout: high_pri_pool_ratio: 0.000 2023-12-11T17:37:19.165 INFO:journalctl@ceph.mon.a.smithi059.stdout: block_cache_compressed: (nil) 2023-12-11T17:37:19.165 INFO:journalctl@ceph.mon.a.smithi059.stdout: persistent_cache: (nil) 2023-12-11T17:37:19.165 INFO:journalctl@ceph.mon.a.smithi059.stdout: block_size: 4096 2023-12-11T17:37:19.165 INFO:journalctl@ceph.mon.a.smithi059.stdout: block_size_deviation: 10 2023-12-11T17:37:19.165 INFO:journalctl@ceph.mon.a.smithi059.stdout: block_restart_interval: 16 2023-12-11T17:37:19.165 INFO:journalctl@ceph.mon.a.smithi059.stdout: index_block_restart_interval: 1 2023-12-11T17:37:19.165 INFO:journalctl@ceph.mon.a.smithi059.stdout: metadata_block_size: 4096 2023-12-11T17:37:19.165 INFO:journalctl@ceph.mon.a.smithi059.stdout: partition_filters: 0 2023-12-11T17:37:19.166 INFO:journalctl@ceph.mon.a.smithi059.stdout: use_delta_encoding: 1 2023-12-11T17:37:19.166 INFO:journalctl@ceph.mon.a.smithi059.stdout: filter_policy: bloomfilter 2023-12-11T17:37:19.166 INFO:journalctl@ceph.mon.a.smithi059.stdout: whole_key_filtering: 1 2023-12-11T17:37:19.166 INFO:journalctl@ceph.mon.a.smithi059.stdout: verify_compression: 0 2023-12-11T17:37:19.166 INFO:journalctl@ceph.mon.a.smithi059.stdout: read_amp_bytes_per_bit: 0 2023-12-11T17:37:19.166 INFO:journalctl@ceph.mon.a.smithi059.stdout: format_version: 5 2023-12-11T17:37:19.166 INFO:journalctl@ceph.mon.a.smithi059.stdout: enable_index_compression: 1 2023-12-11T17:37:19.166 INFO:journalctl@ceph.mon.a.smithi059.stdout: block_align: 0 2023-12-11T17:37:19.166 INFO:journalctl@ceph.mon.a.smithi059.stdout: max_auto_readahead_size: 262144 2023-12-11T17:37:19.166 INFO:journalctl@ceph.mon.a.smithi059.stdout: prepopulate_block_cache: 0 2023-12-11T17:37:19.166 INFO:journalctl@ceph.mon.a.smithi059.stdout: initial_auto_readahead_size: 8192 2023-12-11T17:37:19.166 INFO:journalctl@ceph.mon.a.smithi059.stdout: num_file_reads_for_auto_readahead: 2 2023-12-11T17:37:19.166 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.write_buffer_size: 33554432 2023-12-11T17:37:19.166 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.max_write_buffer_number: 2 2023-12-11T17:37:19.167 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.compression: NoCompression 2023-12-11T17:37:19.167 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.bottommost_compression: Disabled 2023-12-11T17:37:19.167 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.prefix_extractor: nullptr 2023-12-11T17:37:19.167 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-12-11T17:37:19.167 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.num_levels: 7 2023-12-11T17:37:19.167 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-12-11T17:37:19.167 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-12-11T17:37:19.167 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-12-11T17:37:19.167 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-12-11T17:37:19.167 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-12-11T17:37:19.167 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-12-11T17:37:19.167 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-12-11T17:37:19.167 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-12-11T17:37:19.167 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-12-11T17:37:19.167 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-12-11T17:37:19.168 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-12-11T17:37:19.168 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-12-11T17:37:19.168 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.compression_opts.window_bits: -14 2023-12-11T17:37:19.168 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.compression_opts.level: 32767 2023-12-11T17:37:19.168 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.compression_opts.strategy: 0 2023-12-11T17:37:19.168 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-12-11T17:37:19.168 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-12-11T17:37:19.168 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-12-11T17:37:19.168 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-12-11T17:37:19.168 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.compression_opts.enabled: false 2023-12-11T17:37:19.168 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-12-11T17:37:19.168 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-12-11T17:37:19.169 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-12-11T17:37:19.169 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-12-11T17:37:19.169 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.target_file_size_base: 67108864 2023-12-11T17:37:19.169 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.target_file_size_multiplier: 1 2023-12-11T17:37:19.169 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-12-11T17:37:19.169 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-12-11T17:37:19.169 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-12-11T17:37:19.169 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-12-11T17:37:19.169 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-12-11T17:37:19.169 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-12-11T17:37:19.170 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-12-11T17:37:19.170 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-12-11T17:37:19.170 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-12-11T17:37:19.170 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-12-11T17:37:19.170 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-12-11T17:37:19.170 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-12-11T17:37:19.170 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-12-11T17:37:19.170 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.arena_block_size: 1048576 2023-12-11T17:37:19.170 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-12-11T17:37:19.170 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-12-11T17:37:19.170 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.disable_auto_compactions: 0 2023-12-11T17:37:19.171 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-12-11T17:37:19.171 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-12-11T17:37:19.171 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-12-11T17:37:19.171 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-12-11T17:37:19.171 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-12-11T17:37:19.171 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-12-11T17:37:19.171 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-12-11T17:37:19.171 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-12-11T17:37:19.171 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-12-11T17:37:19.171 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-12-11T17:37:19.171 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.table_properties_collectors: 2023-12-11T17:37:19.171 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.inplace_update_support: 0 2023-12-11T17:37:19.171 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.inplace_update_num_locks: 10000 2023-12-11T17:37:19.171 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-12-11T17:37:19.171 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-12-11T17:37:19.172 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.memtable_huge_page_size: 0 2023-12-11T17:37:19.172 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.bloom_locality: 0 2023-12-11T17:37:19.172 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.max_successive_merges: 0 2023-12-11T17:37:19.172 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.optimize_filters_for_hits: 0 2023-12-11T17:37:19.172 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.paranoid_file_checks: 0 2023-12-11T17:37:19.172 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.force_consistency_checks: 1 2023-12-11T17:37:19.172 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.report_bg_io_stats: 0 2023-12-11T17:37:19.172 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.ttl: 2592000 2023-12-11T17:37:19.172 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.periodic_compaction_seconds: 0 2023-12-11T17:37:19.172 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-12-11T17:37:19.172 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-12-11T17:37:19.172 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.enable_blob_files: false 2023-12-11T17:37:19.172 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.min_blob_size: 0 2023-12-11T17:37:19.172 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.blob_file_size: 268435456 2023-12-11T17:37:19.172 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.blob_compression_type: NoCompression 2023-12-11T17:37:19.172 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.enable_blob_garbage_collection: false 2023-12-11T17:37:19.173 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-12-11T17:37:19.173 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-12-11T17:37:19.173 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-12-11T17:37:19.173 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.blob_file_starting_level: 0 2023-12-11T17:37:19.173 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-12-11T17:37:19.173 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2023-12-11T17:37:19.173 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2023-12-11T17:37:19.173 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 0652ac3d-3f3d-4fe2-a081-f11cd1483894 2023-12-11T17:37:19.173 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: EVENT_LOG_v1 {"time_micros": 1702316239019435, "job": 1, "event": "recovery_started", "wal_files": [9]} 2023-12-11T17:37:19.173 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2023-12-11T17:37:19.173 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: EVENT_LOG_v1 {"time_micros": 1702316239020408, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 94070, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 314, "table_properties": {"data_size": 92075, "index_size": 277, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13640, "raw_average_key_size": 49, "raw_value_size": 84640, "raw_average_value_size": 310, "num_data_blocks": 11, "num_entries": 273, "num_filter_entries": 273, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1702316239, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "0652ac3d-3f3d-4fe2-a081-f11cd1483894", "db_session_id": "I6Q1S3JTF9C1TCLOFQ3A", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2023-12-11T17:37:19.173 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: EVENT_LOG_v1 {"time_micros": 1702316239020484, "job": 1, "event": "recovery_finished"} 2023-12-11T17:37:19.173 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2023-12-11T17:37:19.173 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-12-11T17:37:19.173 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55a099b3c000 2023-12-11T17:37:19.174 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: DB pointer 0x55a099b28000 2023-12-11T17:37:19.174 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-12-11T17:37:19.174 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-12-11T17:37:19.174 INFO:journalctl@ceph.mon.a.smithi059.stdout: ** DB Stats ** 2023-12-11T17:37:19.174 INFO:journalctl@ceph.mon.a.smithi059.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-12-11T17:37:19.174 INFO:journalctl@ceph.mon.a.smithi059.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-12-11T17:37:19.174 INFO:journalctl@ceph.mon.a.smithi059.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-12-11T17:37:19.174 INFO:journalctl@ceph.mon.a.smithi059.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-12-11T17:37:19.174 INFO:journalctl@ceph.mon.a.smithi059.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-12-11T17:37:19.174 INFO:journalctl@ceph.mon.a.smithi059.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-12-11T17:37:19.174 INFO:journalctl@ceph.mon.a.smithi059.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-12-11T17:37:19.174 INFO:journalctl@ceph.mon.a.smithi059.stdout: 2023-12-11T17:37:19.174 INFO:journalctl@ceph.mon.a.smithi059.stdout: ** Compaction Stats [default] ** 2023-12-11T17:37:19.174 INFO:journalctl@ceph.mon.a.smithi059.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-12-11T17:37:19.174 INFO:journalctl@ceph.mon.a.smithi059.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-12-11T17:37:19.175 INFO:journalctl@ceph.mon.a.smithi059.stdout: L0 2/0 93.70 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 153.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-12-11T17:37:19.175 INFO:journalctl@ceph.mon.a.smithi059.stdout: Sum 2/0 93.70 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 153.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-12-11T17:37:19.175 INFO:journalctl@ceph.mon.a.smithi059.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 153.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-12-11T17:37:19.175 INFO:journalctl@ceph.mon.a.smithi059.stdout: 2023-12-11T17:37:19.175 INFO:journalctl@ceph.mon.a.smithi059.stdout: ** Compaction Stats [default] ** 2023-12-11T17:37:19.175 INFO:journalctl@ceph.mon.a.smithi059.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-12-11T17:37:19.175 INFO:journalctl@ceph.mon.a.smithi059.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-12-11T17:37:19.175 INFO:journalctl@ceph.mon.a.smithi059.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 153.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-12-11T17:37:19.175 INFO:journalctl@ceph.mon.a.smithi059.stdout: 2023-12-11T17:37:19.175 INFO:journalctl@ceph.mon.a.smithi059.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-12-11T17:37:19.175 INFO:journalctl@ceph.mon.a.smithi059.stdout: 2023-12-11T17:37:19.175 INFO:journalctl@ceph.mon.a.smithi059.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-12-11T17:37:19.175 INFO:journalctl@ceph.mon.a.smithi059.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-12-11T17:37:19.175 INFO:journalctl@ceph.mon.a.smithi059.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-12-11T17:37:19.175 INFO:journalctl@ceph.mon.a.smithi059.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-12-11T17:37:19.176 INFO:journalctl@ceph.mon.a.smithi059.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-12-11T17:37:19.176 INFO:journalctl@ceph.mon.a.smithi059.stdout: AddFile(Keys): cumulative 0, interval 0 2023-12-11T17:37:19.176 INFO:journalctl@ceph.mon.a.smithi059.stdout: Cumulative compaction: 0.00 GB write, 25.05 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-12-11T17:37:19.176 INFO:journalctl@ceph.mon.a.smithi059.stdout: Interval compaction: 0.00 GB write, 25.05 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-12-11T17:37:19.176 INFO:journalctl@ceph.mon.a.smithi059.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-12-11T17:37:19.176 INFO:journalctl@ceph.mon.a.smithi059.stdout: Block cache BinnedLRUCache@0x55a099aa1090#7 capacity: 512.00 MB usage: 1.36 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2023-12-11T17:37:19.176 INFO:journalctl@ceph.mon.a.smithi059.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2023-12-11T17:37:19.176 INFO:journalctl@ceph.mon.a.smithi059.stdout: 2023-12-11T17:37:19.176 INFO:journalctl@ceph.mon.a.smithi059.stdout: ** File Read Latency Histogram By Level [default] ** 2023-12-11T17:37:19.176 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: starting mon.a rank 0 at public addrs [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] at bind addrs [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid d0a07eaa-984b-11ee-95a3-87774f69a715 2023-12-11T17:37:19.176 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 systemd[1]: Started Ceph mon.a for d0a07eaa-984b-11ee-95a3-87774f69a715. 2023-12-11T17:37:19.176 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: mon.a@-1(???) e1 preinit fsid d0a07eaa-984b-11ee-95a3-87774f69a715 2023-12-11T17:37:19.176 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: mon.a@-1(???).mds e1 new map 2023-12-11T17:37:19.176 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: mon.a@-1(???).mds e1 print_map 2023-12-11T17:37:19.176 INFO:journalctl@ceph.mon.a.smithi059.stdout: e1 2023-12-11T17:37:19.177 INFO:journalctl@ceph.mon.a.smithi059.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-12-11T17:37:19.177 INFO:journalctl@ceph.mon.a.smithi059.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-12-11T17:37:19.177 INFO:journalctl@ceph.mon.a.smithi059.stdout: legacy client fscid: -1 2023-12-11T17:37:19.177 INFO:journalctl@ceph.mon.a.smithi059.stdout: 2023-12-11T17:37:19.177 INFO:journalctl@ceph.mon.a.smithi059.stdout: No filesystems configured 2023-12-11T17:37:19.177 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2023-12-11T17:37:19.177 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-12-11T17:37:19.177 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-12-11T17:37:19.177 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-12-11T17:37:19.177 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2023-12-11T17:37:19.177 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: mon.a is new leader, mons a in quorum (ranks 0) 2023-12-11T17:37:19.177 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: monmap e1: 1 mons at {a=[v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0]} removed_ranks: {} disallowed_leaders: {} 2023-12-11T17:37:19.177 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: fsmap 2023-12-11T17:37:19.177 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: osdmap e1: 0 total, 0 up, 0 in 2023-12-11T17:37:19.177 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:19 smithi059 ceph-mon[98847]: mgrmap e1: no daemons active 2023-12-11T17:37:20.175 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:19.619+0000 7f51ae8e7700 1 Processor -- start 2023-12-11T17:37:20.180 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:19.619+0000 7f51ae8e7700 1 -- start start 2023-12-11T17:37:20.180 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:19.620+0000 7f51ae8e7700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f51a8105c10 0x7f51a8106030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:37:20.180 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:19.620+0000 7f51ae8e7700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51a8106600 con 0x7f51a8105c10 2023-12-11T17:37:20.180 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:19.620+0000 7f51a7fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f51a8105c10 0x7f51a8106030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:37:20.180 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:19.620+0000 7f51a7fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f51a8105c10 0x7f51a8106030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59756/0 (socket says 172.21.15.59:59756) 2023-12-11T17:37:20.180 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:19.620+0000 7f51a7fff700 1 -- 172.21.15.59:0/2272129183 learned_addr learned my addr 172.21.15.59:0/2272129183 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:37:20.180 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:19.620+0000 7f51a7fff700 1 -- 172.21.15.59:0/2272129183 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51a8106e10 con 0x7f51a8105c10 2023-12-11T17:37:20.180 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:19.620+0000 7f51a7fff700 1 --2- 172.21.15.59:0/2272129183 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f51a8105c10 0x7f51a8106030 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f51980098a0 tx=0x7f5198009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f33a6ffe363766b2 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:37:20.181 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:19.621+0000 7f51a6ffd700 1 -- 172.21.15.59:0/2272129183 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5198004160 con 0x7f51a8105c10 2023-12-11T17:37:20.181 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:19.621+0000 7f51a6ffd700 1 -- 172.21.15.59:0/2272129183 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f51980042c0 con 0x7f51a8105c10 2023-12-11T17:37:20.181 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:19.621+0000 7f51a6ffd700 1 -- 172.21.15.59:0/2272129183 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5198011a00 con 0x7f51a8105c10 2023-12-11T17:37:20.181 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:19.621+0000 7f51ae8e7700 1 -- 172.21.15.59:0/2272129183 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f51a8105c10 msgr2=0x7f51a8106030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:37:20.181 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:19.621+0000 7f51ae8e7700 1 --2- 172.21.15.59:0/2272129183 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f51a8105c10 0x7f51a8106030 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f51980098a0 tx=0x7f5198009bb0 comp rx=0 tx=0).stop 2023-12-11T17:37:20.181 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:19.621+0000 7f51ae8e7700 1 -- 172.21.15.59:0/2272129183 shutdown_connections 2023-12-11T17:37:20.181 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:19.621+0000 7f51ae8e7700 1 --2- 172.21.15.59:0/2272129183 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f51a8105c10 0x7f51a8106030 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:37:20.181 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:19.621+0000 7f51ae8e7700 1 -- 172.21.15.59:0/2272129183 >> 172.21.15.59:0/2272129183 conn(0x7f51a81011b0 msgr2=0x7f51a81035f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:37:20.181 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:19.621+0000 7f51ae8e7700 1 -- 172.21.15.59:0/2272129183 shutdown_connections 2023-12-11T17:37:20.181 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:19.622+0000 7f51ae8e7700 1 -- 172.21.15.59:0/2272129183 wait complete. 2023-12-11T17:37:20.181 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:19.622+0000 7f51ae8e7700 1 Processor -- start 2023-12-11T17:37:20.182 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:19.622+0000 7f51ae8e7700 1 -- start start 2023-12-11T17:37:20.182 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:19.622+0000 7f51ae8e7700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f51a8105c10 0x7f51a8190e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:37:20.182 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:19.622+0000 7f51ae8e7700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51a818f3f0 con 0x7f51a8105c10 2023-12-11T17:37:20.182 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:19.623+0000 7f51a7fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f51a8105c10 0x7f51a8190e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:37:20.182 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:19.623+0000 7f51a7fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f51a8105c10 0x7f51a8190e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59758/0 (socket says 172.21.15.59:59758) 2023-12-11T17:37:20.182 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:19.623+0000 7f51a7fff700 1 -- 172.21.15.59:0/2266395794 learned_addr learned my addr 172.21.15.59:0/2266395794 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:37:20.182 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:19.623+0000 7f51a7fff700 1 -- 172.21.15.59:0/2266395794 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5198009550 con 0x7f51a8105c10 2023-12-11T17:37:20.182 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:19.623+0000 7f51a7fff700 1 --2- 172.21.15.59:0/2266395794 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f51a8105c10 0x7f51a8190e50 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f5198009130 tx=0x7f5198011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:37:20.182 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:19.623+0000 7f51a57fa700 1 -- 172.21.15.59:0/2266395794 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f51980038b0 con 0x7f51a8105c10 2023-12-11T17:37:20.182 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:19.623+0000 7f51a57fa700 1 -- 172.21.15.59:0/2266395794 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5198003a10 con 0x7f51a8105c10 2023-12-11T17:37:20.182 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:19.623+0000 7f51ae8e7700 1 -- 172.21.15.59:0/2266395794 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51a818f5f0 con 0x7f51a8105c10 2023-12-11T17:37:20.182 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:19.623+0000 7f51a57fa700 1 -- 172.21.15.59:0/2266395794 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5198019550 con 0x7f51a8105c10 2023-12-11T17:37:20.182 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:19.623+0000 7f51ae8e7700 1 -- 172.21.15.59:0/2266395794 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51a818fa90 con 0x7f51a8105c10 2023-12-11T17:37:20.182 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:19.623+0000 7f51a57fa700 1 -- 172.21.15.59:0/2266395794 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f5198019ae0 con 0x7f51a8105c10 2023-12-11T17:37:20.183 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:19.624+0000 7f51a57fa700 1 -- 172.21.15.59:0/2266395794 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f5198019d90 con 0x7f51a8105c10 2023-12-11T17:37:20.183 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:19.624+0000 7f51ae8e7700 1 -- 172.21.15.59:0/2266395794 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51a8194bf0 con 0x7f51a8105c10 2023-12-11T17:37:20.183 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:19.627+0000 7f51a57fa700 1 -- 172.21.15.59:0/2266395794 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f5198003b80 con 0x7f51a8105c10 2023-12-11T17:37:20.183 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:19.754+0000 7f51ae8e7700 1 -- 172.21.15.59:0/2266395794 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7f51a8067940 con 0x7f51a8105c10 2023-12-11T17:37:20.183 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:19.756+0000 7f51a57fa700 1 -- 172.21.15.59:0/2266395794 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7f519801a960 con 0x7f51a8105c10 2023-12-11T17:37:20.183 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:19.757+0000 7f51ae8e7700 1 -- 172.21.15.59:0/2266395794 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f51a8105c10 msgr2=0x7f51a8190e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:37:20.183 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:19.757+0000 7f51ae8e7700 1 --2- 172.21.15.59:0/2266395794 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f51a8105c10 0x7f51a8190e50 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f5198009130 tx=0x7f5198011ad0 comp rx=0 tx=0).stop 2023-12-11T17:37:20.183 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:19.757+0000 7f51ae8e7700 1 -- 172.21.15.59:0/2266395794 shutdown_connections 2023-12-11T17:37:20.183 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:19.757+0000 7f51ae8e7700 1 --2- 172.21.15.59:0/2266395794 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f51a8105c10 0x7f51a8190e50 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:37:20.183 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:19.757+0000 7f51ae8e7700 1 -- 172.21.15.59:0/2266395794 >> 172.21.15.59:0/2266395794 conn(0x7f51a81011b0 msgr2=0x7f51a8102110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:37:20.183 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:19.757+0000 7f51ae8e7700 1 -- 172.21.15.59:0/2266395794 shutdown_connections 2023-12-11T17:37:20.183 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:19.757+0000 7f51ae8e7700 1 -- 172.21.15.59:0/2266395794 wait complete. 2023-12-11T17:37:20.183 INFO:teuthology.orchestra.run.smithi059.stdout:Wrote config to /etc/ceph/ceph.conf 2023-12-11T17:37:20.295 INFO:teuthology.orchestra.run.smithi059.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2023-12-11T17:37:20.295 INFO:teuthology.orchestra.run.smithi059.stdout:Creating mgr... 2023-12-11T17:37:20.295 INFO:teuthology.orchestra.run.smithi059.stdout:Verifying port 0.0.0.0:9283 ... 2023-12-11T17:37:20.296 INFO:teuthology.orchestra.run.smithi059.stdout:Verifying port 0.0.0.0:8765 ... 2023-12-11T17:37:20.481 INFO:teuthology.orchestra.run.smithi059.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-d0a07eaa-984b-11ee-95a3-87774f69a715@mgr.x 2023-12-11T17:37:20.481 INFO:teuthology.orchestra.run.smithi059.stdout:systemctl: stderr Failed to reset failed state of unit ceph-d0a07eaa-984b-11ee-95a3-87774f69a715@mgr.x.service: Unit ceph-d0a07eaa-984b-11ee-95a3-87774f69a715@mgr.x.service not loaded. 2023-12-11T17:37:20.573 INFO:teuthology.orchestra.run.smithi059.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-d0a07eaa-984b-11ee-95a3-87774f69a715.target.wants/ceph-d0a07eaa-984b-11ee-95a3-87774f69a715@mgr.x.service → /etc/systemd/system/ceph-d0a07eaa-984b-11ee-95a3-87774f69a715@.service. 2023-12-11T17:37:20.759 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:37:20 smithi059 systemd[1]: Starting Ceph mgr.x for d0a07eaa-984b-11ee-95a3-87774f69a715... 2023-12-11T17:37:21.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:20 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/2266395794' entity='client.admin' 2023-12-11T17:37:21.155 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:37:20 smithi059 podman[99076]: 2023-12-11T17:37:21.282 INFO:teuthology.orchestra.run.smithi059.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-12-11T17:37:21.282 INFO:teuthology.orchestra.run.smithi059.stdout:systemctl: stdout disabled 2023-12-11T17:37:21.290 INFO:teuthology.orchestra.run.smithi059.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-12-11T17:37:21.291 INFO:teuthology.orchestra.run.smithi059.stdout:systemctl: stdout inactive 2023-12-11T17:37:21.291 INFO:teuthology.orchestra.run.smithi059.stdout:firewalld.service is not enabled 2023-12-11T17:37:21.291 INFO:teuthology.orchestra.run.smithi059.stdout:Not possible to enable service . firewalld.service is not available 2023-12-11T17:37:21.300 INFO:teuthology.orchestra.run.smithi059.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-12-11T17:37:21.300 INFO:teuthology.orchestra.run.smithi059.stdout:systemctl: stdout disabled 2023-12-11T17:37:21.309 INFO:teuthology.orchestra.run.smithi059.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-12-11T17:37:21.309 INFO:teuthology.orchestra.run.smithi059.stdout:systemctl: stdout inactive 2023-12-11T17:37:21.309 INFO:teuthology.orchestra.run.smithi059.stdout:firewalld.service is not enabled 2023-12-11T17:37:21.309 INFO:teuthology.orchestra.run.smithi059.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2023-12-11T17:37:21.309 INFO:teuthology.orchestra.run.smithi059.stdout:Waiting for mgr to start... 2023-12-11T17:37:21.309 INFO:teuthology.orchestra.run.smithi059.stdout:Waiting for mgr... 2023-12-11T17:37:21.461 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:37:21 smithi059 bash[99076]: ffa6f4d3275365494a8e96e6557db5f3044e541dea9bdcb647c46824932e1141 2023-12-11T17:37:21.461 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:37:21 smithi059 systemd[1]: Started Ceph mgr.x for d0a07eaa-984b-11ee-95a3-87774f69a715. 2023-12-11T17:37:21.461 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:37:21 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T17:37:21.361+0000 7f05e064e200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-11T17:37:21.461 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:37:21 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T17:37:21.457+0000 7f05e064e200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-11T17:37:21.899 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:37:21 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T17:37:21.572+0000 7f05e064e200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-11T17:37:21.899 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:37:21 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T17:37:21.684+0000 7f05e064e200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-11T17:37:21.899 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:37:21 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T17:37:21.896+0000 7f05e064e200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-11T17:37:22.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:21 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/1041141241' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-12-11T17:37:22.259 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout 2023-12-11T17:37:22.262 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout { 2023-12-11T17:37:22.262 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "fsid": "d0a07eaa-984b-11ee-95a3-87774f69a715", 2023-12-11T17:37:22.262 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "health": { 2023-12-11T17:37:22.262 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-12-11T17:37:22.262 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-12-11T17:37:22.262 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-12-11T17:37:22.262 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout }, 2023-12-11T17:37:22.262 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-12-11T17:37:22.263 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-12-11T17:37:22.263 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout 0 2023-12-11T17:37:22.263 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout ], 2023-12-11T17:37:22.263 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-12-11T17:37:22.263 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "a" 2023-12-11T17:37:22.263 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout ], 2023-12-11T17:37:22.263 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2023-12-11T17:37:22.263 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "monmap": { 2023-12-11T17:37:22.263 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-11T17:37:22.263 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-12-11T17:37:22.263 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-12-11T17:37:22.263 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout }, 2023-12-11T17:37:22.263 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-12-11T17:37:22.263 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-11T17:37:22.263 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-12-11T17:37:22.264 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-12-11T17:37:22.264 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-12-11T17:37:22.272 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-12-11T17:37:22.272 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-12-11T17:37:22.272 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-12-11T17:37:22.272 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout }, 2023-12-11T17:37:22.272 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-12-11T17:37:22.272 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-12-11T17:37:22.272 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-12-11T17:37:22.272 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-12-11T17:37:22.272 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-12-11T17:37:22.272 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-12-11T17:37:22.272 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-12-11T17:37:22.272 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-12-11T17:37:22.273 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-12-11T17:37:22.273 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout }, 2023-12-11T17:37:22.273 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-12-11T17:37:22.273 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-11T17:37:22.273 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-12-11T17:37:22.273 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-12-11T17:37:22.273 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout }, 2023-12-11T17:37:22.273 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-12-11T17:37:22.273 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "available": false, 2023-12-11T17:37:22.273 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-12-11T17:37:22.273 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "modules": [ 2023-12-11T17:37:22.273 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "iostat", 2023-12-11T17:37:22.273 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "nfs", 2023-12-11T17:37:22.274 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "restful" 2023-12-11T17:37:22.274 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout ], 2023-12-11T17:37:22.274 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "services": {} 2023-12-11T17:37:22.274 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout }, 2023-12-11T17:37:22.274 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-12-11T17:37:22.274 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-11T17:37:22.274 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "modified": "2023-12-11T17:37:14.527145+0000", 2023-12-11T17:37:22.274 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "services": {} 2023-12-11T17:37:22.274 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout }, 2023-12-11T17:37:22.274 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-12-11T17:37:22.274 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout } 2023-12-11T17:37:22.274 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:21.802+0000 7f9a17072700 1 Processor -- start 2023-12-11T17:37:22.274 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:21.802+0000 7f9a17072700 1 -- start start 2023-12-11T17:37:22.274 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:21.802+0000 7f9a17072700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9a10105be0 0x7f9a10106000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:37:22.275 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:21.802+0000 7f9a17072700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a101065d0 con 0x7f9a10105be0 2023-12-11T17:37:22.275 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:21.802+0000 7f9a14e0e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9a10105be0 0x7f9a10106000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:37:22.275 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:21.802+0000 7f9a14e0e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9a10105be0 0x7f9a10106000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59764/0 (socket says 172.21.15.59:59764) 2023-12-11T17:37:22.275 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:21.802+0000 7f9a14e0e700 1 -- 172.21.15.59:0/2951384839 learned_addr learned my addr 172.21.15.59:0/2951384839 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:37:22.275 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:21.803+0000 7f9a14e0e700 1 -- 172.21.15.59:0/2951384839 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a10106de0 con 0x7f9a10105be0 2023-12-11T17:37:22.275 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:21.803+0000 7f9a14e0e700 1 --2- 172.21.15.59:0/2951384839 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9a10105be0 0x7f9a10106000 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f9a040098a0 tx=0x7f9a04009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95c2ae39e9706fed server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:37:22.275 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:21.803+0000 7f9a0f7fe700 1 -- 172.21.15.59:0/2951384839 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9a04004160 con 0x7f9a10105be0 2023-12-11T17:37:22.275 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:21.803+0000 7f9a0f7fe700 1 -- 172.21.15.59:0/2951384839 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9a040042c0 con 0x7f9a10105be0 2023-12-11T17:37:22.275 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:21.803+0000 7f9a0f7fe700 1 -- 172.21.15.59:0/2951384839 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9a04011a00 con 0x7f9a10105be0 2023-12-11T17:37:22.275 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:21.804+0000 7f9a17072700 1 -- 172.21.15.59:0/2951384839 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9a10105be0 msgr2=0x7f9a10106000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:37:22.275 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:21.804+0000 7f9a17072700 1 --2- 172.21.15.59:0/2951384839 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9a10105be0 0x7f9a10106000 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f9a040098a0 tx=0x7f9a04009bb0 comp rx=0 tx=0).stop 2023-12-11T17:37:22.275 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:21.804+0000 7f9a17072700 1 -- 172.21.15.59:0/2951384839 shutdown_connections 2023-12-11T17:37:22.275 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:21.804+0000 7f9a17072700 1 --2- 172.21.15.59:0/2951384839 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9a10105be0 0x7f9a10106000 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:37:22.276 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:21.804+0000 7f9a17072700 1 -- 172.21.15.59:0/2951384839 >> 172.21.15.59:0/2951384839 conn(0x7f9a10101160 msgr2=0x7f9a101035c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:37:22.276 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:21.804+0000 7f9a17072700 1 -- 172.21.15.59:0/2951384839 shutdown_connections 2023-12-11T17:37:22.277 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:21.804+0000 7f9a17072700 1 -- 172.21.15.59:0/2951384839 wait complete. 2023-12-11T17:37:22.277 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:21.804+0000 7f9a17072700 1 Processor -- start 2023-12-11T17:37:22.277 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:21.805+0000 7f9a17072700 1 -- start start 2023-12-11T17:37:22.277 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:21.805+0000 7f9a17072700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9a10105be0 0x7f9a1019ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:37:22.277 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:21.805+0000 7f9a17072700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a1019d350 con 0x7f9a10105be0 2023-12-11T17:37:22.277 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:21.805+0000 7f9a14e0e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9a10105be0 0x7f9a1019ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:37:22.277 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:21.805+0000 7f9a14e0e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9a10105be0 0x7f9a1019ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59766/0 (socket says 172.21.15.59:59766) 2023-12-11T17:37:22.278 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:21.805+0000 7f9a14e0e700 1 -- 172.21.15.59:0/1041141241 learned_addr learned my addr 172.21.15.59:0/1041141241 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:37:22.278 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:21.805+0000 7f9a14e0e700 1 -- 172.21.15.59:0/1041141241 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a04009550 con 0x7f9a10105be0 2023-12-11T17:37:22.278 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:21.805+0000 7f9a14e0e700 1 --2- 172.21.15.59:0/1041141241 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9a10105be0 0x7f9a1019ce10 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9a04009130 tx=0x7f9a04011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:37:22.278 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:21.805+0000 7f9a0dffb700 1 -- 172.21.15.59:0/1041141241 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9a04003ab0 con 0x7f9a10105be0 2023-12-11T17:37:22.278 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:21.806+0000 7f9a0dffb700 1 -- 172.21.15.59:0/1041141241 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9a04003c10 con 0x7f9a10105be0 2023-12-11T17:37:22.278 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:21.806+0000 7f9a17072700 1 -- 172.21.15.59:0/1041141241 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a1019d550 con 0x7f9a10105be0 2023-12-11T17:37:22.278 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:21.806+0000 7f9a0dffb700 1 -- 172.21.15.59:0/1041141241 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9a040247d0 con 0x7f9a10105be0 2023-12-11T17:37:22.278 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:21.806+0000 7f9a17072700 1 -- 172.21.15.59:0/1041141241 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a1019e010 con 0x7f9a10105be0 2023-12-11T17:37:22.278 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:21.806+0000 7f9a0dffb700 1 -- 172.21.15.59:0/1041141241 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f9a04024d50 con 0x7f9a10105be0 2023-12-11T17:37:22.278 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:21.806+0000 7f9a0dffb700 1 -- 172.21.15.59:0/1041141241 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f9a040193c0 con 0x7f9a10105be0 2023-12-11T17:37:22.278 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:21.807+0000 7f9a17072700 1 -- 172.21.15.59:0/1041141241 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a1019d9f0 con 0x7f9a10105be0 2023-12-11T17:37:22.278 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:21.809+0000 7f9a0dffb700 1 -- 172.21.15.59:0/1041141241 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f9a1019d9f0 con 0x7f9a10105be0 2023-12-11T17:37:22.278 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:21.943+0000 7f9a17072700 1 -- 172.21.15.59:0/1041141241 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f9a10067940 con 0x7f9a10105be0 2023-12-11T17:37:22.279 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:21.944+0000 7f9a0dffb700 1 -- 172.21.15.59:0/1041141241 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f9a0401abb0 con 0x7f9a10105be0 2023-12-11T17:37:22.279 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:21.945+0000 7f9a17072700 1 -- 172.21.15.59:0/1041141241 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9a10105be0 msgr2=0x7f9a1019ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:37:22.279 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:21.945+0000 7f9a17072700 1 --2- 172.21.15.59:0/1041141241 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9a10105be0 0x7f9a1019ce10 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9a04009130 tx=0x7f9a04011ad0 comp rx=0 tx=0).stop 2023-12-11T17:37:22.279 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:21.945+0000 7f9a17072700 1 -- 172.21.15.59:0/1041141241 shutdown_connections 2023-12-11T17:37:22.279 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:21.945+0000 7f9a17072700 1 --2- 172.21.15.59:0/1041141241 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9a10105be0 0x7f9a1019ce10 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:37:22.279 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:21.945+0000 7f9a17072700 1 -- 172.21.15.59:0/1041141241 >> 172.21.15.59:0/1041141241 conn(0x7f9a10101160 msgr2=0x7f9a101020a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:37:22.279 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:21.945+0000 7f9a17072700 1 -- 172.21.15.59:0/1041141241 shutdown_connections 2023-12-11T17:37:22.279 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:21.945+0000 7f9a17072700 1 -- 172.21.15.59:0/1041141241 wait complete. 2023-12-11T17:37:22.279 INFO:teuthology.orchestra.run.smithi059.stdout:mgr not available, waiting (1/15)... 2023-12-11T17:37:22.646 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:37:22 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T17:37:22.383+0000 7f05e064e200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-11T17:37:22.646 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:37:22 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T17:37:22.535+0000 7f05e064e200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-11T17:37:22.646 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:37:22 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T17:37:22.642+0000 7f05e064e200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-11T17:37:22.904 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:37:22 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T17:37:22.757+0000 7f05e064e200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-11T17:37:23.654 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:37:23 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T17:37:23.269+0000 7f05e064e200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-11T17:37:24.404 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:37:24 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T17:37:24.067+0000 7f05e064e200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-11T17:37:24.405 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:37:24 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T17:37:24.165+0000 7f05e064e200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-11T17:37:25.404 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:37:25 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T17:37:25.147+0000 7f05e064e200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-11T17:37:25.404 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:37:25 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T17:37:25.258+0000 7f05e064e200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-11T17:37:25.904 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:37:25 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T17:37:25.481+0000 7f05e064e200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-11T17:37:26.275 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:37:25 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T17:37:25.982+0000 7f05e064e200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-11T17:37:26.275 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:37:26 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T17:37:26.088+0000 7f05e064e200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-11T17:37:26.612 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:37:26 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T17:37:26.272+0000 7f05e064e200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-11T17:37:26.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:26 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/2216981070' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-12-11T17:37:26.904 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:37:26 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T17:37:26.608+0000 7f05e064e200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-11T17:37:27.654 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:37:27 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T17:37:27.450+0000 7f05e064e200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-11T17:37:27.654 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:37:27 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T17:37:27.607+0000 7f05e064e200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-11T17:37:28.154 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:37:27 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T17:37:27.854+0000 7f05e064e200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-11T17:37:28.154 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:37:27 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T17:37:27.990+0000 7f05e064e200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-11T17:37:28.654 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:37:28 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T17:37:28.264+0000 7f05e064e200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-11T17:37:29.024 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout 2023-12-11T17:37:29.024 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout { 2023-12-11T17:37:29.024 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "fsid": "d0a07eaa-984b-11ee-95a3-87774f69a715", 2023-12-11T17:37:29.024 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "health": { 2023-12-11T17:37:29.024 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-12-11T17:37:29.024 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-12-11T17:37:29.024 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-12-11T17:37:29.024 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout }, 2023-12-11T17:37:29.024 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-12-11T17:37:29.024 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-12-11T17:37:29.025 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout 0 2023-12-11T17:37:29.025 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout ], 2023-12-11T17:37:29.025 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-12-11T17:37:29.025 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "a" 2023-12-11T17:37:29.025 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout ], 2023-12-11T17:37:29.025 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "quorum_age": 7, 2023-12-11T17:37:29.025 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "monmap": { 2023-12-11T17:37:29.025 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-11T17:37:29.025 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-12-11T17:37:29.025 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-12-11T17:37:29.025 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout }, 2023-12-11T17:37:29.025 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-12-11T17:37:29.025 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-11T17:37:29.025 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-12-11T17:37:29.025 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-12-11T17:37:29.025 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-12-11T17:37:29.034 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-12-11T17:37:29.034 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-12-11T17:37:29.034 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-12-11T17:37:29.034 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout }, 2023-12-11T17:37:29.034 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-12-11T17:37:29.035 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-12-11T17:37:29.035 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-12-11T17:37:29.035 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-12-11T17:37:29.035 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-12-11T17:37:29.035 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-12-11T17:37:29.035 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-12-11T17:37:29.035 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-12-11T17:37:29.035 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-12-11T17:37:29.035 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout }, 2023-12-11T17:37:29.035 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-12-11T17:37:29.035 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-11T17:37:29.035 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-12-11T17:37:29.035 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-12-11T17:37:29.035 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout }, 2023-12-11T17:37:29.035 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-12-11T17:37:29.036 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "available": false, 2023-12-11T17:37:29.036 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-12-11T17:37:29.036 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "modules": [ 2023-12-11T17:37:29.036 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "iostat", 2023-12-11T17:37:29.036 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "nfs", 2023-12-11T17:37:29.036 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "restful" 2023-12-11T17:37:29.036 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout ], 2023-12-11T17:37:29.036 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "services": {} 2023-12-11T17:37:29.036 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout }, 2023-12-11T17:37:29.036 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-12-11T17:37:29.036 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-11T17:37:29.036 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "modified": "2023-12-11T17:37:14.527145+0000", 2023-12-11T17:37:29.036 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "services": {} 2023-12-11T17:37:29.037 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout }, 2023-12-11T17:37:29.037 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-12-11T17:37:29.037 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout } 2023-12-11T17:37:29.037 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:26.594+0000 7fac9027f700 1 Processor -- start 2023-12-11T17:37:29.037 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:26.594+0000 7fac9027f700 1 -- start start 2023-12-11T17:37:29.037 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:26.594+0000 7fac9027f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac88105000 0x7fac88105420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:37:29.037 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:26.594+0000 7fac9027f700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac881059f0 con 0x7fac88105000 2023-12-11T17:37:29.037 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:26.594+0000 7fac8e01b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac88105000 0x7fac88105420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:37:29.038 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:26.594+0000 7fac8e01b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac88105000 0x7fac88105420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59768/0 (socket says 172.21.15.59:59768) 2023-12-11T17:37:29.038 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:26.594+0000 7fac8e01b700 1 -- 172.21.15.59:0/1445707193 learned_addr learned my addr 172.21.15.59:0/1445707193 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:37:29.038 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:26.595+0000 7fac8e01b700 1 -- 172.21.15.59:0/1445707193 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac88107f90 con 0x7fac88105000 2023-12-11T17:37:29.038 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:26.595+0000 7fac8e01b700 1 --2- 172.21.15.59:0/1445707193 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac88105000 0x7fac88105420 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fac840098a0 tx=0x7fac84009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=576b9e555737ac server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:37:29.038 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:26.595+0000 7fac8d019700 1 -- 172.21.15.59:0/1445707193 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fac84004160 con 0x7fac88105000 2023-12-11T17:37:29.038 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:26.595+0000 7fac8d019700 1 -- 172.21.15.59:0/1445707193 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fac840042c0 con 0x7fac88105000 2023-12-11T17:37:29.039 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:26.595+0000 7fac8d019700 1 -- 172.21.15.59:0/1445707193 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fac84011a00 con 0x7fac88105000 2023-12-11T17:37:29.039 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:26.596+0000 7fac9027f700 1 -- 172.21.15.59:0/1445707193 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac88105000 msgr2=0x7fac88105420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:37:29.039 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:26.596+0000 7fac9027f700 1 --2- 172.21.15.59:0/1445707193 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac88105000 0x7fac88105420 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fac840098a0 tx=0x7fac84009bb0 comp rx=0 tx=0).stop 2023-12-11T17:37:29.039 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:26.596+0000 7fac9027f700 1 -- 172.21.15.59:0/1445707193 shutdown_connections 2023-12-11T17:37:29.039 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:26.596+0000 7fac9027f700 1 --2- 172.21.15.59:0/1445707193 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac88105000 0x7fac88105420 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:37:29.039 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:26.596+0000 7fac9027f700 1 -- 172.21.15.59:0/1445707193 >> 172.21.15.59:0/1445707193 conn(0x7fac88100970 msgr2=0x7fac88102d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:37:29.040 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:26.596+0000 7fac9027f700 1 -- 172.21.15.59:0/1445707193 shutdown_connections 2023-12-11T17:37:29.040 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:26.596+0000 7fac9027f700 1 -- 172.21.15.59:0/1445707193 wait complete. 2023-12-11T17:37:29.040 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:26.596+0000 7fac9027f700 1 Processor -- start 2023-12-11T17:37:29.040 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:26.596+0000 7fac9027f700 1 -- start start 2023-12-11T17:37:29.040 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:26.596+0000 7fac9027f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac88105000 0x7fac881079a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:37:29.040 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:26.596+0000 7fac9027f700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac88105f40 con 0x7fac88105000 2023-12-11T17:37:29.040 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:26.597+0000 7fac8e01b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac88105000 0x7fac881079a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:37:29.040 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:26.597+0000 7fac8e01b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac88105000 0x7fac881079a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59770/0 (socket says 172.21.15.59:59770) 2023-12-11T17:37:29.040 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:26.597+0000 7fac8e01b700 1 -- 172.21.15.59:0/2216981070 learned_addr learned my addr 172.21.15.59:0/2216981070 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:37:29.041 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:26.597+0000 7fac8e01b700 1 -- 172.21.15.59:0/2216981070 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac84009550 con 0x7fac88105000 2023-12-11T17:37:29.041 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:26.597+0000 7fac8e01b700 1 --2- 172.21.15.59:0/2216981070 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac88105000 0x7fac881079a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fac84009fb0 tx=0x7fac84011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:37:29.041 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:26.597+0000 7fac7f7fe700 1 -- 172.21.15.59:0/2216981070 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fac84003a40 con 0x7fac88105000 2023-12-11T17:37:29.041 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:26.597+0000 7fac7f7fe700 1 -- 172.21.15.59:0/2216981070 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fac84003ba0 con 0x7fac88105000 2023-12-11T17:37:29.041 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:26.597+0000 7fac7f7fe700 1 -- 172.21.15.59:0/2216981070 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fac8401a750 con 0x7fac88105000 2023-12-11T17:37:29.041 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:26.597+0000 7fac9027f700 1 -- 172.21.15.59:0/2216981070 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac88106140 con 0x7fac88105000 2023-12-11T17:37:29.041 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:26.597+0000 7fac9027f700 1 -- 172.21.15.59:0/2216981070 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac881065e0 con 0x7fac88105000 2023-12-11T17:37:29.041 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:26.598+0000 7fac7f7fe700 1 -- 172.21.15.59:0/2216981070 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fac8401ac80 con 0x7fac88105000 2023-12-11T17:37:29.042 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:26.598+0000 7fac7f7fe700 1 -- 172.21.15.59:0/2216981070 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fac840193c0 con 0x7fac88105000 2023-12-11T17:37:29.042 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:26.599+0000 7fac9027f700 1 -- 172.21.15.59:0/2216981070 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac88062c00 con 0x7fac88105000 2023-12-11T17:37:29.042 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:26.601+0000 7fac7f7fe700 1 -- 172.21.15.59:0/2216981070 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7fac8402de20 con 0x7fac88105000 2023-12-11T17:37:29.042 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:26.735+0000 7fac9027f700 1 -- 172.21.15.59:0/2216981070 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fac88067940 con 0x7fac88105000 2023-12-11T17:37:29.042 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:26.735+0000 7fac7f7fe700 1 -- 172.21.15.59:0/2216981070 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7fac84024b60 con 0x7fac88105000 2023-12-11T17:37:29.042 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:26.737+0000 7fac9027f700 1 -- 172.21.15.59:0/2216981070 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac88105000 msgr2=0x7fac881079a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:37:29.042 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:26.737+0000 7fac9027f700 1 --2- 172.21.15.59:0/2216981070 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac88105000 0x7fac881079a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fac84009fb0 tx=0x7fac84011ad0 comp rx=0 tx=0).stop 2023-12-11T17:37:29.042 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:26.737+0000 7fac9027f700 1 -- 172.21.15.59:0/2216981070 shutdown_connections 2023-12-11T17:37:29.043 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:26.737+0000 7fac9027f700 1 --2- 172.21.15.59:0/2216981070 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac88105000 0x7fac881079a0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:37:29.043 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:26.737+0000 7fac9027f700 1 -- 172.21.15.59:0/2216981070 >> 172.21.15.59:0/2216981070 conn(0x7fac88100970 msgr2=0x7fac881014b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:37:29.043 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:26.737+0000 7fac9027f700 1 -- 172.21.15.59:0/2216981070 shutdown_connections 2023-12-11T17:37:29.043 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:26.737+0000 7fac9027f700 1 -- 172.21.15.59:0/2216981070 wait complete. 2023-12-11T17:37:29.043 INFO:teuthology.orchestra.run.smithi059.stdout:mgr not available, waiting (2/15)... 2023-12-11T17:37:29.654 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:37:29 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T17:37:29.314+0000 7f05e064e200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-11T17:37:30.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:30 smithi059 ceph-mon[98847]: Activating manager daemon x 2023-12-11T17:37:30.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:30 smithi059 ceph-mon[98847]: mgrmap e2: x(active, starting, since 0.00296886s) 2023-12-11T17:37:30.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:30 smithi059 ceph-mon[98847]: from='mgr.14100 172.21.15.59:0/2767400748' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-12-11T17:37:30.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:30 smithi059 ceph-mon[98847]: from='mgr.14100 172.21.15.59:0/2767400748' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-12-11T17:37:30.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:30 smithi059 ceph-mon[98847]: from='mgr.14100 172.21.15.59:0/2767400748' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-12-11T17:37:30.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:30 smithi059 ceph-mon[98847]: from='mgr.14100 172.21.15.59:0/2767400748' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-11T17:37:30.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:30 smithi059 ceph-mon[98847]: from='mgr.14100 172.21.15.59:0/2767400748' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-12-11T17:37:30.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:30 smithi059 ceph-mon[98847]: Manager daemon x is now available 2023-12-11T17:37:30.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:30 smithi059 ceph-mon[98847]: from='mgr.14100 172.21.15.59:0/2767400748' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-12-11T17:37:30.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:30 smithi059 ceph-mon[98847]: from='mgr.14100 172.21.15.59:0/2767400748' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-12-11T17:37:30.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:30 smithi059 ceph-mon[98847]: from='mgr.14100 172.21.15.59:0/2767400748' entity='mgr.x' 2023-12-11T17:37:30.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:30 smithi059 ceph-mon[98847]: from='mgr.14100 172.21.15.59:0/2767400748' entity='mgr.x' 2023-12-11T17:37:30.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:30 smithi059 ceph-mon[98847]: from='mgr.14100 172.21.15.59:0/2767400748' entity='mgr.x' 2023-12-11T17:37:31.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:31 smithi059 ceph-mon[98847]: mgrmap e3: x(active, since 1.01005s) 2023-12-11T17:37:33.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:33 smithi059 ceph-mon[98847]: mgrmap e4: x(active, since 3s) 2023-12-11T17:37:34.381 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout 2023-12-11T17:37:34.384 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout { 2023-12-11T17:37:34.384 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "fsid": "d0a07eaa-984b-11ee-95a3-87774f69a715", 2023-12-11T17:37:34.384 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "health": { 2023-12-11T17:37:34.384 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-12-11T17:37:34.384 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-12-11T17:37:34.384 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-12-11T17:37:34.384 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout }, 2023-12-11T17:37:34.384 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-12-11T17:37:34.384 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-12-11T17:37:34.384 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout 0 2023-12-11T17:37:34.384 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout ], 2023-12-11T17:37:34.385 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-12-11T17:37:34.385 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "a" 2023-12-11T17:37:34.385 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout ], 2023-12-11T17:37:34.385 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "quorum_age": 14, 2023-12-11T17:37:34.385 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "monmap": { 2023-12-11T17:37:34.385 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-11T17:37:34.385 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-12-11T17:37:34.385 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-12-11T17:37:34.385 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout }, 2023-12-11T17:37:34.385 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-12-11T17:37:34.385 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-11T17:37:34.385 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-12-11T17:37:34.385 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-12-11T17:37:34.385 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-12-11T17:37:34.393 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-12-11T17:37:34.393 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-12-11T17:37:34.393 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-12-11T17:37:34.394 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout }, 2023-12-11T17:37:34.394 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-12-11T17:37:34.394 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-12-11T17:37:34.394 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-12-11T17:37:34.394 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-12-11T17:37:34.394 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-12-11T17:37:34.394 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-12-11T17:37:34.394 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-12-11T17:37:34.394 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-12-11T17:37:34.394 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-12-11T17:37:34.394 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout }, 2023-12-11T17:37:34.394 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-12-11T17:37:34.394 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-11T17:37:34.394 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-12-11T17:37:34.394 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-12-11T17:37:34.395 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout }, 2023-12-11T17:37:34.395 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-12-11T17:37:34.395 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "available": true, 2023-12-11T17:37:34.395 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-12-11T17:37:34.395 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "modules": [ 2023-12-11T17:37:34.395 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "iostat", 2023-12-11T17:37:34.395 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "nfs", 2023-12-11T17:37:34.395 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "restful" 2023-12-11T17:37:34.395 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout ], 2023-12-11T17:37:34.395 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "services": {} 2023-12-11T17:37:34.395 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout }, 2023-12-11T17:37:34.395 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-12-11T17:37:34.395 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-11T17:37:34.395 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "modified": "2023-12-11T17:37:14.527145+0000", 2023-12-11T17:37:34.395 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "services": {} 2023-12-11T17:37:34.395 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout }, 2023-12-11T17:37:34.396 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-12-11T17:37:34.396 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout } 2023-12-11T17:37:34.396 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:33.345+0000 7fc8d6323700 1 Processor -- start 2023-12-11T17:37:34.396 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:33.345+0000 7fc8d6323700 1 -- start start 2023-12-11T17:37:34.396 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:33.345+0000 7fc8d6323700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc8d0105be0 0x7fc8d0106000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:37:34.396 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:33.345+0000 7fc8d6323700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8d01065d0 con 0x7fc8d0105be0 2023-12-11T17:37:34.396 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:33.346+0000 7fc8cffff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc8d0105be0 0x7fc8d0106000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:37:34.396 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:33.346+0000 7fc8cffff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc8d0105be0 0x7fc8d0106000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59796/0 (socket says 172.21.15.59:59796) 2023-12-11T17:37:34.396 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:33.346+0000 7fc8cffff700 1 -- 172.21.15.59:0/456570504 learned_addr learned my addr 172.21.15.59:0/456570504 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:37:34.396 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:33.346+0000 7fc8cffff700 1 -- 172.21.15.59:0/456570504 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8d0106de0 con 0x7fc8d0105be0 2023-12-11T17:37:34.396 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:33.346+0000 7fc8cffff700 1 --2- 172.21.15.59:0/456570504 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc8d0105be0 0x7fc8d0106000 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc8c00098a0 tx=0x7fc8c0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7796d60ac150a6e server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:37:34.396 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:33.347+0000 7fc8ceffd700 1 -- 172.21.15.59:0/456570504 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc8c0004160 con 0x7fc8d0105be0 2023-12-11T17:37:34.396 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:33.347+0000 7fc8ceffd700 1 -- 172.21.15.59:0/456570504 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc8c00042c0 con 0x7fc8d0105be0 2023-12-11T17:37:34.396 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:33.347+0000 7fc8ceffd700 1 -- 172.21.15.59:0/456570504 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc8c0011a00 con 0x7fc8d0105be0 2023-12-11T17:37:34.396 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:33.347+0000 7fc8d6323700 1 -- 172.21.15.59:0/456570504 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc8d0105be0 msgr2=0x7fc8d0106000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:37:34.397 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:33.347+0000 7fc8d6323700 1 --2- 172.21.15.59:0/456570504 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc8d0105be0 0x7fc8d0106000 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc8c00098a0 tx=0x7fc8c0009bb0 comp rx=0 tx=0).stop 2023-12-11T17:37:34.397 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:33.347+0000 7fc8d6323700 1 -- 172.21.15.59:0/456570504 shutdown_connections 2023-12-11T17:37:34.397 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:33.347+0000 7fc8d6323700 1 --2- 172.21.15.59:0/456570504 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc8d0105be0 0x7fc8d0106000 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:37:34.397 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:33.347+0000 7fc8d6323700 1 -- 172.21.15.59:0/456570504 >> 172.21.15.59:0/456570504 conn(0x7fc8d0101160 msgr2=0x7fc8d01035c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:37:34.397 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:33.347+0000 7fc8d6323700 1 -- 172.21.15.59:0/456570504 shutdown_connections 2023-12-11T17:37:34.397 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:33.347+0000 7fc8d6323700 1 -- 172.21.15.59:0/456570504 wait complete. 2023-12-11T17:37:34.397 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:33.348+0000 7fc8d6323700 1 Processor -- start 2023-12-11T17:37:34.397 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:33.348+0000 7fc8d6323700 1 -- start start 2023-12-11T17:37:34.397 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:33.348+0000 7fc8d6323700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc8d0105be0 0x7fc8d019d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:37:34.397 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:33.348+0000 7fc8d6323700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8d007ab90 con 0x7fc8d0105be0 2023-12-11T17:37:34.397 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:33.348+0000 7fc8cffff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc8d0105be0 0x7fc8d019d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:37:34.397 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:33.349+0000 7fc8cffff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc8d0105be0 0x7fc8d019d780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59798/0 (socket says 172.21.15.59:59798) 2023-12-11T17:37:34.397 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:33.349+0000 7fc8cffff700 1 -- 172.21.15.59:0/2221415686 learned_addr learned my addr 172.21.15.59:0/2221415686 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:37:34.397 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:33.349+0000 7fc8cffff700 1 -- 172.21.15.59:0/2221415686 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8c0009550 con 0x7fc8d0105be0 2023-12-11T17:37:34.398 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:33.349+0000 7fc8cffff700 1 --2- 172.21.15.59:0/2221415686 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc8d0105be0 0x7fc8d019d780 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fc8c0009fb0 tx=0x7fc8c0011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:37:34.398 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:33.349+0000 7fc8cd7fa700 1 -- 172.21.15.59:0/2221415686 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc8c0003a40 con 0x7fc8d0105be0 2023-12-11T17:37:34.398 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:33.349+0000 7fc8cd7fa700 1 -- 172.21.15.59:0/2221415686 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc8c0003ba0 con 0x7fc8d0105be0 2023-12-11T17:37:34.398 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:33.349+0000 7fc8d6323700 1 -- 172.21.15.59:0/2221415686 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8d007ad90 con 0x7fc8d0105be0 2023-12-11T17:37:34.398 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:33.349+0000 7fc8cd7fa700 1 -- 172.21.15.59:0/2221415686 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc8c001a750 con 0x7fc8d0105be0 2023-12-11T17:37:34.398 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:33.349+0000 7fc8d6323700 1 -- 172.21.15.59:0/2221415686 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8d019de90 con 0x7fc8d0105be0 2023-12-11T17:37:34.398 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:33.350+0000 7fc8cd7fa700 1 -- 172.21.15.59:0/2221415686 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 4) v1 ==== 45569+0+0 (secure 0 0 0) 0x7fc8c001a9d0 con 0x7fc8d0105be0 2023-12-11T17:37:34.398 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:33.350+0000 7fc8d6323700 1 -- 172.21.15.59:0/2221415686 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8d0067940 con 0x7fc8d0105be0 2023-12-11T17:37:34.398 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:33.350+0000 7fc8cd7fa700 1 --2- 172.21.15.59:0/2221415686 >> [v2:172.21.15.59:6800/1672530462,v1:172.21.15.59:6801/1672530462] conn(0x7fc8b80387d0 0x7fc8b803ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:37:34.398 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:33.350+0000 7fc8cf7fe700 1 --2- 172.21.15.59:0/2221415686 >> [v2:172.21.15.59:6800/1672530462,v1:172.21.15.59:6801/1672530462] conn(0x7fc8b80387d0 0x7fc8b803ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:37:34.398 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:33.350+0000 7fc8cd7fa700 1 -- 172.21.15.59:0/2221415686 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fc8c004c4c0 con 0x7fc8d0105be0 2023-12-11T17:37:34.398 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:33.351+0000 7fc8cf7fe700 1 --2- 172.21.15.59:0/2221415686 >> [v2:172.21.15.59:6800/1672530462,v1:172.21.15.59:6801/1672530462] conn(0x7fc8b80387d0 0x7fc8b803ac90 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc8c4006fd0 tx=0x7fc8c4006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:37:34.398 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:33.356+0000 7fc8cd7fa700 1 -- 172.21.15.59:0/2221415686 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fc8c0018db0 con 0x7fc8d0105be0 2023-12-11T17:37:34.398 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:33.879+0000 7fc8d6323700 1 -- 172.21.15.59:0/2221415686 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fc8d010a100 con 0x7fc8d0105be0 2023-12-11T17:37:34.399 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:33.880+0000 7fc8cd7fa700 1 -- 172.21.15.59:0/2221415686 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7fc8c0016be0 con 0x7fc8d0105be0 2023-12-11T17:37:34.399 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:33.882+0000 7fc8d6323700 1 -- 172.21.15.59:0/2221415686 >> [v2:172.21.15.59:6800/1672530462,v1:172.21.15.59:6801/1672530462] conn(0x7fc8b80387d0 msgr2=0x7fc8b803ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:37:34.399 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:33.882+0000 7fc8d6323700 1 --2- 172.21.15.59:0/2221415686 >> [v2:172.21.15.59:6800/1672530462,v1:172.21.15.59:6801/1672530462] conn(0x7fc8b80387d0 0x7fc8b803ac90 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc8c4006fd0 tx=0x7fc8c4006e40 comp rx=0 tx=0).stop 2023-12-11T17:37:34.399 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:33.882+0000 7fc8d6323700 1 -- 172.21.15.59:0/2221415686 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc8d0105be0 msgr2=0x7fc8d019d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:37:34.399 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:33.882+0000 7fc8d6323700 1 --2- 172.21.15.59:0/2221415686 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc8d0105be0 0x7fc8d019d780 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fc8c0009fb0 tx=0x7fc8c0011ad0 comp rx=0 tx=0).stop 2023-12-11T17:37:34.399 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:33.882+0000 7fc8d6323700 1 -- 172.21.15.59:0/2221415686 shutdown_connections 2023-12-11T17:37:34.399 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:33.882+0000 7fc8d6323700 1 --2- 172.21.15.59:0/2221415686 >> [v2:172.21.15.59:6800/1672530462,v1:172.21.15.59:6801/1672530462] conn(0x7fc8b80387d0 0x7fc8b803ac90 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:37:34.399 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:33.882+0000 7fc8d6323700 1 --2- 172.21.15.59:0/2221415686 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc8d0105be0 0x7fc8d019d780 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:37:34.399 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:33.882+0000 7fc8d6323700 1 -- 172.21.15.59:0/2221415686 >> 172.21.15.59:0/2221415686 conn(0x7fc8d0101160 msgr2=0x7fc8d0101cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:37:34.399 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:33.882+0000 7fc8d6323700 1 -- 172.21.15.59:0/2221415686 shutdown_connections 2023-12-11T17:37:34.399 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:33.882+0000 7fc8d6323700 1 -- 172.21.15.59:0/2221415686 wait complete. 2023-12-11T17:37:34.399 INFO:teuthology.orchestra.run.smithi059.stdout:mgr is available 2023-12-11T17:37:34.561 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:34 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/2221415686' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-12-11T17:37:35.545 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:35 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/2307182074' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2023-12-11T17:37:35.635 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout 2023-12-11T17:37:35.636 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout [global] 2023-12-11T17:37:35.636 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout fsid = d0a07eaa-984b-11ee-95a3-87774f69a715 2023-12-11T17:37:35.636 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2023-12-11T17:37:35.636 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2023-12-11T17:37:35.636 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2023-12-11T17:37:35.636 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2023-12-11T17:37:35.636 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout 2023-12-11T17:37:35.636 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout [mgr] 2023-12-11T17:37:35.636 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2023-12-11T17:37:35.636 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout 2023-12-11T17:37:35.636 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout [osd] 2023-12-11T17:37:35.636 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2023-12-11T17:37:35.636 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2023-12-11T17:37:35.637 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2023-12-11T17:37:35.637 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2023-12-11T17:37:35.637 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2023-12-11T17:37:35.637 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:34.829+0000 7f2cb50c7700 1 Processor -- start 2023-12-11T17:37:35.637 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:34.829+0000 7f2cb50c7700 1 -- start start 2023-12-11T17:37:35.637 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:34.829+0000 7f2cb50c7700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2cb01077d0 0x7f2cb0109bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:37:35.637 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:34.829+0000 7f2cb50c7700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cb007ae20 con 0x7f2cb01077d0 2023-12-11T17:37:35.637 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:34.829+0000 7f2caed9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2cb01077d0 0x7f2cb0109bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:37:35.637 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:34.829+0000 7f2caed9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2cb01077d0 0x7f2cb0109bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59802/0 (socket says 172.21.15.59:59802) 2023-12-11T17:37:35.637 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:34.829+0000 7f2caed9d700 1 -- 172.21.15.59:0/4288459303 learned_addr learned my addr 172.21.15.59:0/4288459303 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:37:35.637 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:34.829+0000 7f2caed9d700 1 -- 172.21.15.59:0/4288459303 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cb010a100 con 0x7f2cb01077d0 2023-12-11T17:37:35.637 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:34.830+0000 7f2caed9d700 1 --2- 172.21.15.59:0/4288459303 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2cb01077d0 0x7f2cb0109bc0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2ca00098a0 tx=0x7f2ca0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23c5505ecf2a3ece server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:37:35.637 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:34.830+0000 7f2cadd9b700 1 -- 172.21.15.59:0/4288459303 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2ca0004160 con 0x7f2cb01077d0 2023-12-11T17:37:35.637 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:34.830+0000 7f2cadd9b700 1 -- 172.21.15.59:0/4288459303 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2ca00042c0 con 0x7f2cb01077d0 2023-12-11T17:37:35.638 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:34.830+0000 7f2cadd9b700 1 -- 172.21.15.59:0/4288459303 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2ca0011a00 con 0x7f2cb01077d0 2023-12-11T17:37:35.638 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:34.830+0000 7f2cb50c7700 1 -- 172.21.15.59:0/4288459303 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2cb01077d0 msgr2=0x7f2cb0109bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:37:35.638 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:34.830+0000 7f2cb50c7700 1 --2- 172.21.15.59:0/4288459303 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2cb01077d0 0x7f2cb0109bc0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2ca00098a0 tx=0x7f2ca0009bb0 comp rx=0 tx=0).stop 2023-12-11T17:37:35.638 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:34.830+0000 7f2cb50c7700 1 -- 172.21.15.59:0/4288459303 shutdown_connections 2023-12-11T17:37:35.638 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:34.830+0000 7f2cb50c7700 1 --2- 172.21.15.59:0/4288459303 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2cb01077d0 0x7f2cb0109bc0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:37:35.638 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:34.830+0000 7f2cb50c7700 1 -- 172.21.15.59:0/4288459303 >> 172.21.15.59:0/4288459303 conn(0x7f2cb01011c0 msgr2=0x7f2cb0103600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:37:35.638 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:34.830+0000 7f2cb50c7700 1 -- 172.21.15.59:0/4288459303 shutdown_connections 2023-12-11T17:37:35.638 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:34.830+0000 7f2cb50c7700 1 -- 172.21.15.59:0/4288459303 wait complete. 2023-12-11T17:37:35.638 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:34.831+0000 7f2cb50c7700 1 Processor -- start 2023-12-11T17:37:35.638 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:34.831+0000 7f2cb50c7700 1 -- start start 2023-12-11T17:37:35.638 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:34.831+0000 7f2cb50c7700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2cb01077d0 0x7f2cb019ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:37:35.638 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:34.831+0000 7f2cb50c7700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cb019b060 con 0x7f2cb01077d0 2023-12-11T17:37:35.638 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:34.832+0000 7f2caed9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2cb01077d0 0x7f2cb019ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:37:35.638 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:34.832+0000 7f2caed9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2cb01077d0 0x7f2cb019ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59804/0 (socket says 172.21.15.59:59804) 2023-12-11T17:37:35.639 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:34.832+0000 7f2caed9d700 1 -- 172.21.15.59:0/2307182074 learned_addr learned my addr 172.21.15.59:0/2307182074 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:37:35.639 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:34.832+0000 7f2caed9d700 1 -- 172.21.15.59:0/2307182074 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ca0009550 con 0x7f2cb01077d0 2023-12-11T17:37:35.639 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:34.832+0000 7f2caed9d700 1 --2- 172.21.15.59:0/2307182074 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2cb01077d0 0x7f2cb019ab20 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f2ca0009130 tx=0x7f2ca0011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:37:35.639 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:34.832+0000 7f2c97fff700 1 -- 172.21.15.59:0/2307182074 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2ca00038b0 con 0x7f2cb01077d0 2023-12-11T17:37:35.639 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:34.832+0000 7f2c97fff700 1 -- 172.21.15.59:0/2307182074 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2ca0003a10 con 0x7f2cb01077d0 2023-12-11T17:37:35.639 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:34.832+0000 7f2cb50c7700 1 -- 172.21.15.59:0/2307182074 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cb019b260 con 0x7f2cb01077d0 2023-12-11T17:37:35.641 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:34.832+0000 7f2c97fff700 1 -- 172.21.15.59:0/2307182074 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2ca0019550 con 0x7f2cb01077d0 2023-12-11T17:37:35.641 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:34.832+0000 7f2cb50c7700 1 -- 172.21.15.59:0/2307182074 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cb019bdc0 con 0x7f2cb01077d0 2023-12-11T17:37:35.641 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:34.833+0000 7f2c97fff700 1 -- 172.21.15.59:0/2307182074 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 4) v1 ==== 45569+0+0 (secure 0 0 0) 0x7f2ca0019810 con 0x7f2cb01077d0 2023-12-11T17:37:35.641 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:34.833+0000 7f2c97fff700 1 --2- 172.21.15.59:0/2307182074 >> [v2:172.21.15.59:6800/1672530462,v1:172.21.15.59:6801/1672530462] conn(0x7f2c98038760 0x7f2c9803ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:37:35.641 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:34.833+0000 7f2c97fff700 1 -- 172.21.15.59:0/2307182074 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f2ca004c5b0 con 0x7f2cb01077d0 2023-12-11T17:37:35.641 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:34.833+0000 7f2cb50c7700 1 -- 172.21.15.59:0/2307182074 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cb0062c00 con 0x7f2cb01077d0 2023-12-11T17:37:35.641 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:34.834+0000 7f2cae59c700 1 --2- 172.21.15.59:0/2307182074 >> [v2:172.21.15.59:6800/1672530462,v1:172.21.15.59:6801/1672530462] conn(0x7f2c98038760 0x7f2c9803ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:37:35.641 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:34.834+0000 7f2cae59c700 1 --2- 172.21.15.59:0/2307182074 >> [v2:172.21.15.59:6800/1672530462,v1:172.21.15.59:6801/1672530462] conn(0x7f2c98038760 0x7f2c9803ac20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2ca4006fd0 tx=0x7f2ca4006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:37:35.642 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:34.838+0000 7f2c97fff700 1 -- 172.21.15.59:0/2307182074 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f2ca0019ac0 con 0x7f2cb01077d0 2023-12-11T17:37:35.642 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:35.201+0000 7f2cb50c7700 1 -- 172.21.15.59:0/2307182074 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f2cb010b120 con 0x7f2cb01077d0 2023-12-11T17:37:35.642 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:35.202+0000 7f2c97fff700 1 -- 172.21.15.59:0/2307182074 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7f2ca0020240 con 0x7f2cb01077d0 2023-12-11T17:37:35.642 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:35.204+0000 7f2cb50c7700 1 -- 172.21.15.59:0/2307182074 >> [v2:172.21.15.59:6800/1672530462,v1:172.21.15.59:6801/1672530462] conn(0x7f2c98038760 msgr2=0x7f2c9803ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:37:35.642 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:35.204+0000 7f2cb50c7700 1 --2- 172.21.15.59:0/2307182074 >> [v2:172.21.15.59:6800/1672530462,v1:172.21.15.59:6801/1672530462] conn(0x7f2c98038760 0x7f2c9803ac20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2ca4006fd0 tx=0x7f2ca4006e40 comp rx=0 tx=0).stop 2023-12-11T17:37:35.642 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:35.204+0000 7f2cb50c7700 1 -- 172.21.15.59:0/2307182074 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2cb01077d0 msgr2=0x7f2cb019ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:37:35.642 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:35.204+0000 7f2cb50c7700 1 --2- 172.21.15.59:0/2307182074 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2cb01077d0 0x7f2cb019ab20 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f2ca0009130 tx=0x7f2ca0011ad0 comp rx=0 tx=0).stop 2023-12-11T17:37:35.642 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:35.204+0000 7f2cb50c7700 1 -- 172.21.15.59:0/2307182074 shutdown_connections 2023-12-11T17:37:35.642 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:35.204+0000 7f2cb50c7700 1 --2- 172.21.15.59:0/2307182074 >> [v2:172.21.15.59:6800/1672530462,v1:172.21.15.59:6801/1672530462] conn(0x7f2c98038760 0x7f2c9803ac20 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:37:35.642 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:35.205+0000 7f2cb50c7700 1 --2- 172.21.15.59:0/2307182074 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2cb01077d0 0x7f2cb019ab20 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:37:35.642 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:35.205+0000 7f2cb50c7700 1 -- 172.21.15.59:0/2307182074 >> 172.21.15.59:0/2307182074 conn(0x7f2cb01011c0 msgr2=0x7f2cb0101d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:37:35.642 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:35.205+0000 7f2cb50c7700 1 -- 172.21.15.59:0/2307182074 shutdown_connections 2023-12-11T17:37:35.642 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:35.205+0000 7f2cb50c7700 1 -- 172.21.15.59:0/2307182074 wait complete. 2023-12-11T17:37:35.642 INFO:teuthology.orchestra.run.smithi059.stdout:Enabling cephadm module... 2023-12-11T17:37:36.947 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:36 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/3550095653' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2023-12-11T17:37:36.947 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:37:36 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: ignoring --setuser ceph since I am not root 2023-12-11T17:37:36.947 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:37:36 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: ignoring --setgroup ceph since I am not root 2023-12-11T17:37:36.947 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:37:36 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T17:37:36.847+0000 7f7db7cb4200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-11T17:37:37.083 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:36.115+0000 7f15decc6700 1 Processor -- start 2023-12-11T17:37:37.083 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:36.115+0000 7f15decc6700 1 -- start start 2023-12-11T17:37:37.083 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:36.115+0000 7f15decc6700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f15d8105c10 0x7f15d8106030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:37:37.083 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:36.115+0000 7f15decc6700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15d8106600 con 0x7f15d8105c10 2023-12-11T17:37:37.083 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:36.115+0000 7f15dca62700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f15d8105c10 0x7f15d8106030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:37:37.083 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:36.115+0000 7f15dca62700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f15d8105c10 0x7f15d8106030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59808/0 (socket says 172.21.15.59:59808) 2023-12-11T17:37:37.083 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:36.115+0000 7f15dca62700 1 -- 172.21.15.59:0/474380877 learned_addr learned my addr 172.21.15.59:0/474380877 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:37:37.083 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:36.116+0000 7f15dca62700 1 -- 172.21.15.59:0/474380877 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15d8106e10 con 0x7f15d8105c10 2023-12-11T17:37:37.084 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:36.116+0000 7f15dca62700 1 --2- 172.21.15.59:0/474380877 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f15d8105c10 0x7f15d8106030 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f15cc00bb40 tx=0x7f15cc00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc5833ecb41469f5 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:37:37.084 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:36.116+0000 7f15d77fe700 1 -- 172.21.15.59:0/474380877 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f15cc00dbe0 con 0x7f15d8105c10 2023-12-11T17:37:37.084 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:36.116+0000 7f15d77fe700 1 -- 172.21.15.59:0/474380877 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f15cc0044d0 con 0x7f15d8105c10 2023-12-11T17:37:37.084 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:36.116+0000 7f15d77fe700 1 -- 172.21.15.59:0/474380877 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f15cc010470 con 0x7f15d8105c10 2023-12-11T17:37:37.084 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:36.117+0000 7f15decc6700 1 -- 172.21.15.59:0/474380877 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f15d8105c10 msgr2=0x7f15d8106030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:37:37.084 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:36.117+0000 7f15decc6700 1 --2- 172.21.15.59:0/474380877 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f15d8105c10 0x7f15d8106030 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f15cc00bb40 tx=0x7f15cc00be50 comp rx=0 tx=0).stop 2023-12-11T17:37:37.084 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:36.117+0000 7f15decc6700 1 -- 172.21.15.59:0/474380877 shutdown_connections 2023-12-11T17:37:37.084 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:36.117+0000 7f15decc6700 1 --2- 172.21.15.59:0/474380877 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f15d8105c10 0x7f15d8106030 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:37:37.084 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:36.117+0000 7f15decc6700 1 -- 172.21.15.59:0/474380877 >> 172.21.15.59:0/474380877 conn(0x7f15d81011b0 msgr2=0x7f15d81035f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:37:37.084 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:36.117+0000 7f15decc6700 1 -- 172.21.15.59:0/474380877 shutdown_connections 2023-12-11T17:37:37.084 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:36.117+0000 7f15decc6700 1 -- 172.21.15.59:0/474380877 wait complete. 2023-12-11T17:37:37.084 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:36.117+0000 7f15decc6700 1 Processor -- start 2023-12-11T17:37:37.084 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:36.117+0000 7f15decc6700 1 -- start start 2023-12-11T17:37:37.084 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:36.118+0000 7f15decc6700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f15d8105c10 0x7f15d8198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:37:37.085 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:36.118+0000 7f15decc6700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15d8198e60 con 0x7f15d8105c10 2023-12-11T17:37:37.085 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:36.118+0000 7f15dca62700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f15d8105c10 0x7f15d8198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:37:37.085 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:36.118+0000 7f15dca62700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f15d8105c10 0x7f15d8198920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59810/0 (socket says 172.21.15.59:59810) 2023-12-11T17:37:37.085 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:36.118+0000 7f15dca62700 1 -- 172.21.15.59:0/3550095653 learned_addr learned my addr 172.21.15.59:0/3550095653 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:37:37.085 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:36.118+0000 7f15dca62700 1 -- 172.21.15.59:0/3550095653 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15cc00b7f0 con 0x7f15d8105c10 2023-12-11T17:37:37.085 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:36.118+0000 7f15dca62700 1 --2- 172.21.15.59:0/3550095653 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f15d8105c10 0x7f15d8198920 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f15cc00bfd0 tx=0x7f15cc003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:37:37.085 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:36.118+0000 7f15d5ffb700 1 -- 172.21.15.59:0/3550095653 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f15cc01e070 con 0x7f15d8105c10 2023-12-11T17:37:37.085 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:36.118+0000 7f15d5ffb700 1 -- 172.21.15.59:0/3550095653 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f15cc0042a0 con 0x7f15d8105c10 2023-12-11T17:37:37.085 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:36.118+0000 7f15d5ffb700 1 -- 172.21.15.59:0/3550095653 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f15cc022ac0 con 0x7f15d8105c10 2023-12-11T17:37:37.085 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:36.118+0000 7f15decc6700 1 -- 172.21.15.59:0/3550095653 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15d8199060 con 0x7f15d8105c10 2023-12-11T17:37:37.085 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:36.119+0000 7f15decc6700 1 -- 172.21.15.59:0/3550095653 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15d8199bc0 con 0x7f15d8105c10 2023-12-11T17:37:37.085 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:36.119+0000 7f15d5ffb700 1 -- 172.21.15.59:0/3550095653 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 4) v1 ==== 45569+0+0 (secure 0 0 0) 0x7f15cc010730 con 0x7f15d8105c10 2023-12-11T17:37:37.085 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:36.120+0000 7f15d5ffb700 1 --2- 172.21.15.59:0/3550095653 >> [v2:172.21.15.59:6800/1672530462,v1:172.21.15.59:6801/1672530462] conn(0x7f15c80387d0 0x7f15c803ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:37:37.085 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:36.120+0000 7f15decc6700 1 -- 172.21.15.59:0/3550095653 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15d8199500 con 0x7f15d8105c10 2023-12-11T17:37:37.085 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:36.120+0000 7f15d5ffb700 1 -- 172.21.15.59:0/3550095653 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f15cc04ebf0 con 0x7f15d8105c10 2023-12-11T17:37:37.086 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:36.121+0000 7f15d7fff700 1 --2- 172.21.15.59:0/3550095653 >> [v2:172.21.15.59:6800/1672530462,v1:172.21.15.59:6801/1672530462] conn(0x7f15c80387d0 0x7f15c803ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:37:37.086 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:36.121+0000 7f15d7fff700 1 --2- 172.21.15.59:0/3550095653 >> [v2:172.21.15.59:6800/1672530462,v1:172.21.15.59:6801/1672530462] conn(0x7f15c80387d0 0x7f15c803ac90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f15c4006fd0 tx=0x7f15c4006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:37:37.086 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:36.125+0000 7f15d5ffb700 1 -- 172.21.15.59:0/3550095653 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f15d8199500 con 0x7f15d8105c10 2023-12-11T17:37:37.086 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:36.606+0000 7f15decc6700 1 -- 172.21.15.59:0/3550095653 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7f15d8199500 con 0x7f15d8105c10 2023-12-11T17:37:37.086 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:36.657+0000 7f15d5ffb700 1 -- 172.21.15.59:0/3550095653 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7f15d8199500 con 0x7f15d8105c10 2023-12-11T17:37:37.086 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:36.659+0000 7f15d5ffb700 1 -- 172.21.15.59:0/3550095653 <== mon.0 v2:172.21.15.59:3300/0 8 ==== mgrmap(e 5) v1 ==== 45580+0+0 (secure 0 0 0) 0x7f15cc020dc0 con 0x7f15d8105c10 2023-12-11T17:37:37.086 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:36.661+0000 7f15decc6700 1 -- 172.21.15.59:0/3550095653 >> [v2:172.21.15.59:6800/1672530462,v1:172.21.15.59:6801/1672530462] conn(0x7f15c80387d0 msgr2=0x7f15c803ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:37:37.086 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:36.661+0000 7f15decc6700 1 --2- 172.21.15.59:0/3550095653 >> [v2:172.21.15.59:6800/1672530462,v1:172.21.15.59:6801/1672530462] conn(0x7f15c80387d0 0x7f15c803ac90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f15c4006fd0 tx=0x7f15c4006e40 comp rx=0 tx=0).stop 2023-12-11T17:37:37.086 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:36.661+0000 7f15decc6700 1 -- 172.21.15.59:0/3550095653 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f15d8105c10 msgr2=0x7f15d8198920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:37:37.086 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:36.661+0000 7f15decc6700 1 --2- 172.21.15.59:0/3550095653 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f15d8105c10 0x7f15d8198920 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f15cc00bfd0 tx=0x7f15cc003d40 comp rx=0 tx=0).stop 2023-12-11T17:37:37.086 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:36.661+0000 7f15decc6700 1 -- 172.21.15.59:0/3550095653 shutdown_connections 2023-12-11T17:37:37.086 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:36.661+0000 7f15decc6700 1 --2- 172.21.15.59:0/3550095653 >> [v2:172.21.15.59:6800/1672530462,v1:172.21.15.59:6801/1672530462] conn(0x7f15c80387d0 0x7f15c803ac90 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:37:37.086 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:36.661+0000 7f15decc6700 1 --2- 172.21.15.59:0/3550095653 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f15d8105c10 0x7f15d8198920 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:37:37.087 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:36.661+0000 7f15decc6700 1 -- 172.21.15.59:0/3550095653 >> 172.21.15.59:0/3550095653 conn(0x7f15d81011b0 msgr2=0x7f15d818f5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:37:37.087 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:36.661+0000 7f15decc6700 1 -- 172.21.15.59:0/3550095653 shutdown_connections 2023-12-11T17:37:37.087 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:36.661+0000 7f15decc6700 1 -- 172.21.15.59:0/3550095653 wait complete. 2023-12-11T17:37:37.328 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:37:36 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T17:37:36.944+0000 7f7db7cb4200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-11T17:37:37.328 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:37:37 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T17:37:37.054+0000 7f7db7cb4200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-11T17:37:37.328 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:37:37 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T17:37:37.177+0000 7f7db7cb4200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-11T17:37:37.654 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:37:37 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T17:37:37.389+0000 7f7db7cb4200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-11T17:37:38.028 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:37 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/3550095653' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2023-12-11T17:37:38.028 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:37 smithi059 ceph-mon[98847]: mgrmap e5: x(active, since 7s) 2023-12-11T17:37:38.028 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:37:37 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T17:37:37.870+0000 7f7db7cb4200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-11T17:37:38.404 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:37:38 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T17:37:38.025+0000 7f7db7cb4200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-11T17:37:38.404 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:37:38 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T17:37:38.133+0000 7f7db7cb4200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-11T17:37:38.404 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:37:38 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T17:37:38.253+0000 7f7db7cb4200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-11T17:37:38.539 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout { 2023-12-11T17:37:38.539 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "epoch": 5, 2023-12-11T17:37:38.539 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "available": true, 2023-12-11T17:37:38.539 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "active_name": "x", 2023-12-11T17:37:38.539 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "num_standby": 0 2023-12-11T17:37:38.539 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout } 2023-12-11T17:37:38.539 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:37.633+0000 7f7aae846700 1 Processor -- start 2023-12-11T17:37:38.539 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:37.633+0000 7f7aae846700 1 -- start start 2023-12-11T17:37:38.540 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:37.633+0000 7f7aae846700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7aa8105be0 0x7f7aa8106000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:37:38.540 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:37.633+0000 7f7aae846700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aa81065d0 con 0x7f7aa8105be0 2023-12-11T17:37:38.540 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:37.633+0000 7f7aa7fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7aa8105be0 0x7f7aa8106000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:37:38.540 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:37.633+0000 7f7aa7fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7aa8105be0 0x7f7aa8106000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59824/0 (socket says 172.21.15.59:59824) 2023-12-11T17:37:38.540 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:37.633+0000 7f7aa7fff700 1 -- 172.21.15.59:0/373881269 learned_addr learned my addr 172.21.15.59:0/373881269 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:37:38.540 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:37.634+0000 7f7aa7fff700 1 -- 172.21.15.59:0/373881269 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7aa8106de0 con 0x7f7aa8105be0 2023-12-11T17:37:38.540 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:37.634+0000 7f7aa7fff700 1 --2- 172.21.15.59:0/373881269 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7aa8105be0 0x7f7aa8106000 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f7a980098a0 tx=0x7f7a98009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dca65236e432971 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:37:38.540 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:37.634+0000 7f7aa6ffd700 1 -- 172.21.15.59:0/373881269 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7a98004160 con 0x7f7aa8105be0 2023-12-11T17:37:38.540 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:37.634+0000 7f7aa6ffd700 1 -- 172.21.15.59:0/373881269 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7a980042c0 con 0x7f7aa8105be0 2023-12-11T17:37:38.540 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:37.634+0000 7f7aa6ffd700 1 -- 172.21.15.59:0/373881269 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7a98011a00 con 0x7f7aa8105be0 2023-12-11T17:37:38.540 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:37.634+0000 7f7aae846700 1 -- 172.21.15.59:0/373881269 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7aa8105be0 msgr2=0x7f7aa8106000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:37:38.540 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:37.634+0000 7f7aae846700 1 --2- 172.21.15.59:0/373881269 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7aa8105be0 0x7f7aa8106000 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f7a980098a0 tx=0x7f7a98009bb0 comp rx=0 tx=0).stop 2023-12-11T17:37:38.540 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:37.635+0000 7f7aae846700 1 -- 172.21.15.59:0/373881269 shutdown_connections 2023-12-11T17:37:38.540 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:37.635+0000 7f7aae846700 1 --2- 172.21.15.59:0/373881269 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7aa8105be0 0x7f7aa8106000 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:37:38.540 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:37.635+0000 7f7aae846700 1 -- 172.21.15.59:0/373881269 >> 172.21.15.59:0/373881269 conn(0x7f7aa8101160 msgr2=0x7f7aa81035c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:37:38.541 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:37.635+0000 7f7aae846700 1 -- 172.21.15.59:0/373881269 shutdown_connections 2023-12-11T17:37:38.541 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:37.635+0000 7f7aae846700 1 -- 172.21.15.59:0/373881269 wait complete. 2023-12-11T17:37:38.541 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:37.635+0000 7f7aae846700 1 Processor -- start 2023-12-11T17:37:38.541 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:37.635+0000 7f7aae846700 1 -- start start 2023-12-11T17:37:38.541 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:37.636+0000 7f7aae846700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7aa8105be0 0x7f7aa81988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:37:38.541 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:37.636+0000 7f7aae846700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aa8198e30 con 0x7f7aa8105be0 2023-12-11T17:37:38.541 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:37.636+0000 7f7aa7fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7aa8105be0 0x7f7aa81988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:37:38.541 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:37.636+0000 7f7aa7fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7aa8105be0 0x7f7aa81988f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59826/0 (socket says 172.21.15.59:59826) 2023-12-11T17:37:38.541 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:37.636+0000 7f7aa7fff700 1 -- 172.21.15.59:0/1588604542 learned_addr learned my addr 172.21.15.59:0/1588604542 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:37:38.541 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:37.636+0000 7f7aa7fff700 1 -- 172.21.15.59:0/1588604542 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a98009550 con 0x7f7aa8105be0 2023-12-11T17:37:38.541 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:37.636+0000 7f7aa7fff700 1 --2- 172.21.15.59:0/1588604542 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7aa8105be0 0x7f7aa81988f0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f7a98009130 tx=0x7f7a98011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:37:38.541 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:37.636+0000 7f7aa57fa700 1 -- 172.21.15.59:0/1588604542 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7a98003ab0 con 0x7f7aa8105be0 2023-12-11T17:37:38.541 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:37.636+0000 7f7aae846700 1 -- 172.21.15.59:0/1588604542 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7aa8199030 con 0x7f7aa8105be0 2023-12-11T17:37:38.541 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:37.636+0000 7f7aa57fa700 1 -- 172.21.15.59:0/1588604542 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7a98003c10 con 0x7f7aa8105be0 2023-12-11T17:37:38.542 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:37.636+0000 7f7aa57fa700 1 -- 172.21.15.59:0/1588604542 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7a980247d0 con 0x7f7aa8105be0 2023-12-11T17:37:38.542 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:37.637+0000 7f7aae846700 1 -- 172.21.15.59:0/1588604542 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7aa8199b90 con 0x7f7aa8105be0 2023-12-11T17:37:38.542 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:37.637+0000 7f7aa57fa700 1 -- 172.21.15.59:0/1588604542 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 45580+0+0 (secure 0 0 0) 0x7f7a9802d3f0 con 0x7f7aa8105be0 2023-12-11T17:37:38.542 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:37.638+0000 7f7aa57fa700 1 --2- 172.21.15.59:0/1588604542 >> [v2:172.21.15.59:6800/1672530462,v1:172.21.15.59:6801/1672530462] conn(0x7f7a9003aa60 0x7f7a9003cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:37:38.542 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:37.638+0000 7f7aae846700 1 -- 172.21.15.59:0/1588604542 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7aa81994d0 con 0x7f7aa8105be0 2023-12-11T17:37:38.542 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:37.638+0000 7f7aa57fa700 1 -- 172.21.15.59:0/1588604542 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f7a9804c5f0 con 0x7f7aa8105be0 2023-12-11T17:37:38.542 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:37.638+0000 7f7aa77fe700 1 -- 172.21.15.59:0/1588604542 >> [v2:172.21.15.59:6800/1672530462,v1:172.21.15.59:6801/1672530462] conn(0x7f7a9003aa60 msgr2=0x7f7a9003cf20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.59:6800/1672530462 2023-12-11T17:37:38.542 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:37.638+0000 7f7aa77fe700 1 --2- 172.21.15.59:0/1588604542 >> [v2:172.21.15.59:6800/1672530462,v1:172.21.15.59:6801/1672530462] conn(0x7f7a9003aa60 0x7f7a9003cf20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-11T17:37:38.542 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:37.643+0000 7f7aa57fa700 1 -- 172.21.15.59:0/1588604542 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f7aa81994d0 con 0x7f7aa8105be0 2023-12-11T17:37:38.542 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:37.839+0000 7f7aa77fe700 1 -- 172.21.15.59:0/1588604542 >> [v2:172.21.15.59:6800/1672530462,v1:172.21.15.59:6801/1672530462] conn(0x7f7a9003aa60 msgr2=0x7f7a9003cf20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.59:6800/1672530462 2023-12-11T17:37:38.542 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:37.839+0000 7f7aa77fe700 1 --2- 172.21.15.59:0/1588604542 >> [v2:172.21.15.59:6800/1672530462,v1:172.21.15.59:6801/1672530462] conn(0x7f7a9003aa60 0x7f7a9003cf20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-11T17:37:38.542 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:38.125+0000 7f7aae846700 1 -- 172.21.15.59:0/1588604542 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f7aa81994d0 con 0x7f7aa8105be0 2023-12-11T17:37:38.542 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:38.125+0000 7f7aa57fa700 1 -- 172.21.15.59:0/1588604542 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+88 (secure 0 0 0) 0x7f7aa81994d0 con 0x7f7aa8105be0 2023-12-11T17:37:38.543 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:38.128+0000 7f7aae846700 1 -- 172.21.15.59:0/1588604542 >> [v2:172.21.15.59:6800/1672530462,v1:172.21.15.59:6801/1672530462] conn(0x7f7a9003aa60 msgr2=0x7f7a9003cf20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-11T17:37:38.543 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:38.128+0000 7f7aae846700 1 --2- 172.21.15.59:0/1588604542 >> [v2:172.21.15.59:6800/1672530462,v1:172.21.15.59:6801/1672530462] conn(0x7f7a9003aa60 0x7f7a9003cf20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:37:38.543 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:38.128+0000 7f7aae846700 1 -- 172.21.15.59:0/1588604542 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7aa8105be0 msgr2=0x7f7aa81988f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:37:38.543 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:38.128+0000 7f7aae846700 1 --2- 172.21.15.59:0/1588604542 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7aa8105be0 0x7f7aa81988f0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f7a98009130 tx=0x7f7a98011ad0 comp rx=0 tx=0).stop 2023-12-11T17:37:38.543 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:38.128+0000 7f7aae846700 1 -- 172.21.15.59:0/1588604542 shutdown_connections 2023-12-11T17:37:38.543 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:38.128+0000 7f7aae846700 1 --2- 172.21.15.59:0/1588604542 >> [v2:172.21.15.59:6800/1672530462,v1:172.21.15.59:6801/1672530462] conn(0x7f7a9003aa60 0x7f7a9003cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:37:38.543 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:38.128+0000 7f7aae846700 1 --2- 172.21.15.59:0/1588604542 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7aa8105be0 0x7f7aa81988f0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:37:38.543 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:38.128+0000 7f7aae846700 1 -- 172.21.15.59:0/1588604542 >> 172.21.15.59:0/1588604542 conn(0x7f7aa8101160 msgr2=0x7f7aa818f5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:37:38.543 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:38.128+0000 7f7aae846700 1 -- 172.21.15.59:0/1588604542 shutdown_connections 2023-12-11T17:37:38.543 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:38.128+0000 7f7aae846700 1 -- 172.21.15.59:0/1588604542 wait complete. 2023-12-11T17:37:38.543 INFO:teuthology.orchestra.run.smithi059.stdout:Waiting for the mgr to restart... 2023-12-11T17:37:38.543 INFO:teuthology.orchestra.run.smithi059.stdout:Waiting for mgr epoch 5... 2023-12-11T17:37:38.810 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:38 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/1588604542' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2023-12-11T17:37:38.810 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:37:38 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T17:37:38.778+0000 7f7db7cb4200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-11T17:37:39.904 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:37:39 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T17:37:39.568+0000 7f7db7cb4200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-11T17:37:39.904 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:37:39 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T17:37:39.666+0000 7f7db7cb4200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-11T17:37:40.904 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:37:40 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T17:37:40.641+0000 7f7db7cb4200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-11T17:37:40.904 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:37:40 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T17:37:40.751+0000 7f7db7cb4200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-11T17:37:41.404 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:37:40 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T17:37:40.973+0000 7f7db7cb4200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-11T17:37:41.757 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:37:41 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T17:37:41.468+0000 7f7db7cb4200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-11T17:37:41.757 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:37:41 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T17:37:41.573+0000 7f7db7cb4200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-11T17:37:42.087 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:37:41 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T17:37:41.755+0000 7f7db7cb4200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-11T17:37:42.404 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:37:42 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T17:37:42.085+0000 7f7db7cb4200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-11T17:37:43.313 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:37:42 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T17:37:42.904+0000 7f7db7cb4200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-11T17:37:43.313 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:37:43 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T17:37:43.061+0000 7f7db7cb4200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-11T17:37:43.654 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:37:43 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T17:37:43.310+0000 7f7db7cb4200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-11T17:37:43.654 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:37:43 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T17:37:43.447+0000 7f7db7cb4200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-11T17:37:44.154 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:37:43 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T17:37:43.721+0000 7f7db7cb4200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-11T17:37:45.033 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:37:44 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T17:37:44.760+0000 7f7db7cb4200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-11T17:37:45.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:45 smithi059 ceph-mon[98847]: Active manager daemon x restarted 2023-12-11T17:37:45.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:45 smithi059 ceph-mon[98847]: Activating manager daemon x 2023-12-11T17:37:45.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:45 smithi059 ceph-mon[98847]: osdmap e2: 0 total, 0 up, 0 in 2023-12-11T17:37:45.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:45 smithi059 ceph-mon[98847]: mgrmap e6: x(active, starting, since 0.00358156s) 2023-12-11T17:37:45.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:45 smithi059 ceph-mon[98847]: from='mgr.14118 172.21.15.59:0/103440859' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-11T17:37:45.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:45 smithi059 ceph-mon[98847]: from='mgr.14118 172.21.15.59:0/103440859' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-12-11T17:37:45.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:45 smithi059 ceph-mon[98847]: from='mgr.14118 172.21.15.59:0/103440859' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-12-11T17:37:45.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:45 smithi059 ceph-mon[98847]: from='mgr.14118 172.21.15.59:0/103440859' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-12-11T17:37:45.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:45 smithi059 ceph-mon[98847]: from='mgr.14118 172.21.15.59:0/103440859' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-12-11T17:37:45.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:45 smithi059 ceph-mon[98847]: Manager daemon x is now available 2023-12-11T17:37:45.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:45 smithi059 ceph-mon[98847]: from='mgr.14118 172.21.15.59:0/103440859' entity='mgr.x' 2023-12-11T17:37:45.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:45 smithi059 ceph-mon[98847]: from='mgr.14118 172.21.15.59:0/103440859' entity='mgr.x' 2023-12-11T17:37:45.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:45 smithi059 ceph-mon[98847]: from='mgr.14118 172.21.15.59:0/103440859' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:37:45.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:45 smithi059 ceph-mon[98847]: from='mgr.14118 172.21.15.59:0/103440859' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:37:45.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:45 smithi059 ceph-mon[98847]: from='mgr.14118 172.21.15.59:0/103440859' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:37:45.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:45 smithi059 ceph-mon[98847]: from='mgr.14118 172.21.15.59:0/103440859' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-12-11T17:37:45.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:45 smithi059 ceph-mon[98847]: from='mgr.14118 172.21.15.59:0/103440859' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-12-11T17:37:45.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:45 smithi059 ceph-mon[98847]: from='mgr.14118 172.21.15.59:0/103440859' entity='mgr.x' 2023-12-11T17:37:45.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:45 smithi059 ceph-mon[98847]: from='mgr.14118 172.21.15.59:0/103440859' entity='mgr.x' 2023-12-11T17:37:46.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:46 smithi059 ceph-mon[98847]: Found migration_current of "None". Setting to last migration. 2023-12-11T17:37:46.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:46 smithi059 ceph-mon[98847]: mgrmap e7: x(active, since 1.00561s) 2023-12-11T17:37:46.210 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout { 2023-12-11T17:37:46.210 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2023-12-11T17:37:46.210 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "initialized": true 2023-12-11T17:37:46.210 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout } 2023-12-11T17:37:46.210 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:39.013+0000 7fcae24f6700 1 Processor -- start 2023-12-11T17:37:46.210 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:39.013+0000 7fcae24f6700 1 -- start start 2023-12-11T17:37:46.210 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:39.014+0000 7fcae24f6700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcadc107790 0x7fcadc109b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:37:46.210 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:39.014+0000 7fcae24f6700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcadc07ae20 con 0x7fcadc107790 2023-12-11T17:37:46.211 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:39.014+0000 7fcadbfff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcadc107790 0x7fcadc109b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:37:46.211 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:39.014+0000 7fcadbfff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcadc107790 0x7fcadc109b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59834/0 (socket says 172.21.15.59:59834) 2023-12-11T17:37:46.211 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:39.014+0000 7fcadbfff700 1 -- 172.21.15.59:0/2389583589 learned_addr learned my addr 172.21.15.59:0/2389583589 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:37:46.211 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:39.014+0000 7fcadbfff700 1 -- 172.21.15.59:0/2389583589 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcadc10a0c0 con 0x7fcadc107790 2023-12-11T17:37:46.211 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:39.014+0000 7fcadbfff700 1 --2- 172.21.15.59:0/2389583589 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcadc107790 0x7fcadc109b80 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fcacc0098a0 tx=0x7fcacc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=257acd23ac792081 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:37:46.211 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:39.014+0000 7fcadaffd700 1 -- 172.21.15.59:0/2389583589 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcacc004160 con 0x7fcadc107790 2023-12-11T17:37:46.211 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:39.014+0000 7fcadaffd700 1 -- 172.21.15.59:0/2389583589 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fcacc0042c0 con 0x7fcadc107790 2023-12-11T17:37:46.211 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:39.014+0000 7fcadaffd700 1 -- 172.21.15.59:0/2389583589 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcacc011a00 con 0x7fcadc107790 2023-12-11T17:37:46.211 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:39.015+0000 7fcae24f6700 1 -- 172.21.15.59:0/2389583589 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcadc107790 msgr2=0x7fcadc109b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:37:46.211 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:39.015+0000 7fcae24f6700 1 --2- 172.21.15.59:0/2389583589 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcadc107790 0x7fcadc109b80 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fcacc0098a0 tx=0x7fcacc009bb0 comp rx=0 tx=0).stop 2023-12-11T17:37:46.211 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:39.015+0000 7fcae24f6700 1 -- 172.21.15.59:0/2389583589 shutdown_connections 2023-12-11T17:37:46.211 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:39.015+0000 7fcae24f6700 1 --2- 172.21.15.59:0/2389583589 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcadc107790 0x7fcadc109b80 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:37:46.211 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:39.015+0000 7fcae24f6700 1 -- 172.21.15.59:0/2389583589 >> 172.21.15.59:0/2389583589 conn(0x7fcadc101160 msgr2=0x7fcadc1035c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:37:46.211 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:39.015+0000 7fcae24f6700 1 -- 172.21.15.59:0/2389583589 shutdown_connections 2023-12-11T17:37:46.212 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:39.015+0000 7fcae24f6700 1 -- 172.21.15.59:0/2389583589 wait complete. 2023-12-11T17:37:46.212 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:39.016+0000 7fcae24f6700 1 Processor -- start 2023-12-11T17:37:46.212 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:39.016+0000 7fcae24f6700 1 -- start start 2023-12-11T17:37:46.212 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:39.016+0000 7fcae24f6700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcadc107790 0x7fcadc1071a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:37:46.212 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:39.016+0000 7fcae24f6700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcadc105740 con 0x7fcadc107790 2023-12-11T17:37:46.212 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:39.016+0000 7fcadbfff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcadc107790 0x7fcadc1071a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:37:46.212 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:39.016+0000 7fcadbfff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcadc107790 0x7fcadc1071a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59836/0 (socket says 172.21.15.59:59836) 2023-12-11T17:37:46.212 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:39.016+0000 7fcadbfff700 1 -- 172.21.15.59:0/2983008559 learned_addr learned my addr 172.21.15.59:0/2983008559 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:37:46.212 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:39.017+0000 7fcadbfff700 1 -- 172.21.15.59:0/2983008559 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcacc009550 con 0x7fcadc107790 2023-12-11T17:37:46.212 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:39.017+0000 7fcadbfff700 1 --2- 172.21.15.59:0/2983008559 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcadc107790 0x7fcadc1071a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fcacc009fb0 tx=0x7fcacc011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:37:46.212 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:39.017+0000 7fcad97fa700 1 -- 172.21.15.59:0/2983008559 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcacc003a40 con 0x7fcadc107790 2023-12-11T17:37:46.213 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:39.017+0000 7fcad97fa700 1 -- 172.21.15.59:0/2983008559 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fcacc003ba0 con 0x7fcadc107790 2023-12-11T17:37:46.213 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:39.017+0000 7fcae24f6700 1 -- 172.21.15.59:0/2983008559 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcadc105940 con 0x7fcadc107790 2023-12-11T17:37:46.213 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:39.017+0000 7fcad97fa700 1 -- 172.21.15.59:0/2983008559 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcacc01a750 con 0x7fcadc107790 2023-12-11T17:37:46.213 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:39.017+0000 7fcae24f6700 1 -- 172.21.15.59:0/2983008559 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcadc105de0 con 0x7fcadc107790 2023-12-11T17:37:46.213 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:39.018+0000 7fcad97fa700 1 -- 172.21.15.59:0/2983008559 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 45580+0+0 (secure 0 0 0) 0x7fcacc01a980 con 0x7fcadc107790 2023-12-11T17:37:46.213 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:39.018+0000 7fcad97fa700 1 --2- 172.21.15.59:0/2983008559 >> [v2:172.21.15.59:6800/1672530462,v1:172.21.15.59:6801/1672530462] conn(0x7fcac4038820 0x7fcac403ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:37:46.213 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:39.018+0000 7fcad97fa700 1 -- 172.21.15.59:0/2983008559 --> [v2:172.21.15.59:6800/1672530462,v1:172.21.15.59:6801/1672530462] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcac403b3f0 con 0x7fcac4038820 2023-12-11T17:37:46.213 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:39.018+0000 7fcad97fa700 1 -- 172.21.15.59:0/2983008559 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fcacc04c510 con 0x7fcadc107790 2023-12-11T17:37:46.213 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:39.018+0000 7fcadb7fe700 1 -- 172.21.15.59:0/2983008559 >> [v2:172.21.15.59:6800/1672530462,v1:172.21.15.59:6801/1672530462] conn(0x7fcac4038820 msgr2=0x7fcac403ace0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.59:6800/1672530462 2023-12-11T17:37:46.213 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:39.018+0000 7fcadb7fe700 1 --2- 172.21.15.59:0/2983008559 >> [v2:172.21.15.59:6800/1672530462,v1:172.21.15.59:6801/1672530462] conn(0x7fcac4038820 0x7fcac403ace0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-11T17:37:46.213 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:39.219+0000 7fcadb7fe700 1 -- 172.21.15.59:0/2983008559 >> [v2:172.21.15.59:6800/1672530462,v1:172.21.15.59:6801/1672530462] conn(0x7fcac4038820 msgr2=0x7fcac403ace0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.59:6800/1672530462 2023-12-11T17:37:46.213 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:39.219+0000 7fcadb7fe700 1 --2- 172.21.15.59:0/2983008559 >> [v2:172.21.15.59:6800/1672530462,v1:172.21.15.59:6801/1672530462] conn(0x7fcac4038820 0x7fcac403ace0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-11T17:37:46.213 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:39.620+0000 7fcadb7fe700 1 -- 172.21.15.59:0/2983008559 >> [v2:172.21.15.59:6800/1672530462,v1:172.21.15.59:6801/1672530462] conn(0x7fcac4038820 msgr2=0x7fcac403ace0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.59:6800/1672530462 2023-12-11T17:37:46.213 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:39.620+0000 7fcadb7fe700 1 --2- 172.21.15.59:0/2983008559 >> [v2:172.21.15.59:6800/1672530462,v1:172.21.15.59:6801/1672530462] conn(0x7fcac4038820 0x7fcac403ace0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-11T17:37:46.214 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:40.421+0000 7fcadb7fe700 1 -- 172.21.15.59:0/2983008559 >> [v2:172.21.15.59:6800/1672530462,v1:172.21.15.59:6801/1672530462] conn(0x7fcac4038820 msgr2=0x7fcac403ace0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.59:6800/1672530462 2023-12-11T17:37:46.214 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:40.421+0000 7fcadb7fe700 1 --2- 172.21.15.59:0/2983008559 >> [v2:172.21.15.59:6800/1672530462,v1:172.21.15.59:6801/1672530462] conn(0x7fcac4038820 0x7fcac403ace0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-11T17:37:46.214 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:42.022+0000 7fcadb7fe700 1 -- 172.21.15.59:0/2983008559 >> [v2:172.21.15.59:6800/1672530462,v1:172.21.15.59:6801/1672530462] conn(0x7fcac4038820 msgr2=0x7fcac403ace0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.59:6800/1672530462 2023-12-11T17:37:46.214 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:42.022+0000 7fcadb7fe700 1 --2- 172.21.15.59:0/2983008559 >> [v2:172.21.15.59:6800/1672530462,v1:172.21.15.59:6801/1672530462] conn(0x7fcac4038820 0x7fcac403ace0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-12-11T17:37:46.214 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:44.775+0000 7fcad97fa700 1 -- 172.21.15.59:0/2983008559 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mgrmap(e 6) v1 ==== 45347+0+0 (secure 0 0 0) 0x7fcacc02d910 con 0x7fcadc107790 2023-12-11T17:37:46.214 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:44.775+0000 7fcad97fa700 1 -- 172.21.15.59:0/2983008559 >> [v2:172.21.15.59:6800/1672530462,v1:172.21.15.59:6801/1672530462] conn(0x7fcac4038820 msgr2=0x7fcac403ace0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-11T17:37:46.214 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:44.775+0000 7fcad97fa700 1 --2- 172.21.15.59:0/2983008559 >> [v2:172.21.15.59:6800/1672530462,v1:172.21.15.59:6801/1672530462] conn(0x7fcac4038820 0x7fcac403ace0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:37:46.214 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:45.776+0000 7fcad97fa700 1 -- 172.21.15.59:0/2983008559 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mgrmap(e 7) v1 ==== 45474+0+0 (secure 0 0 0) 0x7fcacc04df20 con 0x7fcadc107790 2023-12-11T17:37:46.214 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:45.776+0000 7fcad97fa700 1 --2- 172.21.15.59:0/2983008559 >> [v2:172.21.15.59:6800/1259916482,v1:172.21.15.59:6801/1259916482] conn(0x7fcac403bff0 0x7fcac403e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:37:46.214 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:45.776+0000 7fcad97fa700 1 -- 172.21.15.59:0/2983008559 --> [v2:172.21.15.59:6800/1259916482,v1:172.21.15.59:6801/1259916482] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcac403b3f0 con 0x7fcac403bff0 2023-12-11T17:37:46.214 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:45.777+0000 7fcadb7fe700 1 --2- 172.21.15.59:0/2983008559 >> [v2:172.21.15.59:6800/1259916482,v1:172.21.15.59:6801/1259916482] conn(0x7fcac403bff0 0x7fcac403e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:37:46.214 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:45.777+0000 7fcadb7fe700 1 --2- 172.21.15.59:0/2983008559 >> [v2:172.21.15.59:6800/1259916482,v1:172.21.15.59:6801/1259916482] conn(0x7fcac403bff0 0x7fcac403e3e0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fcad0003a10 tx=0x7fcad00092b0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:37:46.215 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:45.778+0000 7fcad97fa700 1 -- 172.21.15.59:0/2983008559 <== mgr.14118 v2:172.21.15.59:6800/1259916482 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (secure 0 0 0) 0x7fcac403b3f0 con 0x7fcac403bff0 2023-12-11T17:37:46.215 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:45.789+0000 7fcae24f6700 1 -- 172.21.15.59:0/2983008559 --> [v2:172.21.15.59:6800/1259916482,v1:172.21.15.59:6801/1259916482] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7fcadc062c00 con 0x7fcac403bff0 2023-12-11T17:37:46.215 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:45.790+0000 7fcad97fa700 1 -- 172.21.15.59:0/2983008559 <== mgr.14118 v2:172.21.15.59:6800/1259916482 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7fcadc062c00 con 0x7fcac403bff0 2023-12-11T17:37:46.215 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:45.790+0000 7fcae24f6700 1 -- 172.21.15.59:0/2983008559 >> [v2:172.21.15.59:6800/1259916482,v1:172.21.15.59:6801/1259916482] conn(0x7fcac403bff0 msgr2=0x7fcac403e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:37:46.215 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:45.790+0000 7fcae24f6700 1 --2- 172.21.15.59:0/2983008559 >> [v2:172.21.15.59:6800/1259916482,v1:172.21.15.59:6801/1259916482] conn(0x7fcac403bff0 0x7fcac403e3e0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fcad0003a10 tx=0x7fcad00092b0 comp rx=0 tx=0).stop 2023-12-11T17:37:46.215 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:45.790+0000 7fcae24f6700 1 -- 172.21.15.59:0/2983008559 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcadc107790 msgr2=0x7fcadc1071a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:37:46.215 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:45.790+0000 7fcae24f6700 1 --2- 172.21.15.59:0/2983008559 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcadc107790 0x7fcadc1071a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fcacc009fb0 tx=0x7fcacc011ad0 comp rx=0 tx=0).stop 2023-12-11T17:37:46.215 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:45.790+0000 7fcae24f6700 1 -- 172.21.15.59:0/2983008559 shutdown_connections 2023-12-11T17:37:46.215 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:45.790+0000 7fcae24f6700 1 --2- 172.21.15.59:0/2983008559 >> [v2:172.21.15.59:6800/1259916482,v1:172.21.15.59:6801/1259916482] conn(0x7fcac403bff0 0x7fcac403e3e0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:37:46.215 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:45.790+0000 7fcae24f6700 1 --2- 172.21.15.59:0/2983008559 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcadc107790 0x7fcadc1071a0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:37:46.215 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:45.790+0000 7fcae24f6700 1 --2- 172.21.15.59:0/2983008559 >> [v2:172.21.15.59:6800/1672530462,v1:172.21.15.59:6801/1672530462] conn(0x7fcac4038820 0x7fcac403ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:37:46.215 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:45.790+0000 7fcae24f6700 1 -- 172.21.15.59:0/2983008559 >> 172.21.15.59:0/2983008559 conn(0x7fcadc101160 msgr2=0x7fcadc101c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:37:46.215 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:45.790+0000 7fcae24f6700 1 -- 172.21.15.59:0/2983008559 shutdown_connections 2023-12-11T17:37:46.216 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:45.790+0000 7fcae24f6700 1 -- 172.21.15.59:0/2983008559 wait complete. 2023-12-11T17:37:46.216 INFO:teuthology.orchestra.run.smithi059.stdout:mgr epoch 5 is available 2023-12-11T17:37:46.216 INFO:teuthology.orchestra.run.smithi059.stdout:Setting orchestrator backend to cephadm... 2023-12-11T17:37:47.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:47 smithi059 ceph-mon[98847]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2023-12-11T17:37:47.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:47 smithi059 ceph-mon[98847]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2023-12-11T17:37:47.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:47 smithi059 ceph-mon[98847]: [11/Dec/2023:17:37:46] ENGINE Bus STARTING 2023-12-11T17:37:47.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:47 smithi059 ceph-mon[98847]: [11/Dec/2023:17:37:46] ENGINE Serving on http://172.21.15.59:8765 2023-12-11T17:37:47.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:47 smithi059 ceph-mon[98847]: [11/Dec/2023:17:37:46] ENGINE Serving on https://172.21.15.59:7150 2023-12-11T17:37:47.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:47 smithi059 ceph-mon[98847]: [11/Dec/2023:17:37:46] ENGINE Bus STARTED 2023-12-11T17:37:47.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:47 smithi059 ceph-mon[98847]: from='mgr.14118 172.21.15.59:0/103440859' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:37:47.448 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:46.673+0000 7fe49b0f7700 1 Processor -- start 2023-12-11T17:37:47.453 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:46.673+0000 7fe49b0f7700 1 -- start start 2023-12-11T17:37:47.453 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:46.674+0000 7fe49b0f7700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe494107780 0x7fe494109b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:37:47.453 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:46.674+0000 7fe49b0f7700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe49407ae60 con 0x7fe494107780 2023-12-11T17:37:47.453 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:46.674+0000 7fe498e93700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe494107780 0x7fe494109b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:37:47.453 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:46.674+0000 7fe498e93700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe494107780 0x7fe494109b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59888/0 (socket says 172.21.15.59:59888) 2023-12-11T17:37:47.453 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:46.674+0000 7fe498e93700 1 -- 172.21.15.59:0/965930016 learned_addr learned my addr 172.21.15.59:0/965930016 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:37:47.453 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:46.674+0000 7fe498e93700 1 -- 172.21.15.59:0/965930016 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe49410a0b0 con 0x7fe494107780 2023-12-11T17:37:47.453 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:46.674+0000 7fe498e93700 1 --2- 172.21.15.59:0/965930016 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe494107780 0x7fe494109b70 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe4880098a0 tx=0x7fe488009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=443d448913c93732 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:37:47.453 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:46.674+0000 7fe4937fe700 1 -- 172.21.15.59:0/965930016 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe488004160 con 0x7fe494107780 2023-12-11T17:37:47.453 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:46.674+0000 7fe4937fe700 1 -- 172.21.15.59:0/965930016 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe4880042c0 con 0x7fe494107780 2023-12-11T17:37:47.454 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:46.674+0000 7fe4937fe700 1 -- 172.21.15.59:0/965930016 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe488011a00 con 0x7fe494107780 2023-12-11T17:37:47.454 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:46.675+0000 7fe49b0f7700 1 -- 172.21.15.59:0/965930016 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe494107780 msgr2=0x7fe494109b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:37:47.454 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:46.675+0000 7fe49b0f7700 1 --2- 172.21.15.59:0/965930016 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe494107780 0x7fe494109b70 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe4880098a0 tx=0x7fe488009bb0 comp rx=0 tx=0).stop 2023-12-11T17:37:47.454 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:46.675+0000 7fe49b0f7700 1 -- 172.21.15.59:0/965930016 shutdown_connections 2023-12-11T17:37:47.454 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:46.675+0000 7fe49b0f7700 1 --2- 172.21.15.59:0/965930016 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe494107780 0x7fe494109b70 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:37:47.454 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:46.675+0000 7fe49b0f7700 1 -- 172.21.15.59:0/965930016 >> 172.21.15.59:0/965930016 conn(0x7fe4941011b0 msgr2=0x7fe4941035f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:37:47.454 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:46.675+0000 7fe49b0f7700 1 -- 172.21.15.59:0/965930016 shutdown_connections 2023-12-11T17:37:47.454 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:46.675+0000 7fe49b0f7700 1 -- 172.21.15.59:0/965930016 wait complete. 2023-12-11T17:37:47.454 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:46.676+0000 7fe49b0f7700 1 Processor -- start 2023-12-11T17:37:47.454 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:46.676+0000 7fe49b0f7700 1 -- start start 2023-12-11T17:37:47.454 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:46.676+0000 7fe49b0f7700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe494107780 0x7fe49419ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:37:47.454 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:46.676+0000 7fe49b0f7700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe49419b080 con 0x7fe494107780 2023-12-11T17:37:47.454 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:46.676+0000 7fe498e93700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe494107780 0x7fe49419ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:37:47.455 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:46.676+0000 7fe498e93700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe494107780 0x7fe49419ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59890/0 (socket says 172.21.15.59:59890) 2023-12-11T17:37:47.455 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:46.676+0000 7fe498e93700 1 -- 172.21.15.59:0/3486340424 learned_addr learned my addr 172.21.15.59:0/3486340424 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:37:47.455 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:46.676+0000 7fe498e93700 1 -- 172.21.15.59:0/3486340424 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe488009550 con 0x7fe494107780 2023-12-11T17:37:47.455 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:46.676+0000 7fe498e93700 1 --2- 172.21.15.59:0/3486340424 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe494107780 0x7fe49419ab40 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe488011a20 tx=0x7fe488011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:37:47.455 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:46.677+0000 7fe491ffb700 1 -- 172.21.15.59:0/3486340424 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe488003af0 con 0x7fe494107780 2023-12-11T17:37:47.455 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:46.677+0000 7fe491ffb700 1 -- 172.21.15.59:0/3486340424 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe488003c50 con 0x7fe494107780 2023-12-11T17:37:47.455 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:46.677+0000 7fe49b0f7700 1 -- 172.21.15.59:0/3486340424 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe49419b280 con 0x7fe494107780 2023-12-11T17:37:47.455 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:46.677+0000 7fe491ffb700 1 -- 172.21.15.59:0/3486340424 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe488019760 con 0x7fe494107780 2023-12-11T17:37:47.455 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:46.677+0000 7fe49b0f7700 1 -- 172.21.15.59:0/3486340424 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe49419bde0 con 0x7fe494107780 2023-12-11T17:37:47.455 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:46.678+0000 7fe491ffb700 1 -- 172.21.15.59:0/3486340424 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 7) v1 ==== 45474+0+0 (secure 0 0 0) 0x7fe488019950 con 0x7fe494107780 2023-12-11T17:37:47.455 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:46.678+0000 7fe491ffb700 1 --2- 172.21.15.59:0/3486340424 >> [v2:172.21.15.59:6800/1259916482,v1:172.21.15.59:6801/1259916482] conn(0x7fe484038740 0x7fe48403ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:37:47.455 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:46.678+0000 7fe49b0f7700 1 -- 172.21.15.59:0/3486340424 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe494062c00 con 0x7fe494107780 2023-12-11T17:37:47.455 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:46.678+0000 7fe491ffb700 1 -- 172.21.15.59:0/3486340424 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fe48804c9b0 con 0x7fe494107780 2023-12-11T17:37:47.455 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:46.678+0000 7fe493fff700 1 --2- 172.21.15.59:0/3486340424 >> [v2:172.21.15.59:6800/1259916482,v1:172.21.15.59:6801/1259916482] conn(0x7fe484038740 0x7fe48403ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:37:47.455 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:46.679+0000 7fe493fff700 1 --2- 172.21.15.59:0/3486340424 >> [v2:172.21.15.59:6800/1259916482,v1:172.21.15.59:6801/1259916482] conn(0x7fe484038740 0x7fe48403ac00 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe480006fd0 tx=0x7fe480006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:37:47.456 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:46.683+0000 7fe491ffb700 1 -- 172.21.15.59:0/3486340424 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fe488018350 con 0x7fe494107780 2023-12-11T17:37:47.456 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:47.034+0000 7fe491ffb700 1 -- 172.21.15.59:0/3486340424 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mgrmap(e 8) v1 ==== 45580+0+0 (secure 0 0 0) 0x7fe48800b900 con 0x7fe494107780 2023-12-11T17:37:47.456 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:47.080+0000 7fe49b0f7700 1 -- 172.21.15.59:0/3486340424 --> [v2:172.21.15.59:6800/1259916482,v1:172.21.15.59:6801/1259916482] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7fe4941023d0 con 0x7fe484038740 2023-12-11T17:37:47.456 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:47.085+0000 7fe491ffb700 1 -- 172.21.15.59:0/3486340424 <== mgr.14118 v2:172.21.15.59:6800/1259916482 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fe4941023d0 con 0x7fe484038740 2023-12-11T17:37:47.456 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:47.088+0000 7fe49b0f7700 1 -- 172.21.15.59:0/3486340424 >> [v2:172.21.15.59:6800/1259916482,v1:172.21.15.59:6801/1259916482] conn(0x7fe484038740 msgr2=0x7fe48403ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:37:47.456 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:47.088+0000 7fe49b0f7700 1 --2- 172.21.15.59:0/3486340424 >> [v2:172.21.15.59:6800/1259916482,v1:172.21.15.59:6801/1259916482] conn(0x7fe484038740 0x7fe48403ac00 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe480006fd0 tx=0x7fe480006e40 comp rx=0 tx=0).stop 2023-12-11T17:37:47.456 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:47.088+0000 7fe49b0f7700 1 -- 172.21.15.59:0/3486340424 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe494107780 msgr2=0x7fe49419ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:37:47.456 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:47.088+0000 7fe49b0f7700 1 --2- 172.21.15.59:0/3486340424 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe494107780 0x7fe49419ab40 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe488011a20 tx=0x7fe488011b00 comp rx=0 tx=0).stop 2023-12-11T17:37:47.456 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:47.088+0000 7fe49b0f7700 1 -- 172.21.15.59:0/3486340424 shutdown_connections 2023-12-11T17:37:47.456 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:47.088+0000 7fe49b0f7700 1 --2- 172.21.15.59:0/3486340424 >> [v2:172.21.15.59:6800/1259916482,v1:172.21.15.59:6801/1259916482] conn(0x7fe484038740 0x7fe48403ac00 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:37:47.456 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:47.088+0000 7fe49b0f7700 1 --2- 172.21.15.59:0/3486340424 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe494107780 0x7fe49419ab40 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:37:47.456 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:47.088+0000 7fe49b0f7700 1 -- 172.21.15.59:0/3486340424 >> 172.21.15.59:0/3486340424 conn(0x7fe4941011b0 msgr2=0x7fe494101cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:37:47.456 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:47.088+0000 7fe49b0f7700 1 -- 172.21.15.59:0/3486340424 shutdown_connections 2023-12-11T17:37:47.456 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:47.088+0000 7fe49b0f7700 1 -- 172.21.15.59:0/3486340424 wait complete. 2023-12-11T17:37:48.349 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:48 smithi059 ceph-mon[98847]: mgrmap e8: x(active, since 2s) 2023-12-11T17:37:48.349 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:48 smithi059 ceph-mon[98847]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T17:37:48.349 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:48 smithi059 ceph-mon[98847]: from='mgr.14118 172.21.15.59:0/103440859' entity='mgr.x' 2023-12-11T17:37:48.349 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:48 smithi059 ceph-mon[98847]: from='mgr.14118 172.21.15.59:0/103440859' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:37:48.698 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout value unchanged 2023-12-11T17:37:48.698 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:47.944+0000 7fb522c29700 1 Processor -- start 2023-12-11T17:37:48.698 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:47.944+0000 7fb522c29700 1 -- start start 2023-12-11T17:37:48.698 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:47.944+0000 7fb522c29700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb51c105390 0x7fb51c1057b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:37:48.698 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:47.944+0000 7fb522c29700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb51c105d80 con 0x7fb51c105390 2023-12-11T17:37:48.698 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:47.945+0000 7fb5209c5700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb51c105390 0x7fb51c1057b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:37:48.698 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:47.945+0000 7fb5209c5700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb51c105390 0x7fb51c1057b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59894/0 (socket says 172.21.15.59:59894) 2023-12-11T17:37:48.699 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:47.945+0000 7fb5209c5700 1 -- 172.21.15.59:0/4064093938 learned_addr learned my addr 172.21.15.59:0/4064093938 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:37:48.699 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:47.945+0000 7fb5209c5700 1 -- 172.21.15.59:0/4064093938 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb51c106590 con 0x7fb51c105390 2023-12-11T17:37:48.699 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:47.945+0000 7fb5209c5700 1 --2- 172.21.15.59:0/4064093938 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb51c105390 0x7fb51c1057b0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb51000aa40 tx=0x7fb51000ad50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f43a4dda6eb51d50 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:37:48.699 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:47.945+0000 7fb51b7fe700 1 -- 172.21.15.59:0/4064093938 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb510004160 con 0x7fb51c105390 2023-12-11T17:37:48.699 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:47.945+0000 7fb51b7fe700 1 -- 172.21.15.59:0/4064093938 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb51000cd90 con 0x7fb51c105390 2023-12-11T17:37:48.699 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:47.945+0000 7fb51b7fe700 1 -- 172.21.15.59:0/4064093938 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb510003c80 con 0x7fb51c105390 2023-12-11T17:37:48.699 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:47.946+0000 7fb522c29700 1 -- 172.21.15.59:0/4064093938 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb51c105390 msgr2=0x7fb51c1057b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:37:48.699 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:47.946+0000 7fb522c29700 1 --2- 172.21.15.59:0/4064093938 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb51c105390 0x7fb51c1057b0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb51000aa40 tx=0x7fb51000ad50 comp rx=0 tx=0).stop 2023-12-11T17:37:48.699 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:47.946+0000 7fb522c29700 1 -- 172.21.15.59:0/4064093938 shutdown_connections 2023-12-11T17:37:48.699 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:47.946+0000 7fb522c29700 1 --2- 172.21.15.59:0/4064093938 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb51c105390 0x7fb51c1057b0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:37:48.699 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:47.946+0000 7fb522c29700 1 -- 172.21.15.59:0/4064093938 >> 172.21.15.59:0/4064093938 conn(0x7fb51c100970 msgr2=0x7fb51c102d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:37:48.699 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:47.946+0000 7fb522c29700 1 -- 172.21.15.59:0/4064093938 shutdown_connections 2023-12-11T17:37:48.699 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:47.946+0000 7fb522c29700 1 -- 172.21.15.59:0/4064093938 wait complete. 2023-12-11T17:37:48.699 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:47.947+0000 7fb522c29700 1 Processor -- start 2023-12-11T17:37:48.700 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:47.947+0000 7fb522c29700 1 -- start start 2023-12-11T17:37:48.700 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:47.947+0000 7fb522c29700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb51c105390 0x7fb51c19abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:37:48.700 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:47.947+0000 7fb522c29700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb51c19b100 con 0x7fb51c105390 2023-12-11T17:37:48.700 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:47.947+0000 7fb5209c5700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb51c105390 0x7fb51c19abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:37:48.700 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:47.947+0000 7fb5209c5700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb51c105390 0x7fb51c19abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59896/0 (socket says 172.21.15.59:59896) 2023-12-11T17:37:48.700 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:47.947+0000 7fb5209c5700 1 -- 172.21.15.59:0/2548108375 learned_addr learned my addr 172.21.15.59:0/2548108375 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:37:48.700 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:47.947+0000 7fb5209c5700 1 -- 172.21.15.59:0/2548108375 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb51000a6f0 con 0x7fb51c105390 2023-12-11T17:37:48.700 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:47.948+0000 7fb5209c5700 1 --2- 172.21.15.59:0/2548108375 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb51c105390 0x7fb51c19abc0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fb510004170 tx=0x7fb510004250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:37:48.700 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:47.948+0000 7fb519ffb700 1 -- 172.21.15.59:0/2548108375 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb510010600 con 0x7fb51c105390 2023-12-11T17:37:48.700 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:47.948+0000 7fb519ffb700 1 -- 172.21.15.59:0/2548108375 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb510010c00 con 0x7fb51c105390 2023-12-11T17:37:48.700 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:47.948+0000 7fb522c29700 1 -- 172.21.15.59:0/2548108375 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb51c19b300 con 0x7fb51c105390 2023-12-11T17:37:48.700 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:47.948+0000 7fb519ffb700 1 -- 172.21.15.59:0/2548108375 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb510020990 con 0x7fb51c105390 2023-12-11T17:37:48.701 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:47.948+0000 7fb522c29700 1 -- 172.21.15.59:0/2548108375 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb51c19be60 con 0x7fb51c105390 2023-12-11T17:37:48.701 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:47.949+0000 7fb519ffb700 1 -- 172.21.15.59:0/2548108375 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 8) v1 ==== 45580+0+0 (secure 0 0 0) 0x7fb51002a440 con 0x7fb51c105390 2023-12-11T17:37:48.701 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:47.949+0000 7fb519ffb700 1 --2- 172.21.15.59:0/2548108375 >> [v2:172.21.15.59:6800/1259916482,v1:172.21.15.59:6801/1259916482] conn(0x7fb50c038820 0x7fb50c03ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:37:48.701 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:47.949+0000 7fb519ffb700 1 -- 172.21.15.59:0/2548108375 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fb51004c980 con 0x7fb51c105390 2023-12-11T17:37:48.701 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:47.949+0000 7fb51bfff700 1 --2- 172.21.15.59:0/2548108375 >> [v2:172.21.15.59:6800/1259916482,v1:172.21.15.59:6801/1259916482] conn(0x7fb50c038820 0x7fb50c03ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:37:48.701 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:47.949+0000 7fb522c29700 1 -- 172.21.15.59:0/2548108375 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb51c19b7a0 con 0x7fb51c105390 2023-12-11T17:37:48.701 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:47.949+0000 7fb51bfff700 1 --2- 172.21.15.59:0/2548108375 >> [v2:172.21.15.59:6800/1259916482,v1:172.21.15.59:6801/1259916482] conn(0x7fb50c038820 0x7fb50c03ace0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb508006fd0 tx=0x7fb508006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:37:48.701 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:47.954+0000 7fb519ffb700 1 -- 172.21.15.59:0/2548108375 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fb51c19b7a0 con 0x7fb51c105390 2023-12-11T17:37:48.701 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:48.320+0000 7fb522c29700 1 -- 172.21.15.59:0/2548108375 --> [v2:172.21.15.59:6800/1259916482,v1:172.21.15.59:6801/1259916482] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7fb51c19b7a0 con 0x7fb50c038820 2023-12-11T17:37:48.701 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:48.321+0000 7fb519ffb700 1 -- 172.21.15.59:0/2548108375 <== mgr.14118 v2:172.21.15.59:6800/1259916482 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7fb51c19b7a0 con 0x7fb50c038820 2023-12-11T17:37:48.701 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:48.323+0000 7fb522c29700 1 -- 172.21.15.59:0/2548108375 >> [v2:172.21.15.59:6800/1259916482,v1:172.21.15.59:6801/1259916482] conn(0x7fb50c038820 msgr2=0x7fb50c03ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:37:48.701 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:48.323+0000 7fb522c29700 1 --2- 172.21.15.59:0/2548108375 >> [v2:172.21.15.59:6800/1259916482,v1:172.21.15.59:6801/1259916482] conn(0x7fb50c038820 0x7fb50c03ace0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb508006fd0 tx=0x7fb508006e40 comp rx=0 tx=0).stop 2023-12-11T17:37:48.701 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:48.323+0000 7fb522c29700 1 -- 172.21.15.59:0/2548108375 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb51c105390 msgr2=0x7fb51c19abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:37:48.702 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:48.323+0000 7fb522c29700 1 --2- 172.21.15.59:0/2548108375 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb51c105390 0x7fb51c19abc0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fb510004170 tx=0x7fb510004250 comp rx=0 tx=0).stop 2023-12-11T17:37:48.702 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:48.323+0000 7fb522c29700 1 -- 172.21.15.59:0/2548108375 shutdown_connections 2023-12-11T17:37:48.702 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:48.323+0000 7fb522c29700 1 --2- 172.21.15.59:0/2548108375 >> [v2:172.21.15.59:6800/1259916482,v1:172.21.15.59:6801/1259916482] conn(0x7fb50c038820 0x7fb50c03ace0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:37:48.702 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:48.323+0000 7fb522c29700 1 --2- 172.21.15.59:0/2548108375 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb51c105390 0x7fb51c19abc0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:37:48.702 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:48.323+0000 7fb522c29700 1 -- 172.21.15.59:0/2548108375 >> 172.21.15.59:0/2548108375 conn(0x7fb51c100970 msgr2=0x7fb51c18f6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:37:48.702 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:48.323+0000 7fb522c29700 1 -- 172.21.15.59:0/2548108375 shutdown_connections 2023-12-11T17:37:48.702 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:48.323+0000 7fb522c29700 1 -- 172.21.15.59:0/2548108375 wait complete. 2023-12-11T17:37:48.702 INFO:teuthology.orchestra.run.smithi059.stdout:Generating ssh key... 2023-12-11T17:37:49.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:49 smithi059 ceph-mon[98847]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T17:37:50.654 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:37:50 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: Generating public/private rsa key pair. 2023-12-11T17:37:50.654 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:37:50 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: Your identification has been saved in /tmp/tmpbpa7mocz/key. 2023-12-11T17:37:50.654 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:37:50 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: Your public key has been saved in /tmp/tmpbpa7mocz/key.pub. 2023-12-11T17:37:50.654 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:37:50 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: The key fingerprint is: 2023-12-11T17:37:50.655 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:37:50 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: SHA256:d0Hzb+dXFGxUvSJUZzfoNOqK6TH7fydJlISYTCLqd4Q ceph-d0a07eaa-984b-11ee-95a3-87774f69a715 2023-12-11T17:37:50.655 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:37:50 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: The key's randomart image is: 2023-12-11T17:37:50.655 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:37:50 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: +---[RSA 3072]----+ 2023-12-11T17:37:50.655 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:37:50 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: | . .o.o =.+*=| 2023-12-11T17:37:50.655 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:37:50 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: | . o .+ + Boo=| 2023-12-11T17:37:50.655 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:37:50 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: | . E . . * = o| 2023-12-11T17:37:50.655 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:37:50 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: | . . o * + | 2023-12-11T17:37:50.655 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:37:50 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: | . . .S o + . =| 2023-12-11T17:37:50.655 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:37:50 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: | . . . o . oo| 2023-12-11T17:37:50.655 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:37:50 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: | oo . . . o| 2023-12-11T17:37:50.655 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:37:50 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: | o+. + ..| 2023-12-11T17:37:50.655 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:37:50 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: | .o..... o | 2023-12-11T17:37:50.655 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:37:50 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: +----[SHA256]-----+ 2023-12-11T17:37:50.747 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:49.383+0000 7efcbdfc0700 1 Processor -- start 2023-12-11T17:37:50.748 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:49.383+0000 7efcbdfc0700 1 -- start start 2023-12-11T17:37:50.748 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:49.384+0000 7efcbdfc0700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efcb81077e0 0x7efcb8109bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:37:50.748 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:49.384+0000 7efcbdfc0700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcb807ae00 con 0x7efcb81077e0 2023-12-11T17:37:50.748 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:49.384+0000 7efcb77fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efcb81077e0 0x7efcb8109bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:37:50.748 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:49.384+0000 7efcb77fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efcb81077e0 0x7efcb8109bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59900/0 (socket says 172.21.15.59:59900) 2023-12-11T17:37:50.748 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:49.384+0000 7efcb77fe700 1 -- 172.21.15.59:0/3414968596 learned_addr learned my addr 172.21.15.59:0/3414968596 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:37:50.748 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:49.384+0000 7efcb77fe700 1 -- 172.21.15.59:0/3414968596 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcb810a110 con 0x7efcb81077e0 2023-12-11T17:37:50.748 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:49.384+0000 7efcb77fe700 1 --2- 172.21.15.59:0/3414968596 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efcb81077e0 0x7efcb8109bd0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7efca80098a0 tx=0x7efca8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95a2c4dd0bcc63ea server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:37:50.748 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:49.384+0000 7efcb67fc700 1 -- 172.21.15.59:0/3414968596 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efca8004160 con 0x7efcb81077e0 2023-12-11T17:37:50.748 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:49.384+0000 7efcb67fc700 1 -- 172.21.15.59:0/3414968596 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7efca80042c0 con 0x7efcb81077e0 2023-12-11T17:37:50.748 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:49.384+0000 7efcb67fc700 1 -- 172.21.15.59:0/3414968596 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efca8011a00 con 0x7efcb81077e0 2023-12-11T17:37:50.748 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:49.385+0000 7efcbdfc0700 1 -- 172.21.15.59:0/3414968596 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efcb81077e0 msgr2=0x7efcb8109bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:37:50.748 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:49.385+0000 7efcbdfc0700 1 --2- 172.21.15.59:0/3414968596 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efcb81077e0 0x7efcb8109bd0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7efca80098a0 tx=0x7efca8009bb0 comp rx=0 tx=0).stop 2023-12-11T17:37:50.749 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:49.385+0000 7efcbdfc0700 1 -- 172.21.15.59:0/3414968596 shutdown_connections 2023-12-11T17:37:50.749 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:49.385+0000 7efcbdfc0700 1 --2- 172.21.15.59:0/3414968596 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efcb81077e0 0x7efcb8109bd0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:37:50.749 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:49.385+0000 7efcbdfc0700 1 -- 172.21.15.59:0/3414968596 >> 172.21.15.59:0/3414968596 conn(0x7efcb81011d0 msgr2=0x7efcb81035f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:37:50.749 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:49.385+0000 7efcbdfc0700 1 -- 172.21.15.59:0/3414968596 shutdown_connections 2023-12-11T17:37:50.749 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:49.385+0000 7efcbdfc0700 1 -- 172.21.15.59:0/3414968596 wait complete. 2023-12-11T17:37:50.749 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:49.386+0000 7efcbdfc0700 1 Processor -- start 2023-12-11T17:37:50.749 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:49.386+0000 7efcbdfc0700 1 -- start start 2023-12-11T17:37:50.749 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:49.386+0000 7efcbdfc0700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efcb81077e0 0x7efcb819ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:37:50.749 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:49.386+0000 7efcbdfc0700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcb819b080 con 0x7efcb81077e0 2023-12-11T17:37:50.749 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:49.386+0000 7efcb77fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efcb81077e0 0x7efcb819ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:37:50.749 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:49.386+0000 7efcb77fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efcb81077e0 0x7efcb819ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59902/0 (socket says 172.21.15.59:59902) 2023-12-11T17:37:50.749 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:49.386+0000 7efcb77fe700 1 -- 172.21.15.59:0/23329273 learned_addr learned my addr 172.21.15.59:0/23329273 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:37:50.749 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:49.387+0000 7efcb77fe700 1 -- 172.21.15.59:0/23329273 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efca8009550 con 0x7efcb81077e0 2023-12-11T17:37:50.749 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:49.387+0000 7efcb77fe700 1 --2- 172.21.15.59:0/23329273 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efcb81077e0 0x7efcb819ab40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7efca8009130 tx=0x7efca8011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:37:50.750 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:49.387+0000 7efcb4ff9700 1 -- 172.21.15.59:0/23329273 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efca8003b70 con 0x7efcb81077e0 2023-12-11T17:37:50.750 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:49.387+0000 7efcb4ff9700 1 -- 172.21.15.59:0/23329273 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7efca8003cd0 con 0x7efcb81077e0 2023-12-11T17:37:50.750 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:49.387+0000 7efcbdfc0700 1 -- 172.21.15.59:0/23329273 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcb819b280 con 0x7efcb81077e0 2023-12-11T17:37:50.750 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:49.387+0000 7efcb4ff9700 1 -- 172.21.15.59:0/23329273 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efca80197d0 con 0x7efcb81077e0 2023-12-11T17:37:50.750 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:49.387+0000 7efcbdfc0700 1 -- 172.21.15.59:0/23329273 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcb819bde0 con 0x7efcb81077e0 2023-12-11T17:37:50.750 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:49.388+0000 7efcb4ff9700 1 -- 172.21.15.59:0/23329273 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 8) v1 ==== 45580+0+0 (secure 0 0 0) 0x7efca8019a10 con 0x7efcb81077e0 2023-12-11T17:37:50.750 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:49.388+0000 7efcb4ff9700 1 --2- 172.21.15.59:0/23329273 >> [v2:172.21.15.59:6800/1259916482,v1:172.21.15.59:6801/1259916482] conn(0x7efca0038820 0x7efca003ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:37:50.750 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:49.388+0000 7efcb4ff9700 1 -- 172.21.15.59:0/23329273 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7efca804cab0 con 0x7efcb81077e0 2023-12-11T17:37:50.750 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:49.388+0000 7efcbdfc0700 1 -- 172.21.15.59:0/23329273 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcb8062c00 con 0x7efcb81077e0 2023-12-11T17:37:50.750 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:49.388+0000 7efcb6ffd700 1 --2- 172.21.15.59:0/23329273 >> [v2:172.21.15.59:6800/1259916482,v1:172.21.15.59:6801/1259916482] conn(0x7efca0038820 0x7efca003ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:37:50.750 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:49.389+0000 7efcb6ffd700 1 --2- 172.21.15.59:0/23329273 >> [v2:172.21.15.59:6800/1259916482,v1:172.21.15.59:6801/1259916482] conn(0x7efca0038820 0x7efca003ace0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7efcac006fd0 tx=0x7efcac006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:37:50.750 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:49.393+0000 7efcb4ff9700 1 -- 172.21.15.59:0/23329273 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7efca8018db0 con 0x7efcb81077e0 2023-12-11T17:37:50.750 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:49.761+0000 7efcbdfc0700 1 -- 172.21.15.59:0/23329273 --> [v2:172.21.15.59:6800/1259916482,v1:172.21.15.59:6801/1259916482] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7efcb8102420 con 0x7efca0038820 2023-12-11T17:37:50.750 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:50.361+0000 7efcb4ff9700 1 -- 172.21.15.59:0/23329273 <== mgr.14118 v2:172.21.15.59:6800/1259916482 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7efcb8102420 con 0x7efca0038820 2023-12-11T17:37:50.751 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:50.363+0000 7efcbdfc0700 1 -- 172.21.15.59:0/23329273 >> [v2:172.21.15.59:6800/1259916482,v1:172.21.15.59:6801/1259916482] conn(0x7efca0038820 msgr2=0x7efca003ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:37:50.751 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:50.363+0000 7efcbdfc0700 1 --2- 172.21.15.59:0/23329273 >> [v2:172.21.15.59:6800/1259916482,v1:172.21.15.59:6801/1259916482] conn(0x7efca0038820 0x7efca003ace0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7efcac006fd0 tx=0x7efcac006e40 comp rx=0 tx=0).stop 2023-12-11T17:37:50.751 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:50.363+0000 7efcbdfc0700 1 -- 172.21.15.59:0/23329273 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efcb81077e0 msgr2=0x7efcb819ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:37:50.751 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:50.363+0000 7efcbdfc0700 1 --2- 172.21.15.59:0/23329273 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efcb81077e0 0x7efcb819ab40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7efca8009130 tx=0x7efca8011ad0 comp rx=0 tx=0).stop 2023-12-11T17:37:50.751 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:50.363+0000 7efcbdfc0700 1 -- 172.21.15.59:0/23329273 shutdown_connections 2023-12-11T17:37:50.751 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:50.363+0000 7efcbdfc0700 1 --2- 172.21.15.59:0/23329273 >> [v2:172.21.15.59:6800/1259916482,v1:172.21.15.59:6801/1259916482] conn(0x7efca0038820 0x7efca003ace0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:37:50.751 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:50.363+0000 7efcbdfc0700 1 --2- 172.21.15.59:0/23329273 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efcb81077e0 0x7efcb819ab40 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:37:50.751 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:50.363+0000 7efcbdfc0700 1 -- 172.21.15.59:0/23329273 >> 172.21.15.59:0/23329273 conn(0x7efcb81011d0 msgr2=0x7efcb8101d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:37:50.751 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:50.363+0000 7efcbdfc0700 1 -- 172.21.15.59:0/23329273 shutdown_connections 2023-12-11T17:37:50.751 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:50.364+0000 7efcbdfc0700 1 -- 172.21.15.59:0/23329273 wait complete. 2023-12-11T17:37:51.615 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:51 smithi059 ceph-mon[98847]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T17:37:51.615 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:51 smithi059 ceph-mon[98847]: Generating ssh key... 2023-12-11T17:37:51.616 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:51 smithi059 ceph-mon[98847]: from='mgr.14118 172.21.15.59:0/103440859' entity='mgr.x' 2023-12-11T17:37:51.616 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:51 smithi059 ceph-mon[98847]: from='mgr.14118 172.21.15.59:0/103440859' entity='mgr.x' 2023-12-11T17:37:51.994 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC2X706m8ehaO/BbVOxHH3AThaWP+IN5sgEq68TzOH8k4qte1+vTAM5nclfi8bYYPoqPCJyrIft9nDQ0rwwGMhMc0pLDw+2Ie6Zlhv80pu8vIL4zshzxFi7o2TEmqc3NYqxrrY234BygnfBzOmsQcNEyHJk8IcswJr4aVN8ucE2HdbBxLLNNW5wnPV1pj95SdY5Jo45PEi620bCuWt2D/Qmc+bUDZOCKUYg0GGKOGzTS9+Oo4uDX1PcByW+zPG+AHLaFcmiyJ+3GdvTNvC0R2LsEaWAAdMR5bMftpbRt0Bl/TOqneIQwxEWocC0HKNDRF3g6aqQgwJ8JnJ1moU6Y7P3zzsKbit0tSYBD5geSFJyfuJ6J4dAHQ5E7pbdqpWyWWCB4ibSM00dopweUBbTw5HxTLo2bTbMBZL41RjL2nGHKX3LFQpdHpQqmDCS7N4IMtg5hz8byUvMhDkmqeAvR0S2LVfV8+trGlnDSGTXllm+sw3Y6VUDxflb00QESSncRS0= ceph-d0a07eaa-984b-11ee-95a3-87774f69a715 2023-12-11T17:37:51.994 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:51.210+0000 7fd3dbc74700 1 Processor -- start 2023-12-11T17:37:51.994 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:51.210+0000 7fd3dbc74700 1 -- start start 2023-12-11T17:37:51.994 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:51.210+0000 7fd3dbc74700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd3d4105be0 0x7fd3d4106000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:37:51.994 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:51.210+0000 7fd3dbc74700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3d41065d0 con 0x7fd3d4105be0 2023-12-11T17:37:51.994 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:51.210+0000 7fd3d9a10700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd3d4105be0 0x7fd3d4106000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:37:51.994 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:51.210+0000 7fd3d9a10700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd3d4105be0 0x7fd3d4106000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59906/0 (socket says 172.21.15.59:59906) 2023-12-11T17:37:51.995 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:51.210+0000 7fd3d9a10700 1 -- 172.21.15.59:0/681744823 learned_addr learned my addr 172.21.15.59:0/681744823 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:37:51.995 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:51.210+0000 7fd3d9a10700 1 -- 172.21.15.59:0/681744823 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3d4106de0 con 0x7fd3d4105be0 2023-12-11T17:37:51.995 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:51.211+0000 7fd3d9a10700 1 --2- 172.21.15.59:0/681744823 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd3d4105be0 0x7fd3d4106000 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd3d00098a0 tx=0x7fd3d0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24ee181583269fe5 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:37:51.995 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:51.211+0000 7fd3d8a0e700 1 -- 172.21.15.59:0/681744823 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd3d0004160 con 0x7fd3d4105be0 2023-12-11T17:37:51.995 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:51.211+0000 7fd3d8a0e700 1 -- 172.21.15.59:0/681744823 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd3d00042c0 con 0x7fd3d4105be0 2023-12-11T17:37:51.995 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:51.211+0000 7fd3d8a0e700 1 -- 172.21.15.59:0/681744823 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd3d0011a00 con 0x7fd3d4105be0 2023-12-11T17:37:51.995 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:51.211+0000 7fd3dbc74700 1 -- 172.21.15.59:0/681744823 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd3d4105be0 msgr2=0x7fd3d4106000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:37:51.995 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:51.211+0000 7fd3dbc74700 1 --2- 172.21.15.59:0/681744823 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd3d4105be0 0x7fd3d4106000 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd3d00098a0 tx=0x7fd3d0009bb0 comp rx=0 tx=0).stop 2023-12-11T17:37:51.995 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:51.211+0000 7fd3dbc74700 1 -- 172.21.15.59:0/681744823 shutdown_connections 2023-12-11T17:37:51.995 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:51.211+0000 7fd3dbc74700 1 --2- 172.21.15.59:0/681744823 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd3d4105be0 0x7fd3d4106000 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:37:51.995 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:51.211+0000 7fd3dbc74700 1 -- 172.21.15.59:0/681744823 >> 172.21.15.59:0/681744823 conn(0x7fd3d4101160 msgr2=0x7fd3d41035c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:37:51.995 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:51.212+0000 7fd3dbc74700 1 -- 172.21.15.59:0/681744823 shutdown_connections 2023-12-11T17:37:51.995 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:51.212+0000 7fd3dbc74700 1 -- 172.21.15.59:0/681744823 wait complete. 2023-12-11T17:37:51.996 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:51.212+0000 7fd3dbc74700 1 Processor -- start 2023-12-11T17:37:51.996 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:51.212+0000 7fd3dbc74700 1 -- start start 2023-12-11T17:37:51.996 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:51.212+0000 7fd3dbc74700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd3d4105be0 0x7fd3d419ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:37:51.996 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:51.212+0000 7fd3dbc74700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3d419b0d0 con 0x7fd3d4105be0 2023-12-11T17:37:51.996 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:51.212+0000 7fd3d9a10700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd3d4105be0 0x7fd3d419ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:37:51.996 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:51.212+0000 7fd3d9a10700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd3d4105be0 0x7fd3d419ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59908/0 (socket says 172.21.15.59:59908) 2023-12-11T17:37:51.996 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:51.212+0000 7fd3d9a10700 1 -- 172.21.15.59:0/3993641603 learned_addr learned my addr 172.21.15.59:0/3993641603 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:37:51.996 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:51.213+0000 7fd3d9a10700 1 -- 172.21.15.59:0/3993641603 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3d0009550 con 0x7fd3d4105be0 2023-12-11T17:37:51.996 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:51.213+0000 7fd3d9a10700 1 --2- 172.21.15.59:0/3993641603 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd3d4105be0 0x7fd3d419ab90 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd3d0011710 tx=0x7fd3d0011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:37:51.996 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:51.213+0000 7fd3caffd700 1 -- 172.21.15.59:0/3993641603 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd3d0003aa0 con 0x7fd3d4105be0 2023-12-11T17:37:51.996 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:51.213+0000 7fd3caffd700 1 -- 172.21.15.59:0/3993641603 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd3d0003c00 con 0x7fd3d4105be0 2023-12-11T17:37:51.996 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:51.213+0000 7fd3dbc74700 1 -- 172.21.15.59:0/3993641603 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3d419b2d0 con 0x7fd3d4105be0 2023-12-11T17:37:51.996 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:51.213+0000 7fd3caffd700 1 -- 172.21.15.59:0/3993641603 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd3d0024800 con 0x7fd3d4105be0 2023-12-11T17:37:51.996 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:51.213+0000 7fd3dbc74700 1 -- 172.21.15.59:0/3993641603 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3d419be30 con 0x7fd3d4105be0 2023-12-11T17:37:51.997 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:51.214+0000 7fd3caffd700 1 -- 172.21.15.59:0/3993641603 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 8) v1 ==== 45580+0+0 (secure 0 0 0) 0x7fd3d001a430 con 0x7fd3d4105be0 2023-12-11T17:37:51.997 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:51.214+0000 7fd3dbc74700 1 -- 172.21.15.59:0/3993641603 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3d419b770 con 0x7fd3d4105be0 2023-12-11T17:37:51.997 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:51.214+0000 7fd3caffd700 1 --2- 172.21.15.59:0/3993641603 >> [v2:172.21.15.59:6800/1259916482,v1:172.21.15.59:6801/1259916482] conn(0x7fd3c0038820 0x7fd3c003ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:37:51.997 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:51.214+0000 7fd3caffd700 1 -- 172.21.15.59:0/3993641603 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fd3d004c830 con 0x7fd3d4105be0 2023-12-11T17:37:51.997 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:51.215+0000 7fd3d920f700 1 --2- 172.21.15.59:0/3993641603 >> [v2:172.21.15.59:6800/1259916482,v1:172.21.15.59:6801/1259916482] conn(0x7fd3c0038820 0x7fd3c003ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:37:51.997 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:51.215+0000 7fd3d920f700 1 --2- 172.21.15.59:0/3993641603 >> [v2:172.21.15.59:6800/1259916482,v1:172.21.15.59:6801/1259916482] conn(0x7fd3c0038820 0x7fd3c003ace0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd3c4006fd0 tx=0x7fd3c4006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:37:51.997 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:51.219+0000 7fd3caffd700 1 -- 172.21.15.59:0/3993641603 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fd3d419b770 con 0x7fd3d4105be0 2023-12-11T17:37:51.997 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:51.587+0000 7fd3dbc74700 1 -- 172.21.15.59:0/3993641603 --> [v2:172.21.15.59:6800/1259916482,v1:172.21.15.59:6801/1259916482] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7fd3d419b770 con 0x7fd3c0038820 2023-12-11T17:37:51.997 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:51.588+0000 7fd3caffd700 1 -- 172.21.15.59:0/3993641603 <== mgr.14118 v2:172.21.15.59:6800/1259916482 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7fd3d419b770 con 0x7fd3c0038820 2023-12-11T17:37:51.997 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:51.590+0000 7fd3dbc74700 1 -- 172.21.15.59:0/3993641603 >> [v2:172.21.15.59:6800/1259916482,v1:172.21.15.59:6801/1259916482] conn(0x7fd3c0038820 msgr2=0x7fd3c003ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:37:51.997 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:51.590+0000 7fd3dbc74700 1 --2- 172.21.15.59:0/3993641603 >> [v2:172.21.15.59:6800/1259916482,v1:172.21.15.59:6801/1259916482] conn(0x7fd3c0038820 0x7fd3c003ace0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd3c4006fd0 tx=0x7fd3c4006e40 comp rx=0 tx=0).stop 2023-12-11T17:37:51.997 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:51.590+0000 7fd3dbc74700 1 -- 172.21.15.59:0/3993641603 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd3d4105be0 msgr2=0x7fd3d419ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:37:51.997 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:51.590+0000 7fd3dbc74700 1 --2- 172.21.15.59:0/3993641603 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd3d4105be0 0x7fd3d419ab90 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd3d0011710 tx=0x7fd3d0011ad0 comp rx=0 tx=0).stop 2023-12-11T17:37:51.998 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:51.590+0000 7fd3dbc74700 1 -- 172.21.15.59:0/3993641603 shutdown_connections 2023-12-11T17:37:51.998 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:51.590+0000 7fd3dbc74700 1 --2- 172.21.15.59:0/3993641603 >> [v2:172.21.15.59:6800/1259916482,v1:172.21.15.59:6801/1259916482] conn(0x7fd3c0038820 0x7fd3c003ace0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:37:51.998 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:51.590+0000 7fd3dbc74700 1 --2- 172.21.15.59:0/3993641603 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd3d4105be0 0x7fd3d419ab90 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:37:51.998 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:51.590+0000 7fd3dbc74700 1 -- 172.21.15.59:0/3993641603 >> 172.21.15.59:0/3993641603 conn(0x7fd3d4101160 msgr2=0x7fd3d41020a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:37:51.998 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:51.590+0000 7fd3dbc74700 1 -- 172.21.15.59:0/3993641603 shutdown_connections 2023-12-11T17:37:51.998 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:51.590+0000 7fd3dbc74700 1 -- 172.21.15.59:0/3993641603 wait complete. 2023-12-11T17:37:51.998 INFO:teuthology.orchestra.run.smithi059.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2023-12-11T17:37:51.998 INFO:teuthology.orchestra.run.smithi059.stdout:Adding key to root@localhost authorized_keys... 2023-12-11T17:37:51.998 INFO:teuthology.orchestra.run.smithi059.stdout:Adding host smithi059... 2023-12-11T17:37:52.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:52 smithi059 ceph-mon[98847]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T17:37:54.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:53 smithi059 ceph-mon[98847]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi059", "addr": "172.21.15.59", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T17:37:54.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:53 smithi059 ceph-mon[98847]: Deploying cephadm binary to smithi059 2023-12-11T17:37:58.552 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout Added host 'smithi059' with addr '172.21.15.59' 2023-12-11T17:37:58.553 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:52.508+0000 7fe101f45700 1 Processor -- start 2023-12-11T17:37:58.553 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:52.508+0000 7fe101f45700 1 -- start start 2023-12-11T17:37:58.553 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:52.509+0000 7fe101f45700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe0fc1077c0 0x7fe0fc109bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:37:58.553 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:52.509+0000 7fe101f45700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0fc07ae20 con 0x7fe0fc1077c0 2023-12-11T17:37:58.553 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:52.509+0000 7fe0fb7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe0fc1077c0 0x7fe0fc109bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:37:58.553 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:52.509+0000 7fe0fb7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe0fc1077c0 0x7fe0fc109bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59912/0 (socket says 172.21.15.59:59912) 2023-12-11T17:37:58.553 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:52.509+0000 7fe0fb7fe700 1 -- 172.21.15.59:0/3859015837 learned_addr learned my addr 172.21.15.59:0/3859015837 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:37:58.554 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:52.509+0000 7fe0fb7fe700 1 -- 172.21.15.59:0/3859015837 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0fc10a0f0 con 0x7fe0fc1077c0 2023-12-11T17:37:58.554 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:52.509+0000 7fe0fb7fe700 1 --2- 172.21.15.59:0/3859015837 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe0fc1077c0 0x7fe0fc109bb0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec0098a0 tx=0x7fe0ec009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2001cc662c55d30 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:37:58.554 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:52.509+0000 7fe0fa7fc700 1 -- 172.21.15.59:0/3859015837 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe0ec004160 con 0x7fe0fc1077c0 2023-12-11T17:37:58.554 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:52.509+0000 7fe0fa7fc700 1 -- 172.21.15.59:0/3859015837 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe0ec0042c0 con 0x7fe0fc1077c0 2023-12-11T17:37:58.554 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:52.509+0000 7fe0fa7fc700 1 -- 172.21.15.59:0/3859015837 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe0ec011a00 con 0x7fe0fc1077c0 2023-12-11T17:37:58.554 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:52.510+0000 7fe101f45700 1 -- 172.21.15.59:0/3859015837 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe0fc1077c0 msgr2=0x7fe0fc109bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:37:58.554 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:52.510+0000 7fe101f45700 1 --2- 172.21.15.59:0/3859015837 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe0fc1077c0 0x7fe0fc109bb0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec0098a0 tx=0x7fe0ec009bb0 comp rx=0 tx=0).stop 2023-12-11T17:37:58.554 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:52.510+0000 7fe101f45700 1 -- 172.21.15.59:0/3859015837 shutdown_connections 2023-12-11T17:37:58.554 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:52.510+0000 7fe101f45700 1 --2- 172.21.15.59:0/3859015837 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe0fc1077c0 0x7fe0fc109bb0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:37:58.554 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:52.510+0000 7fe101f45700 1 -- 172.21.15.59:0/3859015837 >> 172.21.15.59:0/3859015837 conn(0x7fe0fc1011b0 msgr2=0x7fe0fc1035f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:37:58.554 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:52.510+0000 7fe101f45700 1 -- 172.21.15.59:0/3859015837 shutdown_connections 2023-12-11T17:37:58.554 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:52.510+0000 7fe101f45700 1 -- 172.21.15.59:0/3859015837 wait complete. 2023-12-11T17:37:58.554 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:52.511+0000 7fe101f45700 1 Processor -- start 2023-12-11T17:37:58.554 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:52.511+0000 7fe101f45700 1 -- start start 2023-12-11T17:37:58.555 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:52.511+0000 7fe101f45700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe0fc1077c0 0x7fe0fc19ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:37:58.555 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:52.512+0000 7fe101f45700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0fc19b0b0 con 0x7fe0fc1077c0 2023-12-11T17:37:58.555 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:52.512+0000 7fe0fb7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe0fc1077c0 0x7fe0fc19ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:37:58.555 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:52.512+0000 7fe0fb7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe0fc1077c0 0x7fe0fc19ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59914/0 (socket says 172.21.15.59:59914) 2023-12-11T17:37:58.555 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:52.512+0000 7fe0fb7fe700 1 -- 172.21.15.59:0/1728417167 learned_addr learned my addr 172.21.15.59:0/1728417167 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:37:58.555 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:52.512+0000 7fe0fb7fe700 1 -- 172.21.15.59:0/1728417167 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0ec009550 con 0x7fe0fc1077c0 2023-12-11T17:37:58.555 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:52.512+0000 7fe0fb7fe700 1 --2- 172.21.15.59:0/1728417167 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe0fc1077c0 0x7fe0fc19ab70 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec009130 tx=0x7fe0ec011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:37:58.555 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:52.512+0000 7fe0f8ff9700 1 -- 172.21.15.59:0/1728417167 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe0ec0038b0 con 0x7fe0fc1077c0 2023-12-11T17:37:58.555 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:52.512+0000 7fe0f8ff9700 1 -- 172.21.15.59:0/1728417167 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe0ec003a10 con 0x7fe0fc1077c0 2023-12-11T17:37:58.555 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:52.512+0000 7fe101f45700 1 -- 172.21.15.59:0/1728417167 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0fc19b2b0 con 0x7fe0fc1077c0 2023-12-11T17:37:58.555 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:52.512+0000 7fe101f45700 1 -- 172.21.15.59:0/1728417167 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0fc19bdc0 con 0x7fe0fc1077c0 2023-12-11T17:37:58.555 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:52.512+0000 7fe0f8ff9700 1 -- 172.21.15.59:0/1728417167 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe0ec019550 con 0x7fe0fc1077c0 2023-12-11T17:37:58.555 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:52.513+0000 7fe0f8ff9700 1 -- 172.21.15.59:0/1728417167 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 8) v1 ==== 45580+0+0 (secure 0 0 0) 0x7fe0ec019840 con 0x7fe0fc1077c0 2023-12-11T17:37:58.555 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:52.513+0000 7fe0f8ff9700 1 --2- 172.21.15.59:0/1728417167 >> [v2:172.21.15.59:6800/1259916482,v1:172.21.15.59:6801/1259916482] conn(0x7fe0e8038800 0x7fe0e803acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:37:58.555 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:52.513+0000 7fe0f8ff9700 1 -- 172.21.15.59:0/1728417167 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fe0ec04c7d0 con 0x7fe0fc1077c0 2023-12-11T17:37:58.555 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:52.513+0000 7fe101f45700 1 -- 172.21.15.59:0/1728417167 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0dc005320 con 0x7fe0fc1077c0 2023-12-11T17:37:58.556 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:52.513+0000 7fe0faffd700 1 --2- 172.21.15.59:0/1728417167 >> [v2:172.21.15.59:6800/1259916482,v1:172.21.15.59:6801/1259916482] conn(0x7fe0e8038800 0x7fe0e803acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:37:58.556 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:52.514+0000 7fe0faffd700 1 --2- 172.21.15.59:0/1728417167 >> [v2:172.21.15.59:6800/1259916482,v1:172.21.15.59:6801/1259916482] conn(0x7fe0e8038800 0x7fe0e803acc0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe0e4006fd0 tx=0x7fe0e4006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:37:58.556 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:52.518+0000 7fe0f8ff9700 1 -- 172.21.15.59:0/1728417167 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fe0ec019b40 con 0x7fe0fc1077c0 2023-12-11T17:37:58.556 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:52.895+0000 7fe101f45700 1 -- 172.21.15.59:0/1728417167 --> [v2:172.21.15.59:6800/1259916482,v1:172.21.15.59:6801/1259916482] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi059", "addr": "172.21.15.59", "target": ["mon-mgr", ""]}) v1 -- 0x7fe0dc000bf0 con 0x7fe0e8038800 2023-12-11T17:37:58.556 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:58.222+0000 7fe0f8ff9700 1 -- 172.21.15.59:0/1728417167 <== mgr.14118 v2:172.21.15.59:6800/1259916482 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7fe0dc000bf0 con 0x7fe0e8038800 2023-12-11T17:37:58.556 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:58.225+0000 7fe101f45700 1 -- 172.21.15.59:0/1728417167 >> [v2:172.21.15.59:6800/1259916482,v1:172.21.15.59:6801/1259916482] conn(0x7fe0e8038800 msgr2=0x7fe0e803acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:37:58.556 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:58.225+0000 7fe101f45700 1 --2- 172.21.15.59:0/1728417167 >> [v2:172.21.15.59:6800/1259916482,v1:172.21.15.59:6801/1259916482] conn(0x7fe0e8038800 0x7fe0e803acc0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe0e4006fd0 tx=0x7fe0e4006e40 comp rx=0 tx=0).stop 2023-12-11T17:37:58.556 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:58.225+0000 7fe101f45700 1 -- 172.21.15.59:0/1728417167 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe0fc1077c0 msgr2=0x7fe0fc19ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:37:58.556 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:58.225+0000 7fe101f45700 1 --2- 172.21.15.59:0/1728417167 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe0fc1077c0 0x7fe0fc19ab70 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec009130 tx=0x7fe0ec011ad0 comp rx=0 tx=0).stop 2023-12-11T17:37:58.556 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:58.225+0000 7fe101f45700 1 -- 172.21.15.59:0/1728417167 shutdown_connections 2023-12-11T17:37:58.556 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:58.225+0000 7fe101f45700 1 --2- 172.21.15.59:0/1728417167 >> [v2:172.21.15.59:6800/1259916482,v1:172.21.15.59:6801/1259916482] conn(0x7fe0e8038800 0x7fe0e803acc0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:37:58.556 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:58.225+0000 7fe101f45700 1 --2- 172.21.15.59:0/1728417167 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe0fc1077c0 0x7fe0fc19ab70 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:37:58.556 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:58.225+0000 7fe101f45700 1 -- 172.21.15.59:0/1728417167 >> 172.21.15.59:0/1728417167 conn(0x7fe0fc1011b0 msgr2=0x7fe0fc101d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:37:58.556 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:58.225+0000 7fe101f45700 1 -- 172.21.15.59:0/1728417167 shutdown_connections 2023-12-11T17:37:58.556 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:58.226+0000 7fe101f45700 1 -- 172.21.15.59:0/1728417167 wait complete. 2023-12-11T17:37:58.557 INFO:teuthology.orchestra.run.smithi059.stdout:Deploying unmanaged mon service... 2023-12-11T17:37:59.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:59 smithi059 ceph-mon[98847]: from='mgr.14118 172.21.15.59:0/103440859' entity='mgr.x' 2023-12-11T17:37:59.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:59 smithi059 ceph-mon[98847]: Added host smithi059 2023-12-11T17:37:59.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:37:59 smithi059 ceph-mon[98847]: from='mgr.14118 172.21.15.59:0/103440859' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:38:00.199 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2023-12-11T17:38:00.199 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:59.249+0000 7f07998a6700 1 Processor -- start 2023-12-11T17:38:00.199 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:59.249+0000 7f07998a6700 1 -- start start 2023-12-11T17:38:00.199 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:59.249+0000 7f07998a6700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0794107790 0x7f0794109b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:38:00.200 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:59.249+0000 7f07998a6700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f079407ae00 con 0x7f0794107790 2023-12-11T17:38:00.200 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:59.250+0000 7f0792ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0794107790 0x7f0794109b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:38:00.200 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:59.250+0000 7f0792ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0794107790 0x7f0794109b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59948/0 (socket says 172.21.15.59:59948) 2023-12-11T17:38:00.200 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:59.250+0000 7f0792ffd700 1 -- 172.21.15.59:0/1079482187 learned_addr learned my addr 172.21.15.59:0/1079482187 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:38:00.200 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:59.250+0000 7f0792ffd700 1 -- 172.21.15.59:0/1079482187 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f079410a0c0 con 0x7f0794107790 2023-12-11T17:38:00.200 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:59.250+0000 7f0792ffd700 1 --2- 172.21.15.59:0/1079482187 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0794107790 0x7f0794109b80 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f07840098a0 tx=0x7f0784009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75347d87bd6aa1c8 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:38:00.200 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:59.250+0000 7f0791ffb700 1 -- 172.21.15.59:0/1079482187 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0784004160 con 0x7f0794107790 2023-12-11T17:38:00.200 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:59.250+0000 7f0791ffb700 1 -- 172.21.15.59:0/1079482187 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f07840042c0 con 0x7f0794107790 2023-12-11T17:38:00.201 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:59.250+0000 7f0791ffb700 1 -- 172.21.15.59:0/1079482187 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0784011a00 con 0x7f0794107790 2023-12-11T17:38:00.201 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:59.251+0000 7f07998a6700 1 -- 172.21.15.59:0/1079482187 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0794107790 msgr2=0x7f0794109b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:38:00.201 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:59.251+0000 7f07998a6700 1 --2- 172.21.15.59:0/1079482187 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0794107790 0x7f0794109b80 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f07840098a0 tx=0x7f0784009bb0 comp rx=0 tx=0).stop 2023-12-11T17:38:00.201 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:59.251+0000 7f07998a6700 1 -- 172.21.15.59:0/1079482187 shutdown_connections 2023-12-11T17:38:00.201 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:59.251+0000 7f07998a6700 1 --2- 172.21.15.59:0/1079482187 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0794107790 0x7f0794109b80 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:38:00.201 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:59.251+0000 7f07998a6700 1 -- 172.21.15.59:0/1079482187 >> 172.21.15.59:0/1079482187 conn(0x7f0794101140 msgr2=0x7f07941035a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:38:00.201 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:59.251+0000 7f07998a6700 1 -- 172.21.15.59:0/1079482187 shutdown_connections 2023-12-11T17:38:00.201 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:59.251+0000 7f07998a6700 1 -- 172.21.15.59:0/1079482187 wait complete. 2023-12-11T17:38:00.202 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:59.252+0000 7f07998a6700 1 Processor -- start 2023-12-11T17:38:00.202 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:59.252+0000 7f07998a6700 1 -- start start 2023-12-11T17:38:00.202 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:59.252+0000 7f07998a6700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0794107790 0x7f079419ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:38:00.202 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:59.252+0000 7f07998a6700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f079419b050 con 0x7f0794107790 2023-12-11T17:38:00.202 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:59.252+0000 7f0792ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0794107790 0x7f079419ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:38:00.202 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:59.252+0000 7f0792ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0794107790 0x7f079419ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59950/0 (socket says 172.21.15.59:59950) 2023-12-11T17:38:00.202 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:59.252+0000 7f0792ffd700 1 -- 172.21.15.59:0/286602306 learned_addr learned my addr 172.21.15.59:0/286602306 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:38:00.202 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:59.252+0000 7f0792ffd700 1 -- 172.21.15.59:0/286602306 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0784009550 con 0x7f0794107790 2023-12-11T17:38:00.202 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:59.253+0000 7f0792ffd700 1 --2- 172.21.15.59:0/286602306 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0794107790 0x7f079419ab10 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f0784009130 tx=0x7f0784011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:38:00.202 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:59.253+0000 7f07988a4700 1 -- 172.21.15.59:0/286602306 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0784003b70 con 0x7f0794107790 2023-12-11T17:38:00.203 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:59.253+0000 7f07988a4700 1 -- 172.21.15.59:0/286602306 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0784003cd0 con 0x7f0794107790 2023-12-11T17:38:00.203 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:59.253+0000 7f07998a6700 1 -- 172.21.15.59:0/286602306 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f079419b250 con 0x7f0794107790 2023-12-11T17:38:00.203 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:59.253+0000 7f07988a4700 1 -- 172.21.15.59:0/286602306 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f07840197d0 con 0x7f0794107790 2023-12-11T17:38:00.203 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:59.253+0000 7f07998a6700 1 -- 172.21.15.59:0/286602306 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f079419bdb0 con 0x7f0794107790 2023-12-11T17:38:00.203 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:59.254+0000 7f07988a4700 1 -- 172.21.15.59:0/286602306 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 8) v1 ==== 45580+0+0 (secure 0 0 0) 0x7f0784019a10 con 0x7f0794107790 2023-12-11T17:38:00.203 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:59.254+0000 7f07988a4700 1 --2- 172.21.15.59:0/286602306 >> [v2:172.21.15.59:6800/1259916482,v1:172.21.15.59:6801/1259916482] conn(0x7f0780038820 0x7f078003ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:38:00.203 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:59.254+0000 7f07988a4700 1 -- 172.21.15.59:0/286602306 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f078404cac0 con 0x7f0794107790 2023-12-11T17:38:00.203 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:59.254+0000 7f07927fc700 1 --2- 172.21.15.59:0/286602306 >> [v2:172.21.15.59:6800/1259916482,v1:172.21.15.59:6801/1259916482] conn(0x7f0780038820 0x7f078003ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:38:00.204 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:59.254+0000 7f07998a6700 1 -- 172.21.15.59:0/286602306 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f079419b6f0 con 0x7f0794107790 2023-12-11T17:38:00.204 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:59.254+0000 7f07927fc700 1 --2- 172.21.15.59:0/286602306 >> [v2:172.21.15.59:6800/1259916482,v1:172.21.15.59:6801/1259916482] conn(0x7f0780038820 0x7f078003ace0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f077c006fd0 tx=0x7f077c006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:38:00.204 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:59.259+0000 7f07988a4700 1 -- 172.21.15.59:0/286602306 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f079419b6f0 con 0x7f0794107790 2023-12-11T17:38:00.204 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:59.650+0000 7f07998a6700 1 -- 172.21.15.59:0/286602306 --> [v2:172.21.15.59:6800/1259916482,v1:172.21.15.59:6801/1259916482] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f079419b6f0 con 0x7f0780038820 2023-12-11T17:38:00.204 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:59.655+0000 7f07988a4700 1 -- 172.21.15.59:0/286602306 <== mgr.14118 v2:172.21.15.59:6800/1259916482 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f079419b6f0 con 0x7f0780038820 2023-12-11T17:38:00.204 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:59.658+0000 7f07998a6700 1 -- 172.21.15.59:0/286602306 >> [v2:172.21.15.59:6800/1259916482,v1:172.21.15.59:6801/1259916482] conn(0x7f0780038820 msgr2=0x7f078003ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:38:00.204 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:59.658+0000 7f07998a6700 1 --2- 172.21.15.59:0/286602306 >> [v2:172.21.15.59:6800/1259916482,v1:172.21.15.59:6801/1259916482] conn(0x7f0780038820 0x7f078003ace0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f077c006fd0 tx=0x7f077c006e40 comp rx=0 tx=0).stop 2023-12-11T17:38:00.205 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:59.658+0000 7f07998a6700 1 -- 172.21.15.59:0/286602306 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0794107790 msgr2=0x7f079419ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:38:00.205 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:59.658+0000 7f07998a6700 1 --2- 172.21.15.59:0/286602306 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0794107790 0x7f079419ab10 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f0784009130 tx=0x7f0784011ad0 comp rx=0 tx=0).stop 2023-12-11T17:38:00.205 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:59.658+0000 7f07998a6700 1 -- 172.21.15.59:0/286602306 shutdown_connections 2023-12-11T17:38:00.205 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:59.658+0000 7f07998a6700 1 --2- 172.21.15.59:0/286602306 >> [v2:172.21.15.59:6800/1259916482,v1:172.21.15.59:6801/1259916482] conn(0x7f0780038820 0x7f078003ace0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:38:00.205 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:59.658+0000 7f07998a6700 1 --2- 172.21.15.59:0/286602306 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0794107790 0x7f079419ab10 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:38:00.205 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:59.658+0000 7f07998a6700 1 -- 172.21.15.59:0/286602306 >> 172.21.15.59:0/286602306 conn(0x7f0794101140 msgr2=0x7f0794101ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:38:00.205 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:59.658+0000 7f07998a6700 1 -- 172.21.15.59:0/286602306 shutdown_connections 2023-12-11T17:38:00.205 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:37:59.658+0000 7f07998a6700 1 -- 172.21.15.59:0/286602306 wait complete. 2023-12-11T17:38:00.206 INFO:teuthology.orchestra.run.smithi059.stdout:Deploying unmanaged mgr service... 2023-12-11T17:38:00.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:00 smithi059 ceph-mon[98847]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T17:38:00.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:00 smithi059 ceph-mon[98847]: Saving service mon spec with placement count:5 2023-12-11T17:38:00.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:00 smithi059 ceph-mon[98847]: from='mgr.14118 172.21.15.59:0/103440859' entity='mgr.x' 2023-12-11T17:38:00.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:00 smithi059 ceph-mon[98847]: from='mgr.14118 172.21.15.59:0/103440859' entity='mgr.x' 2023-12-11T17:38:01.746 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2023-12-11T17:38:01.746 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:00.931+0000 7fac8fd58700 1 Processor -- start 2023-12-11T17:38:01.746 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:00.931+0000 7fac8fd58700 1 -- start start 2023-12-11T17:38:01.746 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:00.932+0000 7fac8fd58700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac88105bc0 0x7fac88105fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:38:01.746 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:00.932+0000 7fac8fd58700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac881065b0 con 0x7fac88105bc0 2023-12-11T17:38:01.746 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:00.932+0000 7fac8daf4700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac88105bc0 0x7fac88105fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:38:01.746 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:00.932+0000 7fac8daf4700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac88105bc0 0x7fac88105fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59954/0 (socket says 172.21.15.59:59954) 2023-12-11T17:38:01.747 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:00.932+0000 7fac8daf4700 1 -- 172.21.15.59:0/248635222 learned_addr learned my addr 172.21.15.59:0/248635222 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:38:01.747 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:00.932+0000 7fac8daf4700 1 -- 172.21.15.59:0/248635222 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac88106dc0 con 0x7fac88105bc0 2023-12-11T17:38:01.747 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:00.932+0000 7fac8daf4700 1 --2- 172.21.15.59:0/248635222 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac88105bc0 0x7fac88105fe0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fac7c0098a0 tx=0x7fac7c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f73a18663163ef2 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:38:01.747 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:00.932+0000 7fac8caf2700 1 -- 172.21.15.59:0/248635222 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fac7c004160 con 0x7fac88105bc0 2023-12-11T17:38:01.747 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:00.932+0000 7fac8caf2700 1 -- 172.21.15.59:0/248635222 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fac7c0042c0 con 0x7fac88105bc0 2023-12-11T17:38:01.747 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:00.932+0000 7fac8caf2700 1 -- 172.21.15.59:0/248635222 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fac7c0119b0 con 0x7fac88105bc0 2023-12-11T17:38:01.747 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:00.933+0000 7fac8fd58700 1 -- 172.21.15.59:0/248635222 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac88105bc0 msgr2=0x7fac88105fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:38:01.747 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:00.933+0000 7fac8fd58700 1 --2- 172.21.15.59:0/248635222 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac88105bc0 0x7fac88105fe0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fac7c0098a0 tx=0x7fac7c009bb0 comp rx=0 tx=0).stop 2023-12-11T17:38:01.747 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:00.933+0000 7fac8fd58700 1 -- 172.21.15.59:0/248635222 shutdown_connections 2023-12-11T17:38:01.747 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:00.933+0000 7fac8fd58700 1 --2- 172.21.15.59:0/248635222 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac88105bc0 0x7fac88105fe0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:38:01.747 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:00.933+0000 7fac8fd58700 1 -- 172.21.15.59:0/248635222 >> 172.21.15.59:0/248635222 conn(0x7fac88101140 msgr2=0x7fac881035a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:38:01.747 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:00.933+0000 7fac8fd58700 1 -- 172.21.15.59:0/248635222 shutdown_connections 2023-12-11T17:38:01.747 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:00.933+0000 7fac8fd58700 1 -- 172.21.15.59:0/248635222 wait complete. 2023-12-11T17:38:01.747 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:00.934+0000 7fac8fd58700 1 Processor -- start 2023-12-11T17:38:01.747 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:00.934+0000 7fac8fd58700 1 -- start start 2023-12-11T17:38:01.747 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:00.934+0000 7fac8fd58700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac88105bc0 0x7fac8819cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:38:01.748 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:00.934+0000 7fac8fd58700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac8819d260 con 0x7fac88105bc0 2023-12-11T17:38:01.748 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:00.934+0000 7fac8daf4700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac88105bc0 0x7fac8819cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:38:01.748 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:00.934+0000 7fac8daf4700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac88105bc0 0x7fac8819cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59956/0 (socket says 172.21.15.59:59956) 2023-12-11T17:38:01.748 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:00.934+0000 7fac8daf4700 1 -- 172.21.15.59:0/489402330 learned_addr learned my addr 172.21.15.59:0/489402330 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:38:01.748 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:00.934+0000 7fac8daf4700 1 -- 172.21.15.59:0/489402330 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac7c009550 con 0x7fac88105bc0 2023-12-11T17:38:01.748 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:00.935+0000 7fac8daf4700 1 --2- 172.21.15.59:0/489402330 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac88105bc0 0x7fac8819cd20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fac7c006af0 tx=0x7fac7c011a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:38:01.748 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:00.935+0000 7fac7affd700 1 -- 172.21.15.59:0/489402330 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fac7c003a40 con 0x7fac88105bc0 2023-12-11T17:38:01.748 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:00.935+0000 7fac7affd700 1 -- 172.21.15.59:0/489402330 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fac7c003ba0 con 0x7fac88105bc0 2023-12-11T17:38:01.748 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:00.935+0000 7fac8fd58700 1 -- 172.21.15.59:0/489402330 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac8819d460 con 0x7fac88105bc0 2023-12-11T17:38:01.748 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:00.935+0000 7fac7affd700 1 -- 172.21.15.59:0/489402330 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fac7c019690 con 0x7fac88105bc0 2023-12-11T17:38:01.748 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:00.935+0000 7fac8fd58700 1 -- 172.21.15.59:0/489402330 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac8819dfc0 con 0x7fac88105bc0 2023-12-11T17:38:01.748 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:00.936+0000 7fac7affd700 1 -- 172.21.15.59:0/489402330 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 8) v1 ==== 45580+0+0 (secure 0 0 0) 0x7fac7c019950 con 0x7fac88105bc0 2023-12-11T17:38:01.748 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:00.936+0000 7fac7affd700 1 --2- 172.21.15.59:0/489402330 >> [v2:172.21.15.59:6800/1259916482,v1:172.21.15.59:6801/1259916482] conn(0x7fac740387d0 0x7fac7403ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:38:01.748 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:00.936+0000 7fac7affd700 1 -- 172.21.15.59:0/489402330 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fac7c04ca20 con 0x7fac88105bc0 2023-12-11T17:38:01.748 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:00.936+0000 7fac8d2f3700 1 --2- 172.21.15.59:0/489402330 >> [v2:172.21.15.59:6800/1259916482,v1:172.21.15.59:6801/1259916482] conn(0x7fac740387d0 0x7fac7403ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:38:01.749 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:00.936+0000 7fac8fd58700 1 -- 172.21.15.59:0/489402330 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac8819d900 con 0x7fac88105bc0 2023-12-11T17:38:01.749 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:00.936+0000 7fac8d2f3700 1 --2- 172.21.15.59:0/489402330 >> [v2:172.21.15.59:6800/1259916482,v1:172.21.15.59:6801/1259916482] conn(0x7fac740387d0 0x7fac7403ac90 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fac84006fd0 tx=0x7fac84006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:38:01.749 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:00.941+0000 7fac7affd700 1 -- 172.21.15.59:0/489402330 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fac8819d900 con 0x7fac88105bc0 2023-12-11T17:38:01.749 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:01.336+0000 7fac8fd58700 1 -- 172.21.15.59:0/489402330 --> [v2:172.21.15.59:6800/1259916482,v1:172.21.15.59:6801/1259916482] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fac8810a300 con 0x7fac740387d0 2023-12-11T17:38:01.749 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:01.343+0000 7fac7affd700 1 -- 172.21.15.59:0/489402330 <== mgr.14118 v2:172.21.15.59:6800/1259916482 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fac8810a300 con 0x7fac740387d0 2023-12-11T17:38:01.749 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:01.346+0000 7fac8fd58700 1 -- 172.21.15.59:0/489402330 >> [v2:172.21.15.59:6800/1259916482,v1:172.21.15.59:6801/1259916482] conn(0x7fac740387d0 msgr2=0x7fac7403ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:38:01.749 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:01.346+0000 7fac8fd58700 1 --2- 172.21.15.59:0/489402330 >> [v2:172.21.15.59:6800/1259916482,v1:172.21.15.59:6801/1259916482] conn(0x7fac740387d0 0x7fac7403ac90 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fac84006fd0 tx=0x7fac84006e40 comp rx=0 tx=0).stop 2023-12-11T17:38:01.749 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:01.346+0000 7fac8fd58700 1 -- 172.21.15.59:0/489402330 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac88105bc0 msgr2=0x7fac8819cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:38:01.749 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:01.346+0000 7fac8fd58700 1 --2- 172.21.15.59:0/489402330 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac88105bc0 0x7fac8819cd20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fac7c006af0 tx=0x7fac7c011a10 comp rx=0 tx=0).stop 2023-12-11T17:38:01.749 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:01.346+0000 7fac8fd58700 1 -- 172.21.15.59:0/489402330 shutdown_connections 2023-12-11T17:38:01.749 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:01.346+0000 7fac8fd58700 1 --2- 172.21.15.59:0/489402330 >> [v2:172.21.15.59:6800/1259916482,v1:172.21.15.59:6801/1259916482] conn(0x7fac740387d0 0x7fac7403ac90 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:38:01.749 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:01.346+0000 7fac8fd58700 1 --2- 172.21.15.59:0/489402330 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac88105bc0 0x7fac8819cd20 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:38:01.749 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:01.346+0000 7fac8fd58700 1 -- 172.21.15.59:0/489402330 >> 172.21.15.59:0/489402330 conn(0x7fac88101140 msgr2=0x7fac881020b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:38:01.749 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:01.346+0000 7fac8fd58700 1 -- 172.21.15.59:0/489402330 shutdown_connections 2023-12-11T17:38:01.750 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:01.346+0000 7fac8fd58700 1 -- 172.21.15.59:0/489402330 wait complete. 2023-12-11T17:38:02.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:02 smithi059 ceph-mon[98847]: from='mgr.14118 172.21.15.59:0/103440859' entity='mgr.x' 2023-12-11T17:38:02.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:02 smithi059 ceph-mon[98847]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T17:38:02.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:02 smithi059 ceph-mon[98847]: Saving service mgr spec with placement count:2 2023-12-11T17:38:02.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:02 smithi059 ceph-mon[98847]: from='mgr.14118 172.21.15.59:0/103440859' entity='mgr.x' 2023-12-11T17:38:02.988 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:02.233+0000 7f81ea905700 1 Processor -- start 2023-12-11T17:38:02.988 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:02.233+0000 7f81ea905700 1 -- start start 2023-12-11T17:38:02.988 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:02.234+0000 7f81ea905700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f81e4105c20 0x7f81e4106040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:38:02.988 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:02.234+0000 7f81ea905700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81e4106610 con 0x7f81e4105c20 2023-12-11T17:38:02.988 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:02.234+0000 7f81e3fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f81e4105c20 0x7f81e4106040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:38:02.988 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:02.234+0000 7f81e3fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f81e4105c20 0x7f81e4106040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59960/0 (socket says 172.21.15.59:59960) 2023-12-11T17:38:02.988 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:02.234+0000 7f81e3fff700 1 -- 172.21.15.59:0/530989569 learned_addr learned my addr 172.21.15.59:0/530989569 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:38:02.989 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:02.234+0000 7f81e3fff700 1 -- 172.21.15.59:0/530989569 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81e4106e20 con 0x7f81e4105c20 2023-12-11T17:38:02.989 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:02.234+0000 7f81e3fff700 1 --2- 172.21.15.59:0/530989569 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f81e4105c20 0x7f81e4106040 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f81d40098a0 tx=0x7f81d4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57f2b6e19fbfa27e server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:38:02.989 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:02.234+0000 7f81e2ffd700 1 -- 172.21.15.59:0/530989569 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f81d4004160 con 0x7f81e4105c20 2023-12-11T17:38:02.989 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:02.235+0000 7f81e2ffd700 1 -- 172.21.15.59:0/530989569 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f81d40042c0 con 0x7f81e4105c20 2023-12-11T17:38:02.989 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:02.235+0000 7f81e2ffd700 1 -- 172.21.15.59:0/530989569 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f81d40119b0 con 0x7f81e4105c20 2023-12-11T17:38:02.989 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:02.235+0000 7f81ea905700 1 -- 172.21.15.59:0/530989569 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f81e4105c20 msgr2=0x7f81e4106040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:38:02.989 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:02.235+0000 7f81ea905700 1 --2- 172.21.15.59:0/530989569 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f81e4105c20 0x7f81e4106040 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f81d40098a0 tx=0x7f81d4009bb0 comp rx=0 tx=0).stop 2023-12-11T17:38:02.989 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:02.235+0000 7f81ea905700 1 -- 172.21.15.59:0/530989569 shutdown_connections 2023-12-11T17:38:02.989 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:02.235+0000 7f81ea905700 1 --2- 172.21.15.59:0/530989569 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f81e4105c20 0x7f81e4106040 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:38:02.989 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:02.235+0000 7f81ea905700 1 -- 172.21.15.59:0/530989569 >> 172.21.15.59:0/530989569 conn(0x7f81e41011c0 msgr2=0x7f81e4103600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:38:02.989 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:02.235+0000 7f81ea905700 1 -- 172.21.15.59:0/530989569 shutdown_connections 2023-12-11T17:38:02.989 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:02.235+0000 7f81ea905700 1 -- 172.21.15.59:0/530989569 wait complete. 2023-12-11T17:38:02.989 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:02.236+0000 7f81ea905700 1 Processor -- start 2023-12-11T17:38:02.989 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:02.236+0000 7f81ea905700 1 -- start start 2023-12-11T17:38:02.989 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:02.236+0000 7f81ea905700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f81e4105c20 0x7f81e419fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:38:02.989 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:02.236+0000 7f81ea905700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81e419b5d0 con 0x7f81e4105c20 2023-12-11T17:38:02.990 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:02.236+0000 7f81e3fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f81e4105c20 0x7f81e419fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:38:02.990 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:02.236+0000 7f81e3fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f81e4105c20 0x7f81e419fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59962/0 (socket says 172.21.15.59:59962) 2023-12-11T17:38:02.990 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:02.236+0000 7f81e3fff700 1 -- 172.21.15.59:0/3935940025 learned_addr learned my addr 172.21.15.59:0/3935940025 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:38:02.990 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:02.237+0000 7f81e3fff700 1 -- 172.21.15.59:0/3935940025 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81d4009550 con 0x7f81e4105c20 2023-12-11T17:38:02.990 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:02.237+0000 7f81e3fff700 1 --2- 172.21.15.59:0/3935940025 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f81e4105c20 0x7f81e419fbe0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f81d4011670 tx=0x7f81d4011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:38:02.990 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:02.237+0000 7f81e17fa700 1 -- 172.21.15.59:0/3935940025 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f81d4003af0 con 0x7f81e4105c20 2023-12-11T17:38:02.990 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:02.237+0000 7f81e17fa700 1 -- 172.21.15.59:0/3935940025 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f81d4003c50 con 0x7f81e4105c20 2023-12-11T17:38:02.990 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:02.237+0000 7f81ea905700 1 -- 172.21.15.59:0/3935940025 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81e407ab90 con 0x7f81e4105c20 2023-12-11T17:38:02.990 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:02.237+0000 7f81e17fa700 1 -- 172.21.15.59:0/3935940025 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f81d4024820 con 0x7f81e4105c20 2023-12-11T17:38:02.990 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:02.237+0000 7f81ea905700 1 -- 172.21.15.59:0/3935940025 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81e41a0120 con 0x7f81e4105c20 2023-12-11T17:38:02.990 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:02.238+0000 7f81e17fa700 1 -- 172.21.15.59:0/3935940025 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 8) v1 ==== 45580+0+0 (secure 0 0 0) 0x7f81d4024a50 con 0x7f81e4105c20 2023-12-11T17:38:02.990 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:02.238+0000 7f81e17fa700 1 --2- 172.21.15.59:0/3935940025 >> [v2:172.21.15.59:6800/1259916482,v1:172.21.15.59:6801/1259916482] conn(0x7f81d0038870 0x7f81d003ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:38:02.990 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:02.238+0000 7f81e17fa700 1 -- 172.21.15.59:0/3935940025 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f81d404c960 con 0x7f81e4105c20 2023-12-11T17:38:02.990 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:02.238+0000 7f81ea905700 1 -- 172.21.15.59:0/3935940025 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81e4067940 con 0x7f81e4105c20 2023-12-11T17:38:02.991 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:02.238+0000 7f81e37fe700 1 --2- 172.21.15.59:0/3935940025 >> [v2:172.21.15.59:6800/1259916482,v1:172.21.15.59:6801/1259916482] conn(0x7f81d0038870 0x7f81d003ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:38:02.991 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:02.239+0000 7f81e37fe700 1 --2- 172.21.15.59:0/3935940025 >> [v2:172.21.15.59:6800/1259916482,v1:172.21.15.59:6801/1259916482] conn(0x7f81d0038870 0x7f81d003ad30 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f81cc006fd0 tx=0x7f81cc006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:38:02.991 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:02.243+0000 7f81e17fa700 1 -- 172.21.15.59:0/3935940025 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f81d4024d50 con 0x7f81e4105c20 2023-12-11T17:38:02.991 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:02.611+0000 7f81ea905700 1 -- 172.21.15.59:0/3935940025 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7f81e410a050 con 0x7f81e4105c20 2023-12-11T17:38:02.991 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:02.614+0000 7f81e17fa700 1 -- 172.21.15.59:0/3935940025 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7f81d402c350 con 0x7f81e4105c20 2023-12-11T17:38:02.991 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:02.616+0000 7f81ea905700 1 -- 172.21.15.59:0/3935940025 >> [v2:172.21.15.59:6800/1259916482,v1:172.21.15.59:6801/1259916482] conn(0x7f81d0038870 msgr2=0x7f81d003ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:38:02.991 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:02.616+0000 7f81ea905700 1 --2- 172.21.15.59:0/3935940025 >> [v2:172.21.15.59:6800/1259916482,v1:172.21.15.59:6801/1259916482] conn(0x7f81d0038870 0x7f81d003ad30 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f81cc006fd0 tx=0x7f81cc006e40 comp rx=0 tx=0).stop 2023-12-11T17:38:02.991 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:02.616+0000 7f81ea905700 1 -- 172.21.15.59:0/3935940025 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f81e4105c20 msgr2=0x7f81e419fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:38:02.991 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:02.616+0000 7f81ea905700 1 --2- 172.21.15.59:0/3935940025 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f81e4105c20 0x7f81e419fbe0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f81d4011670 tx=0x7f81d4011a30 comp rx=0 tx=0).stop 2023-12-11T17:38:02.991 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:02.616+0000 7f81ea905700 1 -- 172.21.15.59:0/3935940025 shutdown_connections 2023-12-11T17:38:02.991 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:02.616+0000 7f81ea905700 1 --2- 172.21.15.59:0/3935940025 >> [v2:172.21.15.59:6800/1259916482,v1:172.21.15.59:6801/1259916482] conn(0x7f81d0038870 0x7f81d003ad30 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:38:02.991 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:02.616+0000 7f81ea905700 1 --2- 172.21.15.59:0/3935940025 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f81e4105c20 0x7f81e419fbe0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:38:02.991 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:02.616+0000 7f81ea905700 1 -- 172.21.15.59:0/3935940025 >> 172.21.15.59:0/3935940025 conn(0x7f81e41011c0 msgr2=0x7f81e4101d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:38:02.991 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:02.616+0000 7f81ea905700 1 -- 172.21.15.59:0/3935940025 shutdown_connections 2023-12-11T17:38:02.991 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:02.616+0000 7f81ea905700 1 -- 172.21.15.59:0/3935940025 wait complete. 2023-12-11T17:38:03.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:03 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/3935940025' entity='client.admin' 2023-12-11T17:38:03.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:03 smithi059 ceph-mon[98847]: from='mgr.14118 172.21.15.59:0/103440859' entity='mgr.x' 2023-12-11T17:38:04.336 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:03.521+0000 7f38a853c700 1 Processor -- start 2023-12-11T17:38:04.336 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:03.521+0000 7f38a853c700 1 -- start start 2023-12-11T17:38:04.336 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:03.522+0000 7f38a853c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f38a0107810 0x7f38a0109c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:38:04.336 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:03.522+0000 7f38a853c700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38a007ad80 con 0x7f38a0107810 2023-12-11T17:38:04.336 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:03.522+0000 7f38a62d8700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f38a0107810 0x7f38a0109c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:38:04.337 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:03.522+0000 7f38a62d8700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f38a0107810 0x7f38a0109c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59966/0 (socket says 172.21.15.59:59966) 2023-12-11T17:38:04.337 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:03.522+0000 7f38a62d8700 1 -- 172.21.15.59:0/1488142190 learned_addr learned my addr 172.21.15.59:0/1488142190 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:38:04.337 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:03.522+0000 7f38a62d8700 1 -- 172.21.15.59:0/1488142190 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38a007aec0 con 0x7f38a0107810 2023-12-11T17:38:04.337 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:03.522+0000 7f38a62d8700 1 --2- 172.21.15.59:0/1488142190 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f38a0107810 0x7f38a0109c00 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f389c0098a0 tx=0x7f389c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a53ef2187a173b46 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:38:04.337 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:03.523+0000 7f38a52d6700 1 -- 172.21.15.59:0/1488142190 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f389c004160 con 0x7f38a0107810 2023-12-11T17:38:04.337 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:03.523+0000 7f38a52d6700 1 -- 172.21.15.59:0/1488142190 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f389c0042c0 con 0x7f38a0107810 2023-12-11T17:38:04.337 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:03.523+0000 7f38a52d6700 1 -- 172.21.15.59:0/1488142190 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f389c0119b0 con 0x7f38a0107810 2023-12-11T17:38:04.337 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:03.523+0000 7f38a853c700 1 -- 172.21.15.59:0/1488142190 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f38a0107810 msgr2=0x7f38a0109c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:38:04.337 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:03.523+0000 7f38a853c700 1 --2- 172.21.15.59:0/1488142190 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f38a0107810 0x7f38a0109c00 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f389c0098a0 tx=0x7f389c009bb0 comp rx=0 tx=0).stop 2023-12-11T17:38:04.337 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:03.523+0000 7f38a853c700 1 -- 172.21.15.59:0/1488142190 shutdown_connections 2023-12-11T17:38:04.337 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:03.523+0000 7f38a853c700 1 --2- 172.21.15.59:0/1488142190 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f38a0107810 0x7f38a0109c00 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:38:04.337 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:03.523+0000 7f38a853c700 1 -- 172.21.15.59:0/1488142190 >> 172.21.15.59:0/1488142190 conn(0x7f38a0101140 msgr2=0x7f38a01035a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:38:04.337 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:03.524+0000 7f38a853c700 1 -- 172.21.15.59:0/1488142190 shutdown_connections 2023-12-11T17:38:04.337 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:03.524+0000 7f38a853c700 1 -- 172.21.15.59:0/1488142190 wait complete. 2023-12-11T17:38:04.337 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:03.524+0000 7f38a853c700 1 Processor -- start 2023-12-11T17:38:04.338 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:03.524+0000 7f38a853c700 1 -- start start 2023-12-11T17:38:04.338 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:03.524+0000 7f38a853c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f38a0107810 0x7f38a019aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:38:04.338 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:03.524+0000 7f38a853c700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38a019b010 con 0x7f38a0107810 2023-12-11T17:38:04.338 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:03.525+0000 7f38a62d8700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f38a0107810 0x7f38a019aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:38:04.338 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:03.525+0000 7f38a62d8700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f38a0107810 0x7f38a019aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59968/0 (socket says 172.21.15.59:59968) 2023-12-11T17:38:04.338 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:03.525+0000 7f38a62d8700 1 -- 172.21.15.59:0/1412475374 learned_addr learned my addr 172.21.15.59:0/1412475374 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:38:04.338 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:03.525+0000 7f38a62d8700 1 -- 172.21.15.59:0/1412475374 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f389c009550 con 0x7f38a0107810 2023-12-11T17:38:04.338 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:03.525+0000 7f38a62d8700 1 --2- 172.21.15.59:0/1412475374 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f38a0107810 0x7f38a019aad0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f389c006af0 tx=0x7f389c011a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:38:04.338 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:03.525+0000 7f38977fe700 1 -- 172.21.15.59:0/1412475374 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f389c003a40 con 0x7f38a0107810 2023-12-11T17:38:04.338 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:03.525+0000 7f38977fe700 1 -- 172.21.15.59:0/1412475374 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f389c003ba0 con 0x7f38a0107810 2023-12-11T17:38:04.338 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:03.525+0000 7f38977fe700 1 -- 172.21.15.59:0/1412475374 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f389c0196c0 con 0x7f38a0107810 2023-12-11T17:38:04.338 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:03.525+0000 7f38a853c700 1 -- 172.21.15.59:0/1412475374 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38a019b210 con 0x7f38a0107810 2023-12-11T17:38:04.338 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:03.525+0000 7f38a853c700 1 -- 172.21.15.59:0/1412475374 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38a019bd70 con 0x7f38a0107810 2023-12-11T17:38:04.338 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:03.526+0000 7f38977fe700 1 -- 172.21.15.59:0/1412475374 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 8) v1 ==== 45580+0+0 (secure 0 0 0) 0x7f389c019980 con 0x7f38a0107810 2023-12-11T17:38:04.339 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:03.527+0000 7f38977fe700 1 --2- 172.21.15.59:0/1412475374 >> [v2:172.21.15.59:6800/1259916482,v1:172.21.15.59:6801/1259916482] conn(0x7f388c0387d0 0x7f388c03ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:38:04.339 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:03.527+0000 7f38977fe700 1 -- 172.21.15.59:0/1412475374 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f389c04c980 con 0x7f38a0107810 2023-12-11T17:38:04.339 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:03.527+0000 7f38a5ad7700 1 --2- 172.21.15.59:0/1412475374 >> [v2:172.21.15.59:6800/1259916482,v1:172.21.15.59:6801/1259916482] conn(0x7f388c0387d0 0x7f388c03ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:38:04.339 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:03.527+0000 7f38a853c700 1 -- 172.21.15.59:0/1412475374 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38a019b6b0 con 0x7f38a0107810 2023-12-11T17:38:04.339 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:03.527+0000 7f38a5ad7700 1 --2- 172.21.15.59:0/1412475374 >> [v2:172.21.15.59:6800/1259916482,v1:172.21.15.59:6801/1259916482] conn(0x7f388c0387d0 0x7f388c03ac90 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3890006fd0 tx=0x7f3890006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:38:04.339 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:03.532+0000 7f38977fe700 1 -- 172.21.15.59:0/1412475374 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f38a019b6b0 con 0x7f38a0107810 2023-12-11T17:38:04.339 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:03.913+0000 7f38a853c700 1 -- 172.21.15.59:0/1412475374 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7f38a019b6b0 con 0x7f38a0107810 2023-12-11T17:38:04.339 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:03.915+0000 7f38977fe700 1 -- 172.21.15.59:0/1412475374 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7f389c0243f0 con 0x7f38a0107810 2023-12-11T17:38:04.339 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:03.918+0000 7f38a853c700 1 -- 172.21.15.59:0/1412475374 >> [v2:172.21.15.59:6800/1259916482,v1:172.21.15.59:6801/1259916482] conn(0x7f388c0387d0 msgr2=0x7f388c03ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:38:04.339 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:03.918+0000 7f38a853c700 1 --2- 172.21.15.59:0/1412475374 >> [v2:172.21.15.59:6800/1259916482,v1:172.21.15.59:6801/1259916482] conn(0x7f388c0387d0 0x7f388c03ac90 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3890006fd0 tx=0x7f3890006e40 comp rx=0 tx=0).stop 2023-12-11T17:38:04.339 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:03.918+0000 7f38a853c700 1 -- 172.21.15.59:0/1412475374 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f38a0107810 msgr2=0x7f38a019aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:38:04.339 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:03.918+0000 7f38a853c700 1 --2- 172.21.15.59:0/1412475374 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f38a0107810 0x7f38a019aad0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f389c006af0 tx=0x7f389c011a10 comp rx=0 tx=0).stop 2023-12-11T17:38:04.339 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:03.918+0000 7f38a853c700 1 -- 172.21.15.59:0/1412475374 shutdown_connections 2023-12-11T17:38:04.339 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:03.918+0000 7f38a853c700 1 --2- 172.21.15.59:0/1412475374 >> [v2:172.21.15.59:6800/1259916482,v1:172.21.15.59:6801/1259916482] conn(0x7f388c0387d0 0x7f388c03ac90 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:38:04.340 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:03.918+0000 7f38a853c700 1 --2- 172.21.15.59:0/1412475374 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f38a0107810 0x7f38a019aad0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:38:04.340 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:03.918+0000 7f38a853c700 1 -- 172.21.15.59:0/1412475374 >> 172.21.15.59:0/1412475374 conn(0x7f38a0101140 msgr2=0x7f38a0101c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:38:04.340 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:03.918+0000 7f38a853c700 1 -- 172.21.15.59:0/1412475374 shutdown_connections 2023-12-11T17:38:04.340 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:03.918+0000 7f38a853c700 1 -- 172.21.15.59:0/1412475374 wait complete. 2023-12-11T17:38:04.340 INFO:teuthology.orchestra.run.smithi059.stdout:Enabling the dashboard module... 2023-12-11T17:38:05.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:04 smithi059 ceph-mon[98847]: from='mgr.14118 172.21.15.59:0/103440859' entity='mgr.x' 2023-12-11T17:38:05.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:04 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/1412475374' entity='client.admin' 2023-12-11T17:38:06.154 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:38:05 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: ignoring --setuser ceph since I am not root 2023-12-11T17:38:06.154 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:38:05 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: ignoring --setgroup ceph since I am not root 2023-12-11T17:38:06.154 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:38:06 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T17:38:06.096+0000 7fabc0506200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-11T17:38:06.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:05 smithi059 ceph-mon[98847]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T17:38:06.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:05 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/1895435568' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2023-12-11T17:38:06.278 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:04.763+0000 7fd9dd983700 1 Processor -- start 2023-12-11T17:38:06.283 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:04.763+0000 7fd9dd983700 1 -- start start 2023-12-11T17:38:06.283 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:04.763+0000 7fd9dd983700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd9d8106d10 0x7fd9d8107130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:38:06.283 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:04.763+0000 7fd9dd983700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d8107700 con 0x7fd9d8106d10 2023-12-11T17:38:06.284 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:04.763+0000 7fd9d6ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd9d8106d10 0x7fd9d8107130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:38:06.284 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:04.763+0000 7fd9d6ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd9d8106d10 0x7fd9d8107130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59972/0 (socket says 172.21.15.59:59972) 2023-12-11T17:38:06.284 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:04.763+0000 7fd9d6ffd700 1 -- 172.21.15.59:0/3892220364 learned_addr learned my addr 172.21.15.59:0/3892220364 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:38:06.284 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:04.763+0000 7fd9d6ffd700 1 -- 172.21.15.59:0/3892220364 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9d8107ec0 con 0x7fd9d8106d10 2023-12-11T17:38:06.284 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:04.764+0000 7fd9d6ffd700 1 --2- 172.21.15.59:0/3892220364 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd9d8106d10 0x7fd9d8107130 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd9c80098a0 tx=0x7fd9c8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abd46f1b6045ede5 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:38:06.284 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:04.764+0000 7fd9d5ffb700 1 -- 172.21.15.59:0/3892220364 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd9c8004160 con 0x7fd9d8106d10 2023-12-11T17:38:06.284 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:04.764+0000 7fd9d5ffb700 1 -- 172.21.15.59:0/3892220364 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd9c80042c0 con 0x7fd9d8106d10 2023-12-11T17:38:06.284 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:04.764+0000 7fd9d5ffb700 1 -- 172.21.15.59:0/3892220364 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd9c80119b0 con 0x7fd9d8106d10 2023-12-11T17:38:06.284 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:04.764+0000 7fd9dd983700 1 -- 172.21.15.59:0/3892220364 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd9d8106d10 msgr2=0x7fd9d8107130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:38:06.284 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:04.764+0000 7fd9dd983700 1 --2- 172.21.15.59:0/3892220364 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd9d8106d10 0x7fd9d8107130 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd9c80098a0 tx=0x7fd9c8009bb0 comp rx=0 tx=0).stop 2023-12-11T17:38:06.284 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:04.765+0000 7fd9dd983700 1 -- 172.21.15.59:0/3892220364 shutdown_connections 2023-12-11T17:38:06.284 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:04.765+0000 7fd9dd983700 1 --2- 172.21.15.59:0/3892220364 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd9d8106d10 0x7fd9d8107130 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:38:06.285 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:04.765+0000 7fd9dd983700 1 -- 172.21.15.59:0/3892220364 >> 172.21.15.59:0/3892220364 conn(0x7fd9d81022d0 msgr2=0x7fd9d81046f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:38:06.285 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:04.765+0000 7fd9dd983700 1 -- 172.21.15.59:0/3892220364 shutdown_connections 2023-12-11T17:38:06.285 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:04.765+0000 7fd9dd983700 1 -- 172.21.15.59:0/3892220364 wait complete. 2023-12-11T17:38:06.285 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:04.765+0000 7fd9dd983700 1 Processor -- start 2023-12-11T17:38:06.285 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:04.766+0000 7fd9dd983700 1 -- start start 2023-12-11T17:38:06.285 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:04.766+0000 7fd9dd983700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd9d8106d10 0x7fd9d819aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:38:06.285 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:04.766+0000 7fd9dd983700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d819b0e0 con 0x7fd9d8106d10 2023-12-11T17:38:06.285 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:04.766+0000 7fd9d6ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd9d8106d10 0x7fd9d819aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:38:06.285 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:04.766+0000 7fd9d6ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd9d8106d10 0x7fd9d819aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59974/0 (socket says 172.21.15.59:59974) 2023-12-11T17:38:06.285 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:04.766+0000 7fd9d6ffd700 1 -- 172.21.15.59:0/1895435568 learned_addr learned my addr 172.21.15.59:0/1895435568 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:38:06.285 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:04.766+0000 7fd9d6ffd700 1 -- 172.21.15.59:0/1895435568 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9c8009550 con 0x7fd9d8106d10 2023-12-11T17:38:06.285 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:04.767+0000 7fd9d6ffd700 1 --2- 172.21.15.59:0/1895435568 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd9d8106d10 0x7fd9d819aba0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd9c8011670 tx=0x7fd9c8011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:38:06.285 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:04.767+0000 7fd9dc981700 1 -- 172.21.15.59:0/1895435568 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd9c8003af0 con 0x7fd9d8106d10 2023-12-11T17:38:06.285 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:04.767+0000 7fd9dc981700 1 -- 172.21.15.59:0/1895435568 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd9c8003c50 con 0x7fd9d8106d10 2023-12-11T17:38:06.285 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:04.767+0000 7fd9dd983700 1 -- 172.21.15.59:0/1895435568 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9d819b2e0 con 0x7fd9d8106d10 2023-12-11T17:38:06.286 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:04.767+0000 7fd9dc981700 1 -- 172.21.15.59:0/1895435568 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd9c8024820 con 0x7fd9d8106d10 2023-12-11T17:38:06.286 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:04.767+0000 7fd9dd983700 1 -- 172.21.15.59:0/1895435568 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9d819be40 con 0x7fd9d8106d10 2023-12-11T17:38:06.286 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:04.768+0000 7fd9dc981700 1 -- 172.21.15.59:0/1895435568 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 8) v1 ==== 45580+0+0 (secure 0 0 0) 0x7fd9c8024a50 con 0x7fd9d8106d10 2023-12-11T17:38:06.286 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:04.768+0000 7fd9dd983700 1 -- 172.21.15.59:0/1895435568 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9c4005320 con 0x7fd9d8106d10 2023-12-11T17:38:06.286 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:04.768+0000 7fd9dc981700 1 --2- 172.21.15.59:0/1895435568 >> [v2:172.21.15.59:6800/1259916482,v1:172.21.15.59:6801/1259916482] conn(0x7fd9c0038870 0x7fd9c003ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:38:06.286 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:04.768+0000 7fd9dc981700 1 -- 172.21.15.59:0/1895435568 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fd9c804c810 con 0x7fd9d8106d10 2023-12-11T17:38:06.286 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:04.768+0000 7fd9d67fc700 1 --2- 172.21.15.59:0/1895435568 >> [v2:172.21.15.59:6800/1259916482,v1:172.21.15.59:6801/1259916482] conn(0x7fd9c0038870 0x7fd9c003ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:38:06.286 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:04.769+0000 7fd9d67fc700 1 --2- 172.21.15.59:0/1895435568 >> [v2:172.21.15.59:6800/1259916482,v1:172.21.15.59:6801/1259916482] conn(0x7fd9c0038870 0x7fd9c003ad30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc006fd0 tx=0x7fd9cc006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:38:06.286 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:04.773+0000 7fd9dc981700 1 -- 172.21.15.59:0/1895435568 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fd9c8024d00 con 0x7fd9d8106d10 2023-12-11T17:38:06.286 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:05.267+0000 7fd9dd983700 1 -- 172.21.15.59:0/1895435568 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7fd9c40055d0 con 0x7fd9d8106d10 2023-12-11T17:38:06.286 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:05.884+0000 7fd9dc981700 1 -- 172.21.15.59:0/1895435568 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7fd9c802c350 con 0x7fd9d8106d10 2023-12-11T17:38:06.286 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:05.886+0000 7fd9dc981700 1 -- 172.21.15.59:0/1895435568 <== mon.0 v2:172.21.15.59:3300/0 8 ==== mgrmap(e 9) v1 ==== 45593+0+0 (secure 0 0 0) 0x7fd9c804ecd0 con 0x7fd9d8106d10 2023-12-11T17:38:06.286 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:05.888+0000 7fd9dd983700 1 -- 172.21.15.59:0/1895435568 >> [v2:172.21.15.59:6800/1259916482,v1:172.21.15.59:6801/1259916482] conn(0x7fd9c0038870 msgr2=0x7fd9c003ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:38:06.287 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:05.888+0000 7fd9dd983700 1 --2- 172.21.15.59:0/1895435568 >> [v2:172.21.15.59:6800/1259916482,v1:172.21.15.59:6801/1259916482] conn(0x7fd9c0038870 0x7fd9c003ad30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc006fd0 tx=0x7fd9cc006e40 comp rx=0 tx=0).stop 2023-12-11T17:38:06.287 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:05.888+0000 7fd9dd983700 1 -- 172.21.15.59:0/1895435568 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd9d8106d10 msgr2=0x7fd9d819aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:38:06.287 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:05.888+0000 7fd9dd983700 1 --2- 172.21.15.59:0/1895435568 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd9d8106d10 0x7fd9d819aba0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd9c8011670 tx=0x7fd9c8011a30 comp rx=0 tx=0).stop 2023-12-11T17:38:06.287 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:05.888+0000 7fd9dd983700 1 -- 172.21.15.59:0/1895435568 shutdown_connections 2023-12-11T17:38:06.287 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:05.888+0000 7fd9dd983700 1 --2- 172.21.15.59:0/1895435568 >> [v2:172.21.15.59:6800/1259916482,v1:172.21.15.59:6801/1259916482] conn(0x7fd9c0038870 0x7fd9c003ad30 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:38:06.287 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:05.888+0000 7fd9dd983700 1 --2- 172.21.15.59:0/1895435568 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd9d8106d10 0x7fd9d819aba0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:38:06.287 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:05.888+0000 7fd9dd983700 1 -- 172.21.15.59:0/1895435568 >> 172.21.15.59:0/1895435568 conn(0x7fd9d81022d0 msgr2=0x7fd9d8102e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:38:06.287 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:05.889+0000 7fd9dd983700 1 -- 172.21.15.59:0/1895435568 shutdown_connections 2023-12-11T17:38:06.287 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:05.889+0000 7fd9dd983700 1 -- 172.21.15.59:0/1895435568 wait complete. 2023-12-11T17:38:06.458 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:38:06 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T17:38:06.193+0000 7fabc0506200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-11T17:38:06.459 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:38:06 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T17:38:06.304+0000 7fabc0506200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-11T17:38:06.459 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:38:06 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T17:38:06.420+0000 7fabc0506200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-11T17:38:06.887 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:38:06 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T17:38:06.635+0000 7fabc0506200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-11T17:38:07.154 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:38:07 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T17:38:07.115+0000 7fabc0506200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-11T17:38:07.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:06 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/1895435568' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2023-12-11T17:38:07.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:06 smithi059 ceph-mon[98847]: mgrmap e9: x(active, since 21s) 2023-12-11T17:38:07.632 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout { 2023-12-11T17:38:07.632 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "epoch": 9, 2023-12-11T17:38:07.632 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "available": true, 2023-12-11T17:38:07.632 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "active_name": "x", 2023-12-11T17:38:07.633 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "num_standby": 0 2023-12-11T17:38:07.633 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout } 2023-12-11T17:38:07.633 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:06.738+0000 7fea5de91700 1 Processor -- start 2023-12-11T17:38:07.633 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:06.738+0000 7fea5de91700 1 -- start start 2023-12-11T17:38:07.633 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:06.738+0000 7fea5de91700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fea581077b0 0x7fea58109ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:38:07.633 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:06.739+0000 7fea5de91700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea5807ae00 con 0x7fea581077b0 2023-12-11T17:38:07.633 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:06.739+0000 7fea577fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fea581077b0 0x7fea58109ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:38:07.633 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:06.739+0000 7fea577fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fea581077b0 0x7fea58109ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59988/0 (socket says 172.21.15.59:59988) 2023-12-11T17:38:07.633 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:06.739+0000 7fea577fe700 1 -- 172.21.15.59:0/2471041362 learned_addr learned my addr 172.21.15.59:0/2471041362 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:38:07.633 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:06.739+0000 7fea577fe700 1 -- 172.21.15.59:0/2471041362 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea5810a0e0 con 0x7fea581077b0 2023-12-11T17:38:07.633 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:06.739+0000 7fea577fe700 1 --2- 172.21.15.59:0/2471041362 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fea581077b0 0x7fea58109ba0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fea48009e80 tx=0x7fea4800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=503e19a6f941d8 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:38:07.633 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:06.739+0000 7fea567fc700 1 -- 172.21.15.59:0/2471041362 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fea4800e070 con 0x7fea581077b0 2023-12-11T17:38:07.633 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:06.739+0000 7fea567fc700 1 -- 172.21.15.59:0/2471041362 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fea4800b8c0 con 0x7fea581077b0 2023-12-11T17:38:07.633 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:06.739+0000 7fea567fc700 1 -- 172.21.15.59:0/2471041362 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fea480043e0 con 0x7fea581077b0 2023-12-11T17:38:07.634 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:06.740+0000 7fea5de91700 1 -- 172.21.15.59:0/2471041362 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fea581077b0 msgr2=0x7fea58109ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:38:07.634 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:06.740+0000 7fea5de91700 1 --2- 172.21.15.59:0/2471041362 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fea581077b0 0x7fea58109ba0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fea48009e80 tx=0x7fea4800b330 comp rx=0 tx=0).stop 2023-12-11T17:38:07.634 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:06.740+0000 7fea5de91700 1 -- 172.21.15.59:0/2471041362 shutdown_connections 2023-12-11T17:38:07.634 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:06.740+0000 7fea5de91700 1 --2- 172.21.15.59:0/2471041362 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fea581077b0 0x7fea58109ba0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:38:07.634 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:06.740+0000 7fea5de91700 1 -- 172.21.15.59:0/2471041362 >> 172.21.15.59:0/2471041362 conn(0x7fea58101160 msgr2=0x7fea581035c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:38:07.634 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:06.740+0000 7fea5de91700 1 -- 172.21.15.59:0/2471041362 shutdown_connections 2023-12-11T17:38:07.634 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:06.740+0000 7fea5de91700 1 -- 172.21.15.59:0/2471041362 wait complete. 2023-12-11T17:38:07.637 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:06.741+0000 7fea5de91700 1 Processor -- start 2023-12-11T17:38:07.637 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:06.741+0000 7fea5de91700 1 -- start start 2023-12-11T17:38:07.637 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:06.742+0000 7fea5de91700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fea581077b0 0x7fea5819aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:38:07.637 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:06.742+0000 7fea5de91700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea5819afa0 con 0x7fea581077b0 2023-12-11T17:38:07.637 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:06.742+0000 7fea577fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fea581077b0 0x7fea5819aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:38:07.637 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:06.742+0000 7fea577fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fea581077b0 0x7fea5819aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59990/0 (socket says 172.21.15.59:59990) 2023-12-11T17:38:07.638 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:06.742+0000 7fea577fe700 1 -- 172.21.15.59:0/2533182869 learned_addr learned my addr 172.21.15.59:0/2533182869 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:38:07.638 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:06.742+0000 7fea577fe700 1 -- 172.21.15.59:0/2533182869 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea48009b30 con 0x7fea581077b0 2023-12-11T17:38:07.638 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:06.742+0000 7fea577fe700 1 --2- 172.21.15.59:0/2533182869 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fea581077b0 0x7fea5819aa60 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fea48009530 tx=0x7fea48003d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:38:07.638 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:06.742+0000 7fea54ff9700 1 -- 172.21.15.59:0/2533182869 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fea4800e040 con 0x7fea581077b0 2023-12-11T17:38:07.638 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:06.742+0000 7fea54ff9700 1 -- 172.21.15.59:0/2533182869 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fea4800bd70 con 0x7fea581077b0 2023-12-11T17:38:07.638 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:06.742+0000 7fea54ff9700 1 -- 172.21.15.59:0/2533182869 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fea4801c700 con 0x7fea581077b0 2023-12-11T17:38:07.638 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:06.742+0000 7fea5de91700 1 -- 172.21.15.59:0/2533182869 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea5819b1a0 con 0x7fea581077b0 2023-12-11T17:38:07.638 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:06.742+0000 7fea5de91700 1 -- 172.21.15.59:0/2533182869 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea5819bd00 con 0x7fea581077b0 2023-12-11T17:38:07.638 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:06.743+0000 7fea54ff9700 1 -- 172.21.15.59:0/2533182869 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 9) v1 ==== 45593+0+0 (secure 0 0 0) 0x7fea48012a80 con 0x7fea581077b0 2023-12-11T17:38:07.638 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:06.743+0000 7fea5de91700 1 -- 172.21.15.59:0/2533182869 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea58062c00 con 0x7fea581077b0 2023-12-11T17:38:07.638 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:06.744+0000 7fea54ff9700 1 --2- 172.21.15.59:0/2533182869 >> [v2:172.21.15.59:6800/1259916482,v1:172.21.15.59:6801/1259916482] conn(0x7fea440388c0 0x7fea4403ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:38:07.638 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:06.744+0000 7fea54ff9700 1 -- 172.21.15.59:0/2533182869 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fea4804c620 con 0x7fea581077b0 2023-12-11T17:38:07.638 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:06.744+0000 7fea56ffd700 1 -- 172.21.15.59:0/2533182869 >> [v2:172.21.15.59:6800/1259916482,v1:172.21.15.59:6801/1259916482] conn(0x7fea440388c0 msgr2=0x7fea4403ad80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.59:6800/1259916482 2023-12-11T17:38:07.638 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:06.744+0000 7fea56ffd700 1 --2- 172.21.15.59:0/2533182869 >> [v2:172.21.15.59:6800/1259916482,v1:172.21.15.59:6801/1259916482] conn(0x7fea440388c0 0x7fea4403ad80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-11T17:38:07.639 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:06.748+0000 7fea54ff9700 1 -- 172.21.15.59:0/2533182869 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fea48012430 con 0x7fea581077b0 2023-12-11T17:38:07.639 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:06.945+0000 7fea56ffd700 1 -- 172.21.15.59:0/2533182869 >> [v2:172.21.15.59:6800/1259916482,v1:172.21.15.59:6801/1259916482] conn(0x7fea440388c0 msgr2=0x7fea4403ad80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.59:6800/1259916482 2023-12-11T17:38:07.639 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:06.945+0000 7fea56ffd700 1 --2- 172.21.15.59:0/2533182869 >> [v2:172.21.15.59:6800/1259916482,v1:172.21.15.59:6801/1259916482] conn(0x7fea440388c0 0x7fea4403ad80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-11T17:38:07.639 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:07.226+0000 7fea5de91700 1 -- 172.21.15.59:0/2533182869 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7fea5819c030 con 0x7fea581077b0 2023-12-11T17:38:07.639 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:07.227+0000 7fea54ff9700 1 -- 172.21.15.59:0/2533182869 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+88 (secure 0 0 0) 0x7fea48017070 con 0x7fea581077b0 2023-12-11T17:38:07.639 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:07.229+0000 7fea5de91700 1 -- 172.21.15.59:0/2533182869 >> [v2:172.21.15.59:6800/1259916482,v1:172.21.15.59:6801/1259916482] conn(0x7fea440388c0 msgr2=0x7fea4403ad80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-11T17:38:07.639 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:07.229+0000 7fea5de91700 1 --2- 172.21.15.59:0/2533182869 >> [v2:172.21.15.59:6800/1259916482,v1:172.21.15.59:6801/1259916482] conn(0x7fea440388c0 0x7fea4403ad80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:38:07.639 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:07.229+0000 7fea5de91700 1 -- 172.21.15.59:0/2533182869 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fea581077b0 msgr2=0x7fea5819aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:38:07.639 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:07.229+0000 7fea5de91700 1 --2- 172.21.15.59:0/2533182869 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fea581077b0 0x7fea5819aa60 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fea48009530 tx=0x7fea48003d00 comp rx=0 tx=0).stop 2023-12-11T17:38:07.639 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:07.229+0000 7fea5de91700 1 -- 172.21.15.59:0/2533182869 shutdown_connections 2023-12-11T17:38:07.639 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:07.229+0000 7fea5de91700 1 --2- 172.21.15.59:0/2533182869 >> [v2:172.21.15.59:6800/1259916482,v1:172.21.15.59:6801/1259916482] conn(0x7fea440388c0 0x7fea4403ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:38:07.639 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:07.229+0000 7fea5de91700 1 --2- 172.21.15.59:0/2533182869 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fea581077b0 0x7fea5819aa60 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:38:07.639 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:07.229+0000 7fea5de91700 1 -- 172.21.15.59:0/2533182869 >> 172.21.15.59:0/2533182869 conn(0x7fea58101160 msgr2=0x7fea58101c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:38:07.639 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:07.229+0000 7fea5de91700 1 -- 172.21.15.59:0/2533182869 shutdown_connections 2023-12-11T17:38:07.639 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:07.229+0000 7fea5de91700 1 -- 172.21.15.59:0/2533182869 wait complete. 2023-12-11T17:38:07.640 INFO:teuthology.orchestra.run.smithi059.stdout:Waiting for the mgr to restart... 2023-12-11T17:38:07.640 INFO:teuthology.orchestra.run.smithi059.stdout:Waiting for mgr epoch 9... 2023-12-11T17:38:07.640 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:38:07 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T17:38:07.271+0000 7fabc0506200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-11T17:38:07.640 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:38:07 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T17:38:07.384+0000 7fabc0506200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-11T17:38:07.640 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:38:07 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T17:38:07.500+0000 7fabc0506200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-11T17:38:08.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:07 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/2533182869' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2023-12-11T17:38:08.154 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:38:08 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T17:38:08.026+0000 7fabc0506200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-11T17:38:09.154 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:38:08 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T17:38:08.814+0000 7fabc0506200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-11T17:38:09.154 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:38:08 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T17:38:08.913+0000 7fabc0506200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-11T17:38:10.154 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:38:09 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T17:38:09.891+0000 7fabc0506200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-11T17:38:10.154 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:38:10 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T17:38:10.002+0000 7fabc0506200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-11T17:38:10.654 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:38:10 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T17:38:10.224+0000 7fabc0506200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-11T17:38:11.014 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:38:10 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T17:38:10.722+0000 7fabc0506200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-11T17:38:11.014 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:38:10 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T17:38:10.829+0000 7fabc0506200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-11T17:38:11.345 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:38:11 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T17:38:11.011+0000 7fabc0506200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-11T17:38:11.654 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:38:11 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T17:38:11.343+0000 7fabc0506200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-11T17:38:12.572 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:38:12 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T17:38:12.163+0000 7fabc0506200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-11T17:38:12.572 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:38:12 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T17:38:12.321+0000 7fabc0506200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-11T17:38:12.904 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:38:12 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T17:38:12.569+0000 7fabc0506200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-11T17:38:12.904 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:38:12 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T17:38:12.706+0000 7fabc0506200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-11T17:38:13.404 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:38:12 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T17:38:12.980+0000 7fabc0506200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-11T17:38:14.404 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:38:14 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T17:38:14.023+0000 7fabc0506200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-11T17:38:15.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:15 smithi059 ceph-mon[98847]: Active manager daemon x restarted 2023-12-11T17:38:15.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:15 smithi059 ceph-mon[98847]: Activating manager daemon x 2023-12-11T17:38:15.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:15 smithi059 ceph-mon[98847]: osdmap e3: 0 total, 0 up, 0 in 2023-12-11T17:38:15.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:15 smithi059 ceph-mon[98847]: mgrmap e10: x(active, starting, since 0.00305308s) 2023-12-11T17:38:15.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:15 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-11T17:38:15.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:15 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-12-11T17:38:15.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:15 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-12-11T17:38:15.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:15 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-12-11T17:38:15.406 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:15 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-12-11T17:38:15.406 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:15 smithi059 ceph-mon[98847]: Manager daemon x is now available 2023-12-11T17:38:15.406 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:15 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:38:15.406 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:15 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:38:15.406 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:15 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-12-11T17:38:15.407 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:15 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-12-11T17:38:15.603 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout { 2023-12-11T17:38:15.603 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2023-12-11T17:38:15.603 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "initialized": true 2023-12-11T17:38:15.603 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout } 2023-12-11T17:38:15.603 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:08.142+0000 7f27c401d700 1 Processor -- start 2023-12-11T17:38:15.603 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:08.143+0000 7f27c401d700 1 -- start start 2023-12-11T17:38:15.603 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:08.143+0000 7f27c401d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27bc1077d0 0x7f27bc109bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:38:15.603 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:08.143+0000 7f27c401d700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27bc07ade0 con 0x7f27bc1077d0 2023-12-11T17:38:15.603 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:08.143+0000 7f27c1db9700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27bc1077d0 0x7f27bc109bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:38:15.603 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:08.143+0000 7f27c1db9700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27bc1077d0 0x7f27bc109bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59998/0 (socket says 172.21.15.59:59998) 2023-12-11T17:38:15.603 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:08.143+0000 7f27c1db9700 1 -- 172.21.15.59:0/64317910 learned_addr learned my addr 172.21.15.59:0/64317910 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:38:15.603 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:08.143+0000 7f27c1db9700 1 -- 172.21.15.59:0/64317910 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27bc10a100 con 0x7f27bc1077d0 2023-12-11T17:38:15.603 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:08.143+0000 7f27c1db9700 1 --2- 172.21.15.59:0/64317910 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27bc1077d0 0x7f27bc109bc0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f27b80098a0 tx=0x7f27b8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b8bd6b6054b9084 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:38:15.604 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:08.144+0000 7f27c0db7700 1 -- 172.21.15.59:0/64317910 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f27b8004160 con 0x7f27bc1077d0 2023-12-11T17:38:15.604 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:08.144+0000 7f27c0db7700 1 -- 172.21.15.59:0/64317910 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f27b80042c0 con 0x7f27bc1077d0 2023-12-11T17:38:15.604 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:08.144+0000 7f27c0db7700 1 -- 172.21.15.59:0/64317910 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f27b80119b0 con 0x7f27bc1077d0 2023-12-11T17:38:15.604 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:08.145+0000 7f27c401d700 1 -- 172.21.15.59:0/64317910 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27bc1077d0 msgr2=0x7f27bc109bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:38:15.604 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:08.145+0000 7f27c401d700 1 --2- 172.21.15.59:0/64317910 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27bc1077d0 0x7f27bc109bc0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f27b80098a0 tx=0x7f27b8009bb0 comp rx=0 tx=0).stop 2023-12-11T17:38:15.604 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:08.145+0000 7f27c401d700 1 -- 172.21.15.59:0/64317910 shutdown_connections 2023-12-11T17:38:15.604 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:08.145+0000 7f27c401d700 1 --2- 172.21.15.59:0/64317910 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27bc1077d0 0x7f27bc109bc0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:38:15.604 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:08.145+0000 7f27c401d700 1 -- 172.21.15.59:0/64317910 >> 172.21.15.59:0/64317910 conn(0x7f27bc101160 msgr2=0x7f27bc1035c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:38:15.604 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:08.145+0000 7f27c401d700 1 -- 172.21.15.59:0/64317910 shutdown_connections 2023-12-11T17:38:15.604 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:08.145+0000 7f27c401d700 1 -- 172.21.15.59:0/64317910 wait complete. 2023-12-11T17:38:15.604 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:08.146+0000 7f27c401d700 1 Processor -- start 2023-12-11T17:38:15.604 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:08.146+0000 7f27c401d700 1 -- start start 2023-12-11T17:38:15.604 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:08.146+0000 7f27c401d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27bc1077d0 0x7f27bc1071e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:38:15.604 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:08.146+0000 7f27c401d700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27bc105780 con 0x7f27bc1077d0 2023-12-11T17:38:15.604 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:08.146+0000 7f27c1db9700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27bc1077d0 0x7f27bc1071e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:38:15.605 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:08.146+0000 7f27c1db9700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27bc1077d0 0x7f27bc1071e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60000/0 (socket says 172.21.15.59:60000) 2023-12-11T17:38:15.605 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:08.146+0000 7f27c1db9700 1 -- 172.21.15.59:0/2382418708 learned_addr learned my addr 172.21.15.59:0/2382418708 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:38:15.605 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:08.146+0000 7f27c1db9700 1 -- 172.21.15.59:0/2382418708 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27b8009550 con 0x7f27bc1077d0 2023-12-11T17:38:15.605 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:08.147+0000 7f27c1db9700 1 --2- 172.21.15.59:0/2382418708 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27bc1077d0 0x7f27bc1071e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f27b8006af0 tx=0x7f27b8011a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:38:15.605 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:08.147+0000 7f27b2ffd700 1 -- 172.21.15.59:0/2382418708 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f27b8003a40 con 0x7f27bc1077d0 2023-12-11T17:38:15.605 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:08.147+0000 7f27b2ffd700 1 -- 172.21.15.59:0/2382418708 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f27b8003ba0 con 0x7f27bc1077d0 2023-12-11T17:38:15.605 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:08.147+0000 7f27c401d700 1 -- 172.21.15.59:0/2382418708 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27bc105980 con 0x7f27bc1077d0 2023-12-11T17:38:15.605 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:08.147+0000 7f27b2ffd700 1 -- 172.21.15.59:0/2382418708 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f27b80196c0 con 0x7f27bc1077d0 2023-12-11T17:38:15.605 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:08.147+0000 7f27c401d700 1 -- 172.21.15.59:0/2382418708 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27bc105e20 con 0x7f27bc1077d0 2023-12-11T17:38:15.605 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:08.148+0000 7f27b2ffd700 1 -- 172.21.15.59:0/2382418708 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 9) v1 ==== 45593+0+0 (secure 0 0 0) 0x7f27b8019980 con 0x7f27bc1077d0 2023-12-11T17:38:15.605 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:08.148+0000 7f27b2ffd700 1 --2- 172.21.15.59:0/2382418708 >> [v2:172.21.15.59:6800/1259916482,v1:172.21.15.59:6801/1259916482] conn(0x7f27a8038820 0x7f27a803ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:38:15.605 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:08.148+0000 7f27c401d700 1 -- 172.21.15.59:0/2382418708 --> [v2:172.21.15.59:6800/1259916482,v1:172.21.15.59:6801/1259916482] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f27bc062c00 con 0x7f27a8038820 2023-12-11T17:38:15.605 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:08.148+0000 7f27b2ffd700 1 -- 172.21.15.59:0/2382418708 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f27b804c850 con 0x7f27bc1077d0 2023-12-11T17:38:15.605 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:08.148+0000 7f27c15b8700 1 -- 172.21.15.59:0/2382418708 >> [v2:172.21.15.59:6800/1259916482,v1:172.21.15.59:6801/1259916482] conn(0x7f27a8038820 msgr2=0x7f27a803ace0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.59:6800/1259916482 2023-12-11T17:38:15.605 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:08.148+0000 7f27c15b8700 1 --2- 172.21.15.59:0/2382418708 >> [v2:172.21.15.59:6800/1259916482,v1:172.21.15.59:6801/1259916482] conn(0x7f27a8038820 0x7f27a803ace0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-11T17:38:15.606 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:08.349+0000 7f27c15b8700 1 -- 172.21.15.59:0/2382418708 >> [v2:172.21.15.59:6800/1259916482,v1:172.21.15.59:6801/1259916482] conn(0x7f27a8038820 msgr2=0x7f27a803ace0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.59:6800/1259916482 2023-12-11T17:38:15.606 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:08.349+0000 7f27c15b8700 1 --2- 172.21.15.59:0/2382418708 >> [v2:172.21.15.59:6800/1259916482,v1:172.21.15.59:6801/1259916482] conn(0x7f27a8038820 0x7f27a803ace0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-11T17:38:15.606 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:08.750+0000 7f27c15b8700 1 -- 172.21.15.59:0/2382418708 >> [v2:172.21.15.59:6800/1259916482,v1:172.21.15.59:6801/1259916482] conn(0x7f27a8038820 msgr2=0x7f27a803ace0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.59:6800/1259916482 2023-12-11T17:38:15.606 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:08.750+0000 7f27c15b8700 1 --2- 172.21.15.59:0/2382418708 >> [v2:172.21.15.59:6800/1259916482,v1:172.21.15.59:6801/1259916482] conn(0x7f27a8038820 0x7f27a803ace0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-11T17:38:15.606 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:09.551+0000 7f27c15b8700 1 -- 172.21.15.59:0/2382418708 >> [v2:172.21.15.59:6800/1259916482,v1:172.21.15.59:6801/1259916482] conn(0x7f27a8038820 msgr2=0x7f27a803ace0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.59:6800/1259916482 2023-12-11T17:38:15.606 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:09.551+0000 7f27c15b8700 1 --2- 172.21.15.59:0/2382418708 >> [v2:172.21.15.59:6800/1259916482,v1:172.21.15.59:6801/1259916482] conn(0x7f27a8038820 0x7f27a803ace0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-11T17:38:15.606 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:11.152+0000 7f27c15b8700 1 -- 172.21.15.59:0/2382418708 >> [v2:172.21.15.59:6800/1259916482,v1:172.21.15.59:6801/1259916482] conn(0x7f27a8038820 msgr2=0x7f27a803ace0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.59:6800/1259916482 2023-12-11T17:38:15.606 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:11.152+0000 7f27c15b8700 1 --2- 172.21.15.59:0/2382418708 >> [v2:172.21.15.59:6800/1259916482,v1:172.21.15.59:6801/1259916482] conn(0x7f27a8038820 0x7f27a803ace0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-12-11T17:38:15.606 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:14.039+0000 7f27b2ffd700 1 -- 172.21.15.59:0/2382418708 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mgrmap(e 10) v1 ==== 45360+0+0 (secure 0 0 0) 0x7f27b8019d20 con 0x7f27bc1077d0 2023-12-11T17:38:15.606 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:14.039+0000 7f27b2ffd700 1 -- 172.21.15.59:0/2382418708 >> [v2:172.21.15.59:6800/1259916482,v1:172.21.15.59:6801/1259916482] conn(0x7f27a8038820 msgr2=0x7f27a803ace0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-11T17:38:15.606 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:14.039+0000 7f27b2ffd700 1 --2- 172.21.15.59:0/2382418708 >> [v2:172.21.15.59:6800/1259916482,v1:172.21.15.59:6801/1259916482] conn(0x7f27a8038820 0x7f27a803ace0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:38:15.606 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:15.040+0000 7f27b2ffd700 1 -- 172.21.15.59:0/2382418708 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mgrmap(e 11) v1 ==== 45487+0+0 (secure 0 0 0) 0x7f27b8021d60 con 0x7f27bc1077d0 2023-12-11T17:38:15.606 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:15.040+0000 7f27b2ffd700 1 --2- 172.21.15.59:0/2382418708 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f27a803bfd0 0x7f27a803e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:38:15.606 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:15.041+0000 7f27b2ffd700 1 -- 172.21.15.59:0/2382418708 --> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f27bc062c00 con 0x7f27a803bfd0 2023-12-11T17:38:15.607 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:15.042+0000 7f27c15b8700 1 --2- 172.21.15.59:0/2382418708 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f27a803bfd0 0x7f27a803e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:38:15.607 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:15.043+0000 7f27c15b8700 1 --2- 172.21.15.59:0/2382418708 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f27a803bfd0 0x7f27a803e3c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f27ac003a10 tx=0x7f27ac0092b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:38:15.607 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:15.043+0000 7f27b2ffd700 1 -- 172.21.15.59:0/2382418708 <== mgr.14150 v2:172.21.15.59:6800/3627728353 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (secure 0 0 0) 0x7f27bc062c00 con 0x7f27a803bfd0 2023-12-11T17:38:15.607 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:15.055+0000 7f27c401d700 1 -- 172.21.15.59:0/2382418708 --> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f27bc067940 con 0x7f27a803bfd0 2023-12-11T17:38:15.607 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:15.055+0000 7f27b2ffd700 1 -- 172.21.15.59:0/2382418708 <== mgr.14150 v2:172.21.15.59:6800/3627728353 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7f27bc067940 con 0x7f27a803bfd0 2023-12-11T17:38:15.607 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:15.055+0000 7f27c401d700 1 -- 172.21.15.59:0/2382418708 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f27a803bfd0 msgr2=0x7f27a803e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:38:15.607 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:15.055+0000 7f27c401d700 1 --2- 172.21.15.59:0/2382418708 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f27a803bfd0 0x7f27a803e3c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f27ac003a10 tx=0x7f27ac0092b0 comp rx=0 tx=0).stop 2023-12-11T17:38:15.607 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:15.055+0000 7f27c401d700 1 -- 172.21.15.59:0/2382418708 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27bc1077d0 msgr2=0x7f27bc1071e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:38:15.607 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:15.055+0000 7f27c401d700 1 --2- 172.21.15.59:0/2382418708 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27bc1077d0 0x7f27bc1071e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f27b8006af0 tx=0x7f27b8011a10 comp rx=0 tx=0).stop 2023-12-11T17:38:15.607 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:15.055+0000 7f27c401d700 1 -- 172.21.15.59:0/2382418708 shutdown_connections 2023-12-11T17:38:15.607 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:15.055+0000 7f27c401d700 1 --2- 172.21.15.59:0/2382418708 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f27a803bfd0 0x7f27a803e3c0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:38:15.607 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:15.055+0000 7f27c401d700 1 --2- 172.21.15.59:0/2382418708 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27bc1077d0 0x7f27bc1071e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:38:15.608 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:15.055+0000 7f27c401d700 1 --2- 172.21.15.59:0/2382418708 >> [v2:172.21.15.59:6800/1259916482,v1:172.21.15.59:6801/1259916482] conn(0x7f27a8038820 0x7f27a803ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:38:15.608 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:15.055+0000 7f27c401d700 1 -- 172.21.15.59:0/2382418708 >> 172.21.15.59:0/2382418708 conn(0x7f27bc101160 msgr2=0x7f27bc101cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:38:15.608 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:15.055+0000 7f27c401d700 1 -- 172.21.15.59:0/2382418708 shutdown_connections 2023-12-11T17:38:15.608 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:15.056+0000 7f27c401d700 1 -- 172.21.15.59:0/2382418708 wait complete. 2023-12-11T17:38:15.608 INFO:teuthology.orchestra.run.smithi059.stdout:mgr epoch 9 is available 2023-12-11T17:38:15.608 INFO:teuthology.orchestra.run.smithi059.stdout:Generating a dashboard self-signed certificate... 2023-12-11T17:38:16.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:16 smithi059 ceph-mon[98847]: [11/Dec/2023:17:38:15] ENGINE Bus STARTING 2023-12-11T17:38:16.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:16 smithi059 ceph-mon[98847]: mgrmap e11: x(active, since 1.00522s) 2023-12-11T17:38:16.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:16 smithi059 ceph-mon[98847]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2023-12-11T17:38:16.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:16 smithi059 ceph-mon[98847]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2023-12-11T17:38:16.155 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:16 smithi059 ceph-mon[98847]: [11/Dec/2023:17:38:15] ENGINE Serving on https://172.21.15.59:7150 2023-12-11T17:38:16.155 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:16 smithi059 ceph-mon[98847]: [11/Dec/2023:17:38:15] ENGINE Serving on http://172.21.15.59:8765 2023-12-11T17:38:16.155 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:16 smithi059 ceph-mon[98847]: [11/Dec/2023:17:38:15] ENGINE Bus STARTED 2023-12-11T17:38:16.155 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:16 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:38:17.049 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2023-12-11T17:38:17.049 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:16.164+0000 7f0209120700 1 Processor -- start 2023-12-11T17:38:17.049 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:16.165+0000 7f0209120700 1 -- start start 2023-12-11T17:38:17.049 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:16.165+0000 7f0209120700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0204107750 0x7f0204109b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:38:17.049 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:16.165+0000 7f0209120700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f020407ae60 con 0x7f0204107750 2023-12-11T17:38:17.049 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:16.165+0000 7f0202d9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0204107750 0x7f0204109b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:38:17.049 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:16.165+0000 7f0202d9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0204107750 0x7f0204109b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60054/0 (socket says 172.21.15.59:60054) 2023-12-11T17:38:17.049 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:16.165+0000 7f0202d9d700 1 -- 172.21.15.59:0/2399071015 learned_addr learned my addr 172.21.15.59:0/2399071015 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:38:17.049 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:16.165+0000 7f0202d9d700 1 -- 172.21.15.59:0/2399071015 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f020410a080 con 0x7f0204107750 2023-12-11T17:38:17.049 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:16.165+0000 7f0202d9d700 1 --2- 172.21.15.59:0/2399071015 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0204107750 0x7f0204109b40 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f01f40098a0 tx=0x7f01f4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d54d27dba3b6433b server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:38:17.050 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:16.166+0000 7f0201d9b700 1 -- 172.21.15.59:0/2399071015 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f01f4004160 con 0x7f0204107750 2023-12-11T17:38:17.050 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:16.166+0000 7f0201d9b700 1 -- 172.21.15.59:0/2399071015 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f01f40042c0 con 0x7f0204107750 2023-12-11T17:38:17.050 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:16.166+0000 7f0201d9b700 1 -- 172.21.15.59:0/2399071015 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f01f40119b0 con 0x7f0204107750 2023-12-11T17:38:17.050 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:16.166+0000 7f0209120700 1 -- 172.21.15.59:0/2399071015 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0204107750 msgr2=0x7f0204109b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:38:17.050 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:16.166+0000 7f0209120700 1 --2- 172.21.15.59:0/2399071015 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0204107750 0x7f0204109b40 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f01f40098a0 tx=0x7f01f4009bb0 comp rx=0 tx=0).stop 2023-12-11T17:38:17.050 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:16.166+0000 7f0209120700 1 -- 172.21.15.59:0/2399071015 shutdown_connections 2023-12-11T17:38:17.050 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:16.166+0000 7f0209120700 1 --2- 172.21.15.59:0/2399071015 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0204107750 0x7f0204109b40 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:38:17.050 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:16.166+0000 7f0209120700 1 -- 172.21.15.59:0/2399071015 >> 172.21.15.59:0/2399071015 conn(0x7f0204101160 msgr2=0x7f02041035c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:38:17.050 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:16.166+0000 7f0209120700 1 -- 172.21.15.59:0/2399071015 shutdown_connections 2023-12-11T17:38:17.050 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:16.166+0000 7f0209120700 1 -- 172.21.15.59:0/2399071015 wait complete. 2023-12-11T17:38:17.050 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:16.167+0000 7f0209120700 1 Processor -- start 2023-12-11T17:38:17.050 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:16.167+0000 7f0209120700 1 -- start start 2023-12-11T17:38:17.050 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:16.168+0000 7f0209120700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0204107750 0x7f0204107110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:38:17.051 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:16.168+0000 7f0209120700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0204105700 con 0x7f0204107750 2023-12-11T17:38:17.051 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:16.168+0000 7f0202d9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0204107750 0x7f0204107110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:38:17.051 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:16.168+0000 7f0202d9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0204107750 0x7f0204107110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60056/0 (socket says 172.21.15.59:60056) 2023-12-11T17:38:17.051 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:16.168+0000 7f0202d9d700 1 -- 172.21.15.59:0/3346676021 learned_addr learned my addr 172.21.15.59:0/3346676021 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:38:17.051 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:16.168+0000 7f0202d9d700 1 -- 172.21.15.59:0/3346676021 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01f4009550 con 0x7f0204107750 2023-12-11T17:38:17.051 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:16.168+0000 7f0202d9d700 1 --2- 172.21.15.59:0/3346676021 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0204107750 0x7f0204107110 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f01f4006af0 tx=0x7f01f4011a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:38:17.051 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:16.168+0000 7f01ebfff700 1 -- 172.21.15.59:0/3346676021 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f01f4003a40 con 0x7f0204107750 2023-12-11T17:38:17.051 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:16.168+0000 7f01ebfff700 1 -- 172.21.15.59:0/3346676021 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f01f4003ba0 con 0x7f0204107750 2023-12-11T17:38:17.051 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:16.169+0000 7f0209120700 1 -- 172.21.15.59:0/3346676021 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0204105900 con 0x7f0204107750 2023-12-11T17:38:17.051 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:16.169+0000 7f01ebfff700 1 -- 172.21.15.59:0/3346676021 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f01f40196c0 con 0x7f0204107750 2023-12-11T17:38:17.051 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:16.169+0000 7f0209120700 1 -- 172.21.15.59:0/3346676021 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0204105da0 con 0x7f0204107750 2023-12-11T17:38:17.051 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:16.170+0000 7f01ebfff700 1 -- 172.21.15.59:0/3346676021 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 11) v1 ==== 45487+0+0 (secure 0 0 0) 0x7f01f4019980 con 0x7f0204107750 2023-12-11T17:38:17.051 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:16.170+0000 7f0209120700 1 -- 172.21.15.59:0/3346676021 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0204067940 con 0x7f0204107750 2023-12-11T17:38:17.052 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:16.170+0000 7f01ebfff700 1 --2- 172.21.15.59:0/3346676021 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f01ec038700 0x7f01ec03abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:38:17.052 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:16.170+0000 7f01ebfff700 1 -- 172.21.15.59:0/3346676021 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f01f404cab0 con 0x7f0204107750 2023-12-11T17:38:17.052 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:16.170+0000 7f020259c700 1 --2- 172.21.15.59:0/3346676021 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f01ec038700 0x7f01ec03abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:38:17.052 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:16.170+0000 7f020259c700 1 --2- 172.21.15.59:0/3346676021 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f01ec038700 0x7f01ec03abc0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f01f8006fd0 tx=0x7f01f8006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:38:17.052 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:16.175+0000 7f01ebfff700 1 -- 172.21.15.59:0/3346676021 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f01f4018760 con 0x7f0204107750 2023-12-11T17:38:17.052 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:16.557+0000 7f0209120700 1 -- 172.21.15.59:0/3346676021 --> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7f02041060f0 con 0x7f01ec038700 2023-12-11T17:38:17.052 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:16.722+0000 7f01ebfff700 1 -- 172.21.15.59:0/3346676021 <== mgr.14150 v2:172.21.15.59:6800/3627728353 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7f02041060f0 con 0x7f01ec038700 2023-12-11T17:38:17.052 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:16.724+0000 7f0209120700 1 -- 172.21.15.59:0/3346676021 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f01ec038700 msgr2=0x7f01ec03abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:38:17.052 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:16.724+0000 7f0209120700 1 --2- 172.21.15.59:0/3346676021 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f01ec038700 0x7f01ec03abc0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f01f8006fd0 tx=0x7f01f8006e40 comp rx=0 tx=0).stop 2023-12-11T17:38:17.052 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:16.724+0000 7f0209120700 1 -- 172.21.15.59:0/3346676021 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0204107750 msgr2=0x7f0204107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:38:17.052 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:16.724+0000 7f0209120700 1 --2- 172.21.15.59:0/3346676021 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0204107750 0x7f0204107110 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f01f4006af0 tx=0x7f01f4011a10 comp rx=0 tx=0).stop 2023-12-11T17:38:17.052 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:16.725+0000 7f0209120700 1 -- 172.21.15.59:0/3346676021 shutdown_connections 2023-12-11T17:38:17.052 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:16.725+0000 7f0209120700 1 --2- 172.21.15.59:0/3346676021 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f01ec038700 0x7f01ec03abc0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:38:17.052 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:16.725+0000 7f0209120700 1 --2- 172.21.15.59:0/3346676021 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0204107750 0x7f0204107110 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:38:17.053 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:16.725+0000 7f0209120700 1 -- 172.21.15.59:0/3346676021 >> 172.21.15.59:0/3346676021 conn(0x7f0204101160 msgr2=0x7f0204101c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:38:17.053 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:16.725+0000 7f0209120700 1 -- 172.21.15.59:0/3346676021 shutdown_connections 2023-12-11T17:38:17.053 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:16.725+0000 7f0209120700 1 -- 172.21.15.59:0/3346676021 wait complete. 2023-12-11T17:38:17.053 INFO:teuthology.orchestra.run.smithi059.stdout:Creating initial admin user... 2023-12-11T17:38:18.065 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:17 smithi059 ceph-mon[98847]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T17:38:18.065 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:17 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:38:18.065 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:17 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:38:18.065 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:17 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:38:18.065 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:17 smithi059 ceph-mon[98847]: mgrmap e12: x(active, since 3s) 2023-12-11T17:38:18.697 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$l3EfqXx15Hw5WKE9Nyk8n.QlutyuxkH5tdZ.f0daGIUhK4Dc4DAcS", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1702316298, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2023-12-11T17:38:18.697 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:17.558+0000 7f94ebc4e700 1 Processor -- start 2023-12-11T17:38:18.697 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:17.558+0000 7f94ebc4e700 1 -- start start 2023-12-11T17:38:18.697 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:17.558+0000 7f94ebc4e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f94e407ce50 0x7f94e407b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:38:18.697 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:17.558+0000 7f94ebc4e700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94e407b880 con 0x7f94e407ce50 2023-12-11T17:38:18.697 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:17.559+0000 7f94e99ea700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f94e407ce50 0x7f94e407b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:38:18.697 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:17.559+0000 7f94e99ea700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f94e407ce50 0x7f94e407b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60060/0 (socket says 172.21.15.59:60060) 2023-12-11T17:38:18.697 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:17.559+0000 7f94e99ea700 1 -- 172.21.15.59:0/2142823487 learned_addr learned my addr 172.21.15.59:0/2142823487 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:38:18.697 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:17.559+0000 7f94e99ea700 1 -- 172.21.15.59:0/2142823487 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94e407b9c0 con 0x7f94e407ce50 2023-12-11T17:38:18.698 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:17.559+0000 7f94e99ea700 1 --2- 172.21.15.59:0/2142823487 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f94e407ce50 0x7f94e407b2b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f94d40098a0 tx=0x7f94d4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddbf8c411e278362 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:38:18.698 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:17.559+0000 7f94e89e8700 1 -- 172.21.15.59:0/2142823487 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f94d4004160 con 0x7f94e407ce50 2023-12-11T17:38:18.698 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:17.559+0000 7f94e89e8700 1 -- 172.21.15.59:0/2142823487 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f94d40042c0 con 0x7f94e407ce50 2023-12-11T17:38:18.698 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:17.559+0000 7f94e89e8700 1 -- 172.21.15.59:0/2142823487 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f94d40119b0 con 0x7f94e407ce50 2023-12-11T17:38:18.698 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:17.560+0000 7f94ebc4e700 1 -- 172.21.15.59:0/2142823487 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f94e407ce50 msgr2=0x7f94e407b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:38:18.698 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:17.560+0000 7f94ebc4e700 1 --2- 172.21.15.59:0/2142823487 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f94e407ce50 0x7f94e407b2b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f94d40098a0 tx=0x7f94d4009bb0 comp rx=0 tx=0).stop 2023-12-11T17:38:18.698 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:17.560+0000 7f94ebc4e700 1 -- 172.21.15.59:0/2142823487 shutdown_connections 2023-12-11T17:38:18.698 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:17.560+0000 7f94ebc4e700 1 --2- 172.21.15.59:0/2142823487 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f94e407ce50 0x7f94e407b2b0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:38:18.698 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:17.560+0000 7f94ebc4e700 1 -- 172.21.15.59:0/2142823487 >> 172.21.15.59:0/2142823487 conn(0x7f94e41022a0 msgr2=0x7f94e41046c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:38:18.698 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:17.560+0000 7f94ebc4e700 1 -- 172.21.15.59:0/2142823487 shutdown_connections 2023-12-11T17:38:18.698 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:17.561+0000 7f94ebc4e700 1 -- 172.21.15.59:0/2142823487 wait complete. 2023-12-11T17:38:18.698 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:17.561+0000 7f94ebc4e700 1 Processor -- start 2023-12-11T17:38:18.698 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:17.561+0000 7f94ebc4e700 1 -- start start 2023-12-11T17:38:18.698 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:17.561+0000 7f94ebc4e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f94e407ce50 0x7f94e419ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:38:18.699 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:17.561+0000 7f94ebc4e700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94e419d350 con 0x7f94e407ce50 2023-12-11T17:38:18.699 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:17.561+0000 7f94e99ea700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f94e407ce50 0x7f94e419ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:38:18.699 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:17.561+0000 7f94e99ea700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f94e407ce50 0x7f94e419ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60062/0 (socket says 172.21.15.59:60062) 2023-12-11T17:38:18.699 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:17.561+0000 7f94e99ea700 1 -- 172.21.15.59:0/3091072388 learned_addr learned my addr 172.21.15.59:0/3091072388 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:38:18.699 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:17.562+0000 7f94e99ea700 1 -- 172.21.15.59:0/3091072388 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94d4009550 con 0x7f94e407ce50 2023-12-11T17:38:18.699 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:17.562+0000 7f94e99ea700 1 --2- 172.21.15.59:0/3091072388 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f94e407ce50 0x7f94e419ce10 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f94d4011670 tx=0x7f94d4011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:38:18.699 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:17.562+0000 7f94daffd700 1 -- 172.21.15.59:0/3091072388 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f94d4003af0 con 0x7f94e407ce50 2023-12-11T17:38:18.699 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:17.562+0000 7f94daffd700 1 -- 172.21.15.59:0/3091072388 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f94d4003c50 con 0x7f94e407ce50 2023-12-11T17:38:18.699 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:17.562+0000 7f94ebc4e700 1 -- 172.21.15.59:0/3091072388 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94e419d550 con 0x7f94e407ce50 2023-12-11T17:38:18.699 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:17.562+0000 7f94daffd700 1 -- 172.21.15.59:0/3091072388 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f94d4024820 con 0x7f94e407ce50 2023-12-11T17:38:18.699 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:17.562+0000 7f94ebc4e700 1 -- 172.21.15.59:0/3091072388 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94e419df70 con 0x7f94e407ce50 2023-12-11T17:38:18.699 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:17.563+0000 7f94daffd700 1 -- 172.21.15.59:0/3091072388 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 12) v1 ==== 45593+0+0 (secure 0 0 0) 0x7f94d4024a50 con 0x7f94e407ce50 2023-12-11T17:38:18.699 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:17.563+0000 7f94daffd700 1 --2- 172.21.15.59:0/3091072388 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f94d00388c0 0x7f94d003ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:38:18.699 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:17.563+0000 7f94ebc4e700 1 -- 172.21.15.59:0/3091072388 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94e4062c00 con 0x7f94e407ce50 2023-12-11T17:38:18.699 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:17.564+0000 7f94daffd700 1 -- 172.21.15.59:0/3091072388 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f94d404c990 con 0x7f94e407ce50 2023-12-11T17:38:18.700 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:17.564+0000 7f94e91e9700 1 --2- 172.21.15.59:0/3091072388 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f94d00388c0 0x7f94d003ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:38:18.700 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:17.564+0000 7f94e91e9700 1 --2- 172.21.15.59:0/3091072388 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f94d00388c0 0x7f94d003ad80 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f94e0006fd0 tx=0x7f94e0006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:38:18.700 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:17.569+0000 7f94daffd700 1 -- 172.21.15.59:0/3091072388 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f94d4010790 con 0x7f94e407ce50 2023-12-11T17:38:18.700 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:17.977+0000 7f94ebc4e700 1 -- 172.21.15.59:0/3091072388 --> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f94e410d340 con 0x7f94d00388c0 2023-12-11T17:38:18.700 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:18.224+0000 7f94daffd700 1 -- 172.21.15.59:0/3091072388 <== mgr.14150 v2:172.21.15.59:6800/3627728353 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7f94e410d340 con 0x7f94d00388c0 2023-12-11T17:38:18.700 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:18.226+0000 7f94ebc4e700 1 -- 172.21.15.59:0/3091072388 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f94d00388c0 msgr2=0x7f94d003ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:38:18.700 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:18.226+0000 7f94ebc4e700 1 --2- 172.21.15.59:0/3091072388 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f94d00388c0 0x7f94d003ad80 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f94e0006fd0 tx=0x7f94e0006e40 comp rx=0 tx=0).stop 2023-12-11T17:38:18.700 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:18.226+0000 7f94ebc4e700 1 -- 172.21.15.59:0/3091072388 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f94e407ce50 msgr2=0x7f94e419ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:38:18.700 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:18.226+0000 7f94ebc4e700 1 --2- 172.21.15.59:0/3091072388 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f94e407ce50 0x7f94e419ce10 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f94d4011670 tx=0x7f94d4011a30 comp rx=0 tx=0).stop 2023-12-11T17:38:18.700 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:18.226+0000 7f94ebc4e700 1 -- 172.21.15.59:0/3091072388 shutdown_connections 2023-12-11T17:38:18.700 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:18.226+0000 7f94ebc4e700 1 --2- 172.21.15.59:0/3091072388 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f94d00388c0 0x7f94d003ad80 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:38:18.700 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:18.226+0000 7f94ebc4e700 1 --2- 172.21.15.59:0/3091072388 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f94e407ce50 0x7f94e419ce10 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:38:18.700 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:18.226+0000 7f94ebc4e700 1 -- 172.21.15.59:0/3091072388 >> 172.21.15.59:0/3091072388 conn(0x7f94e41022a0 msgr2=0x7f94e4102e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:38:18.700 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:18.226+0000 7f94ebc4e700 1 -- 172.21.15.59:0/3091072388 shutdown_connections 2023-12-11T17:38:18.701 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:18.226+0000 7f94ebc4e700 1 -- 172.21.15.59:0/3091072388 wait complete. 2023-12-11T17:38:18.701 INFO:teuthology.orchestra.run.smithi059.stdout:Fetching dashboard port number... 2023-12-11T17:38:19.556 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:19 smithi059 ceph-mon[98847]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T17:38:19.556 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:19 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:38:19.843 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout 8443 2023-12-11T17:38:19.843 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:19.139+0000 7f5f25c05700 1 Processor -- start 2023-12-11T17:38:19.843 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:19.139+0000 7f5f25c05700 1 -- start start 2023-12-11T17:38:19.843 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:19.140+0000 7f5f25c05700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5f20104f60 0x7f5f20105380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:38:19.843 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:19.140+0000 7f5f25c05700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f20105950 con 0x7f5f20104f60 2023-12-11T17:38:19.843 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:19.140+0000 7f5f1f7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5f20104f60 0x7f5f20105380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:38:19.843 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:19.140+0000 7f5f1f7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5f20104f60 0x7f5f20105380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60066/0 (socket says 172.21.15.59:60066) 2023-12-11T17:38:19.843 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:19.140+0000 7f5f1f7fe700 1 -- 172.21.15.59:0/2653368360 learned_addr learned my addr 172.21.15.59:0/2653368360 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:38:19.844 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:19.140+0000 7f5f1f7fe700 1 -- 172.21.15.59:0/2653368360 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f20105a90 con 0x7f5f20104f60 2023-12-11T17:38:19.844 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:19.140+0000 7f5f1f7fe700 1 --2- 172.21.15.59:0/2653368360 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5f20104f60 0x7f5f20105380 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5f100098a0 tx=0x7f5f10009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66af66f870669ee8 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:38:19.844 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:19.140+0000 7f5f1effd700 1 -- 172.21.15.59:0/2653368360 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5f10004160 con 0x7f5f20104f60 2023-12-11T17:38:19.844 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:19.140+0000 7f5f1effd700 1 -- 172.21.15.59:0/2653368360 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5f100042c0 con 0x7f5f20104f60 2023-12-11T17:38:19.844 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:19.141+0000 7f5f1effd700 1 -- 172.21.15.59:0/2653368360 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5f100119b0 con 0x7f5f20104f60 2023-12-11T17:38:19.844 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:19.141+0000 7f5f25c05700 1 -- 172.21.15.59:0/2653368360 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5f20104f60 msgr2=0x7f5f20105380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:38:19.844 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:19.141+0000 7f5f25c05700 1 --2- 172.21.15.59:0/2653368360 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5f20104f60 0x7f5f20105380 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5f100098a0 tx=0x7f5f10009bb0 comp rx=0 tx=0).stop 2023-12-11T17:38:19.844 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:19.141+0000 7f5f25c05700 1 -- 172.21.15.59:0/2653368360 shutdown_connections 2023-12-11T17:38:19.844 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:19.141+0000 7f5f25c05700 1 --2- 172.21.15.59:0/2653368360 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5f20104f60 0x7f5f20105380 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:38:19.844 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:19.141+0000 7f5f25c05700 1 -- 172.21.15.59:0/2653368360 >> 172.21.15.59:0/2653368360 conn(0x7f5f20100880 msgr2=0x7f5f20102d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:38:19.844 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:19.141+0000 7f5f25c05700 1 -- 172.21.15.59:0/2653368360 shutdown_connections 2023-12-11T17:38:19.844 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:19.141+0000 7f5f25c05700 1 -- 172.21.15.59:0/2653368360 wait complete. 2023-12-11T17:38:19.844 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:19.142+0000 7f5f25c05700 1 Processor -- start 2023-12-11T17:38:19.844 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:19.142+0000 7f5f25c05700 1 -- start start 2023-12-11T17:38:19.844 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:19.142+0000 7f5f25c05700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5f20104f60 0x7f5f20107950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:38:19.845 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:19.142+0000 7f5f25c05700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f20105ef0 con 0x7f5f20104f60 2023-12-11T17:38:19.845 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:19.142+0000 7f5f1f7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5f20104f60 0x7f5f20107950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:38:19.845 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:19.142+0000 7f5f1f7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5f20104f60 0x7f5f20107950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60068/0 (socket says 172.21.15.59:60068) 2023-12-11T17:38:19.845 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:19.143+0000 7f5f1f7fe700 1 -- 172.21.15.59:0/2776891666 learned_addr learned my addr 172.21.15.59:0/2776891666 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:38:19.845 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:19.143+0000 7f5f1f7fe700 1 -- 172.21.15.59:0/2776891666 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f10009550 con 0x7f5f20104f60 2023-12-11T17:38:19.845 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:19.143+0000 7f5f1f7fe700 1 --2- 172.21.15.59:0/2776891666 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5f20104f60 0x7f5f20107950 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5f10003e60 tx=0x7f5f10011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:38:19.845 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:19.143+0000 7f5f1d7fa700 1 -- 172.21.15.59:0/2776891666 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5f10003a60 con 0x7f5f20104f60 2023-12-11T17:38:19.845 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:19.143+0000 7f5f1d7fa700 1 -- 172.21.15.59:0/2776891666 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5f10003bc0 con 0x7f5f20104f60 2023-12-11T17:38:19.845 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:19.143+0000 7f5f25c05700 1 -- 172.21.15.59:0/2776891666 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f201060f0 con 0x7f5f20104f60 2023-12-11T17:38:19.845 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:19.143+0000 7f5f1d7fa700 1 -- 172.21.15.59:0/2776891666 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5f100196c0 con 0x7f5f20104f60 2023-12-11T17:38:19.845 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:19.143+0000 7f5f25c05700 1 -- 172.21.15.59:0/2776891666 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f20106590 con 0x7f5f20104f60 2023-12-11T17:38:19.846 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:19.144+0000 7f5f1d7fa700 1 -- 172.21.15.59:0/2776891666 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 12) v1 ==== 45593+0+0 (secure 0 0 0) 0x7f5f10019930 con 0x7f5f20104f60 2023-12-11T17:38:19.846 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:19.144+0000 7f5f25c05700 1 -- 172.21.15.59:0/2776891666 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f20062c00 con 0x7f5f20104f60 2023-12-11T17:38:19.846 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:19.144+0000 7f5f1d7fa700 1 --2- 172.21.15.59:0/2776891666 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f5f0c0388a0 0x7f5f0c03ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:38:19.846 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:19.144+0000 7f5f1d7fa700 1 -- 172.21.15.59:0/2776891666 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f5f1004cd30 con 0x7f5f20104f60 2023-12-11T17:38:19.846 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:19.145+0000 7f5f16ffd700 1 --2- 172.21.15.59:0/2776891666 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f5f0c0388a0 0x7f5f0c03ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:38:19.846 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:19.145+0000 7f5f16ffd700 1 --2- 172.21.15.59:0/2776891666 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f5f0c0388a0 0x7f5f0c03ad60 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f5f08006fd0 tx=0x7f5f08006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:38:19.846 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:19.150+0000 7f5f1d7fa700 1 -- 172.21.15.59:0/2776891666 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f5f10010bc0 con 0x7f5f20104f60 2023-12-11T17:38:19.846 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:19.518+0000 7f5f25c05700 1 -- 172.21.15.59:0/2776891666 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7f5f20105a90 con 0x7f5f20104f60 2023-12-11T17:38:19.846 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:19.518+0000 7f5f1d7fa700 1 -- 172.21.15.59:0/2776891666 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7f5f20105a90 con 0x7f5f20104f60 2023-12-11T17:38:19.846 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:19.520+0000 7f5f25c05700 1 -- 172.21.15.59:0/2776891666 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f5f0c0388a0 msgr2=0x7f5f0c03ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:38:19.846 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:19.520+0000 7f5f25c05700 1 --2- 172.21.15.59:0/2776891666 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f5f0c0388a0 0x7f5f0c03ad60 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f5f08006fd0 tx=0x7f5f08006e40 comp rx=0 tx=0).stop 2023-12-11T17:38:19.846 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:19.521+0000 7f5f25c05700 1 -- 172.21.15.59:0/2776891666 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5f20104f60 msgr2=0x7f5f20107950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:38:19.846 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:19.521+0000 7f5f25c05700 1 --2- 172.21.15.59:0/2776891666 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5f20104f60 0x7f5f20107950 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5f10003e60 tx=0x7f5f10011a30 comp rx=0 tx=0).stop 2023-12-11T17:38:19.846 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:19.521+0000 7f5f25c05700 1 -- 172.21.15.59:0/2776891666 shutdown_connections 2023-12-11T17:38:19.847 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:19.521+0000 7f5f25c05700 1 --2- 172.21.15.59:0/2776891666 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f5f0c0388a0 0x7f5f0c03ad60 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:38:19.847 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:19.521+0000 7f5f25c05700 1 --2- 172.21.15.59:0/2776891666 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5f20104f60 0x7f5f20107950 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:38:19.847 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:19.521+0000 7f5f25c05700 1 -- 172.21.15.59:0/2776891666 >> 172.21.15.59:0/2776891666 conn(0x7f5f20100880 msgr2=0x7f5f201014c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:38:19.847 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:19.521+0000 7f5f25c05700 1 -- 172.21.15.59:0/2776891666 shutdown_connections 2023-12-11T17:38:19.847 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:19.521+0000 7f5f25c05700 1 -- 172.21.15.59:0/2776891666 wait complete. 2023-12-11T17:38:19.851 INFO:teuthology.orchestra.run.smithi059.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-12-11T17:38:19.851 INFO:teuthology.orchestra.run.smithi059.stdout:systemctl: stdout disabled 2023-12-11T17:38:19.858 INFO:teuthology.orchestra.run.smithi059.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-12-11T17:38:19.859 INFO:teuthology.orchestra.run.smithi059.stdout:systemctl: stdout inactive 2023-12-11T17:38:19.859 INFO:teuthology.orchestra.run.smithi059.stdout:firewalld.service is not enabled 2023-12-11T17:38:19.859 INFO:teuthology.orchestra.run.smithi059.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2023-12-11T17:38:19.860 INFO:teuthology.orchestra.run.smithi059.stdout:Ceph Dashboard is now available at: 2023-12-11T17:38:19.860 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-12-11T17:38:19.860 INFO:teuthology.orchestra.run.smithi059.stdout: URL: https://smithi059:8443/ 2023-12-11T17:38:19.860 INFO:teuthology.orchestra.run.smithi059.stdout: User: admin 2023-12-11T17:38:19.860 INFO:teuthology.orchestra.run.smithi059.stdout: Password: v73xvfokxp 2023-12-11T17:38:19.860 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-12-11T17:38:19.861 INFO:teuthology.orchestra.run.smithi059.stdout:Saving cluster configuration to /var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/config directory 2023-12-11T17:38:19.861 INFO:teuthology.orchestra.run.smithi059.stdout:Enabling autotune for osd_memory_target 2023-12-11T17:38:20.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:20 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/2776891666' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2023-12-11T17:38:21.309 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:20.536+0000 7f661a1ae700 1 Processor -- start 2023-12-11T17:38:21.309 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:20.536+0000 7f661a1ae700 1 -- start start 2023-12-11T17:38:21.309 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:20.536+0000 7f661a1ae700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f66141082c0 0x7f66141086e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:38:21.310 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:20.536+0000 7f661a1ae700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6614108cb0 con 0x7f66141082c0 2023-12-11T17:38:21.310 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:20.536+0000 7f66137fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f66141082c0 0x7f66141086e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:38:21.310 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:20.536+0000 7f66137fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f66141082c0 0x7f66141086e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60076/0 (socket says 172.21.15.59:60076) 2023-12-11T17:38:21.310 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:20.536+0000 7f66137fe700 1 -- 172.21.15.59:0/2258809932 learned_addr learned my addr 172.21.15.59:0/2258809932 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:38:21.310 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:20.536+0000 7f66137fe700 1 -- 172.21.15.59:0/2258809932 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6614108df0 con 0x7f66141082c0 2023-12-11T17:38:21.310 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:20.537+0000 7f66137fe700 1 --2- 172.21.15.59:0/2258809932 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f66141082c0 0x7f66141086e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f66040098a0 tx=0x7f6604009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6e48f2bb358640c server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:38:21.310 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:20.537+0000 7f66127fc700 1 -- 172.21.15.59:0/2258809932 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6604004160 con 0x7f66141082c0 2023-12-11T17:38:21.310 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:20.537+0000 7f66127fc700 1 -- 172.21.15.59:0/2258809932 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f66040042c0 con 0x7f66141082c0 2023-12-11T17:38:21.310 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:20.537+0000 7f66127fc700 1 -- 172.21.15.59:0/2258809932 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f66040119b0 con 0x7f66141082c0 2023-12-11T17:38:21.310 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:20.537+0000 7f661a1ae700 1 -- 172.21.15.59:0/2258809932 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f66141082c0 msgr2=0x7f66141086e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:38:21.310 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:20.537+0000 7f661a1ae700 1 --2- 172.21.15.59:0/2258809932 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f66141082c0 0x7f66141086e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f66040098a0 tx=0x7f6604009bb0 comp rx=0 tx=0).stop 2023-12-11T17:38:21.311 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:20.537+0000 7f661a1ae700 1 -- 172.21.15.59:0/2258809932 shutdown_connections 2023-12-11T17:38:21.311 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:20.537+0000 7f661a1ae700 1 --2- 172.21.15.59:0/2258809932 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f66141082c0 0x7f66141086e0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:38:21.311 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:20.537+0000 7f661a1ae700 1 -- 172.21.15.59:0/2258809932 >> 172.21.15.59:0/2258809932 conn(0x7f6614100880 msgr2=0x7f6614102d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:38:21.311 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:20.537+0000 7f661a1ae700 1 -- 172.21.15.59:0/2258809932 shutdown_connections 2023-12-11T17:38:21.311 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:20.538+0000 7f661a1ae700 1 -- 172.21.15.59:0/2258809932 wait complete. 2023-12-11T17:38:21.311 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:20.538+0000 7f661a1ae700 1 Processor -- start 2023-12-11T17:38:21.311 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:20.538+0000 7f661a1ae700 1 -- start start 2023-12-11T17:38:21.311 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:20.539+0000 7f661a1ae700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f66141082c0 0x7f6614107a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:38:21.311 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:20.539+0000 7f661a1ae700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6614106010 con 0x7f66141082c0 2023-12-11T17:38:21.311 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:20.539+0000 7f66137fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f66141082c0 0x7f6614107a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:38:21.311 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:20.539+0000 7f66137fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f66141082c0 0x7f6614107a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60078/0 (socket says 172.21.15.59:60078) 2023-12-11T17:38:21.311 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:20.539+0000 7f66137fe700 1 -- 172.21.15.59:0/2111271587 learned_addr learned my addr 172.21.15.59:0/2111271587 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:38:21.312 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:20.539+0000 7f66137fe700 1 -- 172.21.15.59:0/2111271587 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6604009550 con 0x7f66141082c0 2023-12-11T17:38:21.312 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:20.539+0000 7f66137fe700 1 --2- 172.21.15.59:0/2111271587 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f66141082c0 0x7f6614107a70 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6604003e60 tx=0x7f6604011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:38:21.312 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:20.539+0000 7f6610ff9700 1 -- 172.21.15.59:0/2111271587 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6604003a60 con 0x7f66141082c0 2023-12-11T17:38:21.312 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:20.539+0000 7f6610ff9700 1 -- 172.21.15.59:0/2111271587 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6604003bc0 con 0x7f66141082c0 2023-12-11T17:38:21.312 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:20.539+0000 7f661a1ae700 1 -- 172.21.15.59:0/2111271587 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6614106210 con 0x7f66141082c0 2023-12-11T17:38:21.312 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:20.539+0000 7f6610ff9700 1 -- 172.21.15.59:0/2111271587 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f66040196c0 con 0x7f66141082c0 2023-12-11T17:38:21.312 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:20.539+0000 7f661a1ae700 1 -- 172.21.15.59:0/2111271587 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66141066b0 con 0x7f66141082c0 2023-12-11T17:38:21.312 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:20.540+0000 7f6610ff9700 1 -- 172.21.15.59:0/2111271587 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 13) v1 ==== 45636+0+0 (secure 0 0 0) 0x7f6604019930 con 0x7f66141082c0 2023-12-11T17:38:21.312 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:20.540+0000 7f6610ff9700 1 --2- 172.21.15.59:0/2111271587 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f65fc038940 0x7f65fc03ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:38:21.312 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:20.540+0000 7f6610ff9700 1 -- 172.21.15.59:0/2111271587 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f660404ccb0 con 0x7f66141082c0 2023-12-11T17:38:21.312 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:20.540+0000 7f661a1ae700 1 -- 172.21.15.59:0/2111271587 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6614062c00 con 0x7f66141082c0 2023-12-11T17:38:21.312 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:20.541+0000 7f6612ffd700 1 --2- 172.21.15.59:0/2111271587 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f65fc038940 0x7f65fc03ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:38:21.312 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:20.541+0000 7f6612ffd700 1 --2- 172.21.15.59:0/2111271587 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f65fc038940 0x7f65fc03ae00 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f6608006fd0 tx=0x7f6608006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:38:21.313 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:20.545+0000 7f6610ff9700 1 -- 172.21.15.59:0/2111271587 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f6604019e60 con 0x7f66141082c0 2023-12-11T17:38:21.313 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:20.906+0000 7f661a1ae700 1 -- 172.21.15.59:0/2111271587 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7f6614067940 con 0x7f66141082c0 2023-12-11T17:38:21.313 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:20.907+0000 7f6610ff9700 1 -- 172.21.15.59:0/2111271587 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7f6604010b70 con 0x7f66141082c0 2023-12-11T17:38:21.313 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:20.909+0000 7f661a1ae700 1 -- 172.21.15.59:0/2111271587 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f65fc038940 msgr2=0x7f65fc03ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:38:21.313 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:20.909+0000 7f661a1ae700 1 --2- 172.21.15.59:0/2111271587 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f65fc038940 0x7f65fc03ae00 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f6608006fd0 tx=0x7f6608006e40 comp rx=0 tx=0).stop 2023-12-11T17:38:21.313 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:20.909+0000 7f661a1ae700 1 -- 172.21.15.59:0/2111271587 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f66141082c0 msgr2=0x7f6614107a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:38:21.313 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:20.909+0000 7f661a1ae700 1 --2- 172.21.15.59:0/2111271587 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f66141082c0 0x7f6614107a70 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6604003e60 tx=0x7f6604011a30 comp rx=0 tx=0).stop 2023-12-11T17:38:21.313 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:20.909+0000 7f661a1ae700 1 -- 172.21.15.59:0/2111271587 shutdown_connections 2023-12-11T17:38:21.313 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:20.909+0000 7f661a1ae700 1 --2- 172.21.15.59:0/2111271587 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f65fc038940 0x7f65fc03ae00 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:38:21.313 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:20.909+0000 7f661a1ae700 1 --2- 172.21.15.59:0/2111271587 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f66141082c0 0x7f6614107a70 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:38:21.313 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:20.909+0000 7f661a1ae700 1 -- 172.21.15.59:0/2111271587 >> 172.21.15.59:0/2111271587 conn(0x7f6614100880 msgr2=0x7f66141014c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:38:21.313 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:20.909+0000 7f661a1ae700 1 -- 172.21.15.59:0/2111271587 shutdown_connections 2023-12-11T17:38:21.313 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:20.909+0000 7f661a1ae700 1 -- 172.21.15.59:0/2111271587 wait complete. 2023-12-11T17:38:21.495 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:21 smithi059 ceph-mon[98847]: mgrmap e13: x(active, since 6s) 2023-12-11T17:38:22.656 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:21.762+0000 7f1c4fed6700 1 Processor -- start 2023-12-11T17:38:22.656 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:21.762+0000 7f1c4fed6700 1 -- start start 2023-12-11T17:38:22.656 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:21.762+0000 7f1c4fed6700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c48107810 0x7f1c48109c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:38:22.656 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:21.762+0000 7f1c4fed6700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c4807adc0 con 0x7f1c48107810 2023-12-11T17:38:22.656 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:21.762+0000 7f1c4dc72700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c48107810 0x7f1c48109c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:38:22.657 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:21.762+0000 7f1c4dc72700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c48107810 0x7f1c48109c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60082/0 (socket says 172.21.15.59:60082) 2023-12-11T17:38:22.657 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:21.762+0000 7f1c4dc72700 1 -- 172.21.15.59:0/3616804910 learned_addr learned my addr 172.21.15.59:0/3616804910 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:38:22.657 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:21.763+0000 7f1c4dc72700 1 -- 172.21.15.59:0/3616804910 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c4807af00 con 0x7f1c48107810 2023-12-11T17:38:22.657 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:21.763+0000 7f1c4dc72700 1 --2- 172.21.15.59:0/3616804910 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c48107810 0x7f1c48109c00 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1c440098a0 tx=0x7f1c44009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e21b0a3f3212ed46 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:38:22.657 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:21.763+0000 7f1c4cc70700 1 -- 172.21.15.59:0/3616804910 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1c44004160 con 0x7f1c48107810 2023-12-11T17:38:22.657 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:21.763+0000 7f1c4cc70700 1 -- 172.21.15.59:0/3616804910 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1c440042c0 con 0x7f1c48107810 2023-12-11T17:38:22.657 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:21.763+0000 7f1c4cc70700 1 -- 172.21.15.59:0/3616804910 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1c440119b0 con 0x7f1c48107810 2023-12-11T17:38:22.657 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:21.764+0000 7f1c4fed6700 1 -- 172.21.15.59:0/3616804910 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c48107810 msgr2=0x7f1c48109c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:38:22.657 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:21.764+0000 7f1c4fed6700 1 --2- 172.21.15.59:0/3616804910 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c48107810 0x7f1c48109c00 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1c440098a0 tx=0x7f1c44009bb0 comp rx=0 tx=0).stop 2023-12-11T17:38:22.657 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:21.764+0000 7f1c4fed6700 1 -- 172.21.15.59:0/3616804910 shutdown_connections 2023-12-11T17:38:22.657 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:21.764+0000 7f1c4fed6700 1 --2- 172.21.15.59:0/3616804910 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c48107810 0x7f1c48109c00 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:38:22.657 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:21.764+0000 7f1c4fed6700 1 -- 172.21.15.59:0/3616804910 >> 172.21.15.59:0/3616804910 conn(0x7f1c481011a0 msgr2=0x7f1c481035e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:38:22.657 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:21.764+0000 7f1c4fed6700 1 -- 172.21.15.59:0/3616804910 shutdown_connections 2023-12-11T17:38:22.657 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:21.764+0000 7f1c4fed6700 1 -- 172.21.15.59:0/3616804910 wait complete. 2023-12-11T17:38:22.658 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:21.764+0000 7f1c4fed6700 1 Processor -- start 2023-12-11T17:38:22.658 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:21.764+0000 7f1c4fed6700 1 -- start start 2023-12-11T17:38:22.658 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:21.765+0000 7f1c4fed6700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c48107810 0x7f1c48107220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:38:22.658 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:21.765+0000 7f1c4fed6700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c481057c0 con 0x7f1c48107810 2023-12-11T17:38:22.658 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:21.765+0000 7f1c4dc72700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c48107810 0x7f1c48107220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:38:22.658 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:21.765+0000 7f1c4dc72700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c48107810 0x7f1c48107220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60084/0 (socket says 172.21.15.59:60084) 2023-12-11T17:38:22.658 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:21.765+0000 7f1c4dc72700 1 -- 172.21.15.59:0/627678509 learned_addr learned my addr 172.21.15.59:0/627678509 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:38:22.658 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:21.765+0000 7f1c4dc72700 1 -- 172.21.15.59:0/627678509 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c44009550 con 0x7f1c48107810 2023-12-11T17:38:22.658 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:21.765+0000 7f1c4dc72700 1 --2- 172.21.15.59:0/627678509 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c48107810 0x7f1c48107220 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1c44011670 tx=0x7f1c44011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:38:22.658 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:21.765+0000 7f1c3effd700 1 -- 172.21.15.59:0/627678509 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1c44003b20 con 0x7f1c48107810 2023-12-11T17:38:22.658 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:21.765+0000 7f1c3effd700 1 -- 172.21.15.59:0/627678509 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1c44003c80 con 0x7f1c48107810 2023-12-11T17:38:22.658 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:21.765+0000 7f1c3effd700 1 -- 172.21.15.59:0/627678509 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1c44019770 con 0x7f1c48107810 2023-12-11T17:38:22.658 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:21.765+0000 7f1c4fed6700 1 -- 172.21.15.59:0/627678509 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c481059c0 con 0x7f1c48107810 2023-12-11T17:38:22.658 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:21.766+0000 7f1c4fed6700 1 -- 172.21.15.59:0/627678509 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c48105e60 con 0x7f1c48107810 2023-12-11T17:38:22.658 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:21.766+0000 7f1c3effd700 1 -- 172.21.15.59:0/627678509 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 13) v1 ==== 45636+0+0 (secure 0 0 0) 0x7f1c4401ab60 con 0x7f1c48107810 2023-12-11T17:38:22.659 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:21.767+0000 7f1c3effd700 1 --2- 172.21.15.59:0/627678509 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f1c34038960 0x7f1c3403ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:38:22.659 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:21.767+0000 7f1c3effd700 1 -- 172.21.15.59:0/627678509 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f1c4404cd50 con 0x7f1c48107810 2023-12-11T17:38:22.659 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:21.767+0000 7f1c4fed6700 1 -- 172.21.15.59:0/627678509 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c48067940 con 0x7f1c48107810 2023-12-11T17:38:22.659 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:21.767+0000 7f1c4d471700 1 --2- 172.21.15.59:0/627678509 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f1c34038960 0x7f1c3403ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:38:22.659 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:21.767+0000 7f1c4d471700 1 --2- 172.21.15.59:0/627678509 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f1c34038960 0x7f1c3403ae20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1c38006fd0 tx=0x7f1c38006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:38:22.659 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:21.771+0000 7f1c3effd700 1 -- 172.21.15.59:0/627678509 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f1c4401a430 con 0x7f1c48107810 2023-12-11T17:38:22.659 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:22.353+0000 7f1c4fed6700 1 -- 172.21.15.59:0/627678509 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7f1c4810aeb0 con 0x7f1c48107810 2023-12-11T17:38:22.659 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:22.356+0000 7f1c3effd700 1 -- 172.21.15.59:0/627678509 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) v1 ==== 153+0+0 (secure 0 0 0) 0x7f1c4810aeb0 con 0x7f1c48107810 2023-12-11T17:38:22.659 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:22.358+0000 7f1c4fed6700 1 -- 172.21.15.59:0/627678509 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f1c34038960 msgr2=0x7f1c3403ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:38:22.659 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:22.358+0000 7f1c4fed6700 1 --2- 172.21.15.59:0/627678509 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f1c34038960 0x7f1c3403ae20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1c38006fd0 tx=0x7f1c38006e40 comp rx=0 tx=0).stop 2023-12-11T17:38:22.659 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:22.358+0000 7f1c4fed6700 1 -- 172.21.15.59:0/627678509 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c48107810 msgr2=0x7f1c48107220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:38:22.659 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:22.358+0000 7f1c4fed6700 1 --2- 172.21.15.59:0/627678509 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c48107810 0x7f1c48107220 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1c44011670 tx=0x7f1c44011a30 comp rx=0 tx=0).stop 2023-12-11T17:38:22.659 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:22.358+0000 7f1c4fed6700 1 -- 172.21.15.59:0/627678509 shutdown_connections 2023-12-11T17:38:22.659 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:22.358+0000 7f1c4fed6700 1 --2- 172.21.15.59:0/627678509 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f1c34038960 0x7f1c3403ae20 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:38:22.660 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:22.358+0000 7f1c4fed6700 1 --2- 172.21.15.59:0/627678509 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c48107810 0x7f1c48107220 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:38:22.660 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:22.358+0000 7f1c4fed6700 1 -- 172.21.15.59:0/627678509 >> 172.21.15.59:0/627678509 conn(0x7f1c481011a0 msgr2=0x7f1c48101cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:38:22.660 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:22.358+0000 7f1c4fed6700 1 -- 172.21.15.59:0/627678509 shutdown_connections 2023-12-11T17:38:22.660 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2023-12-11T17:38:22.358+0000 7f1c4fed6700 1 -- 172.21.15.59:0/627678509 wait complete. 2023-12-11T17:38:22.660 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2023-12-11T17:38:22.660 INFO:teuthology.orchestra.run.smithi059.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2023-12-11T17:38:22.660 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-12-11T17:38:22.660 INFO:teuthology.orchestra.run.smithi059.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid d0a07eaa-984b-11ee-95a3-87774f69a715 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2023-12-11T17:38:22.660 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-12-11T17:38:22.660 INFO:teuthology.orchestra.run.smithi059.stdout:Or, if you are only running a single cluster on this host: 2023-12-11T17:38:22.660 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-12-11T17:38:22.660 INFO:teuthology.orchestra.run.smithi059.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2023-12-11T17:38:22.660 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-12-11T17:38:22.660 INFO:teuthology.orchestra.run.smithi059.stdout:Please consider enabling telemetry to help improve Ceph: 2023-12-11T17:38:22.660 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-12-11T17:38:22.661 INFO:teuthology.orchestra.run.smithi059.stdout: ceph telemetry on 2023-12-11T17:38:22.661 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-12-11T17:38:22.661 INFO:teuthology.orchestra.run.smithi059.stdout:For more information see: 2023-12-11T17:38:22.661 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-12-11T17:38:22.661 INFO:teuthology.orchestra.run.smithi059.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2023-12-11T17:38:22.661 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-12-11T17:38:22.661 INFO:teuthology.orchestra.run.smithi059.stdout:Bootstrap complete. 2023-12-11T17:38:22.698 INFO:tasks.cephadm:Fetching config... 2023-12-11T17:38:22.698 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2023-12-11T17:38:22.698 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2023-12-11T17:38:22.715 INFO:tasks.cephadm:Fetching client.admin keyring... 2023-12-11T17:38:22.715 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2023-12-11T17:38:22.715 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2023-12-11T17:38:22.772 INFO:tasks.cephadm:Fetching mon keyring... 2023-12-11T17:38:22.773 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2023-12-11T17:38:22.773 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd if=/var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/mon.a/keyring of=/dev/stdout 2023-12-11T17:38:22.841 INFO:tasks.cephadm:Fetching pub ssh key... 2023-12-11T17:38:22.841 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2023-12-11T17:38:22.841 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2023-12-11T17:38:22.898 INFO:tasks.cephadm:Installing pub ssh key for root users... 2023-12-11T17:38:22.898 DEBUG:teuthology.orchestra.run.smithi059:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC2X706m8ehaO/BbVOxHH3AThaWP+IN5sgEq68TzOH8k4qte1+vTAM5nclfi8bYYPoqPCJyrIft9nDQ0rwwGMhMc0pLDw+2Ie6Zlhv80pu8vIL4zshzxFi7o2TEmqc3NYqxrrY234BygnfBzOmsQcNEyHJk8IcswJr4aVN8ucE2HdbBxLLNNW5wnPV1pj95SdY5Jo45PEi620bCuWt2D/Qmc+bUDZOCKUYg0GGKOGzTS9+Oo4uDX1PcByW+zPG+AHLaFcmiyJ+3GdvTNvC0R2LsEaWAAdMR5bMftpbRt0Bl/TOqneIQwxEWocC0HKNDRF3g6aqQgwJ8JnJ1moU6Y7P3zzsKbit0tSYBD5geSFJyfuJ6J4dAHQ5E7pbdqpWyWWCB4ibSM00dopweUBbTw5HxTLo2bTbMBZL41RjL2nGHKX3LFQpdHpQqmDCS7N4IMtg5hz8byUvMhDkmqeAvR0S2LVfV8+trGlnDSGTXllm+sw3Y6VUDxflb00QESSncRS0= ceph-d0a07eaa-984b-11ee-95a3-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-12-11T17:38:22.981 INFO:teuthology.orchestra.run.smithi059.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC2X706m8ehaO/BbVOxHH3AThaWP+IN5sgEq68TzOH8k4qte1+vTAM5nclfi8bYYPoqPCJyrIft9nDQ0rwwGMhMc0pLDw+2Ie6Zlhv80pu8vIL4zshzxFi7o2TEmqc3NYqxrrY234BygnfBzOmsQcNEyHJk8IcswJr4aVN8ucE2HdbBxLLNNW5wnPV1pj95SdY5Jo45PEi620bCuWt2D/Qmc+bUDZOCKUYg0GGKOGzTS9+Oo4uDX1PcByW+zPG+AHLaFcmiyJ+3GdvTNvC0R2LsEaWAAdMR5bMftpbRt0Bl/TOqneIQwxEWocC0HKNDRF3g6aqQgwJ8JnJ1moU6Y7P3zzsKbit0tSYBD5geSFJyfuJ6J4dAHQ5E7pbdqpWyWWCB4ibSM00dopweUBbTw5HxTLo2bTbMBZL41RjL2nGHKX3LFQpdHpQqmDCS7N4IMtg5hz8byUvMhDkmqeAvR0S2LVfV8+trGlnDSGTXllm+sw3Y6VUDxflb00QESSncRS0= ceph-d0a07eaa-984b-11ee-95a3-87774f69a715 2023-12-11T17:38:22.993 DEBUG:teuthology.orchestra.run.smithi115:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC2X706m8ehaO/BbVOxHH3AThaWP+IN5sgEq68TzOH8k4qte1+vTAM5nclfi8bYYPoqPCJyrIft9nDQ0rwwGMhMc0pLDw+2Ie6Zlhv80pu8vIL4zshzxFi7o2TEmqc3NYqxrrY234BygnfBzOmsQcNEyHJk8IcswJr4aVN8ucE2HdbBxLLNNW5wnPV1pj95SdY5Jo45PEi620bCuWt2D/Qmc+bUDZOCKUYg0GGKOGzTS9+Oo4uDX1PcByW+zPG+AHLaFcmiyJ+3GdvTNvC0R2LsEaWAAdMR5bMftpbRt0Bl/TOqneIQwxEWocC0HKNDRF3g6aqQgwJ8JnJ1moU6Y7P3zzsKbit0tSYBD5geSFJyfuJ6J4dAHQ5E7pbdqpWyWWCB4ibSM00dopweUBbTw5HxTLo2bTbMBZL41RjL2nGHKX3LFQpdHpQqmDCS7N4IMtg5hz8byUvMhDkmqeAvR0S2LVfV8+trGlnDSGTXllm+sw3Y6VUDxflb00QESSncRS0= ceph-d0a07eaa-984b-11ee-95a3-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-12-11T17:38:23.041 INFO:teuthology.orchestra.run.smithi115.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC2X706m8ehaO/BbVOxHH3AThaWP+IN5sgEq68TzOH8k4qte1+vTAM5nclfi8bYYPoqPCJyrIft9nDQ0rwwGMhMc0pLDw+2Ie6Zlhv80pu8vIL4zshzxFi7o2TEmqc3NYqxrrY234BygnfBzOmsQcNEyHJk8IcswJr4aVN8ucE2HdbBxLLNNW5wnPV1pj95SdY5Jo45PEi620bCuWt2D/Qmc+bUDZOCKUYg0GGKOGzTS9+Oo4uDX1PcByW+zPG+AHLaFcmiyJ+3GdvTNvC0R2LsEaWAAdMR5bMftpbRt0Bl/TOqneIQwxEWocC0HKNDRF3g6aqQgwJ8JnJ1moU6Y7P3zzsKbit0tSYBD5geSFJyfuJ6J4dAHQ5E7pbdqpWyWWCB4ibSM00dopweUBbTw5HxTLo2bTbMBZL41RjL2nGHKX3LFQpdHpQqmDCS7N4IMtg5hz8byUvMhDkmqeAvR0S2LVfV8+trGlnDSGTXllm+sw3Y6VUDxflb00QESSncRS0= ceph-d0a07eaa-984b-11ee-95a3-87774f69a715 2023-12-11T17:38:23.053 DEBUG:teuthology.orchestra.run.smithi181:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC2X706m8ehaO/BbVOxHH3AThaWP+IN5sgEq68TzOH8k4qte1+vTAM5nclfi8bYYPoqPCJyrIft9nDQ0rwwGMhMc0pLDw+2Ie6Zlhv80pu8vIL4zshzxFi7o2TEmqc3NYqxrrY234BygnfBzOmsQcNEyHJk8IcswJr4aVN8ucE2HdbBxLLNNW5wnPV1pj95SdY5Jo45PEi620bCuWt2D/Qmc+bUDZOCKUYg0GGKOGzTS9+Oo4uDX1PcByW+zPG+AHLaFcmiyJ+3GdvTNvC0R2LsEaWAAdMR5bMftpbRt0Bl/TOqneIQwxEWocC0HKNDRF3g6aqQgwJ8JnJ1moU6Y7P3zzsKbit0tSYBD5geSFJyfuJ6J4dAHQ5E7pbdqpWyWWCB4ibSM00dopweUBbTw5HxTLo2bTbMBZL41RjL2nGHKX3LFQpdHpQqmDCS7N4IMtg5hz8byUvMhDkmqeAvR0S2LVfV8+trGlnDSGTXllm+sw3Y6VUDxflb00QESSncRS0= ceph-d0a07eaa-984b-11ee-95a3-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-12-11T17:38:23.101 INFO:teuthology.orchestra.run.smithi181.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC2X706m8ehaO/BbVOxHH3AThaWP+IN5sgEq68TzOH8k4qte1+vTAM5nclfi8bYYPoqPCJyrIft9nDQ0rwwGMhMc0pLDw+2Ie6Zlhv80pu8vIL4zshzxFi7o2TEmqc3NYqxrrY234BygnfBzOmsQcNEyHJk8IcswJr4aVN8ucE2HdbBxLLNNW5wnPV1pj95SdY5Jo45PEi620bCuWt2D/Qmc+bUDZOCKUYg0GGKOGzTS9+Oo4uDX1PcByW+zPG+AHLaFcmiyJ+3GdvTNvC0R2LsEaWAAdMR5bMftpbRt0Bl/TOqneIQwxEWocC0HKNDRF3g6aqQgwJ8JnJ1moU6Y7P3zzsKbit0tSYBD5geSFJyfuJ6J4dAHQ5E7pbdqpWyWWCB4ibSM00dopweUBbTw5HxTLo2bTbMBZL41RjL2nGHKX3LFQpdHpQqmDCS7N4IMtg5hz8byUvMhDkmqeAvR0S2LVfV8+trGlnDSGTXllm+sw3Y6VUDxflb00QESSncRS0= ceph-d0a07eaa-984b-11ee-95a3-87774f69a715 2023-12-11T17:38:23.114 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d0a07eaa-984b-11ee-95a3-87774f69a715 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2023-12-11T17:38:23.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:23 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/627678509' entity='client.admin' 2023-12-11T17:38:23.720 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/mon.a/config 2023-12-11T17:38:25.275 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:25.273+0000 7f336b482700 1 -- 172.21.15.59:0/678170953 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f33641019f0 msgr2=0x7f3364103de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:38:25.276 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:25.273+0000 7f336b482700 1 --2- 172.21.15.59:0/678170953 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f33641019f0 0x7f3364103de0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3360009910 tx=0x7f3360009c20 comp rx=0 tx=0).stop 2023-12-11T17:38:25.276 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:25.273+0000 7f336b482700 1 -- 172.21.15.59:0/678170953 shutdown_connections 2023-12-11T17:38:25.276 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:25.273+0000 7f336b482700 1 --2- 172.21.15.59:0/678170953 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f33641019f0 0x7f3364103de0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:38:25.276 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:25.273+0000 7f336b482700 1 -- 172.21.15.59:0/678170953 >> 172.21.15.59:0/678170953 conn(0x7f33640fb400 msgr2=0x7f33640fd840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:38:25.276 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:25.273+0000 7f336b482700 1 -- 172.21.15.59:0/678170953 shutdown_connections 2023-12-11T17:38:25.276 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:25.273+0000 7f336b482700 1 -- 172.21.15.59:0/678170953 wait complete. 2023-12-11T17:38:25.277 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:25.274+0000 7f336b482700 1 Processor -- start 2023-12-11T17:38:25.277 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:25.274+0000 7f336b482700 1 -- start start 2023-12-11T17:38:25.278 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:25.274+0000 7f336b482700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f33641019f0 0x7f33641985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:38:25.278 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:25.274+0000 7f336b482700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3364198ae0 con 0x7f33641019f0 2023-12-11T17:38:25.278 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:25.274+0000 7f336921e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f33641019f0 0x7f33641985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:38:25.278 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:25.274+0000 7f336921e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f33641019f0 0x7f33641985a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60090/0 (socket says 172.21.15.59:60090) 2023-12-11T17:38:25.278 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:25.274+0000 7f336921e700 1 -- 172.21.15.59:0/4152518469 learned_addr learned my addr 172.21.15.59:0/4152518469 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:38:25.278 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:25.274+0000 7f336921e700 1 -- 172.21.15.59:0/4152518469 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33600095f0 con 0x7f33641019f0 2023-12-11T17:38:25.278 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:25.274+0000 7f336921e700 1 --2- 172.21.15.59:0/4152518469 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f33641019f0 0x7f33641985a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3360005b40 tx=0x7f33600057e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:38:25.278 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:25.275+0000 7f335a7fc700 1 -- 172.21.15.59:0/4152518469 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f336001d070 con 0x7f33641019f0 2023-12-11T17:38:25.278 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:25.275+0000 7f336b482700 1 -- 172.21.15.59:0/4152518469 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3364198ce0 con 0x7f33641019f0 2023-12-11T17:38:25.278 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:25.275+0000 7f335a7fc700 1 -- 172.21.15.59:0/4152518469 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f336000be20 con 0x7f33641019f0 2023-12-11T17:38:25.278 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:25.275+0000 7f335a7fc700 1 -- 172.21.15.59:0/4152518469 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3360022d80 con 0x7f33641019f0 2023-12-11T17:38:25.278 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:25.275+0000 7f336b482700 1 -- 172.21.15.59:0/4152518469 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3364199180 con 0x7f33641019f0 2023-12-11T17:38:25.279 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:25.276+0000 7f335a7fc700 1 -- 172.21.15.59:0/4152518469 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 13) v1 ==== 45636+0+0 (secure 0 0 0) 0x7f336000f460 con 0x7f33641019f0 2023-12-11T17:38:25.279 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:25.276+0000 7f336b482700 1 -- 172.21.15.59:0/4152518469 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3364192080 con 0x7f33641019f0 2023-12-11T17:38:25.279 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:25.276+0000 7f335a7fc700 1 --2- 172.21.15.59:0/4152518469 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f33500388c0 0x7f335003ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:38:25.279 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:25.276+0000 7f335a7fc700 1 -- 172.21.15.59:0/4152518469 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f336004ce80 con 0x7f33641019f0 2023-12-11T17:38:25.279 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:25.276+0000 7f3368a1d700 1 --2- 172.21.15.59:0/4152518469 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f33500388c0 0x7f335003ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:38:25.279 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:25.276+0000 7f3368a1d700 1 --2- 172.21.15.59:0/4152518469 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f33500388c0 0x7f335003ad80 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f3354006fd0 tx=0x7f3354006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:38:25.284 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:25.282+0000 7f335a7fc700 1 -- 172.21.15.59:0/4152518469 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f3360017440 con 0x7f33641019f0 2023-12-11T17:38:25.482 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:25.479+0000 7f336b482700 1 -- 172.21.15.59:0/4152518469 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7f3364062c00 con 0x7f33641019f0 2023-12-11T17:38:25.484 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:25.482+0000 7f335a7fc700 1 -- 172.21.15.59:0/4152518469 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7f3364062c00 con 0x7f33641019f0 2023-12-11T17:38:25.487 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:25.484+0000 7f336b482700 1 -- 172.21.15.59:0/4152518469 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f33500388c0 msgr2=0x7f335003ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:38:25.487 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:25.485+0000 7f336b482700 1 --2- 172.21.15.59:0/4152518469 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f33500388c0 0x7f335003ad80 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f3354006fd0 tx=0x7f3354006e40 comp rx=0 tx=0).stop 2023-12-11T17:38:25.487 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:25.485+0000 7f336b482700 1 -- 172.21.15.59:0/4152518469 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f33641019f0 msgr2=0x7f33641985a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:38:25.487 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:25.485+0000 7f336b482700 1 --2- 172.21.15.59:0/4152518469 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f33641019f0 0x7f33641985a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3360005b40 tx=0x7f33600057e0 comp rx=0 tx=0).stop 2023-12-11T17:38:25.487 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:25.485+0000 7f336b482700 1 -- 172.21.15.59:0/4152518469 shutdown_connections 2023-12-11T17:38:25.487 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:25.485+0000 7f336b482700 1 --2- 172.21.15.59:0/4152518469 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f33500388c0 0x7f335003ad80 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:38:25.487 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:25.485+0000 7f336b482700 1 --2- 172.21.15.59:0/4152518469 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f33641019f0 0x7f33641985a0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:38:25.487 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:25.485+0000 7f336b482700 1 -- 172.21.15.59:0/4152518469 >> 172.21.15.59:0/4152518469 conn(0x7f33640fb400 msgr2=0x7f33640fd840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:38:25.487 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:25.485+0000 7f336b482700 1 -- 172.21.15.59:0/4152518469 shutdown_connections 2023-12-11T17:38:25.488 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:25.485+0000 7f336b482700 1 -- 172.21.15.59:0/4152518469 wait complete. 2023-12-11T17:38:25.811 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:25 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:38:25.811 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:25 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:38:25.812 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:25 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi059", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:38:25.812 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:25 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:38:25.812 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:25 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:38:25.812 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:25 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:38:25.812 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:25 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/4152518469' entity='client.admin' 2023-12-11T17:38:25.812 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:25 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:38:25.812 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:25 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:38:25.878 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2023-12-11T17:38:25.878 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d0a07eaa-984b-11ee-95a3-87774f69a715 -- ceph orch client-keyring set client.admin '*' --mode 0755 2023-12-11T17:38:26.490 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/mon.a/config 2023-12-11T17:38:27.587 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:27.584+0000 7ff134b56700 1 -- 172.21.15.59:0/865967764 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff1300ffe60 msgr2=0x7ff130100280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:38:27.587 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:27.584+0000 7ff134b56700 1 --2- 172.21.15.59:0/865967764 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff1300ffe60 0x7ff130100280 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff124009910 tx=0x7ff124009c20 comp rx=0 tx=0).stop 2023-12-11T17:38:27.587 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:27.584+0000 7ff134b56700 1 -- 172.21.15.59:0/865967764 shutdown_connections 2023-12-11T17:38:27.587 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:27.584+0000 7ff134b56700 1 --2- 172.21.15.59:0/865967764 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff1300ffe60 0x7ff130100280 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:38:27.587 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:27.584+0000 7ff134b56700 1 -- 172.21.15.59:0/865967764 >> 172.21.15.59:0/865967764 conn(0x7ff1300fb400 msgr2=0x7ff1300fd840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:38:27.587 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:27.584+0000 7ff134b56700 1 -- 172.21.15.59:0/865967764 shutdown_connections 2023-12-11T17:38:27.588 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:27.584+0000 7ff134b56700 1 -- 172.21.15.59:0/865967764 wait complete. 2023-12-11T17:38:27.589 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:27.585+0000 7ff134b56700 1 Processor -- start 2023-12-11T17:38:27.590 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:27.585+0000 7ff134b56700 1 -- start start 2023-12-11T17:38:27.590 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:27.585+0000 7ff134b56700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff1300ffe60 0x7ff1301952c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:38:27.590 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:27.585+0000 7ff134b56700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff130195800 con 0x7ff1300ffe60 2023-12-11T17:38:27.590 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:27.586+0000 7ff12e59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff1300ffe60 0x7ff1301952c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:38:27.590 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:27.586+0000 7ff12e59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff1300ffe60 0x7ff1301952c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60096/0 (socket says 172.21.15.59:60096) 2023-12-11T17:38:27.590 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:27.586+0000 7ff12e59c700 1 -- 172.21.15.59:0/3852801635 learned_addr learned my addr 172.21.15.59:0/3852801635 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:38:27.590 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:27.586+0000 7ff12e59c700 1 -- 172.21.15.59:0/3852801635 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1240095f0 con 0x7ff1300ffe60 2023-12-11T17:38:27.590 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:27.586+0000 7ff12e59c700 1 --2- 172.21.15.59:0/3852801635 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff1300ffe60 0x7ff1301952c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff124005b40 tx=0x7ff1240057e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:38:27.590 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:27.586+0000 7ff11f7fe700 1 -- 172.21.15.59:0/3852801635 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff12401d070 con 0x7ff1300ffe60 2023-12-11T17:38:27.590 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:27.586+0000 7ff134b56700 1 -- 172.21.15.59:0/3852801635 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff130195a00 con 0x7ff1300ffe60 2023-12-11T17:38:27.591 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:27.586+0000 7ff11f7fe700 1 -- 172.21.15.59:0/3852801635 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff12400be20 con 0x7ff1300ffe60 2023-12-11T17:38:27.591 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:27.586+0000 7ff11f7fe700 1 -- 172.21.15.59:0/3852801635 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff124022d80 con 0x7ff1300ffe60 2023-12-11T17:38:27.591 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:27.586+0000 7ff134b56700 1 -- 172.21.15.59:0/3852801635 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff130195ea0 con 0x7ff1300ffe60 2023-12-11T17:38:27.591 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:27.587+0000 7ff11f7fe700 1 -- 172.21.15.59:0/3852801635 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 13) v1 ==== 45636+0+0 (secure 0 0 0) 0x7ff12400f460 con 0x7ff1300ffe60 2023-12-11T17:38:27.591 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:27.587+0000 7ff11f7fe700 1 --2- 172.21.15.59:0/3852801635 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7ff1180388c0 0x7ff11803ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:38:27.591 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:27.587+0000 7ff11f7fe700 1 -- 172.21.15.59:0/3852801635 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7ff12404cad0 con 0x7ff1300ffe60 2023-12-11T17:38:27.591 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:27.587+0000 7ff134b56700 1 -- 172.21.15.59:0/3852801635 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff13018eb00 con 0x7ff1300ffe60 2023-12-11T17:38:27.591 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:27.587+0000 7ff12dd9b700 1 --2- 172.21.15.59:0/3852801635 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7ff1180388c0 0x7ff11803ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:38:27.591 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:27.588+0000 7ff12dd9b700 1 --2- 172.21.15.59:0/3852801635 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7ff1180388c0 0x7ff11803ad80 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff120006fd0 tx=0x7ff120006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:38:27.594 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:27.592+0000 7ff11f7fe700 1 -- 172.21.15.59:0/3852801635 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7ff124027070 con 0x7ff1300ffe60 2023-12-11T17:38:27.800 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:27.798+0000 7ff134b56700 1 -- 172.21.15.59:0/3852801635 --> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7ff130061a10 con 0x7ff1180388c0 2023-12-11T17:38:27.804 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:27.801+0000 7ff11f7fe700 1 -- 172.21.15.59:0/3852801635 <== mgr.14150 v2:172.21.15.59:6800/3627728353 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7ff130061a10 con 0x7ff1180388c0 2023-12-11T17:38:27.806 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:27.804+0000 7ff134b56700 1 -- 172.21.15.59:0/3852801635 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7ff1180388c0 msgr2=0x7ff11803ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:38:27.806 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:27.804+0000 7ff134b56700 1 --2- 172.21.15.59:0/3852801635 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7ff1180388c0 0x7ff11803ad80 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff120006fd0 tx=0x7ff120006e40 comp rx=0 tx=0).stop 2023-12-11T17:38:27.806 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:27.804+0000 7ff134b56700 1 -- 172.21.15.59:0/3852801635 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff1300ffe60 msgr2=0x7ff1301952c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:38:27.806 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:27.804+0000 7ff134b56700 1 --2- 172.21.15.59:0/3852801635 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff1300ffe60 0x7ff1301952c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff124005b40 tx=0x7ff1240057e0 comp rx=0 tx=0).stop 2023-12-11T17:38:27.806 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:27.804+0000 7ff134b56700 1 -- 172.21.15.59:0/3852801635 shutdown_connections 2023-12-11T17:38:27.807 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:27.804+0000 7ff134b56700 1 --2- 172.21.15.59:0/3852801635 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7ff1180388c0 0x7ff11803ad80 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:38:27.807 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:27.804+0000 7ff134b56700 1 --2- 172.21.15.59:0/3852801635 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff1300ffe60 0x7ff1301952c0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:38:27.807 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:27.804+0000 7ff134b56700 1 -- 172.21.15.59:0/3852801635 >> 172.21.15.59:0/3852801635 conn(0x7ff1300fb400 msgr2=0x7ff1300fbf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:38:27.807 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:27.804+0000 7ff134b56700 1 -- 172.21.15.59:0/3852801635 shutdown_connections 2023-12-11T17:38:27.807 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:27.804+0000 7ff134b56700 1 -- 172.21.15.59:0/3852801635 wait complete. 2023-12-11T17:38:28.266 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi115 2023-12-11T17:38:28.266 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-12-11T17:38:28.266 DEBUG:teuthology.orchestra.run.smithi115:> dd of=/etc/ceph/ceph.conf 2023-12-11T17:38:28.287 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-12-11T17:38:28.287 DEBUG:teuthology.orchestra.run.smithi115:> dd of=/etc/ceph/ceph.client.admin.keyring 2023-12-11T17:38:28.346 INFO:tasks.cephadm:Adding host smithi115 to orchestrator... 2023-12-11T17:38:28.346 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d0a07eaa-984b-11ee-95a3-87774f69a715 -- ceph orch host add smithi115 2023-12-11T17:38:28.962 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/mon.a/config 2023-12-11T17:38:29.113 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:28 smithi059 ceph-mon[98847]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T17:38:29.113 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:28 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:38:29.113 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:28 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:38:29.113 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:28 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:38:29.113 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:28 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:38:29.113 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:28 smithi059 ceph-mon[98847]: Updating smithi059:/etc/ceph/ceph.conf 2023-12-11T17:38:30.017 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:30.012+0000 7f50a3e3b700 1 -- 172.21.15.59:0/2157781342 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f509c105e70 msgr2=0x7f509c106250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:38:30.017 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:30.012+0000 7f50a3e3b700 1 --2- 172.21.15.59:0/2157781342 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f509c105e70 0x7f509c106250 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f508c009910 tx=0x7f508c009c20 comp rx=0 tx=0).stop 2023-12-11T17:38:30.017 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:30.012+0000 7f50a3e3b700 1 -- 172.21.15.59:0/2157781342 shutdown_connections 2023-12-11T17:38:30.017 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:30.012+0000 7f50a3e3b700 1 --2- 172.21.15.59:0/2157781342 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f509c105e70 0x7f509c106250 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:38:30.017 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:30.012+0000 7f50a3e3b700 1 -- 172.21.15.59:0/2157781342 >> 172.21.15.59:0/2157781342 conn(0x7f509c075930 msgr2=0x7f509c075d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:38:30.017 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:30.012+0000 7f50a3e3b700 1 -- 172.21.15.59:0/2157781342 shutdown_connections 2023-12-11T17:38:30.017 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:30.012+0000 7f50a3e3b700 1 -- 172.21.15.59:0/2157781342 wait complete. 2023-12-11T17:38:30.018 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:30.013+0000 7f50a3e3b700 1 Processor -- start 2023-12-11T17:38:30.018 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:30.013+0000 7f50a3e3b700 1 -- start start 2023-12-11T17:38:30.018 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:30.013+0000 7f50a3e3b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f509c105e70 0x7f509c195490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:38:30.018 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:30.013+0000 7f50a3e3b700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f509c069500 con 0x7f509c105e70 2023-12-11T17:38:30.018 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:30.013+0000 7f50a1bd7700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f509c105e70 0x7f509c195490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:38:30.018 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:30.013+0000 7f50a1bd7700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f509c105e70 0x7f509c195490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60102/0 (socket says 172.21.15.59:60102) 2023-12-11T17:38:30.018 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:30.013+0000 7f50a1bd7700 1 -- 172.21.15.59:0/1796762295 learned_addr learned my addr 172.21.15.59:0/1796762295 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:38:30.018 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:30.013+0000 7f50a1bd7700 1 -- 172.21.15.59:0/1796762295 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f508c0095f0 con 0x7f509c105e70 2023-12-11T17:38:30.018 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:30.013+0000 7f50a1bd7700 1 --2- 172.21.15.59:0/1796762295 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f509c105e70 0x7f509c195490 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f508c006010 tx=0x7f508c004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:38:30.018 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:30.014+0000 7f5092ffd700 1 -- 172.21.15.59:0/1796762295 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f508c01d070 con 0x7f509c105e70 2023-12-11T17:38:30.018 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:30.014+0000 7f5092ffd700 1 -- 172.21.15.59:0/1796762295 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f508c00bbb0 con 0x7f509c105e70 2023-12-11T17:38:30.018 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:30.014+0000 7f50a3e3b700 1 -- 172.21.15.59:0/1796762295 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f509c1959d0 con 0x7f509c105e70 2023-12-11T17:38:30.018 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:30.014+0000 7f5092ffd700 1 -- 172.21.15.59:0/1796762295 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f508c003810 con 0x7f509c105e70 2023-12-11T17:38:30.018 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:30.014+0000 7f50a3e3b700 1 -- 172.21.15.59:0/1796762295 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f509c195e50 con 0x7f509c105e70 2023-12-11T17:38:30.019 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:30.014+0000 7f5092ffd700 1 -- 172.21.15.59:0/1796762295 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 13) v1 ==== 45636+0+0 (secure 0 0 0) 0x7f508c00b6f0 con 0x7f509c105e70 2023-12-11T17:38:30.019 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:30.015+0000 7f50a3e3b700 1 -- 172.21.15.59:0/1796762295 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f509c101520 con 0x7f509c105e70 2023-12-11T17:38:30.019 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:30.015+0000 7f5092ffd700 1 --2- 172.21.15.59:0/1796762295 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f50880388c0 0x7f508803ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:38:30.019 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:30.015+0000 7f5092ffd700 1 -- 172.21.15.59:0/1796762295 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f508c04c530 con 0x7f509c105e70 2023-12-11T17:38:30.019 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:30.015+0000 7f50a13d6700 1 --2- 172.21.15.59:0/1796762295 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f50880388c0 0x7f508803ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:38:30.019 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:30.015+0000 7f50a13d6700 1 --2- 172.21.15.59:0/1796762295 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f50880388c0 0x7f508803ad80 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f5098006fd0 tx=0x7f5098006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:38:30.023 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:30.020+0000 7f5092ffd700 1 -- 172.21.15.59:0/1796762295 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f508c029540 con 0x7f509c105e70 2023-12-11T17:38:30.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:29 smithi059 ceph-mon[98847]: Updating smithi059:/var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/config/ceph.conf 2023-12-11T17:38:30.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:29 smithi059 ceph-mon[98847]: Updating smithi059:/etc/ceph/ceph.client.admin.keyring 2023-12-11T17:38:30.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:29 smithi059 ceph-mon[98847]: Updating smithi059:/var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/config/ceph.client.admin.keyring 2023-12-11T17:38:30.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:29 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:38:30.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:29 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:38:30.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:29 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:38:30.227 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:30.224+0000 7f50a3e3b700 1 -- 172.21.15.59:0/1796762295 --> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi115", "target": ["mon-mgr", ""]}) v1 -- 0x7f509c068df0 con 0x7f50880388c0 2023-12-11T17:38:31.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:31 smithi059 ceph-mon[98847]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi115", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T17:38:31.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:31 smithi059 ceph-mon[98847]: Deploying cephadm binary to smithi115 2023-12-11T17:38:35.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:35 smithi059 ceph-mon[98847]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T17:38:35.590 INFO:teuthology.orchestra.run.smithi059.stdout:Added host 'smithi115' with addr '172.21.15.115' 2023-12-11T17:38:35.590 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:35.586+0000 7f5092ffd700 1 -- 172.21.15.59:0/1796762295 <== mgr.14150 v2:172.21.15.59:6800/3627728353 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f509c068df0 con 0x7f50880388c0 2023-12-11T17:38:35.593 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:35.589+0000 7f50a3e3b700 1 -- 172.21.15.59:0/1796762295 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f50880388c0 msgr2=0x7f508803ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:38:35.593 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:35.589+0000 7f50a3e3b700 1 --2- 172.21.15.59:0/1796762295 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f50880388c0 0x7f508803ad80 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f5098006fd0 tx=0x7f5098006e40 comp rx=0 tx=0).stop 2023-12-11T17:38:35.593 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:35.589+0000 7f50a3e3b700 1 -- 172.21.15.59:0/1796762295 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f509c105e70 msgr2=0x7f509c195490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:38:35.593 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:35.589+0000 7f50a3e3b700 1 --2- 172.21.15.59:0/1796762295 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f509c105e70 0x7f509c195490 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f508c006010 tx=0x7f508c004ff0 comp rx=0 tx=0).stop 2023-12-11T17:38:35.593 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:35.589+0000 7f50a3e3b700 1 -- 172.21.15.59:0/1796762295 shutdown_connections 2023-12-11T17:38:35.593 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:35.589+0000 7f50a3e3b700 1 --2- 172.21.15.59:0/1796762295 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f50880388c0 0x7f508803ad80 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:38:35.593 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:35.590+0000 7f50a3e3b700 1 --2- 172.21.15.59:0/1796762295 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f509c105e70 0x7f509c195490 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:38:35.593 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:35.590+0000 7f50a3e3b700 1 -- 172.21.15.59:0/1796762295 >> 172.21.15.59:0/1796762295 conn(0x7f509c075930 msgr2=0x7f509c0fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:38:35.594 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:35.590+0000 7f50a3e3b700 1 -- 172.21.15.59:0/1796762295 shutdown_connections 2023-12-11T17:38:35.594 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:35.590+0000 7f50a3e3b700 1 -- 172.21.15.59:0/1796762295 wait complete. 2023-12-11T17:38:36.029 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d0a07eaa-984b-11ee-95a3-87774f69a715 -- ceph orch host ls --format=json 2023-12-11T17:38:36.638 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/mon.a/config 2023-12-11T17:38:36.844 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:36 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:38:36.844 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:36 smithi059 ceph-mon[98847]: Added host smithi115 2023-12-11T17:38:36.844 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:36 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:38:36.845 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:36 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:38:37.734 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:37.729+0000 7f2301c63700 1 -- 172.21.15.59:0/1771105254 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f22fc105e70 msgr2=0x7f22fc106250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:38:37.734 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:37.729+0000 7f2301c63700 1 --2- 172.21.15.59:0/1771105254 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f22fc105e70 0x7f22fc106250 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f22ec009910 tx=0x7f22ec009c20 comp rx=0 tx=0).stop 2023-12-11T17:38:37.734 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:37.729+0000 7f2301c63700 1 -- 172.21.15.59:0/1771105254 shutdown_connections 2023-12-11T17:38:37.734 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:37.729+0000 7f2301c63700 1 --2- 172.21.15.59:0/1771105254 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f22fc105e70 0x7f22fc106250 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:38:37.734 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:37.729+0000 7f2301c63700 1 -- 172.21.15.59:0/1771105254 >> 172.21.15.59:0/1771105254 conn(0x7f22fc075930 msgr2=0x7f22fc075d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:38:37.734 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:37.729+0000 7f2301c63700 1 -- 172.21.15.59:0/1771105254 shutdown_connections 2023-12-11T17:38:37.734 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:37.729+0000 7f2301c63700 1 -- 172.21.15.59:0/1771105254 wait complete. 2023-12-11T17:38:37.735 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:37.730+0000 7f2301c63700 1 Processor -- start 2023-12-11T17:38:37.735 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:37.730+0000 7f2301c63700 1 -- start start 2023-12-11T17:38:37.735 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:37.730+0000 7f2301c63700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f22fc105e70 0x7f22fc195750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:38:37.735 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:37.730+0000 7f2301c63700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22fc195c90 con 0x7f22fc105e70 2023-12-11T17:38:37.735 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:37.730+0000 7f22fb7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f22fc105e70 0x7f22fc195750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:38:37.735 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:37.730+0000 7f22fb7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f22fc105e70 0x7f22fc195750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60128/0 (socket says 172.21.15.59:60128) 2023-12-11T17:38:37.735 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:37.730+0000 7f22fb7fe700 1 -- 172.21.15.59:0/3915228194 learned_addr learned my addr 172.21.15.59:0/3915228194 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:38:37.735 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:37.731+0000 7f22fb7fe700 1 -- 172.21.15.59:0/3915228194 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22ec0095f0 con 0x7f22fc105e70 2023-12-11T17:38:37.735 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:37.731+0000 7f22fb7fe700 1 --2- 172.21.15.59:0/3915228194 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f22fc105e70 0x7f22fc195750 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f22ec006010 tx=0x7f22ec004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:38:37.736 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:37.731+0000 7f22f8ff9700 1 -- 172.21.15.59:0/3915228194 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f22ec01d070 con 0x7f22fc105e70 2023-12-11T17:38:37.736 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:37.731+0000 7f22f8ff9700 1 -- 172.21.15.59:0/3915228194 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f22ec00bbb0 con 0x7f22fc105e70 2023-12-11T17:38:37.736 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:37.731+0000 7f2301c63700 1 -- 172.21.15.59:0/3915228194 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22fc195ef0 con 0x7f22fc105e70 2023-12-11T17:38:37.737 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:37.731+0000 7f22f8ff9700 1 -- 172.21.15.59:0/3915228194 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f22ec003810 con 0x7f22fc105e70 2023-12-11T17:38:37.737 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:37.731+0000 7f2301c63700 1 -- 172.21.15.59:0/3915228194 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22fc196370 con 0x7f22fc105e70 2023-12-11T17:38:37.737 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:37.732+0000 7f22f8ff9700 1 -- 172.21.15.59:0/3915228194 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 13) v1 ==== 45636+0+0 (secure 0 0 0) 0x7f22ec003970 con 0x7f22fc105e70 2023-12-11T17:38:37.737 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:37.732+0000 7f2301c63700 1 -- 172.21.15.59:0/3915228194 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22fc101520 con 0x7f22fc105e70 2023-12-11T17:38:37.737 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:37.732+0000 7f22f8ff9700 1 --2- 172.21.15.59:0/3915228194 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f22e8038960 0x7f22e803ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:38:37.737 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:37.732+0000 7f22f8ff9700 1 -- 172.21.15.59:0/3915228194 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f22ec04c5b0 con 0x7f22fc105e70 2023-12-11T17:38:37.737 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:37.732+0000 7f22faffd700 1 --2- 172.21.15.59:0/3915228194 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f22e8038960 0x7f22e803ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:38:37.737 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:37.733+0000 7f22faffd700 1 --2- 172.21.15.59:0/3915228194 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f22e8038960 0x7f22e803ae20 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f22e4006fd0 tx=0x7f22e4006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:38:37.740 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:37.737+0000 7f22f8ff9700 1 -- 172.21.15.59:0/3915228194 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f22ec02ad00 con 0x7f22fc105e70 2023-12-11T17:38:37.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:37 smithi059 ceph-mon[98847]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T17:38:37.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:37 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:38:37.950 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:37.947+0000 7f2301c63700 1 -- 172.21.15.59:0/3915228194 --> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f22fc10a1e0 con 0x7f22e8038960 2023-12-11T17:38:37.951 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:37.948+0000 7f22f8ff9700 1 -- 172.21.15.59:0/3915228194 <== mgr.14150 v2:172.21.15.59:6800/3627728353 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+160 (secure 0 0 0) 0x7f22fc10a1e0 con 0x7f22e8038960 2023-12-11T17:38:37.951 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-12-11T17:38:37.951 INFO:teuthology.orchestra.run.smithi059.stdout:[{"addr": "172.21.15.59", "hostname": "smithi059", "labels": [], "status": ""}, {"addr": "172.21.15.115", "hostname": "smithi115", "labels": [], "status": ""}] 2023-12-11T17:38:37.954 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:37.950+0000 7f2301c63700 1 -- 172.21.15.59:0/3915228194 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f22e8038960 msgr2=0x7f22e803ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:38:37.954 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:37.950+0000 7f2301c63700 1 --2- 172.21.15.59:0/3915228194 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f22e8038960 0x7f22e803ae20 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f22e4006fd0 tx=0x7f22e4006e40 comp rx=0 tx=0).stop 2023-12-11T17:38:37.954 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:37.950+0000 7f2301c63700 1 -- 172.21.15.59:0/3915228194 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f22fc105e70 msgr2=0x7f22fc195750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:38:37.954 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:37.950+0000 7f2301c63700 1 --2- 172.21.15.59:0/3915228194 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f22fc105e70 0x7f22fc195750 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f22ec006010 tx=0x7f22ec004ff0 comp rx=0 tx=0).stop 2023-12-11T17:38:37.954 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:37.950+0000 7f2301c63700 1 -- 172.21.15.59:0/3915228194 shutdown_connections 2023-12-11T17:38:37.954 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:37.950+0000 7f2301c63700 1 --2- 172.21.15.59:0/3915228194 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f22e8038960 0x7f22e803ae20 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:38:37.954 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:37.950+0000 7f2301c63700 1 --2- 172.21.15.59:0/3915228194 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f22fc105e70 0x7f22fc195750 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:38:37.954 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:37.950+0000 7f2301c63700 1 -- 172.21.15.59:0/3915228194 >> 172.21.15.59:0/3915228194 conn(0x7f22fc075930 msgr2=0x7f22fc0fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:38:37.954 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:37.950+0000 7f2301c63700 1 -- 172.21.15.59:0/3915228194 shutdown_connections 2023-12-11T17:38:37.954 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:37.950+0000 7f2301c63700 1 -- 172.21.15.59:0/3915228194 wait complete. 2023-12-11T17:38:38.301 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi181 2023-12-11T17:38:38.302 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-12-11T17:38:38.302 DEBUG:teuthology.orchestra.run.smithi181:> dd of=/etc/ceph/ceph.conf 2023-12-11T17:38:38.323 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-12-11T17:38:38.323 DEBUG:teuthology.orchestra.run.smithi181:> dd of=/etc/ceph/ceph.client.admin.keyring 2023-12-11T17:38:38.385 INFO:tasks.cephadm:Adding host smithi181 to orchestrator... 2023-12-11T17:38:38.385 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d0a07eaa-984b-11ee-95a3-87774f69a715 -- ceph orch host add smithi181 2023-12-11T17:38:38.987 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/mon.a/config 2023-12-11T17:38:39.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:38 smithi059 ceph-mon[98847]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-11T17:38:39.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:38 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:38:40.038 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:40.034+0000 7f872d3e7700 1 -- 172.21.15.59:0/290591379 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8728100080 msgr2=0x7f8728100460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:38:40.039 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:40.034+0000 7f872d3e7700 1 --2- 172.21.15.59:0/290591379 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8728100080 0x7f8728100460 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8710009910 tx=0x7f8710009c20 comp rx=0 tx=0).stop 2023-12-11T17:38:40.039 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:40.034+0000 7f872d3e7700 1 -- 172.21.15.59:0/290591379 shutdown_connections 2023-12-11T17:38:40.039 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:40.034+0000 7f872d3e7700 1 --2- 172.21.15.59:0/290591379 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8728100080 0x7f8728100460 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:38:40.039 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:40.034+0000 7f872d3e7700 1 -- 172.21.15.59:0/290591379 >> 172.21.15.59:0/290591379 conn(0x7f87280fb970 msgr2=0x7f87280fdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:38:40.039 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:40.034+0000 7f872d3e7700 1 -- 172.21.15.59:0/290591379 shutdown_connections 2023-12-11T17:38:40.039 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:40.034+0000 7f872d3e7700 1 -- 172.21.15.59:0/290591379 wait complete. 2023-12-11T17:38:40.039 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:40.035+0000 7f872d3e7700 1 Processor -- start 2023-12-11T17:38:40.039 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:40.035+0000 7f872d3e7700 1 -- start start 2023-12-11T17:38:40.039 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:40.035+0000 7f872d3e7700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8728100080 0x7f8728101b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:38:40.039 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:40.035+0000 7f872d3e7700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8728103990 con 0x7f8728100080 2023-12-11T17:38:40.039 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:40.035+0000 7f8726ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8728100080 0x7f8728101b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:38:40.040 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:40.035+0000 7f8726ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8728100080 0x7f8728101b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60134/0 (socket says 172.21.15.59:60134) 2023-12-11T17:38:40.040 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:40.035+0000 7f8726ffd700 1 -- 172.21.15.59:0/2196364843 learned_addr learned my addr 172.21.15.59:0/2196364843 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:38:40.041 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:40.036+0000 7f8726ffd700 1 -- 172.21.15.59:0/2196364843 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87100095f0 con 0x7f8728100080 2023-12-11T17:38:40.041 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:40.036+0000 7f8726ffd700 1 --2- 172.21.15.59:0/2196364843 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8728100080 0x7f8728101b20 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8710004f40 tx=0x7f8710005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:38:40.042 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:40.036+0000 7f871ffff700 1 -- 172.21.15.59:0/2196364843 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f871001d070 con 0x7f8728100080 2023-12-11T17:38:40.042 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:40.036+0000 7f871ffff700 1 -- 172.21.15.59:0/2196364843 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f871000bc10 con 0x7f8728100080 2023-12-11T17:38:40.042 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:40.036+0000 7f871ffff700 1 -- 172.21.15.59:0/2196364843 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f87100037d0 con 0x7f8728100080 2023-12-11T17:38:40.042 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:40.036+0000 7f872d3e7700 1 -- 172.21.15.59:0/2196364843 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8728102060 con 0x7f8728100080 2023-12-11T17:38:40.042 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:40.036+0000 7f872d3e7700 1 -- 172.21.15.59:0/2196364843 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8728102480 con 0x7f8728100080 2023-12-11T17:38:40.042 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:40.037+0000 7f871ffff700 1 -- 172.21.15.59:0/2196364843 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 13) v1 ==== 45636+0+0 (secure 0 0 0) 0x7f8710003a90 con 0x7f8728100080 2023-12-11T17:38:40.042 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:40.037+0000 7f872d3e7700 1 -- 172.21.15.59:0/2196364843 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8728101560 con 0x7f8728100080 2023-12-11T17:38:40.042 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:40.037+0000 7f871ffff700 1 --2- 172.21.15.59:0/2196364843 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f87140388c0 0x7f871403ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:38:40.042 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:40.037+0000 7f871ffff700 1 -- 172.21.15.59:0/2196364843 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f871004c6c0 con 0x7f8728100080 2023-12-11T17:38:40.042 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:40.037+0000 7f87267fc700 1 --2- 172.21.15.59:0/2196364843 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f87140388c0 0x7f871403ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:38:40.042 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:40.038+0000 7f87267fc700 1 --2- 172.21.15.59:0/2196364843 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f87140388c0 0x7f871403ad80 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8718006fd0 tx=0x7f8718006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:38:40.045 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:40.042+0000 7f871ffff700 1 -- 172.21.15.59:0/2196364843 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f8710029770 con 0x7f8728100080 2023-12-11T17:38:40.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:39 smithi059 ceph-mon[98847]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T17:38:40.253 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:40.250+0000 7f872d3e7700 1 -- 172.21.15.59:0/2196364843 --> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi181", "target": ["mon-mgr", ""]}) v1 -- 0x7f87281026c0 con 0x7f87140388c0 2023-12-11T17:38:42.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:41 smithi059 ceph-mon[98847]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T17:38:42.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:41 smithi059 ceph-mon[98847]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi181", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T17:38:42.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:41 smithi059 ceph-mon[98847]: Deploying cephadm binary to smithi181 2023-12-11T17:38:42.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:41 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:38:42.155 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:41 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:38:42.155 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:41 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:38:42.155 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:41 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:38:42.155 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:41 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi115", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:38:42.155 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:41 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:38:42.155 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:41 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:38:43.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:42 smithi059 ceph-mon[98847]: Updating smithi115:/etc/ceph/ceph.conf 2023-12-11T17:38:43.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:42 smithi059 ceph-mon[98847]: Updating smithi115:/var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/config/ceph.conf 2023-12-11T17:38:44.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:43 smithi059 ceph-mon[98847]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T17:38:45.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:45 smithi059 ceph-mon[98847]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T17:38:45.616 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:45.614+0000 7f871ffff700 1 -- 172.21.15.59:0/2196364843 <== mgr.14150 v2:172.21.15.59:6800/3627728353 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f87281026c0 con 0x7f87140388c0 2023-12-11T17:38:45.616 INFO:teuthology.orchestra.run.smithi059.stdout:Added host 'smithi181' with addr '172.21.15.181' 2023-12-11T17:38:45.618 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:45.616+0000 7f872d3e7700 1 -- 172.21.15.59:0/2196364843 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f87140388c0 msgr2=0x7f871403ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:38:45.618 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:45.616+0000 7f872d3e7700 1 --2- 172.21.15.59:0/2196364843 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f87140388c0 0x7f871403ad80 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8718006fd0 tx=0x7f8718006e40 comp rx=0 tx=0).stop 2023-12-11T17:38:45.618 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:45.616+0000 7f872d3e7700 1 -- 172.21.15.59:0/2196364843 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8728100080 msgr2=0x7f8728101b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:38:45.618 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:45.616+0000 7f872d3e7700 1 --2- 172.21.15.59:0/2196364843 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8728100080 0x7f8728101b20 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8710004f40 tx=0x7f8710005020 comp rx=0 tx=0).stop 2023-12-11T17:38:45.618 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:45.616+0000 7f872d3e7700 1 -- 172.21.15.59:0/2196364843 shutdown_connections 2023-12-11T17:38:45.618 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:45.616+0000 7f872d3e7700 1 --2- 172.21.15.59:0/2196364843 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f87140388c0 0x7f871403ad80 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:38:45.618 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:45.616+0000 7f872d3e7700 1 --2- 172.21.15.59:0/2196364843 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8728100080 0x7f8728101b20 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:38:45.619 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:45.616+0000 7f872d3e7700 1 -- 172.21.15.59:0/2196364843 >> 172.21.15.59:0/2196364843 conn(0x7f87280fb970 msgr2=0x7f87280692c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:38:45.619 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:45.616+0000 7f872d3e7700 1 -- 172.21.15.59:0/2196364843 shutdown_connections 2023-12-11T17:38:45.619 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:45.616+0000 7f872d3e7700 1 -- 172.21.15.59:0/2196364843 wait complete. 2023-12-11T17:38:45.963 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d0a07eaa-984b-11ee-95a3-87774f69a715 -- ceph orch host ls --format=json 2023-12-11T17:38:46.565 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/mon.a/config 2023-12-11T17:38:46.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:46 smithi059 ceph-mon[98847]: Updating smithi115:/etc/ceph/ceph.client.admin.keyring 2023-12-11T17:38:46.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:46 smithi059 ceph-mon[98847]: Updating smithi115:/var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/config/ceph.client.admin.keyring 2023-12-11T17:38:46.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:46 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:38:46.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:46 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:38:46.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:46 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:38:46.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:46 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:38:46.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:46 smithi059 ceph-mon[98847]: Added host smithi181 2023-12-11T17:38:46.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:46 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:38:46.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:46 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:38:47.665 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:47.660+0000 7f9732328700 1 -- 172.21.15.59:0/1138575519 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f972c105e70 msgr2=0x7f972c106250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:38:47.665 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:47.660+0000 7f9732328700 1 --2- 172.21.15.59:0/1138575519 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f972c105e70 0x7f972c106250 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f971c009910 tx=0x7f971c009c20 comp rx=0 tx=0).stop 2023-12-11T17:38:47.665 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:47.660+0000 7f9732328700 1 -- 172.21.15.59:0/1138575519 shutdown_connections 2023-12-11T17:38:47.665 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:47.660+0000 7f9732328700 1 --2- 172.21.15.59:0/1138575519 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f972c105e70 0x7f972c106250 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:38:47.665 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:47.660+0000 7f9732328700 1 -- 172.21.15.59:0/1138575519 >> 172.21.15.59:0/1138575519 conn(0x7f972c075930 msgr2=0x7f972c075d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:38:47.665 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:47.660+0000 7f9732328700 1 -- 172.21.15.59:0/1138575519 shutdown_connections 2023-12-11T17:38:47.665 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:47.660+0000 7f9732328700 1 -- 172.21.15.59:0/1138575519 wait complete. 2023-12-11T17:38:47.665 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:47.661+0000 7f9732328700 1 Processor -- start 2023-12-11T17:38:47.665 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:47.661+0000 7f9732328700 1 -- start start 2023-12-11T17:38:47.665 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:47.661+0000 7f9732328700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f972c105e70 0x7f972c19c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:38:47.666 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:47.661+0000 7f9732328700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f972c19ca60 con 0x7f972c105e70 2023-12-11T17:38:47.666 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:47.661+0000 7f972bfff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f972c105e70 0x7f972c19c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:38:47.666 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:47.661+0000 7f972bfff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f972c105e70 0x7f972c19c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60160/0 (socket says 172.21.15.59:60160) 2023-12-11T17:38:47.666 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:47.661+0000 7f972bfff700 1 -- 172.21.15.59:0/2312677274 learned_addr learned my addr 172.21.15.59:0/2312677274 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:38:47.666 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:47.662+0000 7f972bfff700 1 -- 172.21.15.59:0/2312677274 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f971c0095f0 con 0x7f972c105e70 2023-12-11T17:38:47.666 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:47.662+0000 7f972bfff700 1 --2- 172.21.15.59:0/2312677274 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f972c105e70 0x7f972c19c520 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f971c004f40 tx=0x7f971c005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:38:47.666 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:47.662+0000 7f97297fa700 1 -- 172.21.15.59:0/2312677274 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f971c01d070 con 0x7f972c105e70 2023-12-11T17:38:47.666 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:47.662+0000 7f97297fa700 1 -- 172.21.15.59:0/2312677274 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f971c00bbb0 con 0x7f972c105e70 2023-12-11T17:38:47.666 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:47.662+0000 7f9732328700 1 -- 172.21.15.59:0/2312677274 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f972c19ccc0 con 0x7f972c105e70 2023-12-11T17:38:47.666 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:47.662+0000 7f97297fa700 1 -- 172.21.15.59:0/2312677274 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f971c003810 con 0x7f972c105e70 2023-12-11T17:38:47.666 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:47.662+0000 7f9732328700 1 -- 172.21.15.59:0/2312677274 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f972c196660 con 0x7f972c105e70 2023-12-11T17:38:47.667 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:47.663+0000 7f97297fa700 1 -- 172.21.15.59:0/2312677274 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 13) v1 ==== 45636+0+0 (secure 0 0 0) 0x7f971c003a90 con 0x7f972c105e70 2023-12-11T17:38:47.667 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:47.663+0000 7f9732328700 1 -- 172.21.15.59:0/2312677274 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f972c101520 con 0x7f972c105e70 2023-12-11T17:38:47.667 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:47.663+0000 7f97297fa700 1 --2- 172.21.15.59:0/2312677274 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f97140388c0 0x7f971403ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:38:47.667 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:47.663+0000 7f97297fa700 1 -- 172.21.15.59:0/2312677274 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f971c04c590 con 0x7f972c105e70 2023-12-11T17:38:47.667 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:47.663+0000 7f972b7fe700 1 --2- 172.21.15.59:0/2312677274 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f97140388c0 0x7f971403ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:38:47.667 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:47.664+0000 7f972b7fe700 1 --2- 172.21.15.59:0/2312677274 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f97140388c0 0x7f971403ad80 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f9720006fd0 tx=0x7f9720006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:38:47.671 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:47.668+0000 7f97297fa700 1 -- 172.21.15.59:0/2312677274 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f971c029770 con 0x7f972c105e70 2023-12-11T17:38:47.881 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:47.877+0000 7f9732328700 1 -- 172.21.15.59:0/2312677274 --> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f972c197b20 con 0x7f97140388c0 2023-12-11T17:38:47.883 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:47.879+0000 7f97297fa700 1 -- 172.21.15.59:0/2312677274 <== mgr.14150 v2:172.21.15.59:6800/3627728353 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+240 (secure 0 0 0) 0x7f972c197b20 con 0x7f97140388c0 2023-12-11T17:38:47.883 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-12-11T17:38:47.883 INFO:teuthology.orchestra.run.smithi059.stdout:[{"addr": "172.21.15.59", "hostname": "smithi059", "labels": [], "status": ""}, {"addr": "172.21.15.115", "hostname": "smithi115", "labels": [], "status": ""}, {"addr": "172.21.15.181", "hostname": "smithi181", "labels": [], "status": ""}] 2023-12-11T17:38:47.884 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:47.881+0000 7f9732328700 1 -- 172.21.15.59:0/2312677274 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f97140388c0 msgr2=0x7f971403ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:38:47.884 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:47.881+0000 7f9732328700 1 --2- 172.21.15.59:0/2312677274 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f97140388c0 0x7f971403ad80 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f9720006fd0 tx=0x7f9720006e40 comp rx=0 tx=0).stop 2023-12-11T17:38:47.884 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:47.881+0000 7f9732328700 1 -- 172.21.15.59:0/2312677274 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f972c105e70 msgr2=0x7f972c19c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:38:47.884 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:47.881+0000 7f9732328700 1 --2- 172.21.15.59:0/2312677274 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f972c105e70 0x7f972c19c520 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f971c004f40 tx=0x7f971c005020 comp rx=0 tx=0).stop 2023-12-11T17:38:47.884 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:47.881+0000 7f9732328700 1 -- 172.21.15.59:0/2312677274 shutdown_connections 2023-12-11T17:38:47.884 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:47.881+0000 7f9732328700 1 --2- 172.21.15.59:0/2312677274 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f97140388c0 0x7f971403ad80 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:38:47.885 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:47.881+0000 7f9732328700 1 --2- 172.21.15.59:0/2312677274 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f972c105e70 0x7f972c19c520 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:38:47.885 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:47.881+0000 7f9732328700 1 -- 172.21.15.59:0/2312677274 >> 172.21.15.59:0/2312677274 conn(0x7f972c075930 msgr2=0x7f972c18f750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:38:47.885 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:47.881+0000 7f9732328700 1 -- 172.21.15.59:0/2312677274 shutdown_connections 2023-12-11T17:38:47.885 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:47.881+0000 7f9732328700 1 -- 172.21.15.59:0/2312677274 wait complete. 2023-12-11T17:38:48.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:47 smithi059 ceph-mon[98847]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T17:38:48.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:47 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:38:48.220 INFO:tasks.cephadm:Setting crush tunables to default 2023-12-11T17:38:48.220 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d0a07eaa-984b-11ee-95a3-87774f69a715 -- ceph osd crush tunables default 2023-12-11T17:38:48.822 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/mon.a/config 2023-12-11T17:38:49.103 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:48 smithi059 ceph-mon[98847]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-11T17:38:49.103 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:48 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:38:50.050 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:50.046+0000 7f6816a4c700 1 -- 172.21.15.59:0/3056751640 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6810100050 msgr2=0x7f6810100430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:38:50.050 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:50.046+0000 7f6816a4c700 1 --2- 172.21.15.59:0/3056751640 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6810100050 0x7f6810100430 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f6800009910 tx=0x7f6800009c20 comp rx=0 tx=0).stop 2023-12-11T17:38:50.050 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:50.046+0000 7f6816a4c700 1 -- 172.21.15.59:0/3056751640 shutdown_connections 2023-12-11T17:38:50.051 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:50.046+0000 7f6816a4c700 1 --2- 172.21.15.59:0/3056751640 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6810100050 0x7f6810100430 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:38:50.051 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:50.046+0000 7f6816a4c700 1 -- 172.21.15.59:0/3056751640 >> 172.21.15.59:0/3056751640 conn(0x7f68100fb900 msgr2=0x7f68100fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:38:50.051 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:50.046+0000 7f6816a4c700 1 -- 172.21.15.59:0/3056751640 shutdown_connections 2023-12-11T17:38:50.051 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:50.046+0000 7f6816a4c700 1 -- 172.21.15.59:0/3056751640 wait complete. 2023-12-11T17:38:50.051 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:50.046+0000 7f6816a4c700 1 Processor -- start 2023-12-11T17:38:50.051 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:50.046+0000 7f6816a4c700 1 -- start start 2023-12-11T17:38:50.051 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:50.047+0000 7f6816a4c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6810100050 0x7f6810195480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:38:50.051 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:50.047+0000 7f6816a4c700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6810069de0 con 0x7f6810100050 2023-12-11T17:38:50.051 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:50.047+0000 7f680ffff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6810100050 0x7f6810195480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:38:50.051 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:50.047+0000 7f680ffff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6810100050 0x7f6810195480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60166/0 (socket says 172.21.15.59:60166) 2023-12-11T17:38:50.051 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:50.047+0000 7f680ffff700 1 -- 172.21.15.59:0/1648308900 learned_addr learned my addr 172.21.15.59:0/1648308900 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:38:50.052 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:50.047+0000 7f680ffff700 1 -- 172.21.15.59:0/1648308900 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68000095f0 con 0x7f6810100050 2023-12-11T17:38:50.052 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:50.047+0000 7f680ffff700 1 --2- 172.21.15.59:0/1648308900 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6810100050 0x7f6810195480 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6800004f40 tx=0x7f6800005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:38:50.052 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:50.047+0000 7f680d7fa700 1 -- 172.21.15.59:0/1648308900 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f680001d070 con 0x7f6810100050 2023-12-11T17:38:50.052 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:50.048+0000 7f6816a4c700 1 -- 172.21.15.59:0/1648308900 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68101959c0 con 0x7f6810100050 2023-12-11T17:38:50.052 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:50.048+0000 7f680d7fa700 1 -- 172.21.15.59:0/1648308900 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f680000bc10 con 0x7f6810100050 2023-12-11T17:38:50.052 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:50.048+0000 7f680d7fa700 1 -- 172.21.15.59:0/1648308900 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f68000037d0 con 0x7f6810100050 2023-12-11T17:38:50.052 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:50.048+0000 7f6816a4c700 1 -- 172.21.15.59:0/1648308900 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6810195de0 con 0x7f6810100050 2023-12-11T17:38:50.053 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:50.048+0000 7f680d7fa700 1 -- 172.21.15.59:0/1648308900 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 13) v1 ==== 45636+0+0 (secure 0 0 0) 0x7f680000b6f0 con 0x7f6810100050 2023-12-11T17:38:50.053 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:50.049+0000 7f680d7fa700 1 --2- 172.21.15.59:0/1648308900 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f67f80388c0 0x7f67f803ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:38:50.053 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:50.049+0000 7f6816a4c700 1 -- 172.21.15.59:0/1648308900 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6810107ff0 con 0x7f6810100050 2023-12-11T17:38:50.053 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:50.049+0000 7f680d7fa700 1 -- 172.21.15.59:0/1648308900 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f680004c540 con 0x7f6810100050 2023-12-11T17:38:50.053 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:50.049+0000 7f680f7fe700 1 --2- 172.21.15.59:0/1648308900 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f67f80388c0 0x7f67f803ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:38:50.053 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:50.049+0000 7f680f7fe700 1 --2- 172.21.15.59:0/1648308900 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f67f80388c0 0x7f67f803ad80 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f6804006fd0 tx=0x7f6804006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:38:50.056 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:50.054+0000 7f680d7fa700 1 -- 172.21.15.59:0/1648308900 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f6800029360 con 0x7f6810100050 2023-12-11T17:38:50.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:49 smithi059 ceph-mon[98847]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T17:38:50.262 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:50.259+0000 7f6816a4c700 1 -- 172.21.15.59:0/1648308900 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f6810069640 con 0x7f6810100050 2023-12-11T17:38:50.855 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:50.852+0000 7f680d7fa700 1 -- 172.21.15.59:0/1648308900 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7f6800026020 con 0x7f6810100050 2023-12-11T17:38:50.858 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:50.855+0000 7f6816a4c700 1 -- 172.21.15.59:0/1648308900 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f67f80388c0 msgr2=0x7f67f803ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:38:50.858 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:50.855+0000 7f6816a4c700 1 --2- 172.21.15.59:0/1648308900 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f67f80388c0 0x7f67f803ad80 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f6804006fd0 tx=0x7f6804006e40 comp rx=0 tx=0).stop 2023-12-11T17:38:50.858 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:50.855+0000 7f6816a4c700 1 -- 172.21.15.59:0/1648308900 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6810100050 msgr2=0x7f6810195480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:38:50.858 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:50.855+0000 7f6816a4c700 1 --2- 172.21.15.59:0/1648308900 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6810100050 0x7f6810195480 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6800004f40 tx=0x7f6800005020 comp rx=0 tx=0).stop 2023-12-11T17:38:50.858 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:50.856+0000 7f6816a4c700 1 -- 172.21.15.59:0/1648308900 shutdown_connections 2023-12-11T17:38:50.859 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:50.856+0000 7f6816a4c700 1 --2- 172.21.15.59:0/1648308900 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f67f80388c0 0x7f67f803ad80 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:38:50.859 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:50.856+0000 7f6816a4c700 1 --2- 172.21.15.59:0/1648308900 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6810100050 0x7f6810195480 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:38:50.859 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:50.856+0000 7f6816a4c700 1 -- 172.21.15.59:0/1648308900 >> 172.21.15.59:0/1648308900 conn(0x7f68100fb900 msgr2=0x7f6810101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:38:50.859 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:50.856+0000 7f6816a4c700 1 -- 172.21.15.59:0/1648308900 shutdown_connections 2023-12-11T17:38:50.859 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:38:50.856+0000 7f6816a4c700 1 -- 172.21.15.59:0/1648308900 wait complete. 2023-12-11T17:38:50.860 INFO:teuthology.orchestra.run.smithi059.stderr:adjusted tunables profile to default 2023-12-11T17:38:51.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:50 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/1648308900' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-12-11T17:38:51.201 INFO:tasks.cephadm:Adding mon.a on smithi059 2023-12-11T17:38:51.201 INFO:tasks.cephadm:Adding mon.b on smithi115 2023-12-11T17:38:51.201 INFO:tasks.cephadm:Adding mon.c on smithi181 2023-12-11T17:38:51.201 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d0a07eaa-984b-11ee-95a3-87774f69a715 -- ceph orch apply mon '3;smithi059:172.21.15.59=a;smithi115:172.21.15.115=b;smithi181:172.21.15.181=c' 2023-12-11T17:38:51.828 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/config/ceph.conf 2023-12-11T17:38:52.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:51 smithi059 ceph-mon[98847]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T17:38:52.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:51 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/1648308900' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-12-11T17:38:52.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:51 smithi059 ceph-mon[98847]: osdmap e4: 0 total, 0 up, 0 in 2023-12-11T17:38:52.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:51 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:38:52.155 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:51 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:38:52.155 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:51 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:38:52.155 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:51 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:38:52.155 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:51 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi181", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:38:52.155 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:51 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:38:52.155 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:51 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:38:53.015 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:38:53.012+0000 7f713ffa8700 1 -- 172.21.15.181:0/3586356454 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7138105e70 msgr2=0x7f7138106250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:38:53.015 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:38:53.012+0000 7f713ffa8700 1 --2- 172.21.15.181:0/3586356454 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7138105e70 0x7f7138106250 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7128009910 tx=0x7f7128009c20 comp rx=0 tx=0).stop 2023-12-11T17:38:53.015 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:38:53.012+0000 7f713ffa8700 1 -- 172.21.15.181:0/3586356454 shutdown_connections 2023-12-11T17:38:53.015 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:38:53.012+0000 7f713ffa8700 1 --2- 172.21.15.181:0/3586356454 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7138105e70 0x7f7138106250 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:38:53.016 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:38:53.012+0000 7f713ffa8700 1 -- 172.21.15.181:0/3586356454 >> 172.21.15.181:0/3586356454 conn(0x7f7138075930 msgr2=0x7f7138075d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:38:53.016 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:38:53.013+0000 7f713ffa8700 1 -- 172.21.15.181:0/3586356454 shutdown_connections 2023-12-11T17:38:53.016 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:38:53.013+0000 7f713ffa8700 1 -- 172.21.15.181:0/3586356454 wait complete. 2023-12-11T17:38:53.017 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:38:53.013+0000 7f713ffa8700 1 Processor -- start 2023-12-11T17:38:53.017 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:38:53.013+0000 7f713ffa8700 1 -- start start 2023-12-11T17:38:53.017 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:38:53.013+0000 7f713ffa8700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7138105e70 0x7f71381954b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:38:53.017 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:38:53.013+0000 7f713ffa8700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71380694e0 con 0x7f7138105e70 2023-12-11T17:38:53.017 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:38:53.014+0000 7f713dd44700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7138105e70 0x7f71381954b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:38:53.017 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:38:53.014+0000 7f713dd44700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7138105e70 0x7f71381954b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.181:37210/0 (socket says 172.21.15.181:37210) 2023-12-11T17:38:53.017 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:38:53.014+0000 7f713dd44700 1 -- 172.21.15.181:0/3429159938 learned_addr learned my addr 172.21.15.181:0/3429159938 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-12-11T17:38:53.017 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:38:53.014+0000 7f713dd44700 1 -- 172.21.15.181:0/3429159938 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71280095f0 con 0x7f7138105e70 2023-12-11T17:38:53.018 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:38:53.014+0000 7f713dd44700 1 --2- 172.21.15.181:0/3429159938 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7138105e70 0x7f71381954b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7128004f40 tx=0x7f7128005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:38:53.018 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:38:53.014+0000 7f712effd700 1 -- 172.21.15.181:0/3429159938 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f712801d070 con 0x7f7138105e70 2023-12-11T17:38:53.018 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:38:53.014+0000 7f712effd700 1 -- 172.21.15.181:0/3429159938 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f712800bbb0 con 0x7f7138105e70 2023-12-11T17:38:53.018 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:38:53.014+0000 7f713ffa8700 1 -- 172.21.15.181:0/3429159938 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71381959f0 con 0x7f7138105e70 2023-12-11T17:38:53.018 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:38:53.014+0000 7f712effd700 1 -- 172.21.15.181:0/3429159938 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7128003810 con 0x7f7138105e70 2023-12-11T17:38:53.018 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:38:53.014+0000 7f713ffa8700 1 -- 172.21.15.181:0/3429159938 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7138195e70 con 0x7f7138105e70 2023-12-11T17:38:53.019 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:38:53.015+0000 7f712effd700 1 -- 172.21.15.181:0/3429159938 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 13) v1 ==== 45636+0+0 (secure 0 0 0) 0x7f712800bd20 con 0x7f7138105e70 2023-12-11T17:38:53.019 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:38:53.015+0000 7f713ffa8700 1 -- 172.21.15.181:0/3429159938 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7138101520 con 0x7f7138105e70 2023-12-11T17:38:53.019 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:38:53.015+0000 7f712effd700 1 --2- 172.21.15.181:0/3429159938 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f71240388c0 0x7f712403ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:38:53.019 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:38:53.015+0000 7f712effd700 1 -- 172.21.15.181:0/3429159938 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f712804c510 con 0x7f7138105e70 2023-12-11T17:38:53.019 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:38:53.015+0000 7f713d543700 1 --2- 172.21.15.181:0/3429159938 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f71240388c0 0x7f712403ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:38:53.019 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:38:53.016+0000 7f713d543700 1 --2- 172.21.15.181:0/3429159938 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f71240388c0 0x7f712403ad80 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f7134006fd0 tx=0x7f7134006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:38:53.023 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:38:53.021+0000 7f712effd700 1 -- 172.21.15.181:0/3429159938 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f712802ab60 con 0x7f7138105e70 2023-12-11T17:38:53.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:52 smithi059 ceph-mon[98847]: Updating smithi181:/etc/ceph/ceph.conf 2023-12-11T17:38:53.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:52 smithi059 ceph-mon[98847]: Updating smithi181:/var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/config/ceph.conf 2023-12-11T17:38:53.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:52 smithi059 ceph-mon[98847]: Updating smithi181:/etc/ceph/ceph.client.admin.keyring 2023-12-11T17:38:53.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:52 smithi059 ceph-mon[98847]: Updating smithi181:/var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/config/ceph.client.admin.keyring 2023-12-11T17:38:53.155 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:52 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:38:53.155 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:52 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:38:53.155 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:52 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:38:53.222 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:38:53.220+0000 7f713ffa8700 1 -- 172.21.15.181:0/3429159938 --> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi059:172.21.15.59=a;smithi115:172.21.15.115=b;smithi181:172.21.15.181=c", "target": ["mon-mgr", ""]}) v1 -- 0x7f7138068da0 con 0x7f71240388c0 2023-12-11T17:38:53.229 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:38:53.227+0000 7f712effd700 1 -- 172.21.15.181:0/3429159938 <== mgr.14150 v2:172.21.15.59:6800/3627728353 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f7138068da0 con 0x7f71240388c0 2023-12-11T17:38:53.229 INFO:teuthology.orchestra.run.smithi181.stdout:Scheduled mon update... 2023-12-11T17:38:53.231 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:38:53.229+0000 7f713ffa8700 1 -- 172.21.15.181:0/3429159938 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f71240388c0 msgr2=0x7f712403ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:38:53.232 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:38:53.229+0000 7f713ffa8700 1 --2- 172.21.15.181:0/3429159938 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f71240388c0 0x7f712403ad80 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f7134006fd0 tx=0x7f7134006e40 comp rx=0 tx=0).stop 2023-12-11T17:38:53.232 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:38:53.229+0000 7f713ffa8700 1 -- 172.21.15.181:0/3429159938 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7138105e70 msgr2=0x7f71381954b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:38:53.232 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:38:53.229+0000 7f713ffa8700 1 --2- 172.21.15.181:0/3429159938 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7138105e70 0x7f71381954b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7128004f40 tx=0x7f7128005020 comp rx=0 tx=0).stop 2023-12-11T17:38:53.232 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:38:53.229+0000 7f713ffa8700 1 -- 172.21.15.181:0/3429159938 shutdown_connections 2023-12-11T17:38:53.232 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:38:53.229+0000 7f713ffa8700 1 --2- 172.21.15.181:0/3429159938 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f71240388c0 0x7f712403ad80 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:38:53.232 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:38:53.229+0000 7f713ffa8700 1 --2- 172.21.15.181:0/3429159938 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7138105e70 0x7f71381954b0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:38:53.232 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:38:53.229+0000 7f713ffa8700 1 -- 172.21.15.181:0/3429159938 >> 172.21.15.181:0/3429159938 conn(0x7f7138075930 msgr2=0x7f71380fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:38:53.232 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:38:53.229+0000 7f713ffa8700 1 -- 172.21.15.181:0/3429159938 shutdown_connections 2023-12-11T17:38:53.232 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:38:53.229+0000 7f713ffa8700 1 -- 172.21.15.181:0/3429159938 wait complete. 2023-12-11T17:38:53.605 DEBUG:teuthology.orchestra.run.smithi115:mon.b> sudo journalctl -f -n 0 -u ceph-d0a07eaa-984b-11ee-95a3-87774f69a715@mon.b.service 2023-12-11T17:38:53.608 DEBUG:teuthology.orchestra.run.smithi181:mon.c> sudo journalctl -f -n 0 -u ceph-d0a07eaa-984b-11ee-95a3-87774f69a715@mon.c.service 2023-12-11T17:38:53.610 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-12-11T17:38:53.610 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d0a07eaa-984b-11ee-95a3-87774f69a715 -- ceph mon dump -f json 2023-12-11T17:38:53.639 INFO:journalctl@ceph.mon.c.smithi181.stdout:-- Logs begin at Mon 2023-12-11 17:22:43 UTC. -- 2023-12-11T17:38:53.644 INFO:journalctl@ceph.mon.b.smithi115.stdout:-- Logs begin at Mon 2023-12-11 17:22:43 UTC. -- 2023-12-11T17:38:54.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:53 smithi059 ceph-mon[98847]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T17:38:54.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:53 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:38:54.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:53 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:38:54.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:53 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:38:54.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:53 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:38:54.155 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:53 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:38:54.155 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:53 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-12-11T17:38:54.155 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:53 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:38:54.333 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/config/ceph.conf 2023-12-11T17:38:55.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:54 smithi059 ceph-mon[98847]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi059:172.21.15.59=a;smithi115:172.21.15.115=b;smithi181:172.21.15.181=c", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T17:38:55.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:54 smithi059 ceph-mon[98847]: Saving service mon spec with placement smithi059:172.21.15.59=a;smithi115:172.21.15.115=b;smithi181:172.21.15.181=c;count:3 2023-12-11T17:38:55.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:54 smithi059 ceph-mon[98847]: Deploying daemon mon.c on smithi181 2023-12-11T17:38:55.991 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:38:55.987+0000 7f0b8df22700 1 -- 172.21.15.181:0/3014138044 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b88105e70 msgr2=0x7f0b88106250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:38:55.991 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:38:55.987+0000 7f0b8df22700 1 --2- 172.21.15.181:0/3014138044 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b88105e70 0x7f0b88106250 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0b78009910 tx=0x7f0b78009c20 comp rx=0 tx=0).stop 2023-12-11T17:38:55.991 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:38:55.987+0000 7f0b8df22700 1 -- 172.21.15.181:0/3014138044 shutdown_connections 2023-12-11T17:38:55.991 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:38:55.987+0000 7f0b8df22700 1 --2- 172.21.15.181:0/3014138044 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b88105e70 0x7f0b88106250 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:38:55.991 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:38:55.987+0000 7f0b8df22700 1 -- 172.21.15.181:0/3014138044 >> 172.21.15.181:0/3014138044 conn(0x7f0b88075930 msgr2=0x7f0b88075d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:38:55.991 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:38:55.987+0000 7f0b8df22700 1 -- 172.21.15.181:0/3014138044 shutdown_connections 2023-12-11T17:38:55.991 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:38:55.987+0000 7f0b8df22700 1 -- 172.21.15.181:0/3014138044 wait complete. 2023-12-11T17:38:55.991 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:38:55.987+0000 7f0b8df22700 1 Processor -- start 2023-12-11T17:38:55.991 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:38:55.988+0000 7f0b8df22700 1 -- start start 2023-12-11T17:38:55.992 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:38:55.988+0000 7f0b8df22700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b88105e70 0x7f0b881976c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:38:55.992 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:38:55.988+0000 7f0b8df22700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b88069690 con 0x7f0b88105e70 2023-12-11T17:38:55.992 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:38:55.988+0000 7f0b877fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b88105e70 0x7f0b881976c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:38:55.992 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:38:55.988+0000 7f0b877fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b88105e70 0x7f0b881976c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.181:37216/0 (socket says 172.21.15.181:37216) 2023-12-11T17:38:55.992 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:38:55.988+0000 7f0b877fe700 1 -- 172.21.15.181:0/3313239583 learned_addr learned my addr 172.21.15.181:0/3313239583 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-12-11T17:38:55.992 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:38:55.988+0000 7f0b877fe700 1 -- 172.21.15.181:0/3313239583 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b780095f0 con 0x7f0b88105e70 2023-12-11T17:38:55.993 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:38:55.989+0000 7f0b877fe700 1 --2- 172.21.15.181:0/3313239583 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b88105e70 0x7f0b881976c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0b78004f40 tx=0x7f0b78005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:38:55.993 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:38:55.989+0000 7f0b84ff9700 1 -- 172.21.15.181:0/3313239583 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0b7801d070 con 0x7f0b88105e70 2023-12-11T17:38:55.994 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:38:55.989+0000 7f0b8df22700 1 -- 172.21.15.181:0/3313239583 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b88197cc0 con 0x7f0b88105e70 2023-12-11T17:38:55.994 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:38:55.989+0000 7f0b84ff9700 1 -- 172.21.15.181:0/3313239583 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0b7800bc10 con 0x7f0b88105e70 2023-12-11T17:38:55.994 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:38:55.989+0000 7f0b84ff9700 1 -- 172.21.15.181:0/3313239583 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0b780037d0 con 0x7f0b88105e70 2023-12-11T17:38:55.994 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:38:55.989+0000 7f0b8df22700 1 -- 172.21.15.181:0/3313239583 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b88198140 con 0x7f0b88105e70 2023-12-11T17:38:55.994 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:38:55.990+0000 7f0b84ff9700 1 -- 172.21.15.181:0/3313239583 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 13) v1 ==== 45636+0+0 (secure 0 0 0) 0x7f0b7800b6f0 con 0x7f0b88105e70 2023-12-11T17:38:55.994 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:38:55.990+0000 7f0b8df22700 1 -- 172.21.15.181:0/3313239583 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b88101520 con 0x7f0b88105e70 2023-12-11T17:38:55.994 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:38:55.990+0000 7f0b84ff9700 1 --2- 172.21.15.181:0/3313239583 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f0b700388c0 0x7f0b7003ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:38:55.994 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:38:55.990+0000 7f0b84ff9700 1 -- 172.21.15.181:0/3313239583 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f0b7804c5e0 con 0x7f0b88105e70 2023-12-11T17:38:55.994 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:38:55.990+0000 7f0b86ffd700 1 --2- 172.21.15.181:0/3313239583 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f0b700388c0 0x7f0b7003ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:38:55.994 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:38:55.991+0000 7f0b86ffd700 1 --2- 172.21.15.181:0/3313239583 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f0b700388c0 0x7f0b7003ad80 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c006fd0 tx=0x7f0b7c006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:38:55.998 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:38:55.996+0000 7f0b84ff9700 1 -- 172.21.15.181:0/3313239583 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f0b78029540 con 0x7f0b88105e70 2023-12-11T17:38:56.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:55 smithi059 ceph-mon[98847]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T17:38:56.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:38:56.289+0000 7f0b8df22700 1 -- 172.21.15.181:0/3313239583 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f0b88198390 con 0x7f0b88105e70 2023-12-11T17:38:56.293 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:38:56.290+0000 7f0b84ff9700 1 -- 172.21.15.181:0/3313239583 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+736 (secure 0 0 0) 0x7f0b78026070 con 0x7f0b88105e70 2023-12-11T17:38:56.293 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-12-11T17:38:56.294 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":1,"fsid":"d0a07eaa-984b-11ee-95a3-87774f69a715","modified":"2023-12-11T17:37:10.325934Z","created":"2023-12-11T17:37:10.325934Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:3300","nonce":0},{"type":"v1","addr":"172.21.15.59:6789","nonce":0}]},"addr":"172.21.15.59:6789/0","public_addr":"172.21.15.59:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-11T17:38:56.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:38:56.292+0000 7f0b8df22700 1 -- 172.21.15.181:0/3313239583 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f0b700388c0 msgr2=0x7f0b7003ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:38:56.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:38:56.292+0000 7f0b8df22700 1 --2- 172.21.15.181:0/3313239583 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f0b700388c0 0x7f0b7003ad80 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c006fd0 tx=0x7f0b7c006e40 comp rx=0 tx=0).stop 2023-12-11T17:38:56.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:38:56.292+0000 7f0b8df22700 1 -- 172.21.15.181:0/3313239583 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b88105e70 msgr2=0x7f0b881976c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:38:56.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:38:56.292+0000 7f0b8df22700 1 --2- 172.21.15.181:0/3313239583 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b88105e70 0x7f0b881976c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0b78004f40 tx=0x7f0b78005020 comp rx=0 tx=0).stop 2023-12-11T17:38:56.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:38:56.292+0000 7f0b8df22700 1 -- 172.21.15.181:0/3313239583 shutdown_connections 2023-12-11T17:38:56.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:38:56.292+0000 7f0b8df22700 1 --2- 172.21.15.181:0/3313239583 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f0b700388c0 0x7f0b7003ad80 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:38:56.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:38:56.292+0000 7f0b8df22700 1 --2- 172.21.15.181:0/3313239583 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b88105e70 0x7f0b881976c0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:38:56.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:38:56.292+0000 7f0b8df22700 1 -- 172.21.15.181:0/3313239583 >> 172.21.15.181:0/3313239583 conn(0x7f0b88075930 msgr2=0x7f0b880fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:38:56.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:38:56.292+0000 7f0b8df22700 1 -- 172.21.15.181:0/3313239583 shutdown_connections 2023-12-11T17:38:56.295 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:38:56.292+0000 7f0b8df22700 1 -- 172.21.15.181:0/3313239583 wait complete. 2023-12-11T17:38:56.296 INFO:teuthology.orchestra.run.smithi181.stderr:dumped monmap epoch 1 2023-12-11T17:38:57.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:38:56 smithi181 ceph-mon[103627]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2023-12-11T17:38:57.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:38:56 smithi181 systemd[1]: Started Ceph mon.c for d0a07eaa-984b-11ee-95a3-87774f69a715. 2023-12-11T17:38:57.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:38:56 smithi059 ceph-mon[98847]: from='client.? 172.21.15.181:0/3313239583' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-11T17:38:57.682 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-12-11T17:38:57.683 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d0a07eaa-984b-11ee-95a3-87774f69a715 -- ceph mon dump -f json 2023-12-11T17:38:58.282 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/mon.c/config 2023-12-11T17:39:02.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:02 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-11T17:39:02.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:02 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-11T17:39:02.396 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:02 smithi181 ceph-mon[103627]: mon.a calling monitor election 2023-12-11T17:39:02.396 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:02 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-11T17:39:02.396 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:02 smithi181 ceph-mon[103627]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T17:39:02.396 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:02 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-11T17:39:02.396 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:02 smithi181 ceph-mon[103627]: mon.c calling monitor election 2023-12-11T17:39:02.396 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:02 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-11T17:39:02.396 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:02 smithi181 ceph-mon[103627]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T17:39:02.396 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:02 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-11T17:39:02.396 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:02 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-11T17:39:02.396 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:02 smithi181 ceph-mon[103627]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-12-11T17:39:02.396 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:02 smithi181 ceph-mon[103627]: monmap e2: 2 mons at {a=[v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0],c=[v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0]} removed_ranks: {} disallowed_leaders: {} 2023-12-11T17:39:02.397 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:02 smithi181 ceph-mon[103627]: fsmap 2023-12-11T17:39:02.397 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:02 smithi181 ceph-mon[103627]: osdmap e4: 0 total, 0 up, 0 in 2023-12-11T17:39:02.397 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:02 smithi181 ceph-mon[103627]: mgrmap e13: x(active, since 47s) 2023-12-11T17:39:02.397 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:02 smithi181 ceph-mon[103627]: overall HEALTH_OK 2023-12-11T17:39:02.397 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:02 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:02.397 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:02 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:02.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:02 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-11T17:39:02.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:02 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-11T17:39:02.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:02 smithi059 ceph-mon[98847]: mon.a calling monitor election 2023-12-11T17:39:02.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:02 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-11T17:39:02.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:02 smithi059 ceph-mon[98847]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T17:39:02.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:02 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-11T17:39:02.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:02 smithi059 ceph-mon[98847]: mon.c calling monitor election 2023-12-11T17:39:02.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:02 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-11T17:39:02.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:02 smithi059 ceph-mon[98847]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T17:39:02.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:02 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-11T17:39:02.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:02 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-11T17:39:02.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:02 smithi059 ceph-mon[98847]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-12-11T17:39:02.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:02 smithi059 ceph-mon[98847]: monmap e2: 2 mons at {a=[v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0],c=[v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0]} removed_ranks: {} disallowed_leaders: {} 2023-12-11T17:39:02.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:02 smithi059 ceph-mon[98847]: fsmap 2023-12-11T17:39:02.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:02 smithi059 ceph-mon[98847]: osdmap e4: 0 total, 0 up, 0 in 2023-12-11T17:39:02.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:02 smithi059 ceph-mon[98847]: mgrmap e13: x(active, since 47s) 2023-12-11T17:39:02.406 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:02 smithi059 ceph-mon[98847]: overall HEALTH_OK 2023-12-11T17:39:02.406 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:02 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:02.406 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:02 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:02.460 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:02.457+0000 7f8e6ee3d700 1 -- 172.21.15.181:0/1029499770 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8e580035c0 msgr2=0x7f8e58005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:39:02.460 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:02.457+0000 7f8e6ee3d700 1 --2- 172.21.15.181:0/1029499770 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8e580035c0 0x7f8e58005a50 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8e680ffe20 tx=0x7f8e5c00feb0 comp rx=0 tx=0).stop 2023-12-11T17:39:02.460 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:02.457+0000 7f8e6ee3d700 1 -- 172.21.15.181:0/1029499770 shutdown_connections 2023-12-11T17:39:02.460 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:02.457+0000 7f8e6ee3d700 1 --2- 172.21.15.181:0/1029499770 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8e580035c0 0x7f8e58005a50 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:39:02.460 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:02.457+0000 7f8e6ee3d700 1 -- 172.21.15.181:0/1029499770 >> 172.21.15.181:0/1029499770 conn(0x7f8e680fb9e0 msgr2=0x7f8e680fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:39:02.460 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:02.457+0000 7f8e6ee3d700 1 -- 172.21.15.181:0/1029499770 shutdown_connections 2023-12-11T17:39:02.460 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:02.457+0000 7f8e6ee3d700 1 -- 172.21.15.181:0/1029499770 wait complete. 2023-12-11T17:39:02.461 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:02.458+0000 7f8e6ee3d700 1 Processor -- start 2023-12-11T17:39:02.461 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:02.458+0000 7f8e6ee3d700 1 -- start start 2023-12-11T17:39:02.461 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:02.459+0000 7f8e6ee3d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e580035c0 0x7f8e6819c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:39:02.462 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:02.459+0000 7f8e6ee3d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8e6819cb50 0x7f8e68196750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:39:02.462 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:02.459+0000 7f8e6ee3d700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e6819cfd0 con 0x7f8e6819cb50 2023-12-11T17:39:02.462 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:02.459+0000 7f8e6ee3d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e6819d140 con 0x7f8e580035c0 2023-12-11T17:39:02.462 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:02.459+0000 7f8e6cbd9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e580035c0 0x7f8e6819c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:39:02.462 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:02.459+0000 7f8e6cbd9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e580035c0 0x7f8e6819c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34202/0 (socket says 172.21.15.181:34202) 2023-12-11T17:39:02.462 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:02.459+0000 7f8e6cbd9700 1 -- 172.21.15.181:0/1931343638 learned_addr learned my addr 172.21.15.181:0/1931343638 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-12-11T17:39:02.462 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:02.459+0000 7f8e6cbd9700 1 -- 172.21.15.181:0/1931343638 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e580035c0 msgr2=0x7f8e6819c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2023-12-11T17:39:02.462 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:02.459+0000 7f8e6cbd9700 1 -- 172.21.15.181:0/1931343638 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e580035c0 msgr2=0x7f8e6819c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-12-11T17:39:02.463 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:02.459+0000 7f8e6cbd9700 1 --2- 172.21.15.181:0/1931343638 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e580035c0 0x7f8e6819c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-12-11T17:39:02.463 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:02.459+0000 7f8e67fff700 1 --2- 172.21.15.181:0/1931343638 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8e6819cb50 0x7f8e68196750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:39:02.463 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:02.459+0000 7f8e6cbd9700 1 --2- 172.21.15.181:0/1931343638 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e580035c0 0x7f8e6819c610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-11T17:39:02.464 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:02.460+0000 7f8e67fff700 1 -- 172.21.15.181:0/1931343638 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e580035c0 msgr2=0x7f8e6819c610 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-11T17:39:02.464 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:02.460+0000 7f8e67fff700 1 --2- 172.21.15.181:0/1931343638 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e580035c0 0x7f8e6819c610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:39:02.464 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:02.460+0000 7f8e67fff700 1 -- 172.21.15.181:0/1931343638 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e5c00f900 con 0x7f8e6819cb50 2023-12-11T17:39:02.464 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:02.460+0000 7f8e67fff700 1 --2- 172.21.15.181:0/1931343638 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8e6819cb50 0x7f8e68196750 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8e5400b4d0 tx=0x7f8e5400b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:39:02.464 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:02.460+0000 7f8e65ffb700 1 -- 172.21.15.181:0/1931343638 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f8e54011630 con 0x7f8e6819cb50 2023-12-11T17:39:02.464 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:02.460+0000 7f8e65ffb700 1 -- 172.21.15.181:0/1931343638 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8e54011c50 con 0x7f8e6819cb50 2023-12-11T17:39:02.464 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:02.460+0000 7f8e65ffb700 1 -- 172.21.15.181:0/1931343638 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f8e54010d90 con 0x7f8e6819cb50 2023-12-11T17:39:02.464 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:02.460+0000 7f8e6ee3d700 1 -- 172.21.15.181:0/1931343638 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e68196db0 con 0x7f8e6819cb50 2023-12-11T17:39:02.464 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:02.460+0000 7f8e6ee3d700 1 -- 172.21.15.181:0/1931343638 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e681972b0 con 0x7f8e6819cb50 2023-12-11T17:39:02.464 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:02.461+0000 7f8e65ffb700 1 -- 172.21.15.181:0/1931343638 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 13) v1 ==== 45636+0+0 (secure 0 0 0) 0x7f8e54011790 con 0x7f8e6819cb50 2023-12-11T17:39:02.465 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:02.461+0000 7f8e65ffb700 1 --2- 172.21.15.181:0/1931343638 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f8e4c0386a0 0x7f8e4c03ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:39:02.465 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:02.461+0000 7f8e65ffb700 1 -- 172.21.15.181:0/1931343638 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f8e5404ae80 con 0x7f8e6819cb50 2023-12-11T17:39:02.465 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:02.462+0000 7f8e6cbd9700 1 --2- 172.21.15.181:0/1931343638 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f8e4c0386a0 0x7f8e4c03ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:39:02.465 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:02.462+0000 7f8e6ee3d700 1 -- 172.21.15.181:0/1931343638 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e68197580 con 0x7f8e6819cb50 2023-12-11T17:39:02.465 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:02.462+0000 7f8e6cbd9700 1 --2- 172.21.15.181:0/1931343638 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f8e4c0386a0 0x7f8e4c03ab60 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8e5c011510 tx=0x7f8e5c005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:39:02.470 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:02.467+0000 7f8e65ffb700 1 -- 172.21.15.181:0/1931343638 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f8e54017730 con 0x7f8e6819cb50 2023-12-11T17:39:02.733 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:02.731+0000 7f8e6ee3d700 1 -- 172.21.15.181:0/1931343638 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f8e68197c10 con 0x7f8e6819cb50 2023-12-11T17:39:02.734 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:02.731+0000 7f8e65ffb700 1 -- 172.21.15.181:0/1931343638 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1006 (secure 0 0 0) 0x7f8e5403d020 con 0x7f8e6819cb50 2023-12-11T17:39:02.734 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-12-11T17:39:02.734 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":2,"fsid":"d0a07eaa-984b-11ee-95a3-87774f69a715","modified":"2023-12-11T17:38:56.979524Z","created":"2023-12-11T17:37:10.325934Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:3300","nonce":0},{"type":"v1","addr":"172.21.15.59:6789","nonce":0}]},"addr":"172.21.15.59:6789/0","public_addr":"172.21.15.59:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:3300","nonce":0},{"type":"v1","addr":"172.21.15.181:6789","nonce":0}]},"addr":"172.21.15.181:6789/0","public_addr":"172.21.15.181:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2023-12-11T17:39:02.736 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:02.733+0000 7f8e6ee3d700 1 -- 172.21.15.181:0/1931343638 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f8e4c0386a0 msgr2=0x7f8e4c03ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:39:02.736 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:02.733+0000 7f8e6ee3d700 1 --2- 172.21.15.181:0/1931343638 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f8e4c0386a0 0x7f8e4c03ab60 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8e5c011510 tx=0x7f8e5c005960 comp rx=0 tx=0).stop 2023-12-11T17:39:02.736 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:02.733+0000 7f8e6ee3d700 1 -- 172.21.15.181:0/1931343638 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8e6819cb50 msgr2=0x7f8e68196750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:39:02.736 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:02.733+0000 7f8e6ee3d700 1 --2- 172.21.15.181:0/1931343638 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8e6819cb50 0x7f8e68196750 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8e5400b4d0 tx=0x7f8e5400b890 comp rx=0 tx=0).stop 2023-12-11T17:39:02.736 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:02.733+0000 7f8e6ee3d700 1 -- 172.21.15.181:0/1931343638 shutdown_connections 2023-12-11T17:39:02.737 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:02.733+0000 7f8e6ee3d700 1 --2- 172.21.15.181:0/1931343638 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f8e4c0386a0 0x7f8e4c03ab60 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:39:02.737 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:02.733+0000 7f8e6ee3d700 1 --2- 172.21.15.181:0/1931343638 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e580035c0 0x7f8e6819c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:39:02.737 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:02.733+0000 7f8e6ee3d700 1 --2- 172.21.15.181:0/1931343638 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8e6819cb50 0x7f8e68196750 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:39:02.737 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:02.733+0000 7f8e6ee3d700 1 -- 172.21.15.181:0/1931343638 >> 172.21.15.181:0/1931343638 conn(0x7f8e680fb9e0 msgr2=0x7f8e68103b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:39:02.737 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:02.733+0000 7f8e6ee3d700 1 -- 172.21.15.181:0/1931343638 shutdown_connections 2023-12-11T17:39:02.737 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:02.733+0000 7f8e6ee3d700 1 -- 172.21.15.181:0/1931343638 wait complete. 2023-12-11T17:39:02.737 INFO:teuthology.orchestra.run.smithi181.stderr:dumped monmap epoch 2 2023-12-11T17:39:03.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:03 smithi181 ceph-mon[103627]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T17:39:03.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:03 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:03.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:03 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-12-11T17:39:03.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:03 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:39:03.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:03 smithi181 ceph-mon[103627]: Deploying daemon mon.b on smithi115 2023-12-11T17:39:03.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:03 smithi181 ceph-mon[103627]: from='client.? 172.21.15.181:0/1931343638' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-11T17:39:03.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:03 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-11T17:39:03.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:03 smithi059 ceph-mon[98847]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T17:39:03.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:03 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:03.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:03 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-12-11T17:39:03.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:03 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:39:03.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:03 smithi059 ceph-mon[98847]: Deploying daemon mon.b on smithi115 2023-12-11T17:39:03.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:03 smithi059 ceph-mon[98847]: from='client.? 172.21.15.181:0/1931343638' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-11T17:39:03.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:03 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-11T17:39:04.040 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-12-11T17:39:04.041 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d0a07eaa-984b-11ee-95a3-87774f69a715 -- ceph mon dump -f json 2023-12-11T17:39:04.644 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/mon.c/config 2023-12-11T17:39:04.983 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:04 smithi115 systemd[1]: Starting Ceph mon.b for d0a07eaa-984b-11ee-95a3-87774f69a715... 2023-12-11T17:39:05.144 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:05 smithi181 ceph-mon[103627]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T17:39:05.234 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:04 smithi115 podman[102708]: 2023-12-11T17:39:05.234 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: set uid:gid to 167:167 (ceph:ceph) 2023-12-11T17:39:05.234 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: ceph version 19.0.0-119-ga62fda73 (a62fda73efba5520d409bb5a21bd8373289dda5d) reef (dev), process ceph-mon, pid 7 2023-12-11T17:39:05.235 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: pidfile_write: ignore empty --pid-file 2023-12-11T17:39:05.235 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 bash[102708]: 0f99ecd93664d6000a6c9d303cfec50b594611384eea1684a3431db0fd65cb0c 2023-12-11T17:39:05.235 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: load: jerasure load: lrc 2023-12-11T17:39:05.235 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: RocksDB version: 7.9.2 2023-12-11T17:39:05.235 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Git sha 0 2023-12-11T17:39:05.235 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Compile date 2023-12-11 11:29:13 2023-12-11T17:39:05.235 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: DB SUMMARY 2023-12-11T17:39:05.235 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: DB Session ID: CUK9NRDDCTCJNKR7B1H3 2023-12-11T17:39:05.235 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: CURRENT file: CURRENT 2023-12-11T17:39:05.235 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: IDENTITY file: IDENTITY 2023-12-11T17:39:05.235 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2023-12-11T17:39:05.235 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2023-12-11T17:39:05.235 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2023-12-11T17:39:05.235 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.error_if_exists: 0 2023-12-11T17:39:05.236 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.create_if_missing: 0 2023-12-11T17:39:05.236 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.paranoid_checks: 1 2023-12-11T17:39:05.236 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.flush_verify_memtable_count: 1 2023-12-11T17:39:05.236 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-12-11T17:39:05.236 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-12-11T17:39:05.236 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.env: 0x56529e910a60 2023-12-11T17:39:05.236 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.fs: PosixFileSystem 2023-12-11T17:39:05.236 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.info_log: 0x56529fba3740 2023-12-11T17:39:05.236 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.max_file_opening_threads: 16 2023-12-11T17:39:05.236 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.statistics: (nil) 2023-12-11T17:39:05.236 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.use_fsync: 0 2023-12-11T17:39:05.236 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.max_log_file_size: 0 2023-12-11T17:39:05.236 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-12-11T17:39:05.236 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.log_file_time_to_roll: 0 2023-12-11T17:39:05.236 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.keep_log_file_num: 1000 2023-12-11T17:39:05.237 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.recycle_log_file_num: 0 2023-12-11T17:39:05.237 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.allow_fallocate: 1 2023-12-11T17:39:05.237 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.allow_mmap_reads: 0 2023-12-11T17:39:05.237 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.allow_mmap_writes: 0 2023-12-11T17:39:05.237 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.use_direct_reads: 0 2023-12-11T17:39:05.237 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-12-11T17:39:05.237 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.create_missing_column_families: 0 2023-12-11T17:39:05.237 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.db_log_dir: 2023-12-11T17:39:05.237 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.wal_dir: 2023-12-11T17:39:05.237 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.table_cache_numshardbits: 6 2023-12-11T17:39:05.237 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.WAL_ttl_seconds: 0 2023-12-11T17:39:05.237 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.WAL_size_limit_MB: 0 2023-12-11T17:39:05.237 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-12-11T17:39:05.237 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-12-11T17:39:05.237 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.is_fd_close_on_exec: 1 2023-12-11T17:39:05.238 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.advise_random_on_open: 1 2023-12-11T17:39:05.238 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.db_write_buffer_size: 0 2023-12-11T17:39:05.238 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.write_buffer_manager: 0x56529fb59360 2023-12-11T17:39:05.238 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-12-11T17:39:05.238 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-12-11T17:39:05.238 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.use_adaptive_mutex: 0 2023-12-11T17:39:05.238 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.rate_limiter: (nil) 2023-12-11T17:39:05.238 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-12-11T17:39:05.238 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.wal_recovery_mode: 2 2023-12-11T17:39:05.238 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.enable_thread_tracking: 0 2023-12-11T17:39:05.238 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.enable_pipelined_write: 0 2023-12-11T17:39:05.238 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.unordered_write: 0 2023-12-11T17:39:05.238 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-12-11T17:39:05.238 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-12-11T17:39:05.239 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-12-11T17:39:05.239 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-12-11T17:39:05.239 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.row_cache: None 2023-12-11T17:39:05.239 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.wal_filter: None 2023-12-11T17:39:05.239 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-12-11T17:39:05.239 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.allow_ingest_behind: 0 2023-12-11T17:39:05.239 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.two_write_queues: 0 2023-12-11T17:39:05.239 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.manual_wal_flush: 0 2023-12-11T17:39:05.239 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.wal_compression: 0 2023-12-11T17:39:05.239 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.atomic_flush: 0 2023-12-11T17:39:05.239 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-12-11T17:39:05.239 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.persist_stats_to_disk: 0 2023-12-11T17:39:05.239 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.write_dbid_to_manifest: 0 2023-12-11T17:39:05.239 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.log_readahead_size: 0 2023-12-11T17:39:05.239 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-12-11T17:39:05.239 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.best_efforts_recovery: 0 2023-12-11T17:39:05.240 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-12-11T17:39:05.240 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-12-11T17:39:05.240 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.allow_data_in_errors: 0 2023-12-11T17:39:05.240 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.db_host_id: __hostname__ 2023-12-11T17:39:05.240 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.enforce_single_del_contracts: true 2023-12-11T17:39:05.240 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.max_background_jobs: 2 2023-12-11T17:39:05.240 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.max_background_compactions: -1 2023-12-11T17:39:05.240 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.max_subcompactions: 1 2023-12-11T17:39:05.240 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-12-11T17:39:05.240 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-12-11T17:39:05.240 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.delayed_write_rate : 16777216 2023-12-11T17:39:05.240 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.max_total_wal_size: 0 2023-12-11T17:39:05.240 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-12-11T17:39:05.240 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.stats_dump_period_sec: 600 2023-12-11T17:39:05.240 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.stats_persist_period_sec: 600 2023-12-11T17:39:05.241 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-12-11T17:39:05.241 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.max_open_files: -1 2023-12-11T17:39:05.241 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.bytes_per_sync: 0 2023-12-11T17:39:05.241 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.wal_bytes_per_sync: 0 2023-12-11T17:39:05.241 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.strict_bytes_per_sync: 0 2023-12-11T17:39:05.241 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.compaction_readahead_size: 0 2023-12-11T17:39:05.241 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.max_background_flushes: -1 2023-12-11T17:39:05.241 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Compression algorithms supported: 2023-12-11T17:39:05.241 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-12-11T17:39:05.241 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: kZSTD supported: 0 2023-12-11T17:39:05.241 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: kXpressCompression supported: 0 2023-12-11T17:39:05.241 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: kLZ4HCCompression supported: 1 2023-12-11T17:39:05.241 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: kZlibCompression supported: 1 2023-12-11T17:39:05.241 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: kSnappyCompression supported: 1 2023-12-11T17:39:05.241 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: kLZ4Compression supported: 1 2023-12-11T17:39:05.242 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: kBZip2Compression supported: 0 2023-12-11T17:39:05.242 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Fast CRC32 supported: Supported on x86 2023-12-11T17:39:05.242 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: DMutex implementation: pthread_mutex_t 2023-12-11T17:39:05.242 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2023-12-11T17:39:05.242 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-12-11T17:39:05.242 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-12-11T17:39:05.242 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.merge_operator: 2023-12-11T17:39:05.242 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.compaction_filter: None 2023-12-11T17:39:05.242 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.compaction_filter_factory: None 2023-12-11T17:39:05.242 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.sst_partitioner_factory: None 2023-12-11T17:39:05.242 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.memtable_factory: SkipListFactory 2023-12-11T17:39:05.242 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.table_factory: BlockBasedTable 2023-12-11T17:39:05.242 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x56529fba3860) 2023-12-11T17:39:05.242 INFO:journalctl@ceph.mon.b.smithi115.stdout: cache_index_and_filter_blocks: 1 2023-12-11T17:39:05.242 INFO:journalctl@ceph.mon.b.smithi115.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-12-11T17:39:05.243 INFO:journalctl@ceph.mon.b.smithi115.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-12-11T17:39:05.243 INFO:journalctl@ceph.mon.b.smithi115.stdout: pin_top_level_index_and_filter: 1 2023-12-11T17:39:05.243 INFO:journalctl@ceph.mon.b.smithi115.stdout: index_type: 0 2023-12-11T17:39:05.243 INFO:journalctl@ceph.mon.b.smithi115.stdout: data_block_index_type: 0 2023-12-11T17:39:05.243 INFO:journalctl@ceph.mon.b.smithi115.stdout: index_shortening: 1 2023-12-11T17:39:05.243 INFO:journalctl@ceph.mon.b.smithi115.stdout: data_block_hash_table_util_ratio: 0.750000 2023-12-11T17:39:05.243 INFO:journalctl@ceph.mon.b.smithi115.stdout: checksum: 4 2023-12-11T17:39:05.243 INFO:journalctl@ceph.mon.b.smithi115.stdout: no_block_cache: 0 2023-12-11T17:39:05.243 INFO:journalctl@ceph.mon.b.smithi115.stdout: block_cache: 0x56529fbf9090 2023-12-11T17:39:05.243 INFO:journalctl@ceph.mon.b.smithi115.stdout: block_cache_name: BinnedLRUCache 2023-12-11T17:39:05.243 INFO:journalctl@ceph.mon.b.smithi115.stdout: block_cache_options: 2023-12-11T17:39:05.243 INFO:journalctl@ceph.mon.b.smithi115.stdout: capacity : 536870912 2023-12-11T17:39:05.243 INFO:journalctl@ceph.mon.b.smithi115.stdout: num_shard_bits : 4 2023-12-11T17:39:05.243 INFO:journalctl@ceph.mon.b.smithi115.stdout: strict_capacity_limit : 0 2023-12-11T17:39:05.244 INFO:journalctl@ceph.mon.b.smithi115.stdout: high_pri_pool_ratio: 0.000 2023-12-11T17:39:05.244 INFO:journalctl@ceph.mon.b.smithi115.stdout: block_cache_compressed: (nil) 2023-12-11T17:39:05.244 INFO:journalctl@ceph.mon.b.smithi115.stdout: persistent_cache: (nil) 2023-12-11T17:39:05.244 INFO:journalctl@ceph.mon.b.smithi115.stdout: block_size: 4096 2023-12-11T17:39:05.244 INFO:journalctl@ceph.mon.b.smithi115.stdout: block_size_deviation: 10 2023-12-11T17:39:05.244 INFO:journalctl@ceph.mon.b.smithi115.stdout: block_restart_interval: 16 2023-12-11T17:39:05.244 INFO:journalctl@ceph.mon.b.smithi115.stdout: index_block_restart_interval: 1 2023-12-11T17:39:05.244 INFO:journalctl@ceph.mon.b.smithi115.stdout: metadata_block_size: 4096 2023-12-11T17:39:05.244 INFO:journalctl@ceph.mon.b.smithi115.stdout: partition_filters: 0 2023-12-11T17:39:05.244 INFO:journalctl@ceph.mon.b.smithi115.stdout: use_delta_encoding: 1 2023-12-11T17:39:05.244 INFO:journalctl@ceph.mon.b.smithi115.stdout: filter_policy: bloomfilter 2023-12-11T17:39:05.244 INFO:journalctl@ceph.mon.b.smithi115.stdout: whole_key_filtering: 1 2023-12-11T17:39:05.244 INFO:journalctl@ceph.mon.b.smithi115.stdout: verify_compression: 0 2023-12-11T17:39:05.244 INFO:journalctl@ceph.mon.b.smithi115.stdout: read_amp_bytes_per_bit: 0 2023-12-11T17:39:05.244 INFO:journalctl@ceph.mon.b.smithi115.stdout: format_version: 5 2023-12-11T17:39:05.244 INFO:journalctl@ceph.mon.b.smithi115.stdout: enable_index_compression: 1 2023-12-11T17:39:05.245 INFO:journalctl@ceph.mon.b.smithi115.stdout: block_align: 0 2023-12-11T17:39:05.245 INFO:journalctl@ceph.mon.b.smithi115.stdout: max_auto_readahead_size: 262144 2023-12-11T17:39:05.245 INFO:journalctl@ceph.mon.b.smithi115.stdout: prepopulate_block_cache: 0 2023-12-11T17:39:05.245 INFO:journalctl@ceph.mon.b.smithi115.stdout: initial_auto_readahead_size: 8192 2023-12-11T17:39:05.245 INFO:journalctl@ceph.mon.b.smithi115.stdout: num_file_reads_for_auto_readahead: 2 2023-12-11T17:39:05.245 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.write_buffer_size: 33554432 2023-12-11T17:39:05.245 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.max_write_buffer_number: 2 2023-12-11T17:39:05.245 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.compression: NoCompression 2023-12-11T17:39:05.245 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.bottommost_compression: Disabled 2023-12-11T17:39:05.245 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.prefix_extractor: nullptr 2023-12-11T17:39:05.245 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-12-11T17:39:05.245 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.num_levels: 7 2023-12-11T17:39:05.245 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-12-11T17:39:05.245 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-12-11T17:39:05.246 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-12-11T17:39:05.246 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-12-11T17:39:05.246 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-12-11T17:39:05.246 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-12-11T17:39:05.246 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-12-11T17:39:05.246 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-12-11T17:39:05.246 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-12-11T17:39:05.246 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-12-11T17:39:05.246 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-12-11T17:39:05.246 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-12-11T17:39:05.246 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.compression_opts.window_bits: -14 2023-12-11T17:39:05.246 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.compression_opts.level: 32767 2023-12-11T17:39:05.246 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.compression_opts.strategy: 0 2023-12-11T17:39:05.246 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-12-11T17:39:05.246 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-12-11T17:39:05.247 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-12-11T17:39:05.247 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-12-11T17:39:05.247 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.compression_opts.enabled: false 2023-12-11T17:39:05.247 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-12-11T17:39:05.247 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-12-11T17:39:05.247 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-12-11T17:39:05.247 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-12-11T17:39:05.247 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.target_file_size_base: 67108864 2023-12-11T17:39:05.247 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.target_file_size_multiplier: 1 2023-12-11T17:39:05.247 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-12-11T17:39:05.247 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-12-11T17:39:05.247 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-12-11T17:39:05.247 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-12-11T17:39:05.247 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-12-11T17:39:05.247 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-12-11T17:39:05.248 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-12-11T17:39:05.248 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-12-11T17:39:05.248 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-12-11T17:39:05.248 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-12-11T17:39:05.248 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-12-11T17:39:05.248 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-12-11T17:39:05.248 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-12-11T17:39:05.248 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.arena_block_size: 1048576 2023-12-11T17:39:05.248 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-12-11T17:39:05.248 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-12-11T17:39:05.248 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.disable_auto_compactions: 0 2023-12-11T17:39:05.248 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-12-11T17:39:05.248 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-12-11T17:39:05.248 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-12-11T17:39:05.248 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-12-11T17:39:05.249 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-12-11T17:39:05.249 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-12-11T17:39:05.249 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-12-11T17:39:05.249 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-12-11T17:39:05.249 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-12-11T17:39:05.249 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-12-11T17:39:05.249 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.table_properties_collectors: 2023-12-11T17:39:05.249 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.inplace_update_support: 0 2023-12-11T17:39:05.249 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.inplace_update_num_locks: 10000 2023-12-11T17:39:05.249 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-12-11T17:39:05.249 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-12-11T17:39:05.249 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.memtable_huge_page_size: 0 2023-12-11T17:39:05.249 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.bloom_locality: 0 2023-12-11T17:39:05.249 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.max_successive_merges: 0 2023-12-11T17:39:05.249 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.optimize_filters_for_hits: 0 2023-12-11T17:39:05.249 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.paranoid_file_checks: 0 2023-12-11T17:39:05.250 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.force_consistency_checks: 1 2023-12-11T17:39:05.250 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.report_bg_io_stats: 0 2023-12-11T17:39:05.250 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.ttl: 2592000 2023-12-11T17:39:05.250 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.periodic_compaction_seconds: 0 2023-12-11T17:39:05.250 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-12-11T17:39:05.250 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-12-11T17:39:05.250 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.enable_blob_files: false 2023-12-11T17:39:05.250 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.min_blob_size: 0 2023-12-11T17:39:05.250 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.blob_file_size: 268435456 2023-12-11T17:39:05.250 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.blob_compression_type: NoCompression 2023-12-11T17:39:05.250 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.enable_blob_garbage_collection: false 2023-12-11T17:39:05.250 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-12-11T17:39:05.250 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-12-11T17:39:05.250 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-12-11T17:39:05.251 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.blob_file_starting_level: 0 2023-12-11T17:39:05.251 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-12-11T17:39:05.251 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2023-12-11T17:39:05.251 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2023-12-11T17:39:05.251 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: a48ef4fa-3b05-446b-984e-d59b9152907b 2023-12-11T17:39:05.251 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: EVENT_LOG_v1 {"time_micros": 1702316345207737, "job": 1, "event": "recovery_started", "wal_files": [4]} 2023-12-11T17:39:05.251 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2023-12-11T17:39:05.251 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: EVENT_LOG_v1 {"time_micros": 1702316345208145, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1747, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1702316345, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a48ef4fa-3b05-446b-984e-d59b9152907b", "db_session_id": "CUK9NRDDCTCJNKR7B1H3", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2023-12-11T17:39:05.251 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: EVENT_LOG_v1 {"time_micros": 1702316345208213, "job": 1, "event": "recovery_finished"} 2023-12-11T17:39:05.251 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2023-12-11T17:39:05.251 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-12-11T17:39:05.251 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x56529fc94000 2023-12-11T17:39:05.251 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: DB pointer 0x56529fc80000 2023-12-11T17:39:05.252 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-12-11T17:39:05.252 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-12-11T17:39:05.252 INFO:journalctl@ceph.mon.b.smithi115.stdout: ** DB Stats ** 2023-12-11T17:39:05.252 INFO:journalctl@ceph.mon.b.smithi115.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-12-11T17:39:05.252 INFO:journalctl@ceph.mon.b.smithi115.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-12-11T17:39:05.252 INFO:journalctl@ceph.mon.b.smithi115.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-12-11T17:39:05.252 INFO:journalctl@ceph.mon.b.smithi115.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-12-11T17:39:05.252 INFO:journalctl@ceph.mon.b.smithi115.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-12-11T17:39:05.252 INFO:journalctl@ceph.mon.b.smithi115.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-12-11T17:39:05.252 INFO:journalctl@ceph.mon.b.smithi115.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-12-11T17:39:05.252 INFO:journalctl@ceph.mon.b.smithi115.stdout: 2023-12-11T17:39:05.252 INFO:journalctl@ceph.mon.b.smithi115.stdout: ** Compaction Stats [default] ** 2023-12-11T17:39:05.252 INFO:journalctl@ceph.mon.b.smithi115.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-12-11T17:39:05.252 INFO:journalctl@ceph.mon.b.smithi115.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-12-11T17:39:05.252 INFO:journalctl@ceph.mon.b.smithi115.stdout: L0 1/0 1.71 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.3 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-12-11T17:39:05.252 INFO:journalctl@ceph.mon.b.smithi115.stdout: Sum 1/0 1.71 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.3 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-12-11T17:39:05.253 INFO:journalctl@ceph.mon.b.smithi115.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.3 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-12-11T17:39:05.253 INFO:journalctl@ceph.mon.b.smithi115.stdout: 2023-12-11T17:39:05.253 INFO:journalctl@ceph.mon.b.smithi115.stdout: ** Compaction Stats [default] ** 2023-12-11T17:39:05.253 INFO:journalctl@ceph.mon.b.smithi115.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-12-11T17:39:05.253 INFO:journalctl@ceph.mon.b.smithi115.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-12-11T17:39:05.253 INFO:journalctl@ceph.mon.b.smithi115.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.3 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-12-11T17:39:05.253 INFO:journalctl@ceph.mon.b.smithi115.stdout: 2023-12-11T17:39:05.253 INFO:journalctl@ceph.mon.b.smithi115.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-12-11T17:39:05.253 INFO:journalctl@ceph.mon.b.smithi115.stdout: 2023-12-11T17:39:05.253 INFO:journalctl@ceph.mon.b.smithi115.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-12-11T17:39:05.253 INFO:journalctl@ceph.mon.b.smithi115.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-12-11T17:39:05.253 INFO:journalctl@ceph.mon.b.smithi115.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-12-11T17:39:05.253 INFO:journalctl@ceph.mon.b.smithi115.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-12-11T17:39:05.253 INFO:journalctl@ceph.mon.b.smithi115.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-12-11T17:39:05.253 INFO:journalctl@ceph.mon.b.smithi115.stdout: AddFile(Keys): cumulative 0, interval 0 2023-12-11T17:39:05.253 INFO:journalctl@ceph.mon.b.smithi115.stdout: Cumulative compaction: 0.00 GB write, 0.71 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-12-11T17:39:05.254 INFO:journalctl@ceph.mon.b.smithi115.stdout: Interval compaction: 0.00 GB write, 0.71 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-12-11T17:39:05.254 INFO:journalctl@ceph.mon.b.smithi115.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-12-11T17:39:05.254 INFO:journalctl@ceph.mon.b.smithi115.stdout: Block cache BinnedLRUCache@0x56529fbf9090#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2023-12-11T17:39:05.254 INFO:journalctl@ceph.mon.b.smithi115.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2023-12-11T17:39:05.254 INFO:journalctl@ceph.mon.b.smithi115.stdout: 2023-12-11T17:39:05.254 INFO:journalctl@ceph.mon.b.smithi115.stdout: ** File Read Latency Histogram By Level [default] ** 2023-12-11T17:39:05.254 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: mon.b does not exist in monmap, will attempt to join an existing cluster 2023-12-11T17:39:05.254 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: using public_addr v2:172.21.15.115:0/0 -> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] 2023-12-11T17:39:05.254 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: starting mon.b rank -1 at public addrs [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] at bind addrs [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid d0a07eaa-984b-11ee-95a3-87774f69a715 2023-12-11T17:39:05.254 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: mon.b@-1(???) e0 preinit fsid d0a07eaa-984b-11ee-95a3-87774f69a715 2023-12-11T17:39:05.254 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: mon.b@-1(synchronizing).mds e1 new map 2023-12-11T17:39:05.254 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: mon.b@-1(synchronizing).mds e1 print_map 2023-12-11T17:39:05.254 INFO:journalctl@ceph.mon.b.smithi115.stdout: e1 2023-12-11T17:39:05.254 INFO:journalctl@ceph.mon.b.smithi115.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-12-11T17:39:05.255 INFO:journalctl@ceph.mon.b.smithi115.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-12-11T17:39:05.255 INFO:journalctl@ceph.mon.b.smithi115.stdout: legacy client fscid: -1 2023-12-11T17:39:05.255 INFO:journalctl@ceph.mon.b.smithi115.stdout: 2023-12-11T17:39:05.255 INFO:journalctl@ceph.mon.b.smithi115.stdout: No filesystems configured 2023-12-11T17:39:05.255 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2023-12-11T17:39:05.255 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2023-12-11T17:39:05.255 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2023-12-11T17:39:05.255 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2023-12-11T17:39:05.255 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2023-12-11T17:39:05.255 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2023-12-11T17:39:05.255 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2023-12-11T17:39:05.255 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-12-11T17:39:05.255 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-12-11T17:39:05.255 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-12-11T17:39:05.255 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T17:39:05.255 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T17:39:05.255 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: Updating smithi115:/etc/ceph/ceph.client.admin.keyring 2023-12-11T17:39:05.256 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: Updating smithi115:/var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/config/ceph.client.admin.keyring 2023-12-11T17:39:05.256 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:05.256 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:05.256 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:05.256 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:05.256 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: Added host smithi181 2023-12-11T17:39:05.256 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:39:05.256 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:05.256 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T17:39:05.256 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:05.256 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-11T17:39:05.256 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:05.256 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T17:39:05.256 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/1648308900' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-12-11T17:39:05.256 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T17:39:05.256 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/1648308900' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-12-11T17:39:05.257 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: osdmap e4: 0 total, 0 up, 0 in 2023-12-11T17:39:05.257 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:05.257 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:05.257 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:05.257 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:05.257 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi181", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:39:05.257 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:39:05.257 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:39:05.257 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: Updating smithi181:/etc/ceph/ceph.conf 2023-12-11T17:39:05.257 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: Updating smithi181:/var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/config/ceph.conf 2023-12-11T17:39:05.257 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: Updating smithi181:/etc/ceph/ceph.client.admin.keyring 2023-12-11T17:39:05.257 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: Updating smithi181:/var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/config/ceph.client.admin.keyring 2023-12-11T17:39:05.257 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:05.257 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:05.257 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:05.257 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T17:39:05.258 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:05.258 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:39:05.258 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:39:05.258 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:39:05.258 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:05.258 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-12-11T17:39:05.258 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:39:05.258 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi059:172.21.15.59=a;smithi115:172.21.15.115=b;smithi181:172.21.15.181=c", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T17:39:05.258 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: Saving service mon spec with placement smithi059:172.21.15.59=a;smithi115:172.21.15.115=b;smithi181:172.21.15.181=c;count:3 2023-12-11T17:39:05.258 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: Deploying daemon mon.c on smithi181 2023-12-11T17:39:05.258 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T17:39:05.259 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: from='client.? 172.21.15.181:0/3313239583' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-11T17:39:05.259 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-11T17:39:05.259 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-11T17:39:05.259 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: mon.a calling monitor election 2023-12-11T17:39:05.259 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-11T17:39:05.259 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T17:39:05.259 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-11T17:39:05.259 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: mon.c calling monitor election 2023-12-11T17:39:05.259 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-11T17:39:05.260 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T17:39:05.260 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-11T17:39:05.260 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-11T17:39:05.260 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-12-11T17:39:05.260 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: monmap e2: 2 mons at {a=[v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0],c=[v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0]} removed_ranks: {} disallowed_leaders: {} 2023-12-11T17:39:05.260 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: fsmap 2023-12-11T17:39:05.260 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: osdmap e4: 0 total, 0 up, 0 in 2023-12-11T17:39:05.260 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: mgrmap e13: x(active, since 47s) 2023-12-11T17:39:05.260 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: overall HEALTH_OK 2023-12-11T17:39:05.260 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:05.260 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:05.260 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T17:39:05.260 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:05.260 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-12-11T17:39:05.261 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:39:05.261 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: Deploying daemon mon.b on smithi115 2023-12-11T17:39:05.261 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: from='client.? 172.21.15.181:0/1931343638' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-11T17:39:05.261 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-11T17:39:05.261 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T17:39:05.261 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 ceph-mon[102748]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2023-12-11T17:39:05.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:05 smithi059 ceph-mon[98847]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T17:39:05.532 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:05 smithi115 systemd[1]: Started Ceph mon.b for d0a07eaa-984b-11ee-95a3-87774f69a715. 2023-12-11T17:39:10.275 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:10.273+0000 7ffbe4e18700 1 -- 172.21.15.181:0/2939342823 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffbc80035c0 msgr2=0x7ffbc8005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:39:10.276 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:10.273+0000 7ffbe4e18700 1 --2- 172.21.15.181:0/2939342823 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffbc80035c0 0x7ffbc8005a50 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ffbe00ffe60 tx=0x7ffbd400fe60 comp rx=0 tx=0).stop 2023-12-11T17:39:10.276 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:10.273+0000 7ffbe4e18700 1 -- 172.21.15.181:0/2939342823 shutdown_connections 2023-12-11T17:39:10.276 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:10.273+0000 7ffbe4e18700 1 --2- 172.21.15.181:0/2939342823 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffbc80035c0 0x7ffbc8005a50 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:39:10.276 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:10.273+0000 7ffbe4e18700 1 -- 172.21.15.181:0/2939342823 >> 172.21.15.181:0/2939342823 conn(0x7ffbe00fba20 msgr2=0x7ffbe00fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:39:10.276 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:10.273+0000 7ffbe4e18700 1 -- 172.21.15.181:0/2939342823 shutdown_connections 2023-12-11T17:39:10.276 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:10.273+0000 7ffbe4e18700 1 -- 172.21.15.181:0/2939342823 wait complete. 2023-12-11T17:39:10.278 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:10.273+0000 7ffbe4e18700 1 Processor -- start 2023-12-11T17:39:10.278 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:10.274+0000 7ffbe4e18700 1 -- start start 2023-12-11T17:39:10.278 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:10.274+0000 7ffbe4e18700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffbc80035c0 0x7ffbe0197910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:39:10.278 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:10.274+0000 7ffbe4e18700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffbe0197e50 0x7ffbe01982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:39:10.278 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:10.274+0000 7ffbe4e18700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffbe01a0980 0x7ffbe01a2d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:39:10.278 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:10.274+0000 7ffbe4e18700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbe01a3400 con 0x7ffbe0197e50 2023-12-11T17:39:10.278 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:10.274+0000 7ffbe4e18700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbe01a3570 con 0x7ffbe01a0980 2023-12-11T17:39:10.278 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:10.274+0000 7ffbe4e18700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbe01a3730 con 0x7ffbc80035c0 2023-12-11T17:39:10.279 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:10.274+0000 7ffbde59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffbc80035c0 0x7ffbe0197910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:39:10.279 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:10.274+0000 7ffbde59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffbc80035c0 0x7ffbe0197910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34230/0 (socket says 172.21.15.181:34230) 2023-12-11T17:39:10.279 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:10.274+0000 7ffbddd9b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffbe0197e50 0x7ffbe01982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:39:10.279 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:10.274+0000 7ffbde59c700 1 -- 172.21.15.181:0/1343225536 learned_addr learned my addr 172.21.15.181:0/1343225536 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-12-11T17:39:10.279 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:10.274+0000 7ffbde59c700 1 -- 172.21.15.181:0/1343225536 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffbc80035c0 msgr2=0x7ffbe0197910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2023-12-11T17:39:10.279 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:10.274+0000 7ffbde59c700 1 -- 172.21.15.181:0/1343225536 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffbc80035c0 msgr2=0x7ffbe0197910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-12-11T17:39:10.279 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:10.274+0000 7ffbde59c700 1 --2- 172.21.15.181:0/1343225536 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffbc80035c0 0x7ffbe0197910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-12-11T17:39:10.279 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:10.274+0000 7ffbded9d700 1 --2- 172.21.15.181:0/1343225536 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffbe01a0980 0x7ffbe01a2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:39:10.279 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:10.274+0000 7ffbde59c700 1 --2- 172.21.15.181:0/1343225536 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffbc80035c0 0x7ffbe0197910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-11T17:39:10.279 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:10.274+0000 7ffbded9d700 1 -- 172.21.15.181:0/1343225536 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffbe01a0980 msgr2=0x7ffbe01a2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2023-12-11T17:39:10.279 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:10.274+0000 7ffbded9d700 1 -- 172.21.15.181:0/1343225536 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffbe01a0980 msgr2=0x7ffbe01a2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-12-11T17:39:10.279 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:10.274+0000 7ffbded9d700 1 --2- 172.21.15.181:0/1343225536 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffbe01a0980 0x7ffbe01a2d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-12-11T17:39:10.279 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:10.274+0000 7ffbded9d700 1 --2- 172.21.15.181:0/1343225536 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffbe01a0980 0x7ffbe01a2d70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-11T17:39:10.279 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:10.274+0000 7ffbddd9b700 1 -- 172.21.15.181:0/1343225536 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffbc80035c0 msgr2=0x7ffbe0197910 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-11T17:39:10.280 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:10.274+0000 7ffbddd9b700 1 --2- 172.21.15.181:0/1343225536 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffbc80035c0 0x7ffbe0197910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:39:10.280 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:10.275+0000 7ffbddd9b700 1 -- 172.21.15.181:0/1343225536 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffbe01a0980 msgr2=0x7ffbe01a2d70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-11T17:39:10.280 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:10.275+0000 7ffbddd9b700 1 --2- 172.21.15.181:0/1343225536 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffbe01a0980 0x7ffbe01a2d70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:39:10.280 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:10.275+0000 7ffbddd9b700 1 -- 172.21.15.181:0/1343225536 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbd400f8b0 con 0x7ffbe0197e50 2023-12-11T17:39:10.280 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:10.275+0000 7ffbddd9b700 1 --2- 172.21.15.181:0/1343225536 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffbe0197e50 0x7ffbe01982d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ffbd000b550 tx=0x7ffbd000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:39:10.280 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:10.275+0000 7ffbcf7fe700 1 -- 172.21.15.181:0/1343225536 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbd0004230 con 0x7ffbe0197e50 2023-12-11T17:39:10.280 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:10.275+0000 7ffbcf7fe700 1 -- 172.21.15.181:0/1343225536 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffbd0004390 con 0x7ffbe0197e50 2023-12-11T17:39:10.280 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:10.275+0000 7ffbe4e18700 1 -- 172.21.15.181:0/1343225536 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbe01a39b0 con 0x7ffbe0197e50 2023-12-11T17:39:10.280 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:10.275+0000 7ffbcf7fe700 1 -- 172.21.15.181:0/1343225536 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbd0011af0 con 0x7ffbe0197e50 2023-12-11T17:39:10.280 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:10.275+0000 7ffbe4e18700 1 -- 172.21.15.181:0/1343225536 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbe01a3eb0 con 0x7ffbe0197e50 2023-12-11T17:39:10.281 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:10.276+0000 7ffbcf7fe700 1 -- 172.21.15.181:0/1343225536 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 13) v1 ==== 45636+0+0 (secure 0 0 0) 0x7ffbd0011c50 con 0x7ffbe0197e50 2023-12-11T17:39:10.281 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:10.276+0000 7ffbe4e18700 1 -- 172.21.15.181:0/1343225536 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbe0101670 con 0x7ffbe0197e50 2023-12-11T17:39:10.281 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:10.276+0000 7ffbcf7fe700 1 --2- 172.21.15.181:0/1343225536 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7ffbc0038d90 0x7ffbc003b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:39:10.281 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:10.276+0000 7ffbcf7fe700 1 -- 172.21.15.181:0/1343225536 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7ffbd004aef0 con 0x7ffbe0197e50 2023-12-11T17:39:10.281 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:10.277+0000 7ffbde59c700 1 --2- 172.21.15.181:0/1343225536 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7ffbc0038d90 0x7ffbc003b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:39:10.281 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:10.277+0000 7ffbde59c700 1 --2- 172.21.15.181:0/1343225536 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7ffbc0038d90 0x7ffbc003b250 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ffbd4020040 tx=0x7ffbd4001e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:39:10.284 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:10.283+0000 7ffbcf7fe700 1 -- 172.21.15.181:0/1343225536 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7ffbd000f2e0 con 0x7ffbe0197e50 2023-12-11T17:39:10.579 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:10.576+0000 7ffbe4e18700 1 -- 172.21.15.181:0/1343225536 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7ffbe0062c00 con 0x7ffbe0197e50 2023-12-11T17:39:10.580 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:10.577+0000 7ffbcf7fe700 1 -- 172.21.15.181:0/1343225536 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) v1 ==== 95+0+1274 (secure 0 0 0) 0x7ffbd000f2e0 con 0x7ffbe0197e50 2023-12-11T17:39:10.580 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-12-11T17:39:10.580 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":3,"fsid":"d0a07eaa-984b-11ee-95a3-87774f69a715","modified":"2023-12-11T17:39:05.236572Z","created":"2023-12-11T17:37:10.325934Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:3300","nonce":0},{"type":"v1","addr":"172.21.15.59:6789","nonce":0}]},"addr":"172.21.15.59:6789/0","public_addr":"172.21.15.59:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:3300","nonce":0},{"type":"v1","addr":"172.21.15.181:6789","nonce":0}]},"addr":"172.21.15.181:6789/0","public_addr":"172.21.15.181:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:3300","nonce":0},{"type":"v1","addr":"172.21.15.115:6789","nonce":0}]},"addr":"172.21.15.115:6789/0","public_addr":"172.21.15.115:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2023-12-11T17:39:10.582 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:10.580+0000 7ffbe4e18700 1 -- 172.21.15.181:0/1343225536 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7ffbc0038d90 msgr2=0x7ffbc003b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:39:10.582 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:10.580+0000 7ffbe4e18700 1 --2- 172.21.15.181:0/1343225536 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7ffbc0038d90 0x7ffbc003b250 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ffbd4020040 tx=0x7ffbd4001e50 comp rx=0 tx=0).stop 2023-12-11T17:39:10.582 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:10.580+0000 7ffbe4e18700 1 -- 172.21.15.181:0/1343225536 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffbe0197e50 msgr2=0x7ffbe01982d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:39:10.583 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:10.580+0000 7ffbe4e18700 1 --2- 172.21.15.181:0/1343225536 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffbe0197e50 0x7ffbe01982d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ffbd000b550 tx=0x7ffbd000b910 comp rx=0 tx=0).stop 2023-12-11T17:39:10.583 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:10.580+0000 7ffbe4e18700 1 -- 172.21.15.181:0/1343225536 shutdown_connections 2023-12-11T17:39:10.583 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:10.580+0000 7ffbe4e18700 1 --2- 172.21.15.181:0/1343225536 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7ffbc0038d90 0x7ffbc003b250 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:39:10.583 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:10.580+0000 7ffbe4e18700 1 --2- 172.21.15.181:0/1343225536 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffbe01a0980 0x7ffbe01a2d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:39:10.583 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:10.580+0000 7ffbe4e18700 1 --2- 172.21.15.181:0/1343225536 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffbc80035c0 0x7ffbe0197910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:39:10.583 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:10.580+0000 7ffbe4e18700 1 --2- 172.21.15.181:0/1343225536 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffbe0197e50 0x7ffbe01982d0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:39:10.583 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:10.580+0000 7ffbe4e18700 1 -- 172.21.15.181:0/1343225536 >> 172.21.15.181:0/1343225536 conn(0x7ffbe00fba20 msgr2=0x7ffbe0103b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:39:10.583 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:10.580+0000 7ffbe4e18700 1 -- 172.21.15.181:0/1343225536 shutdown_connections 2023-12-11T17:39:10.583 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:10.580+0000 7ffbe4e18700 1 -- 172.21.15.181:0/1343225536 wait complete. 2023-12-11T17:39:10.583 INFO:teuthology.orchestra.run.smithi181.stderr:dumped monmap epoch 3 2023-12-11T17:39:10.603 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:10 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-11T17:39:10.603 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:10 smithi181 ceph-mon[103627]: mon.a calling monitor election 2023-12-11T17:39:10.603 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:10 smithi181 ceph-mon[103627]: mon.c calling monitor election 2023-12-11T17:39:10.603 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:10 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-11T17:39:10.603 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:10 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-11T17:39:10.603 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:10 smithi181 ceph-mon[103627]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T17:39:10.604 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:10 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-11T17:39:10.604 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:10 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-11T17:39:10.604 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:10 smithi181 ceph-mon[103627]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T17:39:10.604 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:10 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-11T17:39:10.604 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:10 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-11T17:39:10.604 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:10 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-11T17:39:10.604 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:10 smithi181 ceph-mon[103627]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-12-11T17:39:10.604 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:10 smithi181 ceph-mon[103627]: monmap e3: 3 mons at {a=[v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0],b=[v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0],c=[v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0]} removed_ranks: {} disallowed_leaders: {} 2023-12-11T17:39:10.604 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:10 smithi181 ceph-mon[103627]: fsmap 2023-12-11T17:39:10.604 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:10 smithi181 ceph-mon[103627]: osdmap e4: 0 total, 0 up, 0 in 2023-12-11T17:39:10.604 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:10 smithi181 ceph-mon[103627]: mgrmap e13: x(active, since 56s) 2023-12-11T17:39:10.604 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:10 smithi181 ceph-mon[103627]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-12-11T17:39:10.604 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:10 smithi181 ceph-mon[103627]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-12-11T17:39:10.604 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:10 smithi181 ceph-mon[103627]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-12-11T17:39:10.605 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:10 smithi181 ceph-mon[103627]: mon.b (rank 2) addr [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] is down (out of quorum) 2023-12-11T17:39:10.605 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:10 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:10.605 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:10 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:10.605 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:10 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:10.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:10 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-11T17:39:10.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:10 smithi059 ceph-mon[98847]: mon.a calling monitor election 2023-12-11T17:39:10.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:10 smithi059 ceph-mon[98847]: mon.c calling monitor election 2023-12-11T17:39:10.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:10 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-11T17:39:10.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:10 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-11T17:39:10.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:10 smithi059 ceph-mon[98847]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T17:39:10.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:10 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-11T17:39:10.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:10 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-11T17:39:10.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:10 smithi059 ceph-mon[98847]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T17:39:10.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:10 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-11T17:39:10.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:10 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-11T17:39:10.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:10 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-11T17:39:10.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:10 smithi059 ceph-mon[98847]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-12-11T17:39:10.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:10 smithi059 ceph-mon[98847]: monmap e3: 3 mons at {a=[v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0],b=[v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0],c=[v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0]} removed_ranks: {} disallowed_leaders: {} 2023-12-11T17:39:10.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:10 smithi059 ceph-mon[98847]: fsmap 2023-12-11T17:39:10.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:10 smithi059 ceph-mon[98847]: osdmap e4: 0 total, 0 up, 0 in 2023-12-11T17:39:10.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:10 smithi059 ceph-mon[98847]: mgrmap e13: x(active, since 56s) 2023-12-11T17:39:10.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:10 smithi059 ceph-mon[98847]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-12-11T17:39:10.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:10 smithi059 ceph-mon[98847]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-12-11T17:39:10.656 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:10 smithi059 ceph-mon[98847]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-12-11T17:39:10.656 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:10 smithi059 ceph-mon[98847]: mon.b (rank 2) addr [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] is down (out of quorum) 2023-12-11T17:39:10.656 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:10 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:10.656 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:10 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:10.656 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:10 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:11.016 INFO:tasks.cephadm:Generating final ceph.conf file... 2023-12-11T17:39:11.017 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d0a07eaa-984b-11ee-95a3-87774f69a715 -- ceph config generate-minimal-conf 2023-12-11T17:39:11.643 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/mon.a/config 2023-12-11T17:39:11.644 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:11 smithi181 ceph-mon[103627]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T17:39:11.645 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:11 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:11.645 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:11 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:39:11.645 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:11 smithi181 ceph-mon[103627]: from='client.? 172.21.15.181:0/1343225536' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-11T17:39:11.645 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:11 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-11T17:39:11.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:11 smithi059 ceph-mon[98847]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T17:39:11.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:11 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:11.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:11 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:39:11.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:11 smithi059 ceph-mon[98847]: from='client.? 172.21.15.181:0/1343225536' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-11T17:39:11.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:11 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-11T17:39:12.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:12 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-11T17:39:12.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:12 smithi115 ceph-mon[102748]: mon.a calling monitor election 2023-12-11T17:39:12.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:12 smithi115 ceph-mon[102748]: mon.c calling monitor election 2023-12-11T17:39:12.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:12 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-11T17:39:12.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:12 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-11T17:39:12.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:12 smithi115 ceph-mon[102748]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T17:39:12.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:12 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-11T17:39:12.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:12 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-11T17:39:12.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:12 smithi115 ceph-mon[102748]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T17:39:12.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:12 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-11T17:39:12.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:12 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-11T17:39:12.534 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:12 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-11T17:39:12.534 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:12 smithi115 ceph-mon[102748]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-12-11T17:39:12.534 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:12 smithi115 ceph-mon[102748]: monmap e3: 3 mons at {a=[v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0],b=[v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0],c=[v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0]} removed_ranks: {} disallowed_leaders: {} 2023-12-11T17:39:12.534 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:12 smithi115 ceph-mon[102748]: fsmap 2023-12-11T17:39:12.534 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:12 smithi115 ceph-mon[102748]: osdmap e4: 0 total, 0 up, 0 in 2023-12-11T17:39:12.534 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:12 smithi115 ceph-mon[102748]: mgrmap e13: x(active, since 56s) 2023-12-11T17:39:12.534 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:12 smithi115 ceph-mon[102748]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-12-11T17:39:12.534 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:12 smithi115 ceph-mon[102748]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-12-11T17:39:12.534 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:12 smithi115 ceph-mon[102748]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-12-11T17:39:12.534 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:12 smithi115 ceph-mon[102748]: mon.b (rank 2) addr [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] is down (out of quorum) 2023-12-11T17:39:12.534 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:12 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:12.534 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:12 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:12.535 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:12 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:12.535 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:12 smithi115 ceph-mon[102748]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T17:39:12.535 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:12 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:12.535 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:12 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:39:12.535 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:12 smithi115 ceph-mon[102748]: from='client.? 172.21.15.181:0/1343225536' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-11T17:39:12.535 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:12 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-11T17:39:12.877 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:39:12.874+0000 7f8be5bd6700 1 -- 172.21.15.59:0/3541288532 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8be0105e70 msgr2=0x7f8be0106250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:39:12.877 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:39:12.874+0000 7f8be5bd6700 1 --2- 172.21.15.59:0/3541288532 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8be0105e70 0x7f8be0106250 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f8bd0009910 tx=0x7f8bd0009c20 comp rx=0 tx=0).stop 2023-12-11T17:39:12.877 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:39:12.875+0000 7f8be5bd6700 1 -- 172.21.15.59:0/3541288532 shutdown_connections 2023-12-11T17:39:12.877 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:39:12.875+0000 7f8be5bd6700 1 --2- 172.21.15.59:0/3541288532 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8be0105e70 0x7f8be0106250 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:39:12.878 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:39:12.875+0000 7f8be5bd6700 1 -- 172.21.15.59:0/3541288532 >> 172.21.15.59:0/3541288532 conn(0x7f8be0075930 msgr2=0x7f8be0075d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:39:12.878 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:39:12.875+0000 7f8be5bd6700 1 -- 172.21.15.59:0/3541288532 shutdown_connections 2023-12-11T17:39:12.878 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:39:12.875+0000 7f8be5bd6700 1 -- 172.21.15.59:0/3541288532 wait complete. 2023-12-11T17:39:12.880 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:39:12.875+0000 7f8be5bd6700 1 Processor -- start 2023-12-11T17:39:12.881 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:39:12.875+0000 7f8be5bd6700 1 -- start start 2023-12-11T17:39:12.881 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:39:12.876+0000 7f8be5bd6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8be0105e70 0x7f8be00fd400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:39:12.881 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:39:12.876+0000 7f8be5bd6700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8be00fd940 0x7f8be00fddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:39:12.881 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:39:12.876+0000 7f8be5bd6700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8be01a2b70 0x7f8be01a4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:39:12.881 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:39:12.876+0000 7f8be5bd6700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8be01a5530 con 0x7f8be01a2b70 2023-12-11T17:39:12.881 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:39:12.876+0000 7f8be5bd6700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8be01a5670 con 0x7f8be00fd940 2023-12-11T17:39:12.881 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:39:12.876+0000 7f8be5bd6700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8be01a5830 con 0x7f8be0105e70 2023-12-11T17:39:12.881 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:39:12.876+0000 7f8bdffff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8be01a2b70 0x7f8be01a4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:39:12.881 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:39:12.876+0000 7f8bdffff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8be01a2b70 0x7f8be01a4f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60176/0 (socket says 172.21.15.59:60176) 2023-12-11T17:39:12.881 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:39:12.876+0000 7f8bdffff700 1 -- 172.21.15.59:0/252868523 learned_addr learned my addr 172.21.15.59:0/252868523 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:39:12.882 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:39:12.876+0000 7f8bdeffd700 1 --2- 172.21.15.59:0/252868523 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8be00fd940 0x7f8be00fddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:39:12.882 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:39:12.876+0000 7f8bdf7fe700 1 --2- 172.21.15.59:0/252868523 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8be0105e70 0x7f8be00fd400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:39:12.882 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:39:12.876+0000 7f8bdffff700 1 -- 172.21.15.59:0/252868523 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8be0105e70 msgr2=0x7f8be00fd400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:39:12.882 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:39:12.876+0000 7f8bdffff700 1 --2- 172.21.15.59:0/252868523 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8be0105e70 0x7f8be00fd400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:39:12.882 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:39:12.876+0000 7f8bdffff700 1 -- 172.21.15.59:0/252868523 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8be00fd940 msgr2=0x7f8be00fddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:39:12.882 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:39:12.876+0000 7f8bdffff700 1 --2- 172.21.15.59:0/252868523 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8be00fd940 0x7f8be00fddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:39:12.882 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:39:12.876+0000 7f8bdffff700 1 -- 172.21.15.59:0/252868523 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bd00095f0 con 0x7f8be01a2b70 2023-12-11T17:39:12.882 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:39:12.876+0000 7f8bdf7fe700 1 --2- 172.21.15.59:0/252868523 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8be0105e70 0x7f8be00fd400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T17:39:12.882 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:39:12.876+0000 7f8bdeffd700 1 --2- 172.21.15.59:0/252868523 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8be00fd940 0x7f8be00fddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T17:39:12.882 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:39:12.876+0000 7f8bdffff700 1 --2- 172.21.15.59:0/252868523 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8be01a2b70 0x7f8be01a4f60 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f8bd800ba50 tx=0x7f8bd800be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:39:12.882 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:39:12.877+0000 7f8bdcff9700 1 -- 172.21.15.59:0/252868523 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bd800cab0 con 0x7f8be01a2b70 2023-12-11T17:39:12.882 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:39:12.877+0000 7f8bdcff9700 1 -- 172.21.15.59:0/252868523 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8bd800cc10 con 0x7f8be01a2b70 2023-12-11T17:39:12.882 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:39:12.877+0000 7f8bdcff9700 1 -- 172.21.15.59:0/252868523 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bd80075b0 con 0x7f8be01a2b70 2023-12-11T17:39:12.883 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:39:12.877+0000 7f8be5bd6700 1 -- 172.21.15.59:0/252868523 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8be01a5b10 con 0x7f8be01a2b70 2023-12-11T17:39:12.883 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:39:12.877+0000 7f8be5bd6700 1 -- 172.21.15.59:0/252868523 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8be01a5fc0 con 0x7f8be01a2b70 2023-12-11T17:39:12.883 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:39:12.878+0000 7f8bdcff9700 1 -- 172.21.15.59:0/252868523 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 13) v1 ==== 45636+0+0 (secure 0 0 0) 0x7f8bd8007710 con 0x7f8be01a2b70 2023-12-11T17:39:12.883 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:39:12.878+0000 7f8bdcff9700 1 --2- 172.21.15.59:0/252868523 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f8bc8038d90 0x7f8bc803b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:39:12.883 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:39:12.878+0000 7f8bdcff9700 1 -- 172.21.15.59:0/252868523 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f8bd804bbd0 con 0x7f8be01a2b70 2023-12-11T17:39:12.883 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:39:12.878+0000 7f8be5bd6700 1 -- 172.21.15.59:0/252868523 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8be0101520 con 0x7f8be01a2b70 2023-12-11T17:39:12.883 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:39:12.878+0000 7f8bdf7fe700 1 --2- 172.21.15.59:0/252868523 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f8bc8038d90 0x7f8bc803b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:39:12.883 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:39:12.878+0000 7f8bdf7fe700 1 --2- 172.21.15.59:0/252868523 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f8bc8038d90 0x7f8bc803b250 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8bd0006010 tx=0x7f8bd000b540 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:39:12.885 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:39:12.883+0000 7f8bdcff9700 1 -- 172.21.15.59:0/252868523 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f8bd801b8b0 con 0x7f8be01a2b70 2023-12-11T17:39:13.108 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:39:13.105+0000 7f8be5bd6700 1 -- 172.21.15.59:0/252868523 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f8be00fe8e0 con 0x7f8be01a2b70 2023-12-11T17:39:13.108 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:39:13.105+0000 7f8bdcff9700 1 -- 172.21.15.59:0/252868523 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) v1 ==== 76+0+275 (secure 0 0 0) 0x7f8bd801b2c0 con 0x7f8be01a2b70 2023-12-11T17:39:13.108 INFO:teuthology.orchestra.run.smithi059.stdout:# minimal ceph.conf for d0a07eaa-984b-11ee-95a3-87774f69a715 2023-12-11T17:39:13.108 INFO:teuthology.orchestra.run.smithi059.stdout:[global] 2023-12-11T17:39:13.108 INFO:teuthology.orchestra.run.smithi059.stdout: fsid = d0a07eaa-984b-11ee-95a3-87774f69a715 2023-12-11T17:39:13.109 INFO:teuthology.orchestra.run.smithi059.stdout: mon_host = [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] 2023-12-11T17:39:13.110 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:39:13.107+0000 7f8be5bd6700 1 -- 172.21.15.59:0/252868523 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f8bc8038d90 msgr2=0x7f8bc803b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:39:13.110 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:39:13.107+0000 7f8be5bd6700 1 --2- 172.21.15.59:0/252868523 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f8bc8038d90 0x7f8bc803b250 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8bd0006010 tx=0x7f8bd000b540 comp rx=0 tx=0).stop 2023-12-11T17:39:13.110 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:39:13.107+0000 7f8be5bd6700 1 -- 172.21.15.59:0/252868523 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8be01a2b70 msgr2=0x7f8be01a4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:39:13.110 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:39:13.107+0000 7f8be5bd6700 1 --2- 172.21.15.59:0/252868523 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8be01a2b70 0x7f8be01a4f60 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f8bd800ba50 tx=0x7f8bd800be10 comp rx=0 tx=0).stop 2023-12-11T17:39:13.110 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:39:13.107+0000 7f8be5bd6700 1 -- 172.21.15.59:0/252868523 shutdown_connections 2023-12-11T17:39:13.110 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:39:13.107+0000 7f8be5bd6700 1 --2- 172.21.15.59:0/252868523 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f8bc8038d90 0x7f8bc803b250 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:39:13.110 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:39:13.107+0000 7f8be5bd6700 1 --2- 172.21.15.59:0/252868523 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8be01a2b70 0x7f8be01a4f60 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:39:13.110 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:39:13.107+0000 7f8be5bd6700 1 --2- 172.21.15.59:0/252868523 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8be0105e70 0x7f8be00fd400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:39:13.110 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:39:13.107+0000 7f8be5bd6700 1 --2- 172.21.15.59:0/252868523 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8be00fd940 0x7f8be00fddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:39:13.110 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:39:13.107+0000 7f8be5bd6700 1 -- 172.21.15.59:0/252868523 >> 172.21.15.59:0/252868523 conn(0x7f8be0075930 msgr2=0x7f8be018f6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:39:13.111 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:39:13.107+0000 7f8be5bd6700 1 -- 172.21.15.59:0/252868523 shutdown_connections 2023-12-11T17:39:13.111 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:39:13.107+0000 7f8be5bd6700 1 -- 172.21.15.59:0/252868523 wait complete. 2023-12-11T17:39:13.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:12 smithi115 ceph-mon[102748]: mon.b calling monitor election 2023-12-11T17:39:13.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:12 smithi115 ceph-mon[102748]: mon.b calling monitor election 2023-12-11T17:39:13.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:12 smithi115 ceph-mon[102748]: mon.c calling monitor election 2023-12-11T17:39:13.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:12 smithi115 ceph-mon[102748]: mon.a calling monitor election 2023-12-11T17:39:13.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:12 smithi115 ceph-mon[102748]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-12-11T17:39:13.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:12 smithi115 ceph-mon[102748]: monmap e3: 3 mons at {a=[v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0],b=[v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0],c=[v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0]} removed_ranks: {} disallowed_leaders: {} 2023-12-11T17:39:13.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:12 smithi115 ceph-mon[102748]: fsmap 2023-12-11T17:39:13.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:12 smithi115 ceph-mon[102748]: osdmap e4: 0 total, 0 up, 0 in 2023-12-11T17:39:13.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:12 smithi115 ceph-mon[102748]: mgrmap e13: x(active, since 58s) 2023-12-11T17:39:13.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:12 smithi115 ceph-mon[102748]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-12-11T17:39:13.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:12 smithi115 ceph-mon[102748]: Cluster is now healthy 2023-12-11T17:39:13.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:12 smithi115 ceph-mon[102748]: overall HEALTH_OK 2023-12-11T17:39:13.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:12 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:13.284 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:12 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:13.284 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:12 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:13.284 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:12 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:13.284 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:12 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:13.284 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:12 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:13.284 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:12 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:13.284 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:12 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:13.284 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:12 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:13.284 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:12 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:13.284 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:12 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:13.284 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:12 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-12-11T17:39:13.284 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:12 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-12-11T17:39:13.284 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:12 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:39:13.307 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:12 smithi059 ceph-mon[98847]: mon.b calling monitor election 2023-12-11T17:39:13.307 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:12 smithi059 ceph-mon[98847]: mon.b calling monitor election 2023-12-11T17:39:13.307 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:12 smithi059 ceph-mon[98847]: mon.c calling monitor election 2023-12-11T17:39:13.307 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:12 smithi059 ceph-mon[98847]: mon.a calling monitor election 2023-12-11T17:39:13.308 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:12 smithi059 ceph-mon[98847]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-12-11T17:39:13.308 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:12 smithi059 ceph-mon[98847]: monmap e3: 3 mons at {a=[v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0],b=[v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0],c=[v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0]} removed_ranks: {} disallowed_leaders: {} 2023-12-11T17:39:13.308 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:12 smithi059 ceph-mon[98847]: fsmap 2023-12-11T17:39:13.308 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:12 smithi059 ceph-mon[98847]: osdmap e4: 0 total, 0 up, 0 in 2023-12-11T17:39:13.308 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:12 smithi059 ceph-mon[98847]: mgrmap e13: x(active, since 58s) 2023-12-11T17:39:13.308 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:12 smithi059 ceph-mon[98847]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-12-11T17:39:13.308 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:12 smithi059 ceph-mon[98847]: Cluster is now healthy 2023-12-11T17:39:13.308 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:12 smithi059 ceph-mon[98847]: overall HEALTH_OK 2023-12-11T17:39:13.308 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:12 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:13.308 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:12 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:13.308 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:12 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:13.308 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:12 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:13.308 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:12 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:13.308 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:12 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:13.309 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:12 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:13.309 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:12 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:13.309 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:12 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:13.309 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:12 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:13.309 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:12 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:13.309 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:12 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-12-11T17:39:13.309 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:12 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-12-11T17:39:13.309 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:12 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:39:13.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:12 smithi181 ceph-mon[103627]: mon.b calling monitor election 2023-12-11T17:39:13.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:12 smithi181 ceph-mon[103627]: mon.b calling monitor election 2023-12-11T17:39:13.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:12 smithi181 ceph-mon[103627]: mon.c calling monitor election 2023-12-11T17:39:13.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:12 smithi181 ceph-mon[103627]: mon.a calling monitor election 2023-12-11T17:39:13.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:12 smithi181 ceph-mon[103627]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-12-11T17:39:13.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:12 smithi181 ceph-mon[103627]: monmap e3: 3 mons at {a=[v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0],b=[v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0],c=[v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0]} removed_ranks: {} disallowed_leaders: {} 2023-12-11T17:39:13.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:12 smithi181 ceph-mon[103627]: fsmap 2023-12-11T17:39:13.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:12 smithi181 ceph-mon[103627]: osdmap e4: 0 total, 0 up, 0 in 2023-12-11T17:39:13.396 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:12 smithi181 ceph-mon[103627]: mgrmap e13: x(active, since 58s) 2023-12-11T17:39:13.396 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:12 smithi181 ceph-mon[103627]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-12-11T17:39:13.396 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:12 smithi181 ceph-mon[103627]: Cluster is now healthy 2023-12-11T17:39:13.396 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:12 smithi181 ceph-mon[103627]: overall HEALTH_OK 2023-12-11T17:39:13.396 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:12 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:13.396 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:12 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:13.396 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:12 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:13.396 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:12 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:13.396 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:12 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:13.396 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:12 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:13.396 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:12 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:13.396 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:12 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:13.396 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:12 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:13.396 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:12 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:13.396 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:12 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:13.396 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:12 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-12-11T17:39:13.397 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:12 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-12-11T17:39:13.397 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:12 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:39:13.707 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2023-12-11T17:39:13.707 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2023-12-11T17:39:13.707 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd of=/etc/ceph/ceph.conf 2023-12-11T17:39:13.738 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2023-12-11T17:39:13.738 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-12-11T17:39:13.807 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-12-11T17:39:13.807 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/etc/ceph/ceph.conf 2023-12-11T17:39:13.843 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-12-11T17:39:13.843 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-12-11T17:39:13.917 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-12-11T17:39:13.917 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/etc/ceph/ceph.conf 2023-12-11T17:39:13.954 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-12-11T17:39:13.954 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-12-11T17:39:14.026 INFO:tasks.cephadm:Adding mgr.x on smithi059 2023-12-11T17:39:14.026 INFO:tasks.cephadm:Adding mgr.y on smithi115 2023-12-11T17:39:14.027 INFO:tasks.cephadm:Adding mgr.z on smithi181 2023-12-11T17:39:14.027 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d0a07eaa-984b-11ee-95a3-87774f69a715 -- ceph orch apply mgr '3;smithi059=x;smithi115=y;smithi181=z' 2023-12-11T17:39:14.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:13 smithi115 ceph-mon[102748]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T17:39:14.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:13 smithi115 ceph-mon[102748]: Updating smithi059:/var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/config/ceph.conf 2023-12-11T17:39:14.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:13 smithi115 ceph-mon[102748]: Updating smithi181:/var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/config/ceph.conf 2023-12-11T17:39:14.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:13 smithi115 ceph-mon[102748]: Updating smithi115:/var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/config/ceph.conf 2023-12-11T17:39:14.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:13 smithi115 ceph-mon[102748]: Reconfiguring mon.a (unknown last config time)... 2023-12-11T17:39:14.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:13 smithi115 ceph-mon[102748]: Reconfiguring daemon mon.a on smithi059 2023-12-11T17:39:14.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:13 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/252868523' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:39:14.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:13 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-11T17:39:14.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:13 smithi181 ceph-mon[103627]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T17:39:14.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:13 smithi181 ceph-mon[103627]: Updating smithi059:/var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/config/ceph.conf 2023-12-11T17:39:14.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:13 smithi181 ceph-mon[103627]: Updating smithi181:/var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/config/ceph.conf 2023-12-11T17:39:14.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:13 smithi181 ceph-mon[103627]: Updating smithi115:/var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/config/ceph.conf 2023-12-11T17:39:14.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:13 smithi181 ceph-mon[103627]: Reconfiguring mon.a (unknown last config time)... 2023-12-11T17:39:14.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:13 smithi181 ceph-mon[103627]: Reconfiguring daemon mon.a on smithi059 2023-12-11T17:39:14.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:13 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/252868523' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:39:14.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:13 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-11T17:39:14.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:13 smithi059 ceph-mon[98847]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T17:39:14.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:13 smithi059 ceph-mon[98847]: Updating smithi059:/var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/config/ceph.conf 2023-12-11T17:39:14.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:13 smithi059 ceph-mon[98847]: Updating smithi181:/var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/config/ceph.conf 2023-12-11T17:39:14.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:13 smithi059 ceph-mon[98847]: Updating smithi115:/var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/config/ceph.conf 2023-12-11T17:39:14.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:13 smithi059 ceph-mon[98847]: Reconfiguring mon.a (unknown last config time)... 2023-12-11T17:39:14.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:13 smithi059 ceph-mon[98847]: Reconfiguring daemon mon.a on smithi059 2023-12-11T17:39:14.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:13 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/252868523' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:39:14.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:13 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-11T17:39:14.658 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/mon.c/config 2023-12-11T17:39:15.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:14 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:15.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:14 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:15.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:14 smithi181 ceph-mon[103627]: Reconfiguring mon.b (monmap changed)... 2023-12-11T17:39:15.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:14 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-12-11T17:39:15.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:14 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-12-11T17:39:15.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:14 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:39:15.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:14 smithi181 ceph-mon[103627]: Reconfiguring daemon mon.b on smithi115 2023-12-11T17:39:15.146 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:14 smithi181 ceph-mon[103627]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T17:39:15.146 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:14 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:15.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:14 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:15.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:14 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:15.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:14 smithi115 ceph-mon[102748]: Reconfiguring mon.b (monmap changed)... 2023-12-11T17:39:15.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:14 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-12-11T17:39:15.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:14 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-12-11T17:39:15.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:14 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:39:15.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:14 smithi115 ceph-mon[102748]: Reconfiguring daemon mon.b on smithi115 2023-12-11T17:39:15.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:14 smithi115 ceph-mon[102748]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T17:39:15.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:14 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:15.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:14 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:15.406 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:14 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:15.406 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:14 smithi059 ceph-mon[98847]: Reconfiguring mon.b (monmap changed)... 2023-12-11T17:39:15.406 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:14 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-12-11T17:39:15.406 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:14 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-12-11T17:39:15.406 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:14 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:39:15.406 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:14 smithi059 ceph-mon[98847]: Reconfiguring daemon mon.b on smithi115 2023-12-11T17:39:15.406 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:14 smithi059 ceph-mon[98847]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T17:39:15.407 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:14 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:15.846 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:15.842+0000 7f71a06a6700 1 -- 172.21.15.181:0/756197379 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f71980fff80 msgr2=0x7f71981003a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:39:15.846 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:15.842+0000 7f71a06a6700 1 --2- 172.21.15.181:0/756197379 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f71980fff80 0x7f71981003a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7194009910 tx=0x7f7194009c20 comp rx=0 tx=0).stop 2023-12-11T17:39:15.846 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:15.842+0000 7f71a06a6700 1 -- 172.21.15.181:0/756197379 shutdown_connections 2023-12-11T17:39:15.846 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:15.842+0000 7f71a06a6700 1 --2- 172.21.15.181:0/756197379 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f71980fff80 0x7f71981003a0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:39:15.846 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:15.842+0000 7f71a06a6700 1 -- 172.21.15.181:0/756197379 >> 172.21.15.181:0/756197379 conn(0x7f71980fb540 msgr2=0x7f71980fd960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:39:15.847 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:15.842+0000 7f71a06a6700 1 -- 172.21.15.181:0/756197379 shutdown_connections 2023-12-11T17:39:15.847 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:15.842+0000 7f71a06a6700 1 -- 172.21.15.181:0/756197379 wait complete. 2023-12-11T17:39:15.847 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:15.843+0000 7f71a06a6700 1 Processor -- start 2023-12-11T17:39:15.847 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:15.843+0000 7f71a06a6700 1 -- start start 2023-12-11T17:39:15.847 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:15.843+0000 7f71a06a6700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f71980fff80 0x7f7198069fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:39:15.847 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:15.843+0000 7f71a06a6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f71981992a0 0x7f719819b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:39:15.847 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:15.843+0000 7f71a06a6700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f719819bbd0 0x7f719819e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:39:15.847 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:15.843+0000 7f71a06a6700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f719819e800 con 0x7f71980fff80 2023-12-11T17:39:15.847 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:15.843+0000 7f71a06a6700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f719819e970 con 0x7f719819bbd0 2023-12-11T17:39:15.847 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:15.843+0000 7f71a06a6700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f719819eb30 con 0x7f71981992a0 2023-12-11T17:39:15.847 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:15.843+0000 7f719e442700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f71980fff80 0x7f7198069fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:39:15.847 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:15.843+0000 7f719dc41700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f71981992a0 0x7f719819b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:39:15.847 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:15.843+0000 7f719e442700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f71980fff80 0x7f7198069fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.181:37284/0 (socket says 172.21.15.181:37284) 2023-12-11T17:39:15.847 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:15.843+0000 7f719e442700 1 -- 172.21.15.181:0/3106827584 learned_addr learned my addr 172.21.15.181:0/3106827584 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-12-11T17:39:15.848 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:15.843+0000 7f719ec43700 1 --2- 172.21.15.181:0/3106827584 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f719819bbd0 0x7f719819e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:39:15.848 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:15.843+0000 7f719dc41700 1 -- 172.21.15.181:0/3106827584 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f719819bbd0 msgr2=0x7f719819e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:39:15.848 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:15.843+0000 7f719dc41700 1 --2- 172.21.15.181:0/3106827584 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f719819bbd0 0x7f719819e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:39:15.848 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:15.843+0000 7f719dc41700 1 -- 172.21.15.181:0/3106827584 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f71980fff80 msgr2=0x7f7198069fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:39:15.848 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:15.843+0000 7f719dc41700 1 --2- 172.21.15.181:0/3106827584 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f71980fff80 0x7f7198069fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:39:15.848 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:15.843+0000 7f719dc41700 1 -- 172.21.15.181:0/3106827584 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71940095f0 con 0x7f71981992a0 2023-12-11T17:39:15.848 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:15.843+0000 7f719e442700 1 --2- 172.21.15.181:0/3106827584 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f71980fff80 0x7f7198069fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T17:39:15.848 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:15.843+0000 7f719ec43700 1 --2- 172.21.15.181:0/3106827584 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f719819bbd0 0x7f719819e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T17:39:15.848 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:15.844+0000 7f719dc41700 1 --2- 172.21.15.181:0/3106827584 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f71981992a0 0x7f719819b690 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f718800b550 tx=0x7f718800b860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:39:15.848 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:15.844+0000 7f718f7fe700 1 -- 172.21.15.181:0/3106827584 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7188004230 con 0x7f71981992a0 2023-12-11T17:39:15.849 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:15.844+0000 7f718f7fe700 1 -- 172.21.15.181:0/3106827584 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7188004390 con 0x7f71981992a0 2023-12-11T17:39:15.849 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:15.844+0000 7f71a06a6700 1 -- 172.21.15.181:0/3106827584 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f719819ee10 con 0x7f71981992a0 2023-12-11T17:39:15.849 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:15.844+0000 7f718f7fe700 1 -- 172.21.15.181:0/3106827584 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7188011af0 con 0x7f71981992a0 2023-12-11T17:39:15.849 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:15.844+0000 7f71a06a6700 1 -- 172.21.15.181:0/3106827584 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f719819f300 con 0x7f71981992a0 2023-12-11T17:39:15.850 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:15.845+0000 7f718f7fe700 1 -- 172.21.15.181:0/3106827584 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 13) v1 ==== 45636+0+0 (secure 0 0 0) 0x7f7188011c50 con 0x7f71981992a0 2023-12-11T17:39:15.850 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:15.845+0000 7f71a06a6700 1 -- 172.21.15.181:0/3106827584 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f719818ec50 con 0x7f71981992a0 2023-12-11T17:39:15.850 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:15.845+0000 7f718f7fe700 1 --2- 172.21.15.181:0/3106827584 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f7184038d40 0x7f718403b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:39:15.850 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:15.845+0000 7f718f7fe700 1 -- 172.21.15.181:0/3106827584 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f718804ae90 con 0x7f71981992a0 2023-12-11T17:39:15.850 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:15.845+0000 7f719e442700 1 --2- 172.21.15.181:0/3106827584 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f7184038d40 0x7f718403b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:39:15.850 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:15.846+0000 7f719e442700 1 --2- 172.21.15.181:0/3106827584 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f7184038d40 0x7f718403b200 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7194005850 tx=0x7f7194005200 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:39:15.852 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:15.850+0000 7f718f7fe700 1 -- 172.21.15.181:0/3106827584 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f718801ae40 con 0x7f71981992a0 2023-12-11T17:39:16.069 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:16.066+0000 7f71a06a6700 1 -- 172.21.15.181:0/3106827584 --> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi059=x;smithi115=y;smithi181=z", "target": ["mon-mgr", ""]}) v1 -- 0x7f719802d480 con 0x7f7184038d40 2023-12-11T17:39:16.076 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:16.073+0000 7f718f7fe700 1 -- 172.21.15.181:0/3106827584 <== mgr.14150 v2:172.21.15.59:6800/3627728353 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f719802d480 con 0x7f7184038d40 2023-12-11T17:39:16.076 INFO:teuthology.orchestra.run.smithi181.stdout:Scheduled mgr update... 2023-12-11T17:39:16.078 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:16.075+0000 7f71a06a6700 1 -- 172.21.15.181:0/3106827584 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f7184038d40 msgr2=0x7f718403b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:39:16.078 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:16.075+0000 7f71a06a6700 1 --2- 172.21.15.181:0/3106827584 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f7184038d40 0x7f718403b200 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7194005850 tx=0x7f7194005200 comp rx=0 tx=0).stop 2023-12-11T17:39:16.078 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:16.075+0000 7f71a06a6700 1 -- 172.21.15.181:0/3106827584 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f71981992a0 msgr2=0x7f719819b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:39:16.078 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:16.075+0000 7f71a06a6700 1 --2- 172.21.15.181:0/3106827584 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f71981992a0 0x7f719819b690 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f718800b550 tx=0x7f718800b860 comp rx=0 tx=0).stop 2023-12-11T17:39:16.078 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:16.075+0000 7f71a06a6700 1 -- 172.21.15.181:0/3106827584 shutdown_connections 2023-12-11T17:39:16.078 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:16.075+0000 7f71a06a6700 1 --2- 172.21.15.181:0/3106827584 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f7184038d40 0x7f718403b200 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:39:16.078 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:16.075+0000 7f71a06a6700 1 --2- 172.21.15.181:0/3106827584 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f719819bbd0 0x7f719819e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:39:16.078 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:16.075+0000 7f71a06a6700 1 --2- 172.21.15.181:0/3106827584 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f71980fff80 0x7f7198069fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:39:16.079 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:16.075+0000 7f71a06a6700 1 --2- 172.21.15.181:0/3106827584 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f71981992a0 0x7f719819b690 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:39:16.079 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:16.075+0000 7f71a06a6700 1 -- 172.21.15.181:0/3106827584 >> 172.21.15.181:0/3106827584 conn(0x7f71980fb540 msgr2=0x7f71980fc150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:39:16.079 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:16.076+0000 7f71a06a6700 1 -- 172.21.15.181:0/3106827584 shutdown_connections 2023-12-11T17:39:16.079 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:39:16.076+0000 7f71a06a6700 1 -- 172.21.15.181:0/3106827584 wait complete. 2023-12-11T17:39:16.222 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:16 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:16.222 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:16 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:16.222 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:16 smithi181 ceph-mon[103627]: Reconfiguring mon.c (monmap changed)... 2023-12-11T17:39:16.222 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:16 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-12-11T17:39:16.222 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:16 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-12-11T17:39:16.222 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:16 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:39:16.223 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:16 smithi181 ceph-mon[103627]: Reconfiguring daemon mon.c on smithi181 2023-12-11T17:39:16.223 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:16 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:16.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:16 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:16.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:16 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:16.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:16 smithi115 ceph-mon[102748]: Reconfiguring mon.c (monmap changed)... 2023-12-11T17:39:16.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:16 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-12-11T17:39:16.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:16 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-12-11T17:39:16.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:16 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:39:16.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:16 smithi115 ceph-mon[102748]: Reconfiguring daemon mon.c on smithi181 2023-12-11T17:39:16.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:16 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:16.647 DEBUG:teuthology.orchestra.run.smithi115:mgr.y> sudo journalctl -f -n 0 -u ceph-d0a07eaa-984b-11ee-95a3-87774f69a715@mgr.y.service 2023-12-11T17:39:16.649 DEBUG:teuthology.orchestra.run.smithi181:mgr.z> sudo journalctl -f -n 0 -u ceph-d0a07eaa-984b-11ee-95a3-87774f69a715@mgr.z.service 2023-12-11T17:39:16.652 INFO:tasks.cephadm:Deploying OSDs... 2023-12-11T17:39:16.652 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2023-12-11T17:39:16.652 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/scratch_devs of=/dev/stdout 2023-12-11T17:39:16.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:16 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:16.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:16 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:16.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:16 smithi059 ceph-mon[98847]: Reconfiguring mon.c (monmap changed)... 2023-12-11T17:39:16.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:16 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-12-11T17:39:16.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:16 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-12-11T17:39:16.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:16 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:39:16.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:16 smithi059 ceph-mon[98847]: Reconfiguring daemon mon.c on smithi181 2023-12-11T17:39:16.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:16 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:16.673 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-12-11T17:39:16.674 DEBUG:teuthology.orchestra.run.smithi059:> stat /dev/vg_nvme/lv_1 2023-12-11T17:39:16.681 INFO:journalctl@ceph.mgr.z.smithi181.stdout:-- Logs begin at Mon 2023-12-11 17:22:43 UTC. -- 2023-12-11T17:39:16.683 INFO:journalctl@ceph.mgr.y.smithi115.stdout:-- Logs begin at Mon 2023-12-11 17:22:43 UTC. -- 2023-12-11T17:39:16.734 INFO:teuthology.orchestra.run.smithi059.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-12-11T17:39:16.734 INFO:teuthology.orchestra.run.smithi059.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-11T17:39:16.734 INFO:teuthology.orchestra.run.smithi059.stdout:Device: 6h/6d Inode: 393272 Links: 1 2023-12-11T17:39:16.734 INFO:teuthology.orchestra.run.smithi059.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-11T17:39:16.734 INFO:teuthology.orchestra.run.smithi059.stdout:Context: system_u:object_r:device_t:s0 2023-12-11T17:39:16.734 INFO:teuthology.orchestra.run.smithi059.stdout:Access: 2023-12-11 17:38:25.456045674 +0000 2023-12-11T17:39:16.735 INFO:teuthology.orchestra.run.smithi059.stdout:Modify: 2023-12-11 17:32:49.370336273 +0000 2023-12-11T17:39:16.735 INFO:teuthology.orchestra.run.smithi059.stdout:Change: 2023-12-11 17:32:49.370336273 +0000 2023-12-11T17:39:16.735 INFO:teuthology.orchestra.run.smithi059.stdout: Birth: - 2023-12-11T17:39:16.735 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-12-11T17:39:16.803 INFO:teuthology.orchestra.run.smithi059.stderr:1+0 records in 2023-12-11T17:39:16.803 INFO:teuthology.orchestra.run.smithi059.stderr:1+0 records out 2023-12-11T17:39:16.803 INFO:teuthology.orchestra.run.smithi059.stderr:512 bytes copied, 0.000245459 s, 2.1 MB/s 2023-12-11T17:39:16.804 DEBUG:teuthology.orchestra.run.smithi059:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-12-11T17:39:16.864 DEBUG:teuthology.orchestra.run.smithi059:> stat /dev/vg_nvme/lv_2 2023-12-11T17:39:16.922 INFO:teuthology.orchestra.run.smithi059.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-12-11T17:39:16.923 INFO:teuthology.orchestra.run.smithi059.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-11T17:39:16.923 INFO:teuthology.orchestra.run.smithi059.stdout:Device: 6h/6d Inode: 385761 Links: 1 2023-12-11T17:39:16.923 INFO:teuthology.orchestra.run.smithi059.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-11T17:39:16.923 INFO:teuthology.orchestra.run.smithi059.stdout:Context: system_u:object_r:device_t:s0 2023-12-11T17:39:16.923 INFO:teuthology.orchestra.run.smithi059.stdout:Access: 2023-12-11 17:38:25.504045031 +0000 2023-12-11T17:39:16.923 INFO:teuthology.orchestra.run.smithi059.stdout:Modify: 2023-12-11 17:32:49.887329442 +0000 2023-12-11T17:39:16.923 INFO:teuthology.orchestra.run.smithi059.stdout:Change: 2023-12-11 17:32:49.887329442 +0000 2023-12-11T17:39:16.923 INFO:teuthology.orchestra.run.smithi059.stdout: Birth: - 2023-12-11T17:39:16.923 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-12-11T17:39:16.990 INFO:teuthology.orchestra.run.smithi059.stderr:1+0 records in 2023-12-11T17:39:16.990 INFO:teuthology.orchestra.run.smithi059.stderr:1+0 records out 2023-12-11T17:39:16.990 INFO:teuthology.orchestra.run.smithi059.stderr:512 bytes copied, 0.000223519 s, 2.3 MB/s 2023-12-11T17:39:16.991 DEBUG:teuthology.orchestra.run.smithi059:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-12-11T17:39:17.049 DEBUG:teuthology.orchestra.run.smithi059:> stat /dev/vg_nvme/lv_3 2023-12-11T17:39:17.108 INFO:teuthology.orchestra.run.smithi059.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-12-11T17:39:17.108 INFO:teuthology.orchestra.run.smithi059.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-11T17:39:17.109 INFO:teuthology.orchestra.run.smithi059.stdout:Device: 6h/6d Inode: 394430 Links: 1 2023-12-11T17:39:17.109 INFO:teuthology.orchestra.run.smithi059.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-11T17:39:17.109 INFO:teuthology.orchestra.run.smithi059.stdout:Context: system_u:object_r:device_t:s0 2023-12-11T17:39:17.109 INFO:teuthology.orchestra.run.smithi059.stdout:Access: 2023-12-11 17:38:25.561044268 +0000 2023-12-11T17:39:17.109 INFO:teuthology.orchestra.run.smithi059.stdout:Modify: 2023-12-11 17:32:50.404322611 +0000 2023-12-11T17:39:17.109 INFO:teuthology.orchestra.run.smithi059.stdout:Change: 2023-12-11 17:32:50.404322611 +0000 2023-12-11T17:39:17.109 INFO:teuthology.orchestra.run.smithi059.stdout: Birth: - 2023-12-11T17:39:17.109 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-12-11T17:39:17.177 INFO:teuthology.orchestra.run.smithi059.stderr:1+0 records in 2023-12-11T17:39:17.177 INFO:teuthology.orchestra.run.smithi059.stderr:1+0 records out 2023-12-11T17:39:17.177 INFO:teuthology.orchestra.run.smithi059.stderr:512 bytes copied, 0.00022414 s, 2.3 MB/s 2023-12-11T17:39:17.179 DEBUG:teuthology.orchestra.run.smithi059:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-12-11T17:39:17.237 DEBUG:teuthology.orchestra.run.smithi059:> stat /dev/vg_nvme/lv_4 2023-12-11T17:39:17.295 INFO:teuthology.orchestra.run.smithi059.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-12-11T17:39:17.295 INFO:teuthology.orchestra.run.smithi059.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-11T17:39:17.295 INFO:teuthology.orchestra.run.smithi059.stdout:Device: 6h/6d Inode: 394527 Links: 1 2023-12-11T17:39:17.295 INFO:teuthology.orchestra.run.smithi059.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-11T17:39:17.295 INFO:teuthology.orchestra.run.smithi059.stdout:Context: system_u:object_r:device_t:s0 2023-12-11T17:39:17.295 INFO:teuthology.orchestra.run.smithi059.stdout:Access: 2023-12-11 17:38:25.606043665 +0000 2023-12-11T17:39:17.295 INFO:teuthology.orchestra.run.smithi059.stdout:Modify: 2023-12-11 17:32:50.923315754 +0000 2023-12-11T17:39:17.296 INFO:teuthology.orchestra.run.smithi059.stdout:Change: 2023-12-11 17:32:50.923315754 +0000 2023-12-11T17:39:17.296 INFO:teuthology.orchestra.run.smithi059.stdout: Birth: - 2023-12-11T17:39:17.296 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-12-11T17:39:17.361 INFO:teuthology.orchestra.run.smithi059.stderr:1+0 records in 2023-12-11T17:39:17.361 INFO:teuthology.orchestra.run.smithi059.stderr:1+0 records out 2023-12-11T17:39:17.361 INFO:teuthology.orchestra.run.smithi059.stderr:512 bytes copied, 0.000174263 s, 2.9 MB/s 2023-12-11T17:39:17.362 DEBUG:teuthology.orchestra.run.smithi059:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-12-11T17:39:17.421 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-12-11T17:39:17.421 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/scratch_devs of=/dev/stdout 2023-12-11T17:39:17.441 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-12-11T17:39:17.442 DEBUG:teuthology.orchestra.run.smithi115:> stat /dev/vg_nvme/lv_1 2023-12-11T17:39:17.502 INFO:teuthology.orchestra.run.smithi115.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-12-11T17:39:17.502 INFO:teuthology.orchestra.run.smithi115.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-11T17:39:17.502 INFO:teuthology.orchestra.run.smithi115.stdout:Device: 6h/6d Inode: 383313 Links: 1 2023-12-11T17:39:17.502 INFO:teuthology.orchestra.run.smithi115.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-11T17:39:17.502 INFO:teuthology.orchestra.run.smithi115.stdout:Context: system_u:object_r:device_t:s0 2023-12-11T17:39:17.502 INFO:teuthology.orchestra.run.smithi115.stdout:Access: 2023-12-11 17:39:04.638398666 +0000 2023-12-11T17:39:17.502 INFO:teuthology.orchestra.run.smithi115.stdout:Modify: 2023-12-11 17:32:27.442947439 +0000 2023-12-11T17:39:17.502 INFO:teuthology.orchestra.run.smithi115.stdout:Change: 2023-12-11 17:32:27.442947439 +0000 2023-12-11T17:39:17.502 INFO:teuthology.orchestra.run.smithi115.stdout: Birth: - 2023-12-11T17:39:17.503 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-12-11T17:39:17.571 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records in 2023-12-11T17:39:17.572 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records out 2023-12-11T17:39:17.572 INFO:teuthology.orchestra.run.smithi115.stderr:512 bytes copied, 0.000235688 s, 2.2 MB/s 2023-12-11T17:39:17.573 DEBUG:teuthology.orchestra.run.smithi115:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-12-11T17:39:17.633 DEBUG:teuthology.orchestra.run.smithi115:> stat /dev/vg_nvme/lv_2 2023-12-11T17:39:17.692 INFO:teuthology.orchestra.run.smithi115.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-12-11T17:39:17.692 INFO:teuthology.orchestra.run.smithi115.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-11T17:39:17.693 INFO:teuthology.orchestra.run.smithi115.stdout:Device: 6h/6d Inode: 383400 Links: 1 2023-12-11T17:39:17.693 INFO:teuthology.orchestra.run.smithi115.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-11T17:39:17.693 INFO:teuthology.orchestra.run.smithi115.stdout:Context: system_u:object_r:device_t:s0 2023-12-11T17:39:17.693 INFO:teuthology.orchestra.run.smithi115.stdout:Access: 2023-12-11 17:39:04.638398666 +0000 2023-12-11T17:39:17.693 INFO:teuthology.orchestra.run.smithi115.stdout:Modify: 2023-12-11 17:32:27.956937459 +0000 2023-12-11T17:39:17.693 INFO:teuthology.orchestra.run.smithi115.stdout:Change: 2023-12-11 17:32:27.956937459 +0000 2023-12-11T17:39:17.693 INFO:teuthology.orchestra.run.smithi115.stdout: Birth: - 2023-12-11T17:39:17.693 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-12-11T17:39:17.760 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records in 2023-12-11T17:39:17.760 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records out 2023-12-11T17:39:17.760 INFO:teuthology.orchestra.run.smithi115.stderr:512 bytes copied, 0.00025913 s, 2.0 MB/s 2023-12-11T17:39:17.762 DEBUG:teuthology.orchestra.run.smithi115:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-12-11T17:39:17.821 DEBUG:teuthology.orchestra.run.smithi115:> stat /dev/vg_nvme/lv_3 2023-12-11T17:39:17.880 INFO:teuthology.orchestra.run.smithi115.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-12-11T17:39:17.880 INFO:teuthology.orchestra.run.smithi115.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-11T17:39:17.880 INFO:teuthology.orchestra.run.smithi115.stdout:Device: 6h/6d Inode: 386855 Links: 1 2023-12-11T17:39:17.880 INFO:teuthology.orchestra.run.smithi115.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-11T17:39:17.881 INFO:teuthology.orchestra.run.smithi115.stdout:Context: system_u:object_r:device_t:s0 2023-12-11T17:39:17.881 INFO:teuthology.orchestra.run.smithi115.stdout:Access: 2023-12-11 17:39:04.638398666 +0000 2023-12-11T17:39:17.881 INFO:teuthology.orchestra.run.smithi115.stdout:Modify: 2023-12-11 17:32:28.453927809 +0000 2023-12-11T17:39:17.881 INFO:teuthology.orchestra.run.smithi115.stdout:Change: 2023-12-11 17:32:28.453927809 +0000 2023-12-11T17:39:17.881 INFO:teuthology.orchestra.run.smithi115.stdout: Birth: - 2023-12-11T17:39:17.881 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-12-11T17:39:17.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:17 smithi059 ceph-mon[98847]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T17:39:17.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:17 smithi059 ceph-mon[98847]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi059=x;smithi115=y;smithi181=z", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T17:39:17.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:17 smithi059 ceph-mon[98847]: Saving service mgr spec with placement smithi059=x;smithi115=y;smithi181=z;count:3 2023-12-11T17:39:17.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:17 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:17.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:17 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:17.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:17 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:39:17.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:17 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:39:17.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:17 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:39:17.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:17 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:17.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:17 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-12-11T17:39:17.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:17 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-12-11T17:39:17.906 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:17 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-12-11T17:39:17.906 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:17 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:39:17.935 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:17 smithi181 ceph-mon[103627]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T17:39:17.935 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:17 smithi181 ceph-mon[103627]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi059=x;smithi115=y;smithi181=z", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T17:39:17.936 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:17 smithi181 ceph-mon[103627]: Saving service mgr spec with placement smithi059=x;smithi115=y;smithi181=z;count:3 2023-12-11T17:39:17.936 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:17 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:17.936 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:17 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:17.936 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:17 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:39:17.936 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:17 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:39:17.936 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:17 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:39:17.936 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:17 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:17.936 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:17 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-12-11T17:39:17.936 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:17 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-12-11T17:39:17.936 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:17 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-12-11T17:39:17.936 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:17 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:39:17.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:17 smithi115 ceph-mon[102748]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T17:39:17.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:17 smithi115 ceph-mon[102748]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi059=x;smithi115=y;smithi181=z", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T17:39:17.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:17 smithi115 ceph-mon[102748]: Saving service mgr spec with placement smithi059=x;smithi115=y;smithi181=z;count:3 2023-12-11T17:39:17.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:17 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:17.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:17 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:17.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:17 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:39:17.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:17 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:39:17.946 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:17 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:39:17.947 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:17 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:17.947 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:17 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-12-11T17:39:17.947 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:17 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-12-11T17:39:17.947 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:17 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-12-11T17:39:17.947 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:17 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:39:17.948 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records in 2023-12-11T17:39:17.948 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records out 2023-12-11T17:39:17.948 INFO:teuthology.orchestra.run.smithi115.stderr:512 bytes copied, 0.000244658 s, 2.1 MB/s 2023-12-11T17:39:17.949 DEBUG:teuthology.orchestra.run.smithi115:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-12-11T17:39:18.009 DEBUG:teuthology.orchestra.run.smithi115:> stat /dev/vg_nvme/lv_4 2023-12-11T17:39:18.068 INFO:teuthology.orchestra.run.smithi115.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-12-11T17:39:18.068 INFO:teuthology.orchestra.run.smithi115.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-11T17:39:18.068 INFO:teuthology.orchestra.run.smithi115.stdout:Device: 6h/6d Inode: 392548 Links: 1 2023-12-11T17:39:18.068 INFO:teuthology.orchestra.run.smithi115.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-11T17:39:18.068 INFO:teuthology.orchestra.run.smithi115.stdout:Context: system_u:object_r:device_t:s0 2023-12-11T17:39:18.068 INFO:teuthology.orchestra.run.smithi115.stdout:Access: 2023-12-11 17:39:04.638398666 +0000 2023-12-11T17:39:18.068 INFO:teuthology.orchestra.run.smithi115.stdout:Modify: 2023-12-11 17:32:28.981917557 +0000 2023-12-11T17:39:18.068 INFO:teuthology.orchestra.run.smithi115.stdout:Change: 2023-12-11 17:32:28.981917557 +0000 2023-12-11T17:39:18.069 INFO:teuthology.orchestra.run.smithi115.stdout: Birth: - 2023-12-11T17:39:18.069 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-12-11T17:39:18.135 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records in 2023-12-11T17:39:18.136 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records out 2023-12-11T17:39:18.136 INFO:teuthology.orchestra.run.smithi115.stderr:512 bytes copied, 0.00021759 s, 2.4 MB/s 2023-12-11T17:39:18.137 DEBUG:teuthology.orchestra.run.smithi115:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-12-11T17:39:18.195 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-12-11T17:39:18.195 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/scratch_devs of=/dev/stdout 2023-12-11T17:39:18.214 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-12-11T17:39:18.214 DEBUG:teuthology.orchestra.run.smithi181:> stat /dev/vg_nvme/lv_1 2023-12-11T17:39:18.272 INFO:teuthology.orchestra.run.smithi181.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-12-11T17:39:18.272 INFO:teuthology.orchestra.run.smithi181.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-11T17:39:18.272 INFO:teuthology.orchestra.run.smithi181.stdout:Device: 6h/6d Inode: 386461 Links: 1 2023-12-11T17:39:18.272 INFO:teuthology.orchestra.run.smithi181.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-11T17:39:18.272 INFO:teuthology.orchestra.run.smithi181.stdout:Context: system_u:object_r:device_t:s0 2023-12-11T17:39:18.272 INFO:teuthology.orchestra.run.smithi181.stdout:Access: 2023-12-11 17:38:56.202322324 +0000 2023-12-11T17:39:18.272 INFO:teuthology.orchestra.run.smithi181.stdout:Modify: 2023-12-11 17:32:26.562948021 +0000 2023-12-11T17:39:18.272 INFO:teuthology.orchestra.run.smithi181.stdout:Change: 2023-12-11 17:32:26.562948021 +0000 2023-12-11T17:39:18.272 INFO:teuthology.orchestra.run.smithi181.stdout: Birth: - 2023-12-11T17:39:18.272 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-12-11T17:39:18.338 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records in 2023-12-11T17:39:18.338 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records out 2023-12-11T17:39:18.338 INFO:teuthology.orchestra.run.smithi181.stderr:512 bytes copied, 0.000218094 s, 2.3 MB/s 2023-12-11T17:39:18.339 DEBUG:teuthology.orchestra.run.smithi181:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-12-11T17:39:18.397 DEBUG:teuthology.orchestra.run.smithi181:> stat /dev/vg_nvme/lv_2 2023-12-11T17:39:18.457 INFO:teuthology.orchestra.run.smithi181.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-12-11T17:39:18.457 INFO:teuthology.orchestra.run.smithi181.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-11T17:39:18.457 INFO:teuthology.orchestra.run.smithi181.stdout:Device: 6h/6d Inode: 385500 Links: 1 2023-12-11T17:39:18.457 INFO:teuthology.orchestra.run.smithi181.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-11T17:39:18.457 INFO:teuthology.orchestra.run.smithi181.stdout:Context: system_u:object_r:device_t:s0 2023-12-11T17:39:18.458 INFO:teuthology.orchestra.run.smithi181.stdout:Access: 2023-12-11 17:38:56.202322324 +0000 2023-12-11T17:39:18.458 INFO:teuthology.orchestra.run.smithi181.stdout:Modify: 2023-12-11 17:32:27.068938185 +0000 2023-12-11T17:39:18.458 INFO:teuthology.orchestra.run.smithi181.stdout:Change: 2023-12-11 17:32:27.068938185 +0000 2023-12-11T17:39:18.458 INFO:teuthology.orchestra.run.smithi181.stdout: Birth: - 2023-12-11T17:39:18.459 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-12-11T17:39:18.523 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records in 2023-12-11T17:39:18.524 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records out 2023-12-11T17:39:18.524 INFO:teuthology.orchestra.run.smithi181.stderr:512 bytes copied, 0.000237877 s, 2.2 MB/s 2023-12-11T17:39:18.525 DEBUG:teuthology.orchestra.run.smithi181:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-12-11T17:39:18.588 DEBUG:teuthology.orchestra.run.smithi181:> stat /dev/vg_nvme/lv_3 2023-12-11T17:39:18.646 INFO:teuthology.orchestra.run.smithi181.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-12-11T17:39:18.647 INFO:teuthology.orchestra.run.smithi181.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-11T17:39:18.647 INFO:teuthology.orchestra.run.smithi181.stdout:Device: 6h/6d Inode: 392971 Links: 1 2023-12-11T17:39:18.647 INFO:teuthology.orchestra.run.smithi181.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-11T17:39:18.647 INFO:teuthology.orchestra.run.smithi181.stdout:Context: system_u:object_r:device_t:s0 2023-12-11T17:39:18.647 INFO:teuthology.orchestra.run.smithi181.stdout:Access: 2023-12-11 17:39:18.597891290 +0000 2023-12-11T17:39:18.647 INFO:teuthology.orchestra.run.smithi181.stdout:Modify: 2023-12-11 17:32:27.581928213 +0000 2023-12-11T17:39:18.647 INFO:teuthology.orchestra.run.smithi181.stdout:Change: 2023-12-11 17:32:27.581928213 +0000 2023-12-11T17:39:18.647 INFO:teuthology.orchestra.run.smithi181.stdout: Birth: - 2023-12-11T17:39:18.647 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-12-11T17:39:18.777 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records in 2023-12-11T17:39:18.778 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records out 2023-12-11T17:39:18.778 INFO:teuthology.orchestra.run.smithi181.stderr:512 bytes copied, 0.000216533 s, 2.4 MB/s 2023-12-11T17:39:18.779 DEBUG:teuthology.orchestra.run.smithi181:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-12-11T17:39:18.797 DEBUG:teuthology.orchestra.run.smithi181:> stat /dev/vg_nvme/lv_4 2023-12-11T17:39:18.857 INFO:teuthology.orchestra.run.smithi181.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-12-11T17:39:18.857 INFO:teuthology.orchestra.run.smithi181.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-11T17:39:18.857 INFO:teuthology.orchestra.run.smithi181.stdout:Device: 6h/6d Inode: 390925 Links: 1 2023-12-11T17:39:18.857 INFO:teuthology.orchestra.run.smithi181.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-11T17:39:18.857 INFO:teuthology.orchestra.run.smithi181.stdout:Context: system_u:object_r:device_t:s0 2023-12-11T17:39:18.858 INFO:teuthology.orchestra.run.smithi181.stdout:Access: 2023-12-11 17:39:18.729888750 +0000 2023-12-11T17:39:18.858 INFO:teuthology.orchestra.run.smithi181.stdout:Modify: 2023-12-11 17:32:28.079918533 +0000 2023-12-11T17:39:18.858 INFO:teuthology.orchestra.run.smithi181.stdout:Change: 2023-12-11 17:32:28.079918533 +0000 2023-12-11T17:39:18.858 INFO:teuthology.orchestra.run.smithi181.stdout: Birth: - 2023-12-11T17:39:18.858 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-12-11T17:39:18.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:18 smithi181 ceph-mon[103627]: Deploying daemon mgr.z on smithi181 2023-12-11T17:39:18.895 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Dec 11 17:39:18 smithi181 systemd[1]: Starting Ceph mgr.z for d0a07eaa-984b-11ee-95a3-87774f69a715... 2023-12-11T17:39:18.924 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records in 2023-12-11T17:39:18.924 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records out 2023-12-11T17:39:18.925 INFO:teuthology.orchestra.run.smithi181.stderr:512 bytes copied, 0.000198434 s, 2.6 MB/s 2023-12-11T17:39:18.926 DEBUG:teuthology.orchestra.run.smithi181:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-12-11T17:39:18.984 INFO:tasks.cephadm:Deploying osd.0 on smithi059 with /dev/vg_nvme/lv_4... 2023-12-11T17:39:18.984 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d0a07eaa-984b-11ee-95a3-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-12-11T17:39:19.014 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:18 smithi059 ceph-mon[98847]: Deploying daemon mgr.z on smithi181 2023-12-11T17:39:19.032 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:18 smithi115 ceph-mon[102748]: Deploying daemon mgr.z on smithi181 2023-12-11T17:39:19.198 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Dec 11 17:39:19 smithi181 podman[105703]: 2023-12-11T17:39:19.598 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/mon.a/config 2023-12-11T17:39:19.621 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Dec 11 17:39:19 smithi181 ceph-mgr[105765]: -- 172.21.15.181:0/3575290614 <== mon.1 v2:172.21.15.181:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x564193f35520 con 0x564193f0a800 2023-12-11T17:39:19.622 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Dec 11 17:39:19 smithi181 bash[105703]: 650264cab3718ffb78dd3929dd7f5e65bbb695911e11d1e13ee94eb79d29abdc 2023-12-11T17:39:19.622 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Dec 11 17:39:19 smithi181 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-z[105741]: 2023-12-11T17:39:19.441+0000 7f88af704200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-11T17:39:19.622 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Dec 11 17:39:19 smithi181 systemd[1]: Started Ceph mgr.z for d0a07eaa-984b-11ee-95a3-87774f69a715. 2023-12-11T17:39:19.622 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Dec 11 17:39:19 smithi181 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-z[105741]: 2023-12-11T17:39:19.528+0000 7f88af704200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-11T17:39:19.622 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Dec 11 17:39:19 smithi181 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-z[105741]: 2023-12-11T17:39:19.619+0000 7f88af704200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-11T17:39:19.883 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Dec 11 17:39:19 smithi181 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-z[105741]: 2023-12-11T17:39:19.713+0000 7f88af704200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-11T17:39:19.883 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:19 smithi181 ceph-mon[103627]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T17:39:19.883 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:19 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:19.883 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:19 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:19.883 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:19 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:19.883 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:19 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-12-11T17:39:19.883 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:19 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-12-11T17:39:19.883 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:19 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-12-11T17:39:19.883 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:19 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:39:19.952 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:19 smithi059 ceph-mon[98847]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T17:39:19.952 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:19 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:19.952 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:19 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:19.952 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:19 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:19.952 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:19 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-12-11T17:39:19.952 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:19 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-12-11T17:39:19.952 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:19 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-12-11T17:39:19.952 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:19 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:39:20.032 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:19 smithi115 ceph-mon[102748]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T17:39:20.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:19 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:20.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:19 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:20.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:19 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:20.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:19 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-12-11T17:39:20.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:19 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-12-11T17:39:20.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:19 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-12-11T17:39:20.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:19 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:39:20.145 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Dec 11 17:39:19 smithi181 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-z[105741]: 2023-12-11T17:39:19.880+0000 7f88af704200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-11T17:39:20.555 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Dec 11 17:39:20 smithi181 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-z[105741]: 2023-12-11T17:39:20.241+0000 7f88af704200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-11T17:39:20.556 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Dec 11 17:39:20 smithi181 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-z[105741]: 2023-12-11T17:39:20.373+0000 7f88af704200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-11T17:39:20.556 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Dec 11 17:39:20 smithi181 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-z[105741]: 2023-12-11T17:39:20.460+0000 7f88af704200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-11T17:39:20.556 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Dec 11 17:39:20 smithi181 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-z[105741]: 2023-12-11T17:39:20.553+0000 7f88af704200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-11T17:39:20.894 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:20 smithi181 ceph-mon[103627]: Deploying daemon mgr.y on smithi115 2023-12-11T17:39:20.972 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:20 smithi115 ceph-mon[102748]: Deploying daemon mgr.y on smithi115 2023-12-11T17:39:21.047 INFO:teuthology.orchestra.run.smithi059.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-12-11T17:39:21.047 INFO:teuthology.orchestra.run.smithi059.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-12-11T17:39:21.047 INFO:teuthology.orchestra.run.smithi059.stderr: stderr: 10+0 records in 2023-12-11T17:39:21.047 INFO:teuthology.orchestra.run.smithi059.stderr:10+0 records out 2023-12-11T17:39:21.047 INFO:teuthology.orchestra.run.smithi059.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.014257 s, 735 MB/s 2023-12-11T17:39:21.047 INFO:teuthology.orchestra.run.smithi059.stderr:--> Zapping successful for: 2023-12-11T17:39:21.082 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:20 smithi059 ceph-mon[98847]: Deploying daemon mgr.y on smithi115 2023-12-11T17:39:21.283 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Dec 11 17:39:21 smithi115 systemd[1]: Starting Ceph mgr.y for d0a07eaa-984b-11ee-95a3-87774f69a715... 2023-12-11T17:39:21.365 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d0a07eaa-984b-11ee-95a3-87774f69a715 -- ceph orch daemon add osd smithi059:vg_nvme/lv_4 2023-12-11T17:39:21.395 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Dec 11 17:39:20 smithi181 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-z[105741]: 2023-12-11T17:39:20.970+0000 7f88af704200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-11T17:39:21.716 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:21 smithi115 ceph-mon[102748]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T17:39:21.717 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Dec 11 17:39:21 smithi115 podman[104185]: 2023-12-11T17:39:21.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:21 smithi181 ceph-mon[103627]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T17:39:21.895 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Dec 11 17:39:21 smithi181 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-z[105741]: 2023-12-11T17:39:21.631+0000 7f88af704200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-11T17:39:21.895 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Dec 11 17:39:21 smithi181 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-z[105741]: 2023-12-11T17:39:21.713+0000 7f88af704200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-11T17:39:22.010 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/mon.a/config 2023-12-11T17:39:22.015 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Dec 11 17:39:21 smithi115 bash[104185]: ac4263c0c699b01e2a8b7ef25729120635e30662a06bef51d460cea22092eb7b 2023-12-11T17:39:22.015 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Dec 11 17:39:21 smithi115 ceph-mgr[104226]: -- 172.21.15.115:0/2949437787 <== mon.2 v2:172.21.15.115:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x563aa9df3520 con 0x563aa9dc8c00 2023-12-11T17:39:22.015 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Dec 11 17:39:21 smithi115 systemd[1]: Started Ceph mgr.y for d0a07eaa-984b-11ee-95a3-87774f69a715. 2023-12-11T17:39:22.015 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Dec 11 17:39:21 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-y[104201]: 2023-12-11T17:39:21.839+0000 7fbb42dd6200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-11T17:39:22.015 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Dec 11 17:39:21 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-y[104201]: 2023-12-11T17:39:21.918+0000 7fbb42dd6200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-11T17:39:22.015 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Dec 11 17:39:22 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-y[104201]: 2023-12-11T17:39:22.013+0000 7fbb42dd6200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-11T17:39:22.155 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:21 smithi059 ceph-mon[98847]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T17:39:22.282 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Dec 11 17:39:22 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-y[104201]: 2023-12-11T17:39:22.110+0000 7fbb42dd6200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-11T17:39:22.680 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Dec 11 17:39:22 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-y[104201]: 2023-12-11T17:39:22.290+0000 7fbb42dd6200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-11T17:39:22.824 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Dec 11 17:39:22 smithi181 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-z[105741]: 2023-12-11T17:39:22.533+0000 7f88af704200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-11T17:39:22.824 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Dec 11 17:39:22 smithi181 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-z[105741]: 2023-12-11T17:39:22.635+0000 7f88af704200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-11T17:39:22.824 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Dec 11 17:39:22 smithi181 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-z[105741]: 2023-12-11T17:39:22.821+0000 7f88af704200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-11T17:39:23.002 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:22 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:23.002 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:22 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:23.002 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:22 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:23.002 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:22 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:23.003 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:22 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:39:23.003 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Dec 11 17:39:22 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-y[104201]: 2023-12-11T17:39:22.678+0000 7fbb42dd6200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-11T17:39:23.003 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Dec 11 17:39:22 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-y[104201]: 2023-12-11T17:39:22.813+0000 7fbb42dd6200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-11T17:39:23.003 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Dec 11 17:39:22 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-y[104201]: 2023-12-11T17:39:22.905+0000 7fbb42dd6200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-11T17:39:23.003 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Dec 11 17:39:23 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-y[104201]: 2023-12-11T17:39:23.000+0000 7fbb42dd6200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-11T17:39:23.105 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:39:23.100+0000 7efc8b489700 1 -- 172.21.15.59:0/3506951704 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc84100510 msgr2=0x7efc84100930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:39:23.105 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:39:23.100+0000 7efc8b489700 1 --2- 172.21.15.59:0/3506951704 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc84100510 0x7efc84100930 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7efc800099c0 tx=0x7efc80009cd0 comp rx=0 tx=0).stop 2023-12-11T17:39:23.105 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:39:23.100+0000 7efc8b489700 1 -- 172.21.15.59:0/3506951704 shutdown_connections 2023-12-11T17:39:23.105 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:39:23.100+0000 7efc8b489700 1 --2- 172.21.15.59:0/3506951704 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc841020b0 0x7efc84108960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:39:23.105 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:39:23.100+0000 7efc8b489700 1 --2- 172.21.15.59:0/3506951704 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc84100510 0x7efc84100930 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:39:23.106 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:39:23.100+0000 7efc8b489700 1 --2- 172.21.15.59:0/3506951704 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efc84101710 0x7efc84101b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:39:23.106 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:39:23.100+0000 7efc8b489700 1 -- 172.21.15.59:0/3506951704 >> 172.21.15.59:0/3506951704 conn(0x7efc840fbad0 msgr2=0x7efc840fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:39:23.106 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:39:23.100+0000 7efc8b489700 1 -- 172.21.15.59:0/3506951704 shutdown_connections 2023-12-11T17:39:23.106 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:39:23.100+0000 7efc8b489700 1 -- 172.21.15.59:0/3506951704 wait complete. 2023-12-11T17:39:23.106 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:39:23.101+0000 7efc8b489700 1 Processor -- start 2023-12-11T17:39:23.106 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:39:23.101+0000 7efc8b489700 1 -- start start 2023-12-11T17:39:23.106 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:39:23.101+0000 7efc8b489700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc84100510 0x7efc84069ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:39:23.106 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:39:23.101+0000 7efc8b489700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efc84101710 0x7efc8419c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:39:23.106 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:39:23.101+0000 7efc8b489700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc841020b0 0x7efc8419e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:39:23.106 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:39:23.101+0000 7efc8b489700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc8419f030 con 0x7efc84100510 2023-12-11T17:39:23.106 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:39:23.101+0000 7efc8b489700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc8419f1a0 con 0x7efc84101710 2023-12-11T17:39:23.106 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:39:23.101+0000 7efc8b489700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc8419f360 con 0x7efc841020b0 2023-12-11T17:39:23.106 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:39:23.101+0000 7efc88a24700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efc84101710 0x7efc8419c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:39:23.106 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:39:23.101+0000 7efc89a26700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc841020b0 0x7efc8419e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:39:23.106 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:39:23.102+0000 7efc88a24700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efc84101710 0x7efc8419c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.59:56682/0 (socket says 172.21.15.59:56682) 2023-12-11T17:39:23.107 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:39:23.102+0000 7efc88a24700 1 -- 172.21.15.59:0/1481138515 learned_addr learned my addr 172.21.15.59:0/1481138515 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:39:23.107 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:39:23.102+0000 7efc88a24700 1 -- 172.21.15.59:0/1481138515 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc841020b0 msgr2=0x7efc8419e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:39:23.107 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:39:23.102+0000 7efc88a24700 1 --2- 172.21.15.59:0/1481138515 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc841020b0 0x7efc8419e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:39:23.107 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:39:23.102+0000 7efc88a24700 1 -- 172.21.15.59:0/1481138515 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc84100510 msgr2=0x7efc84069ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:39:23.107 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:39:23.102+0000 7efc88a24700 1 --2- 172.21.15.59:0/1481138515 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc84100510 0x7efc84069ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:39:23.107 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:39:23.102+0000 7efc88a24700 1 -- 172.21.15.59:0/1481138515 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc800095f0 con 0x7efc84101710 2023-12-11T17:39:23.107 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:39:23.102+0000 7efc88a24700 1 --2- 172.21.15.59:0/1481138515 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efc84101710 0x7efc8419c250 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7efc7800e9a0 tx=0x7efc7800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:39:23.107 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:39:23.102+0000 7efc767fc700 1 -- 172.21.15.59:0/1481138515 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc78009790 con 0x7efc84101710 2023-12-11T17:39:23.107 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:39:23.102+0000 7efc767fc700 1 -- 172.21.15.59:0/1481138515 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efc7800cbd0 con 0x7efc84101710 2023-12-11T17:39:23.107 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:39:23.102+0000 7efc8b489700 1 -- 172.21.15.59:0/1481138515 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc8419f640 con 0x7efc84101710 2023-12-11T17:39:23.107 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:39:23.102+0000 7efc767fc700 1 -- 172.21.15.59:0/1481138515 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc78018680 con 0x7efc84101710 2023-12-11T17:39:23.107 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:39:23.102+0000 7efc8b489700 1 -- 172.21.15.59:0/1481138515 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc8419fb40 con 0x7efc84101710 2023-12-11T17:39:23.107 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:39:23.103+0000 7efc767fc700 1 -- 172.21.15.59:0/1481138515 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 13) v1 ==== 45636+0+0 (secure 0 0 0) 0x7efc780187e0 con 0x7efc84101710 2023-12-11T17:39:23.108 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:39:23.103+0000 7efc767fc700 1 --2- 172.21.15.59:0/1481138515 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7efc70038af0 0x7efc7003afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:39:23.108 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:39:23.103+0000 7efc767fc700 1 -- 172.21.15.59:0/1481138515 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7efc78014070 con 0x7efc84101710 2023-12-11T17:39:23.108 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:39:23.103+0000 7efc8b489700 1 -- 172.21.15.59:0/1481138515 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc84107a40 con 0x7efc84101710 2023-12-11T17:39:23.108 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:39:23.103+0000 7efc89225700 1 --2- 172.21.15.59:0/1481138515 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7efc70038af0 0x7efc7003afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:39:23.108 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:39:23.104+0000 7efc89225700 1 --2- 172.21.15.59:0/1481138515 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7efc70038af0 0x7efc7003afb0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7efc800058e0 tx=0x7efc800036f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:39:23.111 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:39:23.109+0000 7efc767fc700 1 -- 172.21.15.59:0/1481138515 <== mon.2 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7efc7801fd30 con 0x7efc84101710 2023-12-11T17:39:23.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:22 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:23.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:22 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:23.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:22 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:23.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:22 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:23.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:22 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:39:23.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:22 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:23.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:22 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:23.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:22 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:23.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:22 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:23.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:22 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:39:23.321 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:39:23.318+0000 7efc8b489700 1 -- 172.21.15.59:0/1481138515 --> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi059:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7efc84061a10 con 0x7efc70038af0 2023-12-11T17:39:23.519 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Dec 11 17:39:23 smithi181 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-z[105741]: 2023-12-11T17:39:23.263+0000 7f88af704200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-11T17:39:23.519 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Dec 11 17:39:23 smithi181 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-z[105741]: 2023-12-11T17:39:23.359+0000 7f88af704200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-11T17:39:23.519 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Dec 11 17:39:23 smithi181 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-z[105741]: 2023-12-11T17:39:23.516+0000 7f88af704200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-11T17:39:23.782 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Dec 11 17:39:23 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-y[104201]: 2023-12-11T17:39:23.446+0000 7fbb42dd6200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-11T17:39:24.145 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Dec 11 17:39:23 smithi181 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-z[105741]: 2023-12-11T17:39:23.799+0000 7f88af704200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-11T17:39:24.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:23 smithi181 ceph-mon[103627]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T17:39:24.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:23 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-11T17:39:24.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:23 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-11T17:39:24.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:23 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:39:24.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:23 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:24.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:23 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:24.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:23 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:24.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:23 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:24.146 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:23 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:39:24.146 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:23 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:39:24.146 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:23 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:24.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:23 smithi059 ceph-mon[98847]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T17:39:24.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:23 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-11T17:39:24.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:23 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-11T17:39:24.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:23 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:39:24.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:23 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:24.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:23 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:24.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:23 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:24.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:23 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:24.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:23 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:39:24.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:23 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:39:24.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:23 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:24.211 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Dec 11 17:39:24 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-y[104201]: 2023-12-11T17:39:24.127+0000 7fbb42dd6200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-11T17:39:24.211 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:23 smithi115 ceph-mon[102748]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T17:39:24.212 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:23 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-11T17:39:24.212 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:23 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-11T17:39:24.212 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:23 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:39:24.212 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:23 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:24.212 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:23 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:24.212 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:23 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:24.212 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:23 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:24.212 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:23 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:39:24.212 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:23 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:39:24.212 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:23 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:24.532 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Dec 11 17:39:24 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-y[104201]: 2023-12-11T17:39:24.208+0000 7fbb42dd6200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-11T17:39:24.827 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Dec 11 17:39:24 smithi181 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-z[105741]: 2023-12-11T17:39:24.501+0000 7f88af704200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-11T17:39:24.828 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Dec 11 17:39:24 smithi181 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-z[105741]: 2023-12-11T17:39:24.630+0000 7f88af704200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-11T17:39:25.134 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:24 smithi059 ceph-mon[98847]: from='client.24119 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi059:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T17:39:25.134 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:24 smithi059 ceph-mon[98847]: Reconfiguring mgr.x (unknown last config time)... 2023-12-11T17:39:25.134 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:24 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-12-11T17:39:25.134 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:24 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-12-11T17:39:25.135 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:24 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:39:25.135 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:24 smithi059 ceph-mon[98847]: Reconfiguring daemon mgr.x on smithi059 2023-12-11T17:39:25.135 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:24 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:25.144 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Dec 11 17:39:24 smithi181 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-z[105741]: 2023-12-11T17:39:24.825+0000 7f88af704200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-11T17:39:25.145 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Dec 11 17:39:24 smithi181 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-z[105741]: 2023-12-11T17:39:24.937+0000 7f88af704200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-11T17:39:25.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:24 smithi181 ceph-mon[103627]: from='client.24119 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi059:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T17:39:25.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:24 smithi181 ceph-mon[103627]: Reconfiguring mgr.x (unknown last config time)... 2023-12-11T17:39:25.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:24 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-12-11T17:39:25.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:24 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-12-11T17:39:25.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:24 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:39:25.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:24 smithi181 ceph-mon[103627]: Reconfiguring daemon mgr.x on smithi059 2023-12-11T17:39:25.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:24 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:25.239 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Dec 11 17:39:24 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-y[104201]: 2023-12-11T17:39:24.975+0000 7fbb42dd6200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-11T17:39:25.239 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Dec 11 17:39:25 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-y[104201]: 2023-12-11T17:39:25.065+0000 7fbb42dd6200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-11T17:39:25.239 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:24 smithi115 ceph-mon[102748]: from='client.24119 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi059:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T17:39:25.240 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:24 smithi115 ceph-mon[102748]: Reconfiguring mgr.x (unknown last config time)... 2023-12-11T17:39:25.240 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:24 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-12-11T17:39:25.240 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:24 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-12-11T17:39:25.240 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:24 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:39:25.240 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:24 smithi115 ceph-mon[102748]: Reconfiguring daemon mgr.x on smithi059 2023-12-11T17:39:25.240 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:24 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:25.532 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Dec 11 17:39:25 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-y[104201]: 2023-12-11T17:39:25.237+0000 7fbb42dd6200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-11T17:39:25.644 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Dec 11 17:39:25 smithi181 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-z[105741]: 2023-12-11T17:39:25.153+0000 7f88af704200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-11T17:39:26.033 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Dec 11 17:39:25 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-y[104201]: 2023-12-11T17:39:25.653+0000 7fbb42dd6200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-11T17:39:26.033 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Dec 11 17:39:25 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-y[104201]: 2023-12-11T17:39:25.739+0000 7fbb42dd6200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-11T17:39:26.033 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Dec 11 17:39:25 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-y[104201]: 2023-12-11T17:39:25.885+0000 7fbb42dd6200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-11T17:39:26.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:25 smithi115 ceph-mon[102748]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T17:39:26.034 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:25 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:26.034 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:25 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:26.034 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:25 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:39:26.034 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:25 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:39:26.034 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:25 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:39:26.034 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:25 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:26.145 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Dec 11 17:39:26 smithi181 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-z[105741]: 2023-12-11T17:39:26.020+0000 7f88af704200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-11T17:39:26.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:25 smithi181 ceph-mon[103627]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T17:39:26.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:25 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:26.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:25 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:26.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:25 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:39:26.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:25 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:39:26.146 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:25 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:39:26.146 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:25 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:26.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:25 smithi059 ceph-mon[98847]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T17:39:26.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:25 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:26.155 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:25 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:26.155 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:25 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:39:26.155 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:25 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:39:26.155 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:25 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:39:26.155 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:25 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:26.533 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Dec 11 17:39:26 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-y[104201]: 2023-12-11T17:39:26.146+0000 7fbb42dd6200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-11T17:39:26.892 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:39:26.889+0000 7efc767fc700 1 -- 172.21.15.59:0/1481138515 <== mon.2 v2:172.21.15.115:3300/0 7 ==== mgrmap(e 14) v1 ==== 90793+0+0 (secure 0 0 0) 0x7efc7800cd40 con 0x7efc84101710 2023-12-11T17:39:27.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:26 smithi181 ceph-mon[103627]: Standby manager daemon z started 2023-12-11T17:39:27.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:26 smithi181 ceph-mon[103627]: from='mgr.? 172.21.15.181:0/44430644' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-12-11T17:39:27.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:26 smithi181 ceph-mon[103627]: from='mgr.? 172.21.15.181:0/44430644' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-12-11T17:39:27.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:26 smithi181 ceph-mon[103627]: from='mgr.? 172.21.15.181:0/44430644' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-12-11T17:39:27.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:26 smithi181 ceph-mon[103627]: from='mgr.? 172.21.15.181:0/44430644' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-12-11T17:39:27.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:26 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/358565371' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3c0b3218-4b2b-40f8-8bef-ed8cd5633607"}]: dispatch 2023-12-11T17:39:27.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:26 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/358565371' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3c0b3218-4b2b-40f8-8bef-ed8cd5633607"}]': finished 2023-12-11T17:39:27.146 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:26 smithi181 ceph-mon[103627]: osdmap e5: 1 total, 0 up, 1 in 2023-12-11T17:39:27.146 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:26 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-11T17:39:27.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:26 smithi059 ceph-mon[98847]: Standby manager daemon z started 2023-12-11T17:39:27.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:26 smithi059 ceph-mon[98847]: from='mgr.? 172.21.15.181:0/44430644' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-12-11T17:39:27.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:26 smithi059 ceph-mon[98847]: from='mgr.? 172.21.15.181:0/44430644' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-12-11T17:39:27.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:26 smithi059 ceph-mon[98847]: from='mgr.? 172.21.15.181:0/44430644' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-12-11T17:39:27.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:26 smithi059 ceph-mon[98847]: from='mgr.? 172.21.15.181:0/44430644' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-12-11T17:39:27.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:26 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/358565371' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3c0b3218-4b2b-40f8-8bef-ed8cd5633607"}]: dispatch 2023-12-11T17:39:27.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:26 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/358565371' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3c0b3218-4b2b-40f8-8bef-ed8cd5633607"}]': finished 2023-12-11T17:39:27.155 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:26 smithi059 ceph-mon[98847]: osdmap e5: 1 total, 0 up, 1 in 2023-12-11T17:39:27.155 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:26 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-11T17:39:27.163 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Dec 11 17:39:26 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-y[104201]: 2023-12-11T17:39:26.835+0000 7fbb42dd6200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-11T17:39:27.164 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Dec 11 17:39:26 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-y[104201]: 2023-12-11T17:39:26.968+0000 7fbb42dd6200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-11T17:39:27.164 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:26 smithi115 ceph-mon[102748]: Standby manager daemon z started 2023-12-11T17:39:27.164 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:26 smithi115 ceph-mon[102748]: from='mgr.? 172.21.15.181:0/44430644' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-12-11T17:39:27.164 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:26 smithi115 ceph-mon[102748]: from='mgr.? 172.21.15.181:0/44430644' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-12-11T17:39:27.164 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:26 smithi115 ceph-mon[102748]: from='mgr.? 172.21.15.181:0/44430644' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-12-11T17:39:27.164 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:26 smithi115 ceph-mon[102748]: from='mgr.? 172.21.15.181:0/44430644' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-12-11T17:39:27.164 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:26 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/358565371' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3c0b3218-4b2b-40f8-8bef-ed8cd5633607"}]: dispatch 2023-12-11T17:39:27.164 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:26 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/358565371' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3c0b3218-4b2b-40f8-8bef-ed8cd5633607"}]': finished 2023-12-11T17:39:27.164 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:26 smithi115 ceph-mon[102748]: osdmap e5: 1 total, 0 up, 1 in 2023-12-11T17:39:27.164 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:26 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-11T17:39:27.493 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Dec 11 17:39:27 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-y[104201]: 2023-12-11T17:39:27.161+0000 7fbb42dd6200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-11T17:39:27.493 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Dec 11 17:39:27 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-y[104201]: 2023-12-11T17:39:27.274+0000 7fbb42dd6200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-11T17:39:27.782 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Dec 11 17:39:27 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-y[104201]: 2023-12-11T17:39:27.490+0000 7fbb42dd6200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-11T17:39:28.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:27 smithi181 ceph-mon[103627]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T17:39:28.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:27 smithi181 ceph-mon[103627]: mgrmap e14: x(active, since 72s), standbys: z 2023-12-11T17:39:28.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:27 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-12-11T17:39:28.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:27 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/813722418' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-11T17:39:28.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:27 smithi059 ceph-mon[98847]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T17:39:28.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:27 smithi059 ceph-mon[98847]: mgrmap e14: x(active, since 72s), standbys: z 2023-12-11T17:39:28.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:27 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-12-11T17:39:28.155 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:27 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/813722418' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-11T17:39:28.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:27 smithi115 ceph-mon[102748]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T17:39:28.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:27 smithi115 ceph-mon[102748]: mgrmap e14: x(active, since 72s), standbys: z 2023-12-11T17:39:28.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:27 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-12-11T17:39:28.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:27 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/813722418' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-11T17:39:28.782 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Dec 11 17:39:28 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-y[104201]: 2023-12-11T17:39:28.337+0000 7fbb42dd6200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-11T17:39:28.903 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:39:28.900+0000 7efc767fc700 1 -- 172.21.15.59:0/1481138515 <== mon.2 v2:172.21.15.115:3300/0 8 ==== mgrmap(e 15) v1 ==== 135950+0+0 (secure 0 0 0) 0x7efc7801cce0 con 0x7efc84101710 2023-12-11T17:39:29.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:28 smithi059 ceph-mon[98847]: Standby manager daemon y started 2023-12-11T17:39:29.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:28 smithi059 ceph-mon[98847]: from='mgr.? 172.21.15.115:0/2830210567' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-12-11T17:39:29.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:28 smithi059 ceph-mon[98847]: from='mgr.? 172.21.15.115:0/2830210567' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-12-11T17:39:29.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:28 smithi059 ceph-mon[98847]: from='mgr.? 172.21.15.115:0/2830210567' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-12-11T17:39:29.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:28 smithi059 ceph-mon[98847]: from='mgr.? 172.21.15.115:0/2830210567' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-12-11T17:39:29.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:28 smithi115 ceph-mon[102748]: Standby manager daemon y started 2023-12-11T17:39:29.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:28 smithi115 ceph-mon[102748]: from='mgr.? 172.21.15.115:0/2830210567' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-12-11T17:39:29.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:28 smithi115 ceph-mon[102748]: from='mgr.? 172.21.15.115:0/2830210567' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-12-11T17:39:29.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:28 smithi115 ceph-mon[102748]: from='mgr.? 172.21.15.115:0/2830210567' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-12-11T17:39:29.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:28 smithi115 ceph-mon[102748]: from='mgr.? 172.21.15.115:0/2830210567' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-12-11T17:39:29.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:28 smithi181 ceph-mon[103627]: Standby manager daemon y started 2023-12-11T17:39:29.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:28 smithi181 ceph-mon[103627]: from='mgr.? 172.21.15.115:0/2830210567' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-12-11T17:39:29.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:28 smithi181 ceph-mon[103627]: from='mgr.? 172.21.15.115:0/2830210567' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-12-11T17:39:29.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:28 smithi181 ceph-mon[103627]: from='mgr.? 172.21.15.115:0/2830210567' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-12-11T17:39:29.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:28 smithi181 ceph-mon[103627]: from='mgr.? 172.21.15.115:0/2830210567' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-12-11T17:39:30.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:29 smithi059 ceph-mon[98847]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T17:39:30.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:29 smithi059 ceph-mon[98847]: mgrmap e15: x(active, since 74s), standbys: z, y 2023-12-11T17:39:30.155 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:29 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-12-11T17:39:30.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:29 smithi115 ceph-mon[102748]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T17:39:30.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:29 smithi115 ceph-mon[102748]: mgrmap e15: x(active, since 74s), standbys: z, y 2023-12-11T17:39:30.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:29 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-12-11T17:39:30.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:29 smithi181 ceph-mon[103627]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T17:39:30.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:29 smithi181 ceph-mon[103627]: mgrmap e15: x(active, since 74s), standbys: z, y 2023-12-11T17:39:30.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:29 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-12-11T17:39:32.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:31 smithi059 ceph-mon[98847]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T17:39:32.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:31 smithi115 ceph-mon[102748]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T17:39:32.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:31 smithi181 ceph-mon[103627]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T17:39:34.057 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:33 smithi059 ceph-mon[98847]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T17:39:34.057 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:33 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-12-11T17:39:34.057 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:33 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:39:34.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:33 smithi115 ceph-mon[102748]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T17:39:34.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:33 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-12-11T17:39:34.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:33 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:39:34.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:33 smithi181 ceph-mon[103627]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T17:39:34.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:33 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-12-11T17:39:34.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:33 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:39:35.188 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:34 smithi059 ceph-mon[98847]: Deploying daemon osd.0 on smithi059 2023-12-11T17:39:35.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:34 smithi115 ceph-mon[102748]: Deploying daemon osd.0 on smithi059 2023-12-11T17:39:35.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:34 smithi181 ceph-mon[103627]: Deploying daemon osd.0 on smithi059 2023-12-11T17:39:36.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:35 smithi115 ceph-mon[102748]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T17:39:36.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:35 smithi181 ceph-mon[103627]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T17:39:36.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:35 smithi059 ceph-mon[98847]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T17:39:38.119 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:37 smithi059 ceph-mon[98847]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T17:39:38.119 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:37 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:38.119 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:37 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:38.119 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:37 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:39:38.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:37 smithi115 ceph-mon[102748]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T17:39:38.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:37 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:38.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:37 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:38.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:37 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:39:38.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:37 smithi181 ceph-mon[103627]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T17:39:38.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:37 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:38.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:37 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:38.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:37 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:39:40.215 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:39 smithi059 ceph-mon[98847]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T17:39:40.215 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:39 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:40.215 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:39 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:40.215 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:39 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:39:40.215 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:39 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:39:40.215 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:39 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:40.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:39 smithi115 ceph-mon[102748]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T17:39:40.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:39 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:40.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:39 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:40.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:39 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:39:40.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:39 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:39:40.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:39 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:40.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:39 smithi181 ceph-mon[103627]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T17:39:40.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:39 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:40.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:39 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:40.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:39 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:39:40.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:39 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:39:40.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:39 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:40.480 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:39:40.478+0000 7efc767fc700 1 -- 172.21.15.59:0/1481138515 <== mgr.14150 v2:172.21.15.59:6800/3627728353 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7efc84061a10 con 0x7efc70038af0 2023-12-11T17:39:40.481 INFO:teuthology.orchestra.run.smithi059.stdout:Created osd(s) 0 on host 'smithi059' 2023-12-11T17:39:40.483 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:39:40.480+0000 7efc8b489700 1 -- 172.21.15.59:0/1481138515 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7efc70038af0 msgr2=0x7efc7003afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:39:40.483 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:39:40.480+0000 7efc8b489700 1 --2- 172.21.15.59:0/1481138515 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7efc70038af0 0x7efc7003afb0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7efc800058e0 tx=0x7efc800036f0 comp rx=0 tx=0).stop 2023-12-11T17:39:40.483 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:39:40.480+0000 7efc8b489700 1 -- 172.21.15.59:0/1481138515 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efc84101710 msgr2=0x7efc8419c250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:39:40.483 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:39:40.480+0000 7efc8b489700 1 --2- 172.21.15.59:0/1481138515 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efc84101710 0x7efc8419c250 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7efc7800e9a0 tx=0x7efc7800ecb0 comp rx=0 tx=0).stop 2023-12-11T17:39:40.483 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:39:40.480+0000 7efc8b489700 1 -- 172.21.15.59:0/1481138515 shutdown_connections 2023-12-11T17:39:40.483 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:39:40.480+0000 7efc8b489700 1 --2- 172.21.15.59:0/1481138515 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7efc70038af0 0x7efc7003afb0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:39:40.483 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:39:40.480+0000 7efc8b489700 1 --2- 172.21.15.59:0/1481138515 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc841020b0 0x7efc8419e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:39:40.483 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:39:40.480+0000 7efc8b489700 1 --2- 172.21.15.59:0/1481138515 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc84100510 0x7efc84069ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:39:40.483 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:39:40.480+0000 7efc8b489700 1 --2- 172.21.15.59:0/1481138515 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efc84101710 0x7efc8419c250 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:39:40.483 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:39:40.480+0000 7efc8b489700 1 -- 172.21.15.59:0/1481138515 >> 172.21.15.59:0/1481138515 conn(0x7efc840fbad0 msgr2=0x7efc84104b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:39:40.483 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:39:40.480+0000 7efc8b489700 1 -- 172.21.15.59:0/1481138515 shutdown_connections 2023-12-11T17:39:40.483 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:39:40.480+0000 7efc8b489700 1 -- 172.21.15.59:0/1481138515 wait complete. 2023-12-11T17:39:40.801 DEBUG:teuthology.orchestra.run.smithi059:osd.0> sudo journalctl -f -n 0 -u ceph-d0a07eaa-984b-11ee-95a3-87774f69a715@osd.0.service 2023-12-11T17:39:40.804 INFO:tasks.cephadm:Deploying osd.1 on smithi115 with /dev/vg_nvme/lv_4... 2023-12-11T17:39:40.804 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d0a07eaa-984b-11ee-95a3-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-12-11T17:39:40.835 INFO:journalctl@ceph.osd.0.smithi059.stdout:-- Logs begin at Mon 2023-12-11 17:22:26 UTC. -- 2023-12-11T17:39:41.404 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/mon.b/config 2023-12-11T17:39:41.652 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:41 smithi059 ceph-mon[98847]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T17:39:41.652 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:41 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:41.652 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:41 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:41.652 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:41 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:39:41.736 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:41 smithi115 ceph-mon[102748]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T17:39:41.736 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:41 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:41.736 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:41 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:41.736 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:41 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:39:41.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:41 smithi181 ceph-mon[103627]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T17:39:41.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:41 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:41.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:41 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:41.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:41 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:39:41.904 INFO:journalctl@ceph.osd.0.smithi059.stdout:Dec 11 17:39:41 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-0[117294]: 2023-12-11T17:39:41.648+0000 7efd04d69640 -1 osd.0 0 log_to_monitors true 2023-12-11T17:39:42.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:42 smithi115 ceph-mon[102748]: from='osd.0 [v2:172.21.15.59:6802/1040759398,v1:172.21.15.59:6803/1040759398]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-12-11T17:39:42.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:42 smithi181 ceph-mon[103627]: from='osd.0 [v2:172.21.15.59:6802/1040759398,v1:172.21.15.59:6803/1040759398]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-12-11T17:39:42.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:42 smithi059 ceph-mon[98847]: from='osd.0 [v2:172.21.15.59:6802/1040759398,v1:172.21.15.59:6803/1040759398]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-12-11T17:39:42.942 INFO:teuthology.orchestra.run.smithi115.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-12-11T17:39:42.943 INFO:teuthology.orchestra.run.smithi115.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-12-11T17:39:42.943 INFO:teuthology.orchestra.run.smithi115.stderr: stderr: 10+0 records in 2023-12-11T17:39:42.943 INFO:teuthology.orchestra.run.smithi115.stderr:10+0 records out 2023-12-11T17:39:42.943 INFO:teuthology.orchestra.run.smithi115.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0146078 s, 718 MB/s 2023-12-11T17:39:42.943 INFO:teuthology.orchestra.run.smithi115.stderr: stderr: 2023-12-11T17:39:42.943 INFO:teuthology.orchestra.run.smithi115.stderr:--> Zapping successful for: 2023-12-11T17:39:43.352 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d0a07eaa-984b-11ee-95a3-87774f69a715 -- ceph orch daemon add osd smithi115:vg_nvme/lv_4 2023-12-11T17:39:43.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:43 smithi115 ceph-mon[102748]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T17:39:43.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:43 smithi115 ceph-mon[102748]: from='osd.0 [v2:172.21.15.59:6802/1040759398,v1:172.21.15.59:6803/1040759398]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-12-11T17:39:43.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:43 smithi115 ceph-mon[102748]: osdmap e6: 1 total, 0 up, 1 in 2023-12-11T17:39:43.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:43 smithi115 ceph-mon[102748]: from='osd.0 [v2:172.21.15.59:6802/1040759398,v1:172.21.15.59:6803/1040759398]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2023-12-11T17:39:43.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:43 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-11T17:39:43.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:43 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:43.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:43 smithi181 ceph-mon[103627]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T17:39:43.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:43 smithi181 ceph-mon[103627]: from='osd.0 [v2:172.21.15.59:6802/1040759398,v1:172.21.15.59:6803/1040759398]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-12-11T17:39:43.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:43 smithi181 ceph-mon[103627]: osdmap e6: 1 total, 0 up, 1 in 2023-12-11T17:39:43.896 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:43 smithi181 ceph-mon[103627]: from='osd.0 [v2:172.21.15.59:6802/1040759398,v1:172.21.15.59:6803/1040759398]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2023-12-11T17:39:43.896 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:43 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-11T17:39:43.896 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:43 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:43.904 INFO:journalctl@ceph.osd.0.smithi059.stdout:Dec 11 17:39:43 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-0[117294]: 2023-12-11T17:39:43.484+0000 7efcfa3e0700 -1 osd.0 0 waiting for initial osdmap 2023-12-11T17:39:43.904 INFO:journalctl@ceph.osd.0.smithi059.stdout:Dec 11 17:39:43 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-0[117294]: 2023-12-11T17:39:43.487+0000 7efcf39d5700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-11T17:39:43.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:43 smithi059 ceph-mon[98847]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T17:39:43.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:43 smithi059 ceph-mon[98847]: from='osd.0 [v2:172.21.15.59:6802/1040759398,v1:172.21.15.59:6803/1040759398]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-12-11T17:39:43.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:43 smithi059 ceph-mon[98847]: osdmap e6: 1 total, 0 up, 1 in 2023-12-11T17:39:43.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:43 smithi059 ceph-mon[98847]: from='osd.0 [v2:172.21.15.59:6802/1040759398,v1:172.21.15.59:6803/1040759398]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2023-12-11T17:39:43.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:43 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-11T17:39:43.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:43 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:43.942 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/mon.b/config 2023-12-11T17:39:44.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:44 smithi115 ceph-mon[102748]: Detected new or changed devices on smithi059 2023-12-11T17:39:44.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:44 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:44.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:44 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:39:44.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:44 smithi115 ceph-mon[102748]: Adjusting osd_memory_target on smithi059 to 17173M 2023-12-11T17:39:44.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:44 smithi115 ceph-mon[102748]: from='osd.0 [v2:172.21.15.59:6802/1040759398,v1:172.21.15.59:6803/1040759398]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]': finished 2023-12-11T17:39:44.784 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:44 smithi115 ceph-mon[102748]: osdmap e7: 1 total, 0 up, 1 in 2023-12-11T17:39:44.784 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:44 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-11T17:39:44.784 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:44 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:44.784 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:44 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-11T17:39:44.784 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:44 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:39:44.784 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:44 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:39:44.784 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:44 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:44.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:44 smithi181 ceph-mon[103627]: Detected new or changed devices on smithi059 2023-12-11T17:39:44.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:44 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:44.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:44 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:39:44.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:44 smithi181 ceph-mon[103627]: Adjusting osd_memory_target on smithi059 to 17173M 2023-12-11T17:39:44.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:44 smithi181 ceph-mon[103627]: from='osd.0 [v2:172.21.15.59:6802/1040759398,v1:172.21.15.59:6803/1040759398]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]': finished 2023-12-11T17:39:44.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:44 smithi181 ceph-mon[103627]: osdmap e7: 1 total, 0 up, 1 in 2023-12-11T17:39:44.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:44 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-11T17:39:44.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:44 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:44.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:44 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-11T17:39:44.896 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:44 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:39:44.896 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:44 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:39:44.896 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:44 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:44.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:44 smithi059 ceph-mon[98847]: Detected new or changed devices on smithi059 2023-12-11T17:39:44.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:44 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:44.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:44 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:39:44.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:44 smithi059 ceph-mon[98847]: Adjusting osd_memory_target on smithi059 to 17173M 2023-12-11T17:39:44.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:44 smithi059 ceph-mon[98847]: from='osd.0 [v2:172.21.15.59:6802/1040759398,v1:172.21.15.59:6803/1040759398]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]': finished 2023-12-11T17:39:44.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:44 smithi059 ceph-mon[98847]: osdmap e7: 1 total, 0 up, 1 in 2023-12-11T17:39:44.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:44 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-11T17:39:44.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:44 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:44.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:44 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-11T17:39:44.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:44 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:39:44.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:44 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:39:44.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:44 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:39:45.233 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:39:45.230+0000 7f0c9bfe0700 1 -- 172.21.15.115:0/1330230940 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c94101140 msgr2=0x7f0c941089e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:39:45.233 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:39:45.230+0000 7f0c9bfe0700 1 --2- 172.21.15.115:0/1330230940 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c94101140 0x7f0c941089e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0c8c00b0a0 tx=0x7f0c8c00b3b0 comp rx=0 tx=0).stop 2023-12-11T17:39:45.233 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:39:45.230+0000 7f0c9bfe0700 1 -- 172.21.15.115:0/1330230940 shutdown_connections 2023-12-11T17:39:45.233 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:39:45.230+0000 7f0c9bfe0700 1 --2- 172.21.15.115:0/1330230940 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c94101140 0x7f0c941089e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:39:45.233 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:39:45.230+0000 7f0c9bfe0700 1 --2- 172.21.15.115:0/1330230940 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0c94101b20 0x7f0c94101f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:39:45.234 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:39:45.230+0000 7f0c9bfe0700 1 --2- 172.21.15.115:0/1330230940 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c94100780 0x7f0c94100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:39:45.234 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:39:45.230+0000 7f0c9bfe0700 1 -- 172.21.15.115:0/1330230940 >> 172.21.15.115:0/1330230940 conn(0x7f0c940fb230 msgr2=0x7f0c940fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:39:45.234 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:39:45.230+0000 7f0c9bfe0700 1 -- 172.21.15.115:0/1330230940 shutdown_connections 2023-12-11T17:39:45.234 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:39:45.230+0000 7f0c9bfe0700 1 -- 172.21.15.115:0/1330230940 wait complete. 2023-12-11T17:39:45.240 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:39:45.231+0000 7f0c9bfe0700 1 Processor -- start 2023-12-11T17:39:45.240 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:39:45.231+0000 7f0c9bfe0700 1 -- start start 2023-12-11T17:39:45.240 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:39:45.231+0000 7f0c9bfe0700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c94100780 0x7f0c94100180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:39:45.240 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:39:45.231+0000 7f0c9bfe0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c94101140 0x7f0c9419c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:39:45.240 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:39:45.231+0000 7f0c9bfe0700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0c94101b20 0x7f0c9419e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:39:45.240 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:39:45.231+0000 7f0c9bfe0700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c9419f0b0 con 0x7f0c94101b20 2023-12-11T17:39:45.240 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:39:45.231+0000 7f0c9bfe0700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c9419f220 con 0x7f0c94100780 2023-12-11T17:39:45.240 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:39:45.231+0000 7f0c9bfe0700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c9419f3e0 con 0x7f0c94101140 2023-12-11T17:39:45.240 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:39:45.231+0000 7f0c99d7c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c94100780 0x7f0c94100180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:39:45.240 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:39:45.231+0000 7f0c99d7c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c94100780 0x7f0c94100180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:54656/0 (socket says 172.21.15.115:54656) 2023-12-11T17:39:45.241 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:39:45.231+0000 7f0c99d7c700 1 -- 172.21.15.115:0/1601348378 learned_addr learned my addr 172.21.15.115:0/1601348378 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-11T17:39:45.241 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:39:45.231+0000 7f0c9957b700 1 --2- 172.21.15.115:0/1601348378 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c94101140 0x7f0c9419c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:39:45.241 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:39:45.231+0000 7f0c9a57d700 1 --2- 172.21.15.115:0/1601348378 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0c94101b20 0x7f0c9419e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:39:45.241 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:39:45.231+0000 7f0c99d7c700 1 -- 172.21.15.115:0/1601348378 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c94101140 msgr2=0x7f0c9419c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:39:45.241 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:39:45.231+0000 7f0c99d7c700 1 --2- 172.21.15.115:0/1601348378 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c94101140 0x7f0c9419c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:39:45.241 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:39:45.231+0000 7f0c99d7c700 1 -- 172.21.15.115:0/1601348378 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0c94101b20 msgr2=0x7f0c9419e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:39:45.241 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:39:45.231+0000 7f0c99d7c700 1 --2- 172.21.15.115:0/1601348378 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0c94101b20 0x7f0c9419e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:39:45.241 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:39:45.231+0000 7f0c99d7c700 1 -- 172.21.15.115:0/1601348378 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c8c009d00 con 0x7f0c94100780 2023-12-11T17:39:45.241 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:39:45.231+0000 7f0c9957b700 1 --2- 172.21.15.115:0/1601348378 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c94101140 0x7f0c9419c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T17:39:45.241 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:39:45.231+0000 7f0c9a57d700 1 --2- 172.21.15.115:0/1601348378 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0c94101b20 0x7f0c9419e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-11T17:39:45.241 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:39:45.232+0000 7f0c99d7c700 1 --2- 172.21.15.115:0/1601348378 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c94100780 0x7f0c94100180 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f0c8800d980 tx=0x7f0c8800dd40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:39:45.241 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:39:45.232+0000 7f0c86ffd700 1 -- 172.21.15.115:0/1601348378 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c8800de60 con 0x7f0c94100780 2023-12-11T17:39:45.241 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:39:45.232+0000 7f0c86ffd700 1 -- 172.21.15.115:0/1601348378 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0c880044e0 con 0x7f0c94100780 2023-12-11T17:39:45.241 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:39:45.232+0000 7f0c9bfe0700 1 -- 172.21.15.115:0/1601348378 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c9419f6c0 con 0x7f0c94100780 2023-12-11T17:39:45.242 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:39:45.232+0000 7f0c86ffd700 1 -- 172.21.15.115:0/1601348378 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c88003be0 con 0x7f0c94100780 2023-12-11T17:39:45.242 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:39:45.232+0000 7f0c9bfe0700 1 -- 172.21.15.115:0/1601348378 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c9419fbc0 con 0x7f0c94100780 2023-12-11T17:39:45.242 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:39:45.233+0000 7f0c9bfe0700 1 -- 172.21.15.115:0/1601348378 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c94107950 con 0x7f0c94100780 2023-12-11T17:39:45.242 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:39:45.234+0000 7f0c86ffd700 1 -- 172.21.15.115:0/1601348378 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 15) v1 ==== 135950+0+0 (secure 0 0 0) 0x7f0c88009bb0 con 0x7f0c94100780 2023-12-11T17:39:45.242 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:39:45.234+0000 7f0c86ffd700 1 --2- 172.21.15.115:0/1601348378 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f0c800a13f0 0x7f0c800a38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:39:45.242 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:39:45.234+0000 7f0c86ffd700 1 -- 172.21.15.115:0/1601348378 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1767+0+0 (secure 0 0 0) 0x7f0c880cafe0 con 0x7f0c94100780 2023-12-11T17:39:45.242 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:39:45.234+0000 7f0c9957b700 1 --2- 172.21.15.115:0/1601348378 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f0c800a13f0 0x7f0c800a38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:39:45.242 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:39:45.235+0000 7f0c9957b700 1 --2- 172.21.15.115:0/1601348378 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f0c800a13f0 0x7f0c800a38b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f0c9000bcf0 tx=0x7f0c9000b2c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:39:45.242 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:39:45.239+0000 7f0c86ffd700 1 -- 172.21.15.115:0/1601348378 <== mon.2 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f0c880fe2f0 con 0x7f0c94100780 2023-12-11T17:39:45.438 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:39:45.435+0000 7f0c9bfe0700 1 -- 172.21.15.115:0/1601348378 --> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f0c94061a10 con 0x7f0c800a13f0 2023-12-11T17:39:45.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:45 smithi115 ceph-mon[102748]: purged_snaps scrub starts 2023-12-11T17:39:45.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:45 smithi115 ceph-mon[102748]: purged_snaps scrub ok 2023-12-11T17:39:45.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:45 smithi115 ceph-mon[102748]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T17:39:45.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:45 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-11T17:39:45.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:45 smithi115 ceph-mon[102748]: osd.0 [v2:172.21.15.59:6802/1040759398,v1:172.21.15.59:6803/1040759398] boot 2023-12-11T17:39:45.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:45 smithi115 ceph-mon[102748]: osdmap e8: 1 total, 1 up, 1 in 2023-12-11T17:39:45.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:45 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-11T17:39:45.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:45 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-11T17:39:45.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:45 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-11T17:39:45.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:45 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:39:45.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:45 smithi181 ceph-mon[103627]: purged_snaps scrub starts 2023-12-11T17:39:45.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:45 smithi181 ceph-mon[103627]: purged_snaps scrub ok 2023-12-11T17:39:45.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:45 smithi181 ceph-mon[103627]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T17:39:45.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:45 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-11T17:39:45.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:45 smithi181 ceph-mon[103627]: osd.0 [v2:172.21.15.59:6802/1040759398,v1:172.21.15.59:6803/1040759398] boot 2023-12-11T17:39:45.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:45 smithi181 ceph-mon[103627]: osdmap e8: 1 total, 1 up, 1 in 2023-12-11T17:39:45.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:45 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-11T17:39:45.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:45 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-11T17:39:45.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:45 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-11T17:39:45.896 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:45 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:39:45.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:45 smithi059 ceph-mon[98847]: purged_snaps scrub starts 2023-12-11T17:39:45.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:45 smithi059 ceph-mon[98847]: purged_snaps scrub ok 2023-12-11T17:39:45.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:45 smithi059 ceph-mon[98847]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-11T17:39:45.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:45 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-11T17:39:45.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:45 smithi059 ceph-mon[98847]: osd.0 [v2:172.21.15.59:6802/1040759398,v1:172.21.15.59:6803/1040759398] boot 2023-12-11T17:39:45.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:45 smithi059 ceph-mon[98847]: osdmap e8: 1 total, 1 up, 1 in 2023-12-11T17:39:45.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:45 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-11T17:39:45.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:45 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-11T17:39:45.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:45 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-11T17:39:45.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:45 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:39:46.621 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:46 smithi115 ceph-mon[102748]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T17:39:46.621 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:46 smithi115 ceph-mon[102748]: osdmap e9: 1 total, 1 up, 1 in 2023-12-11T17:39:46.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:46 smithi181 ceph-mon[103627]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T17:39:46.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:46 smithi181 ceph-mon[103627]: osdmap e9: 1 total, 1 up, 1 in 2023-12-11T17:39:46.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:46 smithi059 ceph-mon[98847]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T17:39:46.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:46 smithi059 ceph-mon[98847]: osdmap e9: 1 total, 1 up, 1 in 2023-12-11T17:39:47.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:47 smithi115 ceph-mon[102748]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-11T17:39:47.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:47 smithi181 ceph-mon[103627]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-11T17:39:47.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:47 smithi059 ceph-mon[98847]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-11T17:39:48.782 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:48 smithi115 ceph-mon[102748]: from='client.? 172.21.15.115:0/4258814662' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2ead7578-765b-4f5a-a72c-863731ed69e6"}]: dispatch 2023-12-11T17:39:48.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:48 smithi115 ceph-mon[102748]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2ead7578-765b-4f5a-a72c-863731ed69e6"}]: dispatch 2023-12-11T17:39:48.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:48 smithi115 ceph-mon[102748]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2ead7578-765b-4f5a-a72c-863731ed69e6"}]': finished 2023-12-11T17:39:48.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:48 smithi115 ceph-mon[102748]: osdmap e10: 2 total, 1 up, 2 in 2023-12-11T17:39:48.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:48 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-11T17:39:48.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:48 smithi181 ceph-mon[103627]: from='client.? 172.21.15.115:0/4258814662' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2ead7578-765b-4f5a-a72c-863731ed69e6"}]: dispatch 2023-12-11T17:39:48.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:48 smithi181 ceph-mon[103627]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2ead7578-765b-4f5a-a72c-863731ed69e6"}]: dispatch 2023-12-11T17:39:48.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:48 smithi181 ceph-mon[103627]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2ead7578-765b-4f5a-a72c-863731ed69e6"}]': finished 2023-12-11T17:39:48.896 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:48 smithi181 ceph-mon[103627]: osdmap e10: 2 total, 1 up, 2 in 2023-12-11T17:39:48.896 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:48 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-11T17:39:48.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:48 smithi059 ceph-mon[98847]: from='client.? 172.21.15.115:0/4258814662' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2ead7578-765b-4f5a-a72c-863731ed69e6"}]: dispatch 2023-12-11T17:39:48.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:48 smithi059 ceph-mon[98847]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2ead7578-765b-4f5a-a72c-863731ed69e6"}]: dispatch 2023-12-11T17:39:48.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:48 smithi059 ceph-mon[98847]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2ead7578-765b-4f5a-a72c-863731ed69e6"}]': finished 2023-12-11T17:39:48.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:48 smithi059 ceph-mon[98847]: osdmap e10: 2 total, 1 up, 2 in 2023-12-11T17:39:48.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:48 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-11T17:39:49.782 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:49 smithi115 ceph-mon[102748]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-11T17:39:49.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:49 smithi115 ceph-mon[102748]: from='client.? 172.21.15.115:0/2220359653' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-11T17:39:49.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:49 smithi181 ceph-mon[103627]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-11T17:39:49.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:49 smithi181 ceph-mon[103627]: from='client.? 172.21.15.115:0/2220359653' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-11T17:39:49.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:49 smithi059 ceph-mon[98847]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-11T17:39:49.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:49 smithi059 ceph-mon[98847]: from='client.? 172.21.15.115:0/2220359653' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-11T17:39:51.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:51 smithi115 ceph-mon[102748]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-11T17:39:51.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:51 smithi181 ceph-mon[103627]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-11T17:39:51.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:51 smithi059 ceph-mon[98847]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-11T17:39:53.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:53 smithi115 ceph-mon[102748]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-11T17:39:53.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:53 smithi181 ceph-mon[103627]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-11T17:39:53.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:53 smithi059 ceph-mon[98847]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-11T17:39:55.352 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:55 smithi115 ceph-mon[102748]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-11T17:39:55.352 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:55 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-12-11T17:39:55.352 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:55 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:39:55.352 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:55 smithi115 ceph-mon[102748]: Deploying daemon osd.1 on smithi115 2023-12-11T17:39:55.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:55 smithi181 ceph-mon[103627]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-11T17:39:55.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:55 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-12-11T17:39:55.396 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:55 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:39:55.396 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:55 smithi181 ceph-mon[103627]: Deploying daemon osd.1 on smithi115 2023-12-11T17:39:55.408 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:55 smithi059 ceph-mon[98847]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-11T17:39:55.408 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:55 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-12-11T17:39:55.409 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:55 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:39:55.409 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:55 smithi059 ceph-mon[98847]: Deploying daemon osd.1 on smithi115 2023-12-11T17:39:57.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:57 smithi115 ceph-mon[102748]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-11T17:39:57.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:57 smithi181 ceph-mon[103627]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-11T17:39:57.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:57 smithi059 ceph-mon[98847]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-11T17:40:00.011 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:59 smithi115 ceph-mon[102748]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-11T17:40:00.011 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:59 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:40:00.011 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:59 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:40:00.011 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:39:59 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:40:00.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:59 smithi181 ceph-mon[103627]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-11T17:40:00.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:59 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:40:00.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:59 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:40:00.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:39:59 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:40:00.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:59 smithi059 ceph-mon[98847]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-11T17:40:00.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:59 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:40:00.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:59 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:40:00.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:39:59 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:40:01.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:00 smithi181 ceph-mon[103627]: overall HEALTH_OK 2023-12-11T17:40:01.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:00 smithi059 ceph-mon[98847]: overall HEALTH_OK 2023-12-11T17:40:01.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:00 smithi115 ceph-mon[102748]: overall HEALTH_OK 2023-12-11T17:40:01.757 INFO:teuthology.orchestra.run.smithi115.stdout:Created osd(s) 1 on host 'smithi115' 2023-12-11T17:40:01.757 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:40:01.753+0000 7f0c86ffd700 1 -- 172.21.15.115:0/1601348378 <== mgr.14150 v2:172.21.15.59:6800/3627728353 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f0c94061a10 con 0x7f0c800a13f0 2023-12-11T17:40:01.759 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:40:01.756+0000 7f0c9bfe0700 1 -- 172.21.15.115:0/1601348378 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f0c800a13f0 msgr2=0x7f0c800a38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:40:01.759 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:40:01.756+0000 7f0c9bfe0700 1 --2- 172.21.15.115:0/1601348378 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f0c800a13f0 0x7f0c800a38b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f0c9000bcf0 tx=0x7f0c9000b2c0 comp rx=0 tx=0).stop 2023-12-11T17:40:01.759 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:40:01.756+0000 7f0c9bfe0700 1 -- 172.21.15.115:0/1601348378 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c94100780 msgr2=0x7f0c94100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:40:01.759 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:40:01.756+0000 7f0c9bfe0700 1 --2- 172.21.15.115:0/1601348378 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c94100780 0x7f0c94100180 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f0c8800d980 tx=0x7f0c8800dd40 comp rx=0 tx=0).stop 2023-12-11T17:40:01.759 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:40:01.756+0000 7f0c9bfe0700 1 -- 172.21.15.115:0/1601348378 shutdown_connections 2023-12-11T17:40:01.760 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:40:01.756+0000 7f0c9bfe0700 1 --2- 172.21.15.115:0/1601348378 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f0c800a13f0 0x7f0c800a38b0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:40:01.760 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:40:01.756+0000 7f0c9bfe0700 1 --2- 172.21.15.115:0/1601348378 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0c94101b20 0x7f0c9419e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:40:01.760 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:40:01.756+0000 7f0c9bfe0700 1 --2- 172.21.15.115:0/1601348378 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c94100780 0x7f0c94100180 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:40:01.760 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:40:01.756+0000 7f0c9bfe0700 1 --2- 172.21.15.115:0/1601348378 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c94101140 0x7f0c9419c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:40:01.760 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:40:01.756+0000 7f0c9bfe0700 1 -- 172.21.15.115:0/1601348378 >> 172.21.15.115:0/1601348378 conn(0x7f0c940fb230 msgr2=0x7f0c94104a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:40:01.760 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:40:01.756+0000 7f0c9bfe0700 1 -- 172.21.15.115:0/1601348378 shutdown_connections 2023-12-11T17:40:01.760 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:40:01.756+0000 7f0c9bfe0700 1 -- 172.21.15.115:0/1601348378 wait complete. 2023-12-11T17:40:02.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:01 smithi115 ceph-mon[102748]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-11T17:40:02.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:01 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:40:02.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:01 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:40:02.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:01 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:40:02.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:01 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:40:02.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:01 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:40:02.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:01 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:40:02.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:01 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:40:02.034 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:01 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:40:02.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:01 smithi181 ceph-mon[103627]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-11T17:40:02.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:01 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:40:02.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:01 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:40:02.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:01 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:40:02.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:01 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:40:02.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:01 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:40:02.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:01 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:40:02.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:01 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:40:02.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:01 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:40:02.153 DEBUG:teuthology.orchestra.run.smithi115:osd.1> sudo journalctl -f -n 0 -u ceph-d0a07eaa-984b-11ee-95a3-87774f69a715@osd.1.service 2023-12-11T17:40:02.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:01 smithi059 ceph-mon[98847]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-11T17:40:02.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:01 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:40:02.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:01 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:40:02.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:01 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:40:02.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:01 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:40:02.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:01 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:40:02.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:01 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:40:02.155 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:01 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:40:02.155 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:01 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:40:02.157 INFO:tasks.cephadm:Deploying osd.2 on smithi181 with /dev/vg_nvme/lv_4... 2023-12-11T17:40:02.157 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d0a07eaa-984b-11ee-95a3-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-12-11T17:40:02.187 INFO:journalctl@ceph.osd.1.smithi115.stdout:-- Logs begin at Mon 2023-12-11 17:22:43 UTC. -- 2023-12-11T17:40:02.763 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/mon.c/config 2023-12-11T17:40:03.283 INFO:journalctl@ceph.osd.1.smithi115.stdout:Dec 11 17:40:02 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-1[107812]: 2023-12-11T17:40:02.853+0000 7f3c03521640 -1 osd.1 0 log_to_monitors true 2023-12-11T17:40:04.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:03 smithi181 ceph-mon[103627]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-11T17:40:04.146 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:03 smithi181 ceph-mon[103627]: from='osd.1 [v2:172.21.15.115:6800/2776354245,v1:172.21.15.115:6801/2776354245]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-12-11T17:40:04.146 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:03 smithi181 ceph-mon[103627]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-12-11T17:40:04.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:03 smithi059 ceph-mon[98847]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-11T17:40:04.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:03 smithi059 ceph-mon[98847]: from='osd.1 [v2:172.21.15.115:6800/2776354245,v1:172.21.15.115:6801/2776354245]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-12-11T17:40:04.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:03 smithi059 ceph-mon[98847]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-12-11T17:40:04.254 INFO:teuthology.orchestra.run.smithi181.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-12-11T17:40:04.255 INFO:teuthology.orchestra.run.smithi181.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-12-11T17:40:04.255 INFO:teuthology.orchestra.run.smithi181.stderr: stderr: 10+0 records in 2023-12-11T17:40:04.255 INFO:teuthology.orchestra.run.smithi181.stderr:10+0 records out 2023-12-11T17:40:04.255 INFO:teuthology.orchestra.run.smithi181.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0113155 s, 927 MB/s 2023-12-11T17:40:04.255 INFO:teuthology.orchestra.run.smithi181.stderr: stderr: 2023-12-11T17:40:04.255 INFO:teuthology.orchestra.run.smithi181.stderr:--> Zapping successful for: 2023-12-11T17:40:04.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:03 smithi115 ceph-mon[102748]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-11T17:40:04.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:03 smithi115 ceph-mon[102748]: from='osd.1 [v2:172.21.15.115:6800/2776354245,v1:172.21.15.115:6801/2776354245]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-12-11T17:40:04.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:03 smithi115 ceph-mon[102748]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-12-11T17:40:04.611 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d0a07eaa-984b-11ee-95a3-87774f69a715 -- ceph orch daemon add osd smithi181:vg_nvme/lv_4 2023-12-11T17:40:05.033 INFO:journalctl@ceph.osd.1.smithi115.stdout:Dec 11 17:40:04 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-1[107812]: 2023-12-11T17:40:04.860+0000 7f3bf8b98700 -1 osd.1 0 waiting for initial osdmap 2023-12-11T17:40:05.033 INFO:journalctl@ceph.osd.1.smithi115.stdout:Dec 11 17:40:04 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-1[107812]: 2023-12-11T17:40:04.863+0000 7f3bf218d700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-11T17:40:05.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:04 smithi115 ceph-mon[102748]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-12-11T17:40:05.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:04 smithi115 ceph-mon[102748]: from='osd.1 [v2:172.21.15.115:6800/2776354245,v1:172.21.15.115:6801/2776354245]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2023-12-11T17:40:05.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:04 smithi115 ceph-mon[102748]: osdmap e11: 2 total, 1 up, 2 in 2023-12-11T17:40:05.034 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:04 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-11T17:40:05.034 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:04 smithi115 ceph-mon[102748]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2023-12-11T17:40:05.034 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:04 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:40:05.034 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:04 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:40:05.034 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:04 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:40:05.034 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:04 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:40:05.034 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:04 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:40:05.034 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:04 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:40:05.035 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:04 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:40:05.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:04 smithi181 ceph-mon[103627]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-12-11T17:40:05.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:04 smithi181 ceph-mon[103627]: from='osd.1 [v2:172.21.15.115:6800/2776354245,v1:172.21.15.115:6801/2776354245]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2023-12-11T17:40:05.146 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:04 smithi181 ceph-mon[103627]: osdmap e11: 2 total, 1 up, 2 in 2023-12-11T17:40:05.146 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:04 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-11T17:40:05.146 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:04 smithi181 ceph-mon[103627]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2023-12-11T17:40:05.146 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:04 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:40:05.146 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:04 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:40:05.146 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:04 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:40:05.146 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:04 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:40:05.146 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:04 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:40:05.146 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:04 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:40:05.147 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:04 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:40:05.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:04 smithi059 ceph-mon[98847]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-12-11T17:40:05.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:04 smithi059 ceph-mon[98847]: from='osd.1 [v2:172.21.15.115:6800/2776354245,v1:172.21.15.115:6801/2776354245]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2023-12-11T17:40:05.155 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:04 smithi059 ceph-mon[98847]: osdmap e11: 2 total, 1 up, 2 in 2023-12-11T17:40:05.155 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:04 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-11T17:40:05.155 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:04 smithi059 ceph-mon[98847]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2023-12-11T17:40:05.155 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:04 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:40:05.155 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:04 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:40:05.155 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:04 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:40:05.155 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:04 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:40:05.156 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:04 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:40:05.156 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:04 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:40:05.156 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:04 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:40:05.208 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/mon.c/config 2023-12-11T17:40:06.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:05 smithi181 ceph-mon[103627]: purged_snaps scrub starts 2023-12-11T17:40:06.146 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:05 smithi181 ceph-mon[103627]: purged_snaps scrub ok 2023-12-11T17:40:06.146 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:05 smithi181 ceph-mon[103627]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-11T17:40:06.146 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:05 smithi181 ceph-mon[103627]: Detected new or changed devices on smithi115 2023-12-11T17:40:06.146 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:05 smithi181 ceph-mon[103627]: Adjusting osd_memory_target on smithi115 to 17168M 2023-12-11T17:40:06.146 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:05 smithi181 ceph-mon[103627]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2023-12-11T17:40:06.146 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:05 smithi181 ceph-mon[103627]: osdmap e12: 2 total, 1 up, 2 in 2023-12-11T17:40:06.146 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:05 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-11T17:40:06.146 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:05 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-11T17:40:06.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:05 smithi059 ceph-mon[98847]: purged_snaps scrub starts 2023-12-11T17:40:06.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:05 smithi059 ceph-mon[98847]: purged_snaps scrub ok 2023-12-11T17:40:06.155 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:05 smithi059 ceph-mon[98847]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-11T17:40:06.155 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:05 smithi059 ceph-mon[98847]: Detected new or changed devices on smithi115 2023-12-11T17:40:06.155 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:05 smithi059 ceph-mon[98847]: Adjusting osd_memory_target on smithi115 to 17168M 2023-12-11T17:40:06.155 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:05 smithi059 ceph-mon[98847]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2023-12-11T17:40:06.155 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:05 smithi059 ceph-mon[98847]: osdmap e12: 2 total, 1 up, 2 in 2023-12-11T17:40:06.155 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:05 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-11T17:40:06.155 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:05 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-11T17:40:06.284 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:05 smithi115 ceph-mon[102748]: purged_snaps scrub starts 2023-12-11T17:40:06.284 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:05 smithi115 ceph-mon[102748]: purged_snaps scrub ok 2023-12-11T17:40:06.285 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:05 smithi115 ceph-mon[102748]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-11T17:40:06.285 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:05 smithi115 ceph-mon[102748]: Detected new or changed devices on smithi115 2023-12-11T17:40:06.285 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:05 smithi115 ceph-mon[102748]: Adjusting osd_memory_target on smithi115 to 17168M 2023-12-11T17:40:06.285 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:05 smithi115 ceph-mon[102748]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2023-12-11T17:40:06.285 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:05 smithi115 ceph-mon[102748]: osdmap e12: 2 total, 1 up, 2 in 2023-12-11T17:40:06.285 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:05 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-11T17:40:06.285 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:05 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-11T17:40:06.456 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:40:06.453+0000 7fc08cf2f700 1 -- 172.21.15.181:0/3341866903 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc0880ff8d0 msgr2=0x7fc0880ffcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:40:06.456 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:40:06.453+0000 7fc08cf2f700 1 --2- 172.21.15.181:0/3341866903 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc0880ff8d0 0x7fc0880ffcf0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc07c009990 tx=0x7fc07c009ca0 comp rx=0 tx=0).stop 2023-12-11T17:40:06.456 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:40:06.453+0000 7fc08cf2f700 1 -- 172.21.15.181:0/3341866903 shutdown_connections 2023-12-11T17:40:06.456 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:40:06.453+0000 7fc08cf2f700 1 --2- 172.21.15.181:0/3341866903 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc088103250 0x7fc088109b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:40:06.456 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:40:06.453+0000 7fc08cf2f700 1 --2- 172.21.15.181:0/3341866903 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc0880ff8d0 0x7fc0880ffcf0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:40:06.456 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:40:06.453+0000 7fc08cf2f700 1 --2- 172.21.15.181:0/3341866903 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0881028b0 0x7fc088102d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:40:06.456 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:40:06.453+0000 7fc08cf2f700 1 -- 172.21.15.181:0/3341866903 >> 172.21.15.181:0/3341866903 conn(0x7fc0880fb230 msgr2=0x7fc0880fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:40:06.456 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:40:06.453+0000 7fc08cf2f700 1 -- 172.21.15.181:0/3341866903 shutdown_connections 2023-12-11T17:40:06.456 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:40:06.453+0000 7fc08cf2f700 1 -- 172.21.15.181:0/3341866903 wait complete. 2023-12-11T17:40:06.458 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:40:06.454+0000 7fc08cf2f700 1 Processor -- start 2023-12-11T17:40:06.458 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:40:06.454+0000 7fc08cf2f700 1 -- start start 2023-12-11T17:40:06.459 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:40:06.454+0000 7fc08cf2f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc0880ff8d0 0x7fc088197660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:40:06.459 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:40:06.454+0000 7fc08cf2f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0881028b0 0x7fc088197ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:40:06.459 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:40:06.454+0000 7fc08cf2f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc088103250 0x7fc08819ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:40:06.459 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:40:06.454+0000 7fc08cf2f700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc08819f1a0 con 0x7fc088103250 2023-12-11T17:40:06.459 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:40:06.454+0000 7fc08cf2f700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc08819f2e0 con 0x7fc0881028b0 2023-12-11T17:40:06.459 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:40:06.454+0000 7fc08cf2f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc08819f470 con 0x7fc0880ff8d0 2023-12-11T17:40:06.459 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:40:06.454+0000 7fc08659c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc0880ff8d0 0x7fc088197660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:40:06.459 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:40:06.454+0000 7fc08659c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc0880ff8d0 0x7fc088197660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34268/0 (socket says 172.21.15.181:34268) 2023-12-11T17:40:06.459 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:40:06.454+0000 7fc08659c700 1 -- 172.21.15.181:0/1422867507 learned_addr learned my addr 172.21.15.181:0/1422867507 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-12-11T17:40:06.459 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:40:06.454+0000 7fc085d9b700 1 --2- 172.21.15.181:0/1422867507 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0881028b0 0x7fc088197ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:40:06.459 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:40:06.454+0000 7fc086d9d700 1 --2- 172.21.15.181:0/1422867507 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc088103250 0x7fc08819ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:40:06.459 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:40:06.454+0000 7fc08659c700 1 -- 172.21.15.181:0/1422867507 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0881028b0 msgr2=0x7fc088197ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:40:06.459 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:40:06.454+0000 7fc08659c700 1 --2- 172.21.15.181:0/1422867507 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0881028b0 0x7fc088197ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:40:06.460 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:40:06.454+0000 7fc08659c700 1 -- 172.21.15.181:0/1422867507 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc088103250 msgr2=0x7fc08819ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:40:06.460 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:40:06.454+0000 7fc08659c700 1 --2- 172.21.15.181:0/1422867507 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc088103250 0x7fc08819ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:40:06.460 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:40:06.454+0000 7fc08659c700 1 -- 172.21.15.181:0/1422867507 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc07c0095f0 con 0x7fc0880ff8d0 2023-12-11T17:40:06.460 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:40:06.454+0000 7fc085d9b700 1 --2- 172.21.15.181:0/1422867507 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0881028b0 0x7fc088197ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T17:40:06.460 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:40:06.454+0000 7fc086d9d700 1 --2- 172.21.15.181:0/1422867507 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc088103250 0x7fc08819ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T17:40:06.460 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:40:06.454+0000 7fc08659c700 1 --2- 172.21.15.181:0/1422867507 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc0880ff8d0 0x7fc088197660 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc07c00c010 tx=0x7fc07c00f7b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:40:06.460 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:40:06.455+0000 7fc0777fe700 1 -- 172.21.15.181:0/1422867507 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc07c01d070 con 0x7fc0880ff8d0 2023-12-11T17:40:06.460 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:40:06.455+0000 7fc0777fe700 1 -- 172.21.15.181:0/1422867507 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc07c00fd50 con 0x7fc0880ff8d0 2023-12-11T17:40:06.460 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:40:06.455+0000 7fc08cf2f700 1 -- 172.21.15.181:0/1422867507 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc08819f6f0 con 0x7fc0880ff8d0 2023-12-11T17:40:06.460 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:40:06.455+0000 7fc0777fe700 1 -- 172.21.15.181:0/1422867507 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc07c0177b0 con 0x7fc0880ff8d0 2023-12-11T17:40:06.460 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:40:06.455+0000 7fc08cf2f700 1 -- 172.21.15.181:0/1422867507 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc08819fb00 con 0x7fc0880ff8d0 2023-12-11T17:40:06.460 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:40:06.456+0000 7fc08cf2f700 1 -- 172.21.15.181:0/1422867507 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc088108bb0 con 0x7fc0880ff8d0 2023-12-11T17:40:06.461 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:40:06.457+0000 7fc0777fe700 1 -- 172.21.15.181:0/1422867507 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 15) v1 ==== 135950+0+0 (secure 0 0 0) 0x7fc07c017910 con 0x7fc0880ff8d0 2023-12-11T17:40:06.461 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:40:06.457+0000 7fc0777fe700 1 --2- 172.21.15.181:0/1422867507 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fc0700a13f0 0x7fc0700a38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:40:06.461 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:40:06.457+0000 7fc0777fe700 1 -- 172.21.15.181:0/1422867507 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 2309+0+0 (secure 0 0 0) 0x7fc07c0cce90 con 0x7fc0880ff8d0 2023-12-11T17:40:06.461 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:40:06.458+0000 7fc085d9b700 1 --2- 172.21.15.181:0/1422867507 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fc0700a13f0 0x7fc0700a38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:40:06.461 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:40:06.458+0000 7fc085d9b700 1 --2- 172.21.15.181:0/1422867507 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fc0700a13f0 0x7fc0700a38b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc0780099d0 tx=0x7fc078008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:40:06.463 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:40:06.461+0000 7fc0777fe700 1 -- 172.21.15.181:0/1422867507 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fc07c097dd0 con 0x7fc0880ff8d0 2023-12-11T17:40:06.659 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:40:06.656+0000 7fc08cf2f700 1 -- 172.21.15.181:0/1422867507 --> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fc088061a10 con 0x7fc0700a13f0 2023-12-11T17:40:07.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:06 smithi181 ceph-mon[103627]: osd.1 [v2:172.21.15.115:6800/2776354245,v1:172.21.15.115:6801/2776354245] boot 2023-12-11T17:40:07.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:06 smithi181 ceph-mon[103627]: osdmap e13: 2 total, 2 up, 2 in 2023-12-11T17:40:07.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:06 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-11T17:40:07.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:06 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-11T17:40:07.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:06 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-11T17:40:07.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:06 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:40:07.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:06 smithi059 ceph-mon[98847]: osd.1 [v2:172.21.15.115:6800/2776354245,v1:172.21.15.115:6801/2776354245] boot 2023-12-11T17:40:07.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:06 smithi059 ceph-mon[98847]: osdmap e13: 2 total, 2 up, 2 in 2023-12-11T17:40:07.155 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:06 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-11T17:40:07.155 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:06 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-11T17:40:07.155 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:06 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-11T17:40:07.155 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:06 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:40:07.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:06 smithi115 ceph-mon[102748]: osd.1 [v2:172.21.15.115:6800/2776354245,v1:172.21.15.115:6801/2776354245] boot 2023-12-11T17:40:07.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:06 smithi115 ceph-mon[102748]: osdmap e13: 2 total, 2 up, 2 in 2023-12-11T17:40:07.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:06 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-11T17:40:07.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:06 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-11T17:40:07.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:06 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-11T17:40:07.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:06 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:40:08.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:07 smithi181 ceph-mon[103627]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-11T17:40:08.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:07 smithi181 ceph-mon[103627]: from='client.24154 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T17:40:08.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:07 smithi181 ceph-mon[103627]: osdmap e14: 2 total, 2 up, 2 in 2023-12-11T17:40:08.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:07 smithi059 ceph-mon[98847]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-11T17:40:08.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:07 smithi059 ceph-mon[98847]: from='client.24154 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T17:40:08.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:07 smithi059 ceph-mon[98847]: osdmap e14: 2 total, 2 up, 2 in 2023-12-11T17:40:08.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:07 smithi115 ceph-mon[102748]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-11T17:40:08.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:07 smithi115 ceph-mon[102748]: from='client.24154 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T17:40:08.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:07 smithi115 ceph-mon[102748]: osdmap e14: 2 total, 2 up, 2 in 2023-12-11T17:40:10.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:09 smithi181 ceph-mon[103627]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-11T17:40:10.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:09 smithi181 ceph-mon[103627]: from='client.? 172.21.15.181:0/456343036' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8cb1d7ec-53cc-4672-b0b6-70d73797c44f"}]: dispatch 2023-12-11T17:40:10.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:09 smithi181 ceph-mon[103627]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8cb1d7ec-53cc-4672-b0b6-70d73797c44f"}]: dispatch 2023-12-11T17:40:10.146 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:09 smithi181 ceph-mon[103627]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8cb1d7ec-53cc-4672-b0b6-70d73797c44f"}]': finished 2023-12-11T17:40:10.146 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:09 smithi181 ceph-mon[103627]: osdmap e15: 3 total, 2 up, 3 in 2023-12-11T17:40:10.146 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:09 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-11T17:40:10.146 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:09 smithi181 ceph-mon[103627]: from='client.? 172.21.15.181:0/3035130931' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-11T17:40:10.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:09 smithi059 ceph-mon[98847]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-11T17:40:10.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:09 smithi059 ceph-mon[98847]: from='client.? 172.21.15.181:0/456343036' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8cb1d7ec-53cc-4672-b0b6-70d73797c44f"}]: dispatch 2023-12-11T17:40:10.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:09 smithi059 ceph-mon[98847]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8cb1d7ec-53cc-4672-b0b6-70d73797c44f"}]: dispatch 2023-12-11T17:40:10.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:09 smithi059 ceph-mon[98847]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8cb1d7ec-53cc-4672-b0b6-70d73797c44f"}]': finished 2023-12-11T17:40:10.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:09 smithi059 ceph-mon[98847]: osdmap e15: 3 total, 2 up, 3 in 2023-12-11T17:40:10.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:09 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-11T17:40:10.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:09 smithi059 ceph-mon[98847]: from='client.? 172.21.15.181:0/3035130931' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-11T17:40:10.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:09 smithi115 ceph-mon[102748]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-11T17:40:10.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:09 smithi115 ceph-mon[102748]: from='client.? 172.21.15.181:0/456343036' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8cb1d7ec-53cc-4672-b0b6-70d73797c44f"}]: dispatch 2023-12-11T17:40:10.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:09 smithi115 ceph-mon[102748]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8cb1d7ec-53cc-4672-b0b6-70d73797c44f"}]: dispatch 2023-12-11T17:40:10.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:09 smithi115 ceph-mon[102748]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8cb1d7ec-53cc-4672-b0b6-70d73797c44f"}]': finished 2023-12-11T17:40:10.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:09 smithi115 ceph-mon[102748]: osdmap e15: 3 total, 2 up, 3 in 2023-12-11T17:40:10.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:09 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-11T17:40:10.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:09 smithi115 ceph-mon[102748]: from='client.? 172.21.15.181:0/3035130931' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-11T17:40:12.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:11 smithi181 ceph-mon[103627]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-11T17:40:12.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:11 smithi059 ceph-mon[98847]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-11T17:40:12.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:11 smithi115 ceph-mon[102748]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-11T17:40:14.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:13 smithi181 ceph-mon[103627]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-11T17:40:14.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:13 smithi059 ceph-mon[98847]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-11T17:40:14.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:13 smithi115 ceph-mon[102748]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-11T17:40:15.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:15 smithi181 ceph-mon[103627]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-11T17:40:15.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:15 smithi059 ceph-mon[98847]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-11T17:40:15.532 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:15 smithi115 ceph-mon[102748]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-11T17:40:16.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:16 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-12-11T17:40:16.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:16 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:40:16.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:16 smithi181 ceph-mon[103627]: Deploying daemon osd.2 on smithi181 2023-12-11T17:40:16.406 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:16 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-12-11T17:40:16.406 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:16 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:40:16.406 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:16 smithi059 ceph-mon[98847]: Deploying daemon osd.2 on smithi181 2023-12-11T17:40:16.532 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:16 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-12-11T17:40:16.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:16 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:40:16.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:16 smithi115 ceph-mon[102748]: Deploying daemon osd.2 on smithi181 2023-12-11T17:40:17.374 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:17 smithi181 ceph-mon[103627]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-11T17:40:17.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:17 smithi059 ceph-mon[98847]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-11T17:40:17.532 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:17 smithi115 ceph-mon[102748]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-11T17:40:19.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:19 smithi181 ceph-mon[103627]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-11T17:40:19.406 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:19 smithi059 ceph-mon[98847]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-11T17:40:19.532 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:19 smithi115 ceph-mon[102748]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-11T17:40:21.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:20 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:40:21.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:20 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:40:21.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:20 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:40:21.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:20 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:40:21.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:20 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:40:21.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:20 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:40:21.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:20 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:40:21.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:20 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:40:21.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:20 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:40:22.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:21 smithi181 ceph-mon[103627]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-11T17:40:22.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:21 smithi115 ceph-mon[102748]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-11T17:40:22.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:21 smithi059 ceph-mon[98847]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-11T17:40:22.875 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:40:22.872+0000 7fc0777fe700 1 -- 172.21.15.181:0/1422867507 <== mgr.14150 v2:172.21.15.59:6800/3627728353 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fc088061a10 con 0x7fc0700a13f0 2023-12-11T17:40:22.875 INFO:teuthology.orchestra.run.smithi181.stdout:Created osd(s) 2 on host 'smithi181' 2023-12-11T17:40:22.877 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:40:22.874+0000 7fc08cf2f700 1 -- 172.21.15.181:0/1422867507 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fc0700a13f0 msgr2=0x7fc0700a38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:40:22.877 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:40:22.874+0000 7fc08cf2f700 1 --2- 172.21.15.181:0/1422867507 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fc0700a13f0 0x7fc0700a38b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc0780099d0 tx=0x7fc078008040 comp rx=0 tx=0).stop 2023-12-11T17:40:22.877 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:40:22.874+0000 7fc08cf2f700 1 -- 172.21.15.181:0/1422867507 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc0880ff8d0 msgr2=0x7fc088197660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:40:22.878 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:40:22.874+0000 7fc08cf2f700 1 --2- 172.21.15.181:0/1422867507 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc0880ff8d0 0x7fc088197660 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc07c00c010 tx=0x7fc07c00f7b0 comp rx=0 tx=0).stop 2023-12-11T17:40:22.878 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:40:22.874+0000 7fc08cf2f700 1 -- 172.21.15.181:0/1422867507 shutdown_connections 2023-12-11T17:40:22.878 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:40:22.874+0000 7fc08cf2f700 1 --2- 172.21.15.181:0/1422867507 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fc0700a13f0 0x7fc0700a38b0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:40:22.878 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:40:22.874+0000 7fc08cf2f700 1 --2- 172.21.15.181:0/1422867507 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc088103250 0x7fc08819ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:40:22.878 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:40:22.874+0000 7fc08cf2f700 1 --2- 172.21.15.181:0/1422867507 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc0880ff8d0 0x7fc088197660 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:40:22.878 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:40:22.874+0000 7fc08cf2f700 1 --2- 172.21.15.181:0/1422867507 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0881028b0 0x7fc088197ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:40:22.878 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:40:22.874+0000 7fc08cf2f700 1 -- 172.21.15.181:0/1422867507 >> 172.21.15.181:0/1422867507 conn(0x7fc0880fb230 msgr2=0x7fc088105cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:40:22.878 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:40:22.874+0000 7fc08cf2f700 1 -- 172.21.15.181:0/1422867507 shutdown_connections 2023-12-11T17:40:22.878 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:40:22.874+0000 7fc08cf2f700 1 -- 172.21.15.181:0/1422867507 wait complete. 2023-12-11T17:40:23.037 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:22 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:40:23.037 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:22 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:40:23.037 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:22 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:40:23.037 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:22 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:40:23.037 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:22 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:40:23.037 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:22 smithi181 ceph-mon[103627]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-11T17:40:23.037 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:22 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:40:23.037 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:22 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:40:23.037 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:22 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:40:23.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:22 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:40:23.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:22 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:40:23.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:22 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:40:23.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:22 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:40:23.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:22 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:40:23.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:22 smithi115 ceph-mon[102748]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-11T17:40:23.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:22 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:40:23.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:22 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:40:23.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:22 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:40:23.301 DEBUG:teuthology.orchestra.run.smithi181:osd.2> sudo journalctl -f -n 0 -u ceph-d0a07eaa-984b-11ee-95a3-87774f69a715@osd.2.service 2023-12-11T17:40:23.303 INFO:tasks.cephadm:Deploying osd.3 on smithi059 with /dev/vg_nvme/lv_3... 2023-12-11T17:40:23.303 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d0a07eaa-984b-11ee-95a3-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-12-11T17:40:23.332 INFO:journalctl@ceph.osd.2.smithi181.stdout:-- Logs begin at Mon 2023-12-11 17:22:43 UTC. -- 2023-12-11T17:40:23.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:22 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:40:23.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:22 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:40:23.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:22 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:40:23.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:22 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:40:23.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:22 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:40:23.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:22 smithi059 ceph-mon[98847]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-11T17:40:23.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:22 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:40:23.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:22 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:40:23.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:22 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:40:23.920 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/mon.a/config 2023-12-11T17:40:24.120 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:24 smithi181 ceph-mon[103627]: from='osd.2 [v2:172.21.15.181:6800/4287675481,v1:172.21.15.181:6801/4287675481]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-12-11T17:40:24.120 INFO:journalctl@ceph.osd.2.smithi181.stdout:Dec 11 17:40:23 smithi181 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-2[109416]: 2023-12-11T17:40:23.902+0000 7fe2f34d0640 -1 osd.2 0 log_to_monitors true 2023-12-11T17:40:24.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:24 smithi115 ceph-mon[102748]: from='osd.2 [v2:172.21.15.181:6800/4287675481,v1:172.21.15.181:6801/4287675481]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-12-11T17:40:24.350 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:24 smithi059 ceph-mon[98847]: from='osd.2 [v2:172.21.15.181:6800/4287675481,v1:172.21.15.181:6801/4287675481]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-12-11T17:40:25.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:25 smithi059 ceph-mon[98847]: from='osd.2 [v2:172.21.15.181:6800/4287675481,v1:172.21.15.181:6801/4287675481]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-12-11T17:40:25.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:25 smithi059 ceph-mon[98847]: osdmap e16: 3 total, 2 up, 3 in 2023-12-11T17:40:25.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:25 smithi059 ceph-mon[98847]: from='osd.2 [v2:172.21.15.181:6800/4287675481,v1:172.21.15.181:6801/4287675481]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2023-12-11T17:40:25.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:25 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-11T17:40:25.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:25 smithi059 ceph-mon[98847]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-11T17:40:25.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:25 smithi115 ceph-mon[102748]: from='osd.2 [v2:172.21.15.181:6800/4287675481,v1:172.21.15.181:6801/4287675481]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-12-11T17:40:25.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:25 smithi115 ceph-mon[102748]: osdmap e16: 3 total, 2 up, 3 in 2023-12-11T17:40:25.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:25 smithi115 ceph-mon[102748]: from='osd.2 [v2:172.21.15.181:6800/4287675481,v1:172.21.15.181:6801/4287675481]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2023-12-11T17:40:25.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:25 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-11T17:40:25.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:25 smithi115 ceph-mon[102748]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-11T17:40:25.371 INFO:teuthology.orchestra.run.smithi059.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-12-11T17:40:25.371 INFO:teuthology.orchestra.run.smithi059.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-12-11T17:40:25.371 INFO:teuthology.orchestra.run.smithi059.stderr: stderr: 10+0 records in 2023-12-11T17:40:25.371 INFO:teuthology.orchestra.run.smithi059.stderr:10+0 records out 2023-12-11T17:40:25.371 INFO:teuthology.orchestra.run.smithi059.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0108648 s, 965 MB/s 2023-12-11T17:40:25.371 INFO:teuthology.orchestra.run.smithi059.stderr:--> Zapping successful for: 2023-12-11T17:40:25.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:25 smithi181 ceph-mon[103627]: from='osd.2 [v2:172.21.15.181:6800/4287675481,v1:172.21.15.181:6801/4287675481]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-12-11T17:40:25.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:25 smithi181 ceph-mon[103627]: osdmap e16: 3 total, 2 up, 3 in 2023-12-11T17:40:25.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:25 smithi181 ceph-mon[103627]: from='osd.2 [v2:172.21.15.181:6800/4287675481,v1:172.21.15.181:6801/4287675481]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2023-12-11T17:40:25.396 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:25 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-11T17:40:25.396 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:25 smithi181 ceph-mon[103627]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-11T17:40:25.396 INFO:journalctl@ceph.osd.2.smithi181.stdout:Dec 11 17:40:25 smithi181 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-2[109416]: 2023-12-11T17:40:25.022+0000 7fe2e8b47700 -1 osd.2 0 waiting for initial osdmap 2023-12-11T17:40:25.396 INFO:journalctl@ceph.osd.2.smithi181.stdout:Dec 11 17:40:25 smithi181 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-2[109416]: 2023-12-11T17:40:25.027+0000 7fe2e293d700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-11T17:40:25.686 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d0a07eaa-984b-11ee-95a3-87774f69a715 -- ceph orch daemon add osd smithi059:vg_nvme/lv_3 2023-12-11T17:40:26.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:26 smithi115 ceph-mon[102748]: from='osd.2 [v2:172.21.15.181:6800/4287675481,v1:172.21.15.181:6801/4287675481]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]': finished 2023-12-11T17:40:26.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:26 smithi115 ceph-mon[102748]: osdmap e17: 3 total, 2 up, 3 in 2023-12-11T17:40:26.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:26 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-11T17:40:26.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:26 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-11T17:40:26.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:26 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:40:26.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:26 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:40:26.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:26 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:40:26.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:26 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:40:26.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:26 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:40:26.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:26 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:40:26.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:26 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:40:26.334 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/mon.a/config 2023-12-11T17:40:26.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:26 smithi181 ceph-mon[103627]: from='osd.2 [v2:172.21.15.181:6800/4287675481,v1:172.21.15.181:6801/4287675481]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]': finished 2023-12-11T17:40:26.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:26 smithi181 ceph-mon[103627]: osdmap e17: 3 total, 2 up, 3 in 2023-12-11T17:40:26.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:26 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-11T17:40:26.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:26 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-11T17:40:26.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:26 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:40:26.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:26 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:40:26.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:26 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:40:26.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:26 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:40:26.396 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:26 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:40:26.396 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:26 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:40:26.396 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:26 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:40:26.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:26 smithi059 ceph-mon[98847]: from='osd.2 [v2:172.21.15.181:6800/4287675481,v1:172.21.15.181:6801/4287675481]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]': finished 2023-12-11T17:40:26.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:26 smithi059 ceph-mon[98847]: osdmap e17: 3 total, 2 up, 3 in 2023-12-11T17:40:26.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:26 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-11T17:40:26.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:26 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-11T17:40:26.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:26 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:40:26.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:26 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:40:26.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:26 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:40:26.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:26 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:40:26.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:26 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:40:26.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:26 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:40:26.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:26 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:40:27.155 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:27 smithi059 ceph-mon[98847]: purged_snaps scrub starts 2023-12-11T17:40:27.156 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:27 smithi059 ceph-mon[98847]: purged_snaps scrub ok 2023-12-11T17:40:27.156 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:27 smithi059 ceph-mon[98847]: Detected new or changed devices on smithi181 2023-12-11T17:40:27.157 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:27 smithi059 ceph-mon[98847]: Adjusting osd_memory_target on smithi181 to 17168M 2023-12-11T17:40:27.157 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:27 smithi059 ceph-mon[98847]: osd.2 [v2:172.21.15.181:6800/4287675481,v1:172.21.15.181:6801/4287675481] boot 2023-12-11T17:40:27.157 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:27 smithi059 ceph-mon[98847]: osdmap e18: 3 total, 3 up, 3 in 2023-12-11T17:40:27.157 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:27 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-11T17:40:27.158 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:27 smithi059 ceph-mon[98847]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-11T17:40:27.158 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:27 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-12-11T17:40:27.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:27 smithi181 ceph-mon[103627]: purged_snaps scrub starts 2023-12-11T17:40:27.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:27 smithi181 ceph-mon[103627]: purged_snaps scrub ok 2023-12-11T17:40:27.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:27 smithi181 ceph-mon[103627]: Detected new or changed devices on smithi181 2023-12-11T17:40:27.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:27 smithi181 ceph-mon[103627]: Adjusting osd_memory_target on smithi181 to 17168M 2023-12-11T17:40:27.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:27 smithi181 ceph-mon[103627]: osd.2 [v2:172.21.15.181:6800/4287675481,v1:172.21.15.181:6801/4287675481] boot 2023-12-11T17:40:27.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:27 smithi181 ceph-mon[103627]: osdmap e18: 3 total, 3 up, 3 in 2023-12-11T17:40:27.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:27 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-11T17:40:27.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:27 smithi181 ceph-mon[103627]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-11T17:40:27.396 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:27 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-12-11T17:40:27.417 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:40:27.414+0000 7fe461582700 1 -- 172.21.15.59:0/430803258 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe45c102c00 msgr2=0x7fe45c103020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:40:27.417 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:40:27.414+0000 7fe461582700 1 --2- 172.21.15.59:0/430803258 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe45c102c00 0x7fe45c103020 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fe44c009990 tx=0x7fe44c009ca0 comp rx=0 tx=0).stop 2023-12-11T17:40:27.417 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:40:27.414+0000 7fe461582700 1 -- 172.21.15.59:0/430803258 shutdown_connections 2023-12-11T17:40:27.418 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:40:27.414+0000 7fe461582700 1 --2- 172.21.15.59:0/430803258 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe45c100020 0x7fe45c109bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:40:27.418 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:40:27.415+0000 7fe461582700 1 --2- 172.21.15.59:0/430803258 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe45c102c00 0x7fe45c103020 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:40:27.418 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:40:27.415+0000 7fe461582700 1 --2- 172.21.15.59:0/430803258 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe45c0ff680 0x7fe45c0ffae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:40:27.418 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:40:27.415+0000 7fe461582700 1 -- 172.21.15.59:0/430803258 >> 172.21.15.59:0/430803258 conn(0x7fe45c0fb260 msgr2=0x7fe45c0fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:40:27.418 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:40:27.415+0000 7fe461582700 1 -- 172.21.15.59:0/430803258 shutdown_connections 2023-12-11T17:40:27.418 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:40:27.415+0000 7fe461582700 1 -- 172.21.15.59:0/430803258 wait complete. 2023-12-11T17:40:27.420 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:40:27.415+0000 7fe461582700 1 Processor -- start 2023-12-11T17:40:27.420 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:40:27.415+0000 7fe461582700 1 -- start start 2023-12-11T17:40:27.420 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:40:27.416+0000 7fe461582700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe45c0ff680 0x7fe45c1022b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:40:27.420 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:40:27.416+0000 7fe461582700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe45c100020 0x7fe45c100950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:40:27.420 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:40:27.416+0000 7fe461582700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe45c102c00 0x7fe45c100e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:40:27.420 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:40:27.416+0000 7fe461582700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe45c101630 con 0x7fe45c0ff680 2023-12-11T17:40:27.420 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:40:27.416+0000 7fe461582700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe45c1017a0 con 0x7fe45c102c00 2023-12-11T17:40:27.420 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:40:27.416+0000 7fe461582700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe45c1a15c0 con 0x7fe45c100020 2023-12-11T17:40:27.420 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:40:27.416+0000 7fe45affd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe45c0ff680 0x7fe45c1022b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:40:27.420 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:40:27.416+0000 7fe45affd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe45c0ff680 0x7fe45c1022b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60260/0 (socket says 172.21.15.59:60260) 2023-12-11T17:40:27.421 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:40:27.416+0000 7fe45affd700 1 -- 172.21.15.59:0/373835384 learned_addr learned my addr 172.21.15.59:0/373835384 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:40:27.421 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:40:27.416+0000 7fe45a7fc700 1 --2- 172.21.15.59:0/373835384 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe45c100020 0x7fe45c100950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:40:27.421 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:40:27.416+0000 7fe45b7fe700 1 --2- 172.21.15.59:0/373835384 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe45c102c00 0x7fe45c100e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:40:27.421 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:40:27.416+0000 7fe45affd700 1 -- 172.21.15.59:0/373835384 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe45c100020 msgr2=0x7fe45c100950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:40:27.421 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:40:27.416+0000 7fe45affd700 1 --2- 172.21.15.59:0/373835384 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe45c100020 0x7fe45c100950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:40:27.421 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:40:27.416+0000 7fe45affd700 1 -- 172.21.15.59:0/373835384 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe45c102c00 msgr2=0x7fe45c100e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:40:27.421 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:40:27.416+0000 7fe45affd700 1 --2- 172.21.15.59:0/373835384 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe45c102c00 0x7fe45c100e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:40:27.421 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:40:27.416+0000 7fe45affd700 1 -- 172.21.15.59:0/373835384 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe44c0095f0 con 0x7fe45c0ff680 2023-12-11T17:40:27.421 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:40:27.416+0000 7fe45b7fe700 1 --2- 172.21.15.59:0/373835384 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe45c102c00 0x7fe45c100e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-11T17:40:27.421 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:40:27.416+0000 7fe45affd700 1 --2- 172.21.15.59:0/373835384 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe45c0ff680 0x7fe45c1022b0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe44c0098e0 tx=0x7fe44c00f7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:40:27.421 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:40:27.416+0000 7fe443fff700 1 -- 172.21.15.59:0/373835384 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe44c01d070 con 0x7fe45c0ff680 2023-12-11T17:40:27.421 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:40:27.416+0000 7fe461582700 1 -- 172.21.15.59:0/373835384 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe45c1a17f0 con 0x7fe45c0ff680 2023-12-11T17:40:27.421 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:40:27.416+0000 7fe443fff700 1 -- 172.21.15.59:0/373835384 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe44c00fca0 con 0x7fe45c0ff680 2023-12-11T17:40:27.421 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:40:27.416+0000 7fe443fff700 1 -- 172.21.15.59:0/373835384 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe44c017720 con 0x7fe45c0ff680 2023-12-11T17:40:27.421 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:40:27.416+0000 7fe461582700 1 -- 172.21.15.59:0/373835384 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe45c1a1c60 con 0x7fe45c0ff680 2023-12-11T17:40:27.421 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:40:27.417+0000 7fe461582700 1 -- 172.21.15.59:0/373835384 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe45c108c20 con 0x7fe45c0ff680 2023-12-11T17:40:27.422 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:40:27.419+0000 7fe443fff700 1 -- 172.21.15.59:0/373835384 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 15) v1 ==== 135950+0+0 (secure 0 0 0) 0x7fe44c017880 con 0x7fe45c0ff680 2023-12-11T17:40:27.422 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:40:27.419+0000 7fe443fff700 1 --2- 172.21.15.59:0/373835384 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fe4440a13f0 0x7fe4440a38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:40:27.422 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:40:27.419+0000 7fe443fff700 1 -- 172.21.15.59:0/373835384 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 3226+0+0 (secure 0 0 0) 0x7fe44c0ccfa0 con 0x7fe45c0ff680 2023-12-11T17:40:27.422 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:40:27.419+0000 7fe45a7fc700 1 --2- 172.21.15.59:0/373835384 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fe4440a13f0 0x7fe4440a38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:40:27.423 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:40:27.420+0000 7fe45a7fc700 1 --2- 172.21.15.59:0/373835384 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fe4440a13f0 0x7fe4440a38b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe450005fd0 tx=0x7fe45000a560 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:40:27.425 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:40:27.423+0000 7fe443fff700 1 -- 172.21.15.59:0/373835384 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fe44c097b50 con 0x7fe45c0ff680 2023-12-11T17:40:27.532 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:27 smithi115 ceph-mon[102748]: purged_snaps scrub starts 2023-12-11T17:40:27.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:27 smithi115 ceph-mon[102748]: purged_snaps scrub ok 2023-12-11T17:40:27.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:27 smithi115 ceph-mon[102748]: Detected new or changed devices on smithi181 2023-12-11T17:40:27.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:27 smithi115 ceph-mon[102748]: Adjusting osd_memory_target on smithi181 to 17168M 2023-12-11T17:40:27.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:27 smithi115 ceph-mon[102748]: osd.2 [v2:172.21.15.181:6800/4287675481,v1:172.21.15.181:6801/4287675481] boot 2023-12-11T17:40:27.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:27 smithi115 ceph-mon[102748]: osdmap e18: 3 total, 3 up, 3 in 2023-12-11T17:40:27.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:27 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-11T17:40:27.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:27 smithi115 ceph-mon[102748]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-11T17:40:27.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:27 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-12-11T17:40:27.635 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:40:27.633+0000 7fe461582700 1 -- 172.21.15.59:0/373835384 --> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi059:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fe45c061a10 con 0x7fe4440a13f0 2023-12-11T17:40:28.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:28 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-12-11T17:40:28.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:28 smithi181 ceph-mon[103627]: osdmap e19: 3 total, 3 up, 3 in 2023-12-11T17:40:28.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:28 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-12-11T17:40:28.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:28 smithi181 ceph-mon[103627]: from='client.14271 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi059:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T17:40:28.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:28 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-11T17:40:28.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:28 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-11T17:40:28.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:28 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:40:28.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:28 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-12-11T17:40:28.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:28 smithi059 ceph-mon[98847]: osdmap e19: 3 total, 3 up, 3 in 2023-12-11T17:40:28.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:28 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-12-11T17:40:28.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:28 smithi059 ceph-mon[98847]: from='client.14271 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi059:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T17:40:28.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:28 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-11T17:40:28.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:28 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-11T17:40:28.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:28 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:40:28.532 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:28 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-12-11T17:40:28.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:28 smithi115 ceph-mon[102748]: osdmap e19: 3 total, 3 up, 3 in 2023-12-11T17:40:28.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:28 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-12-11T17:40:28.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:28 smithi115 ceph-mon[102748]: from='client.14271 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi059:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T17:40:28.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:28 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-11T17:40:28.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:28 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-11T17:40:28.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:28 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:40:29.059 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:40:29.057+0000 7fe443fff700 1 -- 172.21.15.59:0/373835384 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7fe44c097f20 con 0x7fe45c0ff680 2023-12-11T17:40:29.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:29 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-12-11T17:40:29.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:29 smithi181 ceph-mon[103627]: osdmap e20: 3 total, 3 up, 3 in 2023-12-11T17:40:29.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:29 smithi181 ceph-mon[103627]: pgmap v77: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-12-11T17:40:29.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:29 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-12-11T17:40:29.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:29 smithi059 ceph-mon[98847]: osdmap e20: 3 total, 3 up, 3 in 2023-12-11T17:40:29.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:29 smithi059 ceph-mon[98847]: pgmap v77: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-12-11T17:40:29.405 INFO:journalctl@ceph.osd.0.smithi059.stdout:Dec 11 17:40:29 smithi059 sudo[123714]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-12-11T17:40:29.405 INFO:journalctl@ceph.osd.0.smithi059.stdout:Dec 11 17:40:29 smithi059 sudo[123714]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-12-11T17:40:29.405 INFO:journalctl@ceph.osd.0.smithi059.stdout:Dec 11 17:40:29 smithi059 sudo[123714]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-12-11T17:40:29.405 INFO:journalctl@ceph.osd.0.smithi059.stdout:Dec 11 17:40:29 smithi059 sudo[123714]: pam_unix(sudo:session): session closed for user root 2023-12-11T17:40:29.532 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:29 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-12-11T17:40:29.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:29 smithi115 ceph-mon[102748]: osdmap e20: 3 total, 3 up, 3 in 2023-12-11T17:40:29.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:29 smithi115 ceph-mon[102748]: pgmap v77: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-12-11T17:40:29.905 INFO:journalctl@ceph.osd.0.smithi059.stdout:Dec 11 17:40:29 smithi059 sudo[123750]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-12-11T17:40:29.905 INFO:journalctl@ceph.osd.0.smithi059.stdout:Dec 11 17:40:29 smithi059 sudo[123750]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-12-11T17:40:29.905 INFO:journalctl@ceph.osd.0.smithi059.stdout:Dec 11 17:40:29 smithi059 sudo[123750]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-12-11T17:40:29.905 INFO:journalctl@ceph.osd.0.smithi059.stdout:Dec 11 17:40:29 smithi059 sudo[123750]: pam_unix(sudo:session): session closed for user root 2023-12-11T17:40:30.006 INFO:journalctl@ceph.osd.1.smithi115.stdout:Dec 11 17:40:29 smithi115 sudo[113315]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-12-11T17:40:30.006 INFO:journalctl@ceph.osd.1.smithi115.stdout:Dec 11 17:40:29 smithi115 sudo[113315]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-12-11T17:40:30.006 INFO:journalctl@ceph.osd.1.smithi115.stdout:Dec 11 17:40:29 smithi115 sudo[113315]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-12-11T17:40:30.007 INFO:journalctl@ceph.osd.1.smithi115.stdout:Dec 11 17:40:29 smithi115 sudo[113315]: pam_unix(sudo:session): session closed for user root 2023-12-11T17:40:30.283 INFO:journalctl@ceph.osd.1.smithi115.stdout:Dec 11 17:40:30 smithi115 sudo[113318]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-12-11T17:40:30.283 INFO:journalctl@ceph.osd.1.smithi115.stdout:Dec 11 17:40:30 smithi115 sudo[113318]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-12-11T17:40:30.283 INFO:journalctl@ceph.osd.1.smithi115.stdout:Dec 11 17:40:30 smithi115 sudo[113318]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-12-11T17:40:30.283 INFO:journalctl@ceph.osd.1.smithi115.stdout:Dec 11 17:40:30 smithi115 sudo[113318]: pam_unix(sudo:session): session closed for user root 2023-12-11T17:40:30.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:30 smithi115 ceph-mon[102748]: osdmap e21: 3 total, 3 up, 3 in 2023-12-11T17:40:30.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:30 smithi115 ceph-mon[102748]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-12-11T17:40:30.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:30 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:40:30.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:30 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/991677140' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5d64d9af-831b-4d0a-8ace-a452169351d6"}]: dispatch 2023-12-11T17:40:30.284 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:30 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/991677140' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5d64d9af-831b-4d0a-8ace-a452169351d6"}]': finished 2023-12-11T17:40:30.284 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:30 smithi115 ceph-mon[102748]: osdmap e22: 4 total, 3 up, 4 in 2023-12-11T17:40:30.284 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:30 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-11T17:40:30.284 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:30 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:40:30.395 INFO:journalctl@ceph.osd.2.smithi181.stdout:Dec 11 17:40:30 smithi181 sudo[114083]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-12-11T17:40:30.395 INFO:journalctl@ceph.osd.2.smithi181.stdout:Dec 11 17:40:30 smithi181 sudo[114083]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-12-11T17:40:30.395 INFO:journalctl@ceph.osd.2.smithi181.stdout:Dec 11 17:40:30 smithi181 sudo[114083]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-12-11T17:40:30.395 INFO:journalctl@ceph.osd.2.smithi181.stdout:Dec 11 17:40:30 smithi181 sudo[114083]: pam_unix(sudo:session): session closed for user root 2023-12-11T17:40:30.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:30 smithi181 ceph-mon[103627]: osdmap e21: 3 total, 3 up, 3 in 2023-12-11T17:40:30.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:30 smithi181 ceph-mon[103627]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-12-11T17:40:30.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:30 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:40:30.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:30 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/991677140' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5d64d9af-831b-4d0a-8ace-a452169351d6"}]: dispatch 2023-12-11T17:40:30.396 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:30 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/991677140' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5d64d9af-831b-4d0a-8ace-a452169351d6"}]': finished 2023-12-11T17:40:30.396 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:30 smithi181 ceph-mon[103627]: osdmap e22: 4 total, 3 up, 4 in 2023-12-11T17:40:30.396 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:30 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-11T17:40:30.396 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:30 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:40:30.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:30 smithi059 ceph-mon[98847]: osdmap e21: 3 total, 3 up, 3 in 2023-12-11T17:40:30.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:30 smithi059 ceph-mon[98847]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-12-11T17:40:30.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:30 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:40:30.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:30 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/991677140' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5d64d9af-831b-4d0a-8ace-a452169351d6"}]: dispatch 2023-12-11T17:40:30.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:30 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/991677140' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5d64d9af-831b-4d0a-8ace-a452169351d6"}]': finished 2023-12-11T17:40:30.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:30 smithi059 ceph-mon[98847]: osdmap e22: 4 total, 3 up, 4 in 2023-12-11T17:40:30.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:30 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-11T17:40:30.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:30 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:40:30.895 INFO:journalctl@ceph.osd.2.smithi181.stdout:Dec 11 17:40:30 smithi181 sudo[114086]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-12-11T17:40:30.895 INFO:journalctl@ceph.osd.2.smithi181.stdout:Dec 11 17:40:30 smithi181 sudo[114086]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-12-11T17:40:30.895 INFO:journalctl@ceph.osd.2.smithi181.stdout:Dec 11 17:40:30 smithi181 sudo[114086]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-12-11T17:40:30.895 INFO:journalctl@ceph.osd.2.smithi181.stdout:Dec 11 17:40:30 smithi181 sudo[114086]: pam_unix(sudo:session): session closed for user root 2023-12-11T17:40:31.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:30 smithi059 sudo[123829]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-12-11T17:40:31.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:30 smithi059 sudo[123829]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-12-11T17:40:31.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:30 smithi059 sudo[123829]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-12-11T17:40:31.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:31 smithi059 sudo[123829]: pam_unix(sudo:session): session closed for user root 2023-12-11T17:40:31.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:31 smithi059 sudo[123832]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-12-11T17:40:31.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:31 smithi059 sudo[123832]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-12-11T17:40:31.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:31 smithi059 sudo[123832]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-12-11T17:40:31.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:31 smithi059 sudo[123832]: pam_unix(sudo:session): session closed for user root 2023-12-11T17:40:31.946 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:31 smithi181 sudo[114089]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-12-11T17:40:31.947 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:31 smithi181 sudo[114089]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-12-11T17:40:31.947 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:31 smithi181 sudo[114089]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-12-11T17:40:31.947 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:31 smithi181 sudo[114089]: pam_unix(sudo:session): session closed for user root 2023-12-11T17:40:32.207 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:31 smithi115 ceph-mon[102748]: pgmap v80: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-12-11T17:40:32.207 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:31 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:40:32.207 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:31 smithi115 ceph-mon[102748]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-12-11T17:40:32.207 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:31 smithi115 ceph-mon[102748]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-12-11T17:40:32.207 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:31 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-11T17:40:32.207 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:31 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/734729778' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-11T17:40:32.207 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:31 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-11T17:40:32.207 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:31 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-11T17:40:32.208 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:31 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-11T17:40:32.208 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:31 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-11T17:40:32.208 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:31 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-11T17:40:32.208 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:32 smithi115 sudo[113321]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-12-11T17:40:32.208 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:32 smithi115 sudo[113321]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-12-11T17:40:32.208 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:32 smithi115 sudo[113321]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-12-11T17:40:32.208 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:32 smithi115 sudo[113321]: pam_unix(sudo:session): session closed for user root 2023-12-11T17:40:32.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:31 smithi181 sudo[114092]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-12-11T17:40:32.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:31 smithi181 sudo[114092]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-12-11T17:40:32.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:31 smithi181 sudo[114092]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-12-11T17:40:32.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:31 smithi181 sudo[114092]: pam_unix(sudo:session): session closed for user root 2023-12-11T17:40:32.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:31 smithi181 ceph-mon[103627]: pgmap v80: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-12-11T17:40:32.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:31 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:40:32.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:31 smithi181 ceph-mon[103627]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-12-11T17:40:32.396 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:31 smithi181 ceph-mon[103627]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-12-11T17:40:32.396 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:31 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-11T17:40:32.396 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:31 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/734729778' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-11T17:40:32.396 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:31 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-11T17:40:32.396 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:31 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-11T17:40:32.396 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:31 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-11T17:40:32.396 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:31 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-11T17:40:32.396 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:31 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-11T17:40:32.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:31 smithi059 ceph-mon[98847]: pgmap v80: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-12-11T17:40:32.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:31 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:40:32.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:31 smithi059 ceph-mon[98847]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-12-11T17:40:32.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:31 smithi059 ceph-mon[98847]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-12-11T17:40:32.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:31 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-11T17:40:32.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:31 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/734729778' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-11T17:40:32.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:31 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-11T17:40:32.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:31 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-11T17:40:32.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:31 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-11T17:40:32.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:31 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-11T17:40:32.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:31 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-11T17:40:32.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:32 smithi115 sudo[113324]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-12-11T17:40:32.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:32 smithi115 sudo[113324]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-12-11T17:40:32.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:32 smithi115 sudo[113324]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-12-11T17:40:32.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:32 smithi115 sudo[113324]: pam_unix(sudo:session): session closed for user root 2023-12-11T17:40:33.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:32 smithi115 ceph-mon[102748]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-12-11T17:40:33.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:32 smithi115 ceph-mon[102748]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-12-11T17:40:33.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:32 smithi115 ceph-mon[102748]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-12-11T17:40:33.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:32 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-11T17:40:33.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:32 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-11T17:40:33.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:32 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-11T17:40:33.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:32 smithi115 ceph-mon[102748]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-12-11T17:40:33.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:32 smithi181 ceph-mon[103627]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-12-11T17:40:33.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:32 smithi181 ceph-mon[103627]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-12-11T17:40:33.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:32 smithi181 ceph-mon[103627]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-12-11T17:40:33.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:32 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-11T17:40:33.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:32 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-11T17:40:33.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:32 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-11T17:40:33.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:32 smithi181 ceph-mon[103627]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-12-11T17:40:33.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:32 smithi059 ceph-mon[98847]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-12-11T17:40:33.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:32 smithi059 ceph-mon[98847]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-12-11T17:40:33.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:32 smithi059 ceph-mon[98847]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-12-11T17:40:33.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:32 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-11T17:40:33.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:32 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-11T17:40:33.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:32 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-11T17:40:33.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:32 smithi059 ceph-mon[98847]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-12-11T17:40:34.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:33 smithi115 ceph-mon[102748]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-12-11T17:40:34.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:33 smithi181 ceph-mon[103627]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-12-11T17:40:34.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:33 smithi059 ceph-mon[98847]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-12-11T17:40:35.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:35 smithi059 ceph-mon[98847]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-12-11T17:40:35.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:35 smithi181 ceph-mon[103627]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-12-11T17:40:35.532 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:35 smithi115 ceph-mon[102748]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-12-11T17:40:37.233 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:37 smithi059 ceph-mon[98847]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-12-11T17:40:37.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:37 smithi181 ceph-mon[103627]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-12-11T17:40:37.532 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:37 smithi115 ceph-mon[102748]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-12-11T17:40:38.282 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:38 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-12-11T17:40:38.282 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:38 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:40:38.282 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:38 smithi059 ceph-mon[98847]: Deploying daemon osd.3 on smithi059 2023-12-11T17:40:38.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:38 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-12-11T17:40:38.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:38 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:40:38.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:38 smithi181 ceph-mon[103627]: Deploying daemon osd.3 on smithi059 2023-12-11T17:40:38.532 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:38 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-12-11T17:40:38.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:38 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:40:38.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:38 smithi115 ceph-mon[102748]: Deploying daemon osd.3 on smithi059 2023-12-11T17:40:39.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:39 smithi181 ceph-mon[103627]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2023-12-11T17:40:39.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:39 smithi059 ceph-mon[98847]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2023-12-11T17:40:39.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:39 smithi115 ceph-mon[102748]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2023-12-11T17:40:41.377 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:41 smithi059 ceph-mon[98847]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2023-12-11T17:40:41.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:41 smithi181 ceph-mon[103627]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2023-12-11T17:40:41.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:41 smithi115 ceph-mon[102748]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2023-12-11T17:40:42.815 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:42 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:40:42.815 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:42 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:40:42.815 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:42 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:40:42.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:42 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:40:42.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:42 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:40:42.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:42 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:40:43.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:42 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:40:43.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:42 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:40:43.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:42 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:40:43.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:43 smithi181 ceph-mon[103627]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2023-12-11T17:40:43.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:43 smithi059 ceph-mon[98847]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2023-12-11T17:40:44.032 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:43 smithi115 ceph-mon[102748]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2023-12-11T17:40:44.439 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:40:44.437+0000 7fe443fff700 1 -- 172.21.15.59:0/373835384 <== mgr.14150 v2:172.21.15.59:6800/3627728353 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fe45c061a10 con 0x7fe4440a13f0 2023-12-11T17:40:44.439 INFO:teuthology.orchestra.run.smithi059.stdout:Created osd(s) 3 on host 'smithi059' 2023-12-11T17:40:44.442 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:40:44.439+0000 7fe461582700 1 -- 172.21.15.59:0/373835384 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fe4440a13f0 msgr2=0x7fe4440a38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:40:44.442 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:40:44.439+0000 7fe461582700 1 --2- 172.21.15.59:0/373835384 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fe4440a13f0 0x7fe4440a38b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe450005fd0 tx=0x7fe45000a560 comp rx=0 tx=0).stop 2023-12-11T17:40:44.442 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:40:44.439+0000 7fe461582700 1 -- 172.21.15.59:0/373835384 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe45c0ff680 msgr2=0x7fe45c1022b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:40:44.442 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:40:44.439+0000 7fe461582700 1 --2- 172.21.15.59:0/373835384 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe45c0ff680 0x7fe45c1022b0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe44c0098e0 tx=0x7fe44c00f7b0 comp rx=0 tx=0).stop 2023-12-11T17:40:44.442 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:40:44.439+0000 7fe461582700 1 -- 172.21.15.59:0/373835384 shutdown_connections 2023-12-11T17:40:44.442 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:40:44.439+0000 7fe461582700 1 --2- 172.21.15.59:0/373835384 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fe4440a13f0 0x7fe4440a38b0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:40:44.442 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:40:44.439+0000 7fe461582700 1 --2- 172.21.15.59:0/373835384 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe45c102c00 0x7fe45c100e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:40:44.442 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:40:44.439+0000 7fe461582700 1 --2- 172.21.15.59:0/373835384 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe45c0ff680 0x7fe45c1022b0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:40:44.442 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:40:44.439+0000 7fe461582700 1 --2- 172.21.15.59:0/373835384 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe45c100020 0x7fe45c100950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:40:44.442 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:40:44.439+0000 7fe461582700 1 -- 172.21.15.59:0/373835384 >> 172.21.15.59:0/373835384 conn(0x7fe45c0fb260 msgr2=0x7fe45c105d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:40:44.442 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:40:44.439+0000 7fe461582700 1 -- 172.21.15.59:0/373835384 shutdown_connections 2023-12-11T17:40:44.442 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:40:44.440+0000 7fe461582700 1 -- 172.21.15.59:0/373835384 wait complete. 2023-12-11T17:40:44.836 DEBUG:teuthology.orchestra.run.smithi059:osd.3> sudo journalctl -f -n 0 -u ceph-d0a07eaa-984b-11ee-95a3-87774f69a715@osd.3.service 2023-12-11T17:40:44.839 INFO:tasks.cephadm:Deploying osd.4 on smithi115 with /dev/vg_nvme/lv_3... 2023-12-11T17:40:44.839 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d0a07eaa-984b-11ee-95a3-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-12-11T17:40:44.877 INFO:journalctl@ceph.osd.3.smithi059.stdout:-- Logs begin at Mon 2023-12-11 17:22:26 UTC. -- 2023-12-11T17:40:45.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:44 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:40:45.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:44 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:40:45.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:44 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:40:45.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:44 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:40:45.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:44 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:40:45.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:44 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:40:45.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:44 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:40:45.155 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:44 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:40:45.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:44 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:40:45.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:44 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:40:45.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:44 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:40:45.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:44 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:40:45.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:44 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:40:45.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:44 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:40:45.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:44 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:40:45.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:44 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:40:45.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:44 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:40:45.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:44 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:40:45.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:44 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:40:45.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:44 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:40:45.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:44 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:40:45.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:44 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:40:45.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:44 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:40:45.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:44 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:40:45.439 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/mon.b/config 2023-12-11T17:40:45.904 INFO:journalctl@ceph.osd.3.smithi059.stdout:Dec 11 17:40:45 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-3[126028]: 2023-12-11T17:40:45.587+0000 7fe39a958640 -1 osd.3 0 log_to_monitors true 2023-12-11T17:40:46.153 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:45 smithi115 ceph-mon[102748]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2023-12-11T17:40:46.154 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:45 smithi115 ceph-mon[102748]: from='osd.3 [v2:172.21.15.59:6810/219909995,v1:172.21.15.59:6811/219909995]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-12-11T17:40:46.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:45 smithi181 ceph-mon[103627]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2023-12-11T17:40:46.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:45 smithi181 ceph-mon[103627]: from='osd.3 [v2:172.21.15.59:6810/219909995,v1:172.21.15.59:6811/219909995]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-12-11T17:40:46.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:45 smithi059 ceph-mon[98847]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2023-12-11T17:40:46.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:45 smithi059 ceph-mon[98847]: from='osd.3 [v2:172.21.15.59:6810/219909995,v1:172.21.15.59:6811/219909995]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-12-11T17:40:46.913 INFO:teuthology.orchestra.run.smithi115.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-12-11T17:40:46.913 INFO:teuthology.orchestra.run.smithi115.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-12-11T17:40:46.913 INFO:teuthology.orchestra.run.smithi115.stderr: stderr: 10+0 records in 2023-12-11T17:40:46.913 INFO:teuthology.orchestra.run.smithi115.stderr:10+0 records out 2023-12-11T17:40:46.913 INFO:teuthology.orchestra.run.smithi115.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0127021 s, 826 MB/s 2023-12-11T17:40:46.913 INFO:teuthology.orchestra.run.smithi115.stderr: stderr: 2023-12-11T17:40:46.913 INFO:teuthology.orchestra.run.smithi115.stderr:--> Zapping successful for: 2023-12-11T17:40:47.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:47 smithi115 ceph-mon[102748]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2023-12-11T17:40:47.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:47 smithi115 ceph-mon[102748]: from='osd.3 [v2:172.21.15.59:6810/219909995,v1:172.21.15.59:6811/219909995]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-12-11T17:40:47.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:47 smithi115 ceph-mon[102748]: osdmap e23: 4 total, 3 up, 4 in 2023-12-11T17:40:47.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:47 smithi115 ceph-mon[102748]: from='osd.3 [v2:172.21.15.59:6810/219909995,v1:172.21.15.59:6811/219909995]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2023-12-11T17:40:47.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:47 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-11T17:40:47.285 INFO:journalctl@ceph.osd.3.smithi059.stdout:Dec 11 17:40:47 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-3[126028]: 2023-12-11T17:40:47.075+0000 7fe38ffcf700 -1 osd.3 0 waiting for initial osdmap 2023-12-11T17:40:47.285 INFO:journalctl@ceph.osd.3.smithi059.stdout:Dec 11 17:40:47 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-3[126028]: 2023-12-11T17:40:47.081+0000 7fe38bdc9700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-11T17:40:47.286 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:47 smithi059 ceph-mon[98847]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2023-12-11T17:40:47.286 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:47 smithi059 ceph-mon[98847]: from='osd.3 [v2:172.21.15.59:6810/219909995,v1:172.21.15.59:6811/219909995]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-12-11T17:40:47.286 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:47 smithi059 ceph-mon[98847]: osdmap e23: 4 total, 3 up, 4 in 2023-12-11T17:40:47.286 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:47 smithi059 ceph-mon[98847]: from='osd.3 [v2:172.21.15.59:6810/219909995,v1:172.21.15.59:6811/219909995]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2023-12-11T17:40:47.286 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:47 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-11T17:40:47.297 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d0a07eaa-984b-11ee-95a3-87774f69a715 -- ceph orch daemon add osd smithi115:vg_nvme/lv_3 2023-12-11T17:40:47.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:47 smithi181 ceph-mon[103627]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2023-12-11T17:40:47.667 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:47 smithi181 ceph-mon[103627]: from='osd.3 [v2:172.21.15.59:6810/219909995,v1:172.21.15.59:6811/219909995]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-12-11T17:40:47.667 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:47 smithi181 ceph-mon[103627]: osdmap e23: 4 total, 3 up, 4 in 2023-12-11T17:40:47.667 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:47 smithi181 ceph-mon[103627]: from='osd.3 [v2:172.21.15.59:6810/219909995,v1:172.21.15.59:6811/219909995]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2023-12-11T17:40:47.667 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:47 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-11T17:40:47.896 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/mon.b/config 2023-12-11T17:40:48.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:48 smithi181 ceph-mon[103627]: from='osd.3 [v2:172.21.15.59:6810/219909995,v1:172.21.15.59:6811/219909995]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]': finished 2023-12-11T17:40:48.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:48 smithi181 ceph-mon[103627]: osdmap e24: 4 total, 3 up, 4 in 2023-12-11T17:40:48.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:48 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-11T17:40:48.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:48 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-11T17:40:48.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:48 smithi181 ceph-mon[103627]: Detected new or changed devices on smithi059 2023-12-11T17:40:48.396 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:48 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:40:48.396 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:48 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:40:48.396 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:48 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:40:48.396 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:48 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:40:48.396 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:48 smithi181 ceph-mon[103627]: Adjusting osd_memory_target on smithi059 to 8586M 2023-12-11T17:40:48.396 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:48 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:40:48.396 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:48 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:40:48.396 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:48 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:40:48.396 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:48 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:40:48.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:48 smithi059 ceph-mon[98847]: from='osd.3 [v2:172.21.15.59:6810/219909995,v1:172.21.15.59:6811/219909995]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]': finished 2023-12-11T17:40:48.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:48 smithi059 ceph-mon[98847]: osdmap e24: 4 total, 3 up, 4 in 2023-12-11T17:40:48.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:48 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-11T17:40:48.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:48 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-11T17:40:48.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:48 smithi059 ceph-mon[98847]: Detected new or changed devices on smithi059 2023-12-11T17:40:48.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:48 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:40:48.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:48 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:40:48.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:48 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:40:48.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:48 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:40:48.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:48 smithi059 ceph-mon[98847]: Adjusting osd_memory_target on smithi059 to 8586M 2023-12-11T17:40:48.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:48 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:40:48.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:48 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:40:48.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:48 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:40:48.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:48 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:40:48.424 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:48 smithi115 ceph-mon[102748]: from='osd.3 [v2:172.21.15.59:6810/219909995,v1:172.21.15.59:6811/219909995]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]': finished 2023-12-11T17:40:48.424 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:48 smithi115 ceph-mon[102748]: osdmap e24: 4 total, 3 up, 4 in 2023-12-11T17:40:48.424 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:48 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-11T17:40:48.424 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:48 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-11T17:40:48.424 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:48 smithi115 ceph-mon[102748]: Detected new or changed devices on smithi059 2023-12-11T17:40:48.424 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:48 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:40:48.424 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:48 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:40:48.424 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:48 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:40:48.424 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:48 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:40:48.424 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:48 smithi115 ceph-mon[102748]: Adjusting osd_memory_target on smithi059 to 8586M 2023-12-11T17:40:48.424 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:48 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:40:48.425 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:48 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:40:48.425 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:48 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:40:48.425 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:48 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:40:49.114 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:40:49.111+0000 7fe37f0e7700 1 -- 172.21.15.115:0/4093460482 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe3781007e0 msgr2=0x7fe378100c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:40:49.114 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:40:49.111+0000 7fe37f0e7700 1 --2- 172.21.15.115:0/4093460482 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe3781007e0 0x7fe378100c40 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fe3640098f0 tx=0x7fe364009c00 comp rx=0 tx=0).stop 2023-12-11T17:40:49.114 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:40:49.111+0000 7fe37f0e7700 1 -- 172.21.15.115:0/4093460482 shutdown_connections 2023-12-11T17:40:49.114 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:40:49.111+0000 7fe37f0e7700 1 --2- 172.21.15.115:0/4093460482 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe378101180 0x7fe378108a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:40:49.114 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:40:49.111+0000 7fe37f0e7700 1 --2- 172.21.15.115:0/4093460482 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe378101aa0 0x7fe378101ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:40:49.114 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:40:49.111+0000 7fe37f0e7700 1 --2- 172.21.15.115:0/4093460482 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe3781007e0 0x7fe378100c40 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:40:49.114 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:40:49.111+0000 7fe37f0e7700 1 -- 172.21.15.115:0/4093460482 >> 172.21.15.115:0/4093460482 conn(0x7fe3780fb230 msgr2=0x7fe3780fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:40:49.114 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:40:49.111+0000 7fe37f0e7700 1 -- 172.21.15.115:0/4093460482 shutdown_connections 2023-12-11T17:40:49.114 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:40:49.111+0000 7fe37f0e7700 1 -- 172.21.15.115:0/4093460482 wait complete. 2023-12-11T17:40:49.121 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:40:49.112+0000 7fe37f0e7700 1 Processor -- start 2023-12-11T17:40:49.121 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:40:49.112+0000 7fe37f0e7700 1 -- start start 2023-12-11T17:40:49.121 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:40:49.112+0000 7fe37f0e7700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe3781007e0 0x7fe378196490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:40:49.121 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:40:49.112+0000 7fe37f0e7700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe378101180 0x7fe3781969d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:40:49.121 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:40:49.112+0000 7fe37f0e7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe378101aa0 0x7fe37819da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:40:49.121 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:40:49.112+0000 7fe37f0e7700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe37819dfd0 con 0x7fe378101180 2023-12-11T17:40:49.121 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:40:49.112+0000 7fe37f0e7700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe37819e110 con 0x7fe3781007e0 2023-12-11T17:40:49.121 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:40:49.112+0000 7fe37f0e7700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe37819e2a0 con 0x7fe378101aa0 2023-12-11T17:40:49.122 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:40:49.112+0000 7fe37ce83700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe3781007e0 0x7fe378196490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:40:49.122 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:40:49.112+0000 7fe37ce83700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe3781007e0 0x7fe378196490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:54734/0 (socket says 172.21.15.115:54734) 2023-12-11T17:40:49.122 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:40:49.112+0000 7fe37ce83700 1 -- 172.21.15.115:0/480438360 learned_addr learned my addr 172.21.15.115:0/480438360 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-11T17:40:49.122 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:40:49.112+0000 7fe377fff700 1 --2- 172.21.15.115:0/480438360 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe378101180 0x7fe3781969d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:40:49.122 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:40:49.112+0000 7fe37ce83700 1 -- 172.21.15.115:0/480438360 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe378101aa0 msgr2=0x7fe37819da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:40:49.122 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:40:49.112+0000 7fe37d684700 1 --2- 172.21.15.115:0/480438360 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe378101aa0 0x7fe37819da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:40:49.122 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:40:49.112+0000 7fe37ce83700 1 --2- 172.21.15.115:0/480438360 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe378101aa0 0x7fe37819da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:40:49.122 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:40:49.112+0000 7fe37ce83700 1 -- 172.21.15.115:0/480438360 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe378101180 msgr2=0x7fe3781969d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:40:49.122 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:40:49.112+0000 7fe37ce83700 1 --2- 172.21.15.115:0/480438360 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe378101180 0x7fe3781969d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:40:49.122 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:40:49.112+0000 7fe37ce83700 1 -- 172.21.15.115:0/480438360 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe364009520 con 0x7fe3781007e0 2023-12-11T17:40:49.122 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:40:49.112+0000 7fe377fff700 1 --2- 172.21.15.115:0/480438360 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe378101180 0x7fe3781969d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T17:40:49.123 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:40:49.112+0000 7fe37d684700 1 --2- 172.21.15.115:0/480438360 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe378101aa0 0x7fe37819da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-11T17:40:49.123 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:40:49.112+0000 7fe37ce83700 1 --2- 172.21.15.115:0/480438360 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe3781007e0 0x7fe378196490 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe36c00ea80 tx=0x7fe36c00ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:40:49.123 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:40:49.113+0000 7fe375ffb700 1 -- 172.21.15.115:0/480438360 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe36c013070 con 0x7fe3781007e0 2023-12-11T17:40:49.123 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:40:49.113+0000 7fe37f0e7700 1 -- 172.21.15.115:0/480438360 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3780ff730 con 0x7fe3781007e0 2023-12-11T17:40:49.123 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:40:49.113+0000 7fe375ffb700 1 -- 172.21.15.115:0/480438360 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe36c004020 con 0x7fe3781007e0 2023-12-11T17:40:49.123 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:40:49.113+0000 7fe375ffb700 1 -- 172.21.15.115:0/480438360 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe36c003c70 con 0x7fe3781007e0 2023-12-11T17:40:49.123 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:40:49.113+0000 7fe37f0e7700 1 -- 172.21.15.115:0/480438360 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3780ffc80 con 0x7fe3781007e0 2023-12-11T17:40:49.123 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:40:49.114+0000 7fe37f0e7700 1 -- 172.21.15.115:0/480438360 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe37810c650 con 0x7fe3781007e0 2023-12-11T17:40:49.123 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:40:49.115+0000 7fe375ffb700 1 -- 172.21.15.115:0/480438360 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7fe36c004190 con 0x7fe3781007e0 2023-12-11T17:40:49.123 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:40:49.115+0000 7fe375ffb700 1 --2- 172.21.15.115:0/480438360 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fe3680a1480 0x7fe3680a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:40:49.123 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:40:49.116+0000 7fe375ffb700 1 -- 172.21.15.115:0/480438360 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 3698+0+0 (secure 0 0 0) 0x7fe36c0cd2f0 con 0x7fe3781007e0 2023-12-11T17:40:49.123 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:40:49.116+0000 7fe377fff700 1 --2- 172.21.15.115:0/480438360 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fe3680a1480 0x7fe3680a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:40:49.123 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:40:49.116+0000 7fe377fff700 1 --2- 172.21.15.115:0/480438360 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fe3680a1480 0x7fe3680a3940 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe36400fa40 tx=0x7fe364019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:40:49.124 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:40:49.120+0000 7fe375ffb700 1 -- 172.21.15.115:0/480438360 <== mon.2 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fe36c097b50 con 0x7fe3781007e0 2023-12-11T17:40:49.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:49 smithi115 ceph-mon[102748]: purged_snaps scrub starts 2023-12-11T17:40:49.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:49 smithi115 ceph-mon[102748]: purged_snaps scrub ok 2023-12-11T17:40:49.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:49 smithi115 ceph-mon[102748]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-12-11T17:40:49.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:49 smithi115 ceph-mon[102748]: osd.3 [v2:172.21.15.59:6810/219909995,v1:172.21.15.59:6811/219909995] boot 2023-12-11T17:40:49.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:49 smithi115 ceph-mon[102748]: osdmap e25: 4 total, 4 up, 4 in 2023-12-11T17:40:49.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:49 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-11T17:40:49.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:49 smithi115 ceph-mon[102748]: osdmap e26: 4 total, 4 up, 4 in 2023-12-11T17:40:49.319 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:40:49.316+0000 7fe37f0e7700 1 -- 172.21.15.115:0/480438360 --> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fe3780ffea0 con 0x7fe3680a1480 2023-12-11T17:40:49.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:49 smithi181 ceph-mon[103627]: purged_snaps scrub starts 2023-12-11T17:40:49.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:49 smithi181 ceph-mon[103627]: purged_snaps scrub ok 2023-12-11T17:40:49.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:49 smithi181 ceph-mon[103627]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-12-11T17:40:49.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:49 smithi181 ceph-mon[103627]: osd.3 [v2:172.21.15.59:6810/219909995,v1:172.21.15.59:6811/219909995] boot 2023-12-11T17:40:49.396 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:49 smithi181 ceph-mon[103627]: osdmap e25: 4 total, 4 up, 4 in 2023-12-11T17:40:49.396 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:49 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-11T17:40:49.396 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:49 smithi181 ceph-mon[103627]: osdmap e26: 4 total, 4 up, 4 in 2023-12-11T17:40:49.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:49 smithi059 ceph-mon[98847]: purged_snaps scrub starts 2023-12-11T17:40:49.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:49 smithi059 ceph-mon[98847]: purged_snaps scrub ok 2023-12-11T17:40:49.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:49 smithi059 ceph-mon[98847]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-12-11T17:40:49.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:49 smithi059 ceph-mon[98847]: osd.3 [v2:172.21.15.59:6810/219909995,v1:172.21.15.59:6811/219909995] boot 2023-12-11T17:40:49.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:49 smithi059 ceph-mon[98847]: osdmap e25: 4 total, 4 up, 4 in 2023-12-11T17:40:49.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:49 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-11T17:40:49.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:49 smithi059 ceph-mon[98847]: osdmap e26: 4 total, 4 up, 4 in 2023-12-11T17:40:50.373 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:50 smithi115 ceph-mon[102748]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T17:40:50.373 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:50 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-11T17:40:50.373 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:50 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-11T17:40:50.373 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:50 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:40:50.374 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:50 smithi115 ceph-mon[102748]: osdmap e27: 4 total, 4 up, 4 in 2023-12-11T17:40:50.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:50 smithi181 ceph-mon[103627]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T17:40:50.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:50 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-11T17:40:50.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:50 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-11T17:40:50.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:50 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:40:50.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:50 smithi181 ceph-mon[103627]: osdmap e27: 4 total, 4 up, 4 in 2023-12-11T17:40:50.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:50 smithi059 ceph-mon[98847]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T17:40:50.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:50 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-11T17:40:50.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:50 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-11T17:40:50.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:50 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:40:50.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:50 smithi059 ceph-mon[98847]: osdmap e27: 4 total, 4 up, 4 in 2023-12-11T17:40:51.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:51 smithi181 ceph-mon[103627]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-12-11T17:40:51.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:51 smithi059 ceph-mon[98847]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-12-11T17:40:51.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:51 smithi115 ceph-mon[102748]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-12-11T17:40:52.396 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:52 smithi181 ceph-mon[103627]: osdmap e28: 4 total, 4 up, 4 in 2023-12-11T17:40:52.396 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:52 smithi181 ceph-mon[103627]: from='client.? 172.21.15.115:0/2438726628' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ef8fad28-1db5-4cba-b5f2-10e54e8e1549"}]: dispatch 2023-12-11T17:40:52.396 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:52 smithi181 ceph-mon[103627]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ef8fad28-1db5-4cba-b5f2-10e54e8e1549"}]: dispatch 2023-12-11T17:40:52.396 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:52 smithi181 ceph-mon[103627]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ef8fad28-1db5-4cba-b5f2-10e54e8e1549"}]': finished 2023-12-11T17:40:52.396 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:52 smithi181 ceph-mon[103627]: osdmap e29: 5 total, 4 up, 5 in 2023-12-11T17:40:52.396 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:52 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-11T17:40:52.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:52 smithi059 ceph-mon[98847]: osdmap e28: 4 total, 4 up, 4 in 2023-12-11T17:40:52.406 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:52 smithi059 ceph-mon[98847]: from='client.? 172.21.15.115:0/2438726628' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ef8fad28-1db5-4cba-b5f2-10e54e8e1549"}]: dispatch 2023-12-11T17:40:52.406 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:52 smithi059 ceph-mon[98847]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ef8fad28-1db5-4cba-b5f2-10e54e8e1549"}]: dispatch 2023-12-11T17:40:52.406 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:52 smithi059 ceph-mon[98847]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ef8fad28-1db5-4cba-b5f2-10e54e8e1549"}]': finished 2023-12-11T17:40:52.406 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:52 smithi059 ceph-mon[98847]: osdmap e29: 5 total, 4 up, 5 in 2023-12-11T17:40:52.406 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:52 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-11T17:40:52.534 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:52 smithi115 ceph-mon[102748]: osdmap e28: 4 total, 4 up, 4 in 2023-12-11T17:40:52.534 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:52 smithi115 ceph-mon[102748]: from='client.? 172.21.15.115:0/2438726628' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ef8fad28-1db5-4cba-b5f2-10e54e8e1549"}]: dispatch 2023-12-11T17:40:52.535 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:52 smithi115 ceph-mon[102748]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ef8fad28-1db5-4cba-b5f2-10e54e8e1549"}]: dispatch 2023-12-11T17:40:52.535 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:52 smithi115 ceph-mon[102748]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ef8fad28-1db5-4cba-b5f2-10e54e8e1549"}]': finished 2023-12-11T17:40:52.535 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:52 smithi115 ceph-mon[102748]: osdmap e29: 5 total, 4 up, 5 in 2023-12-11T17:40:52.535 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:52 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-11T17:40:53.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:53 smithi181 ceph-mon[103627]: pgmap v98: 1 pgs: 1 peering; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-12-11T17:40:53.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:53 smithi181 ceph-mon[103627]: from='client.? 172.21.15.115:0/3768288896' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-11T17:40:53.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:53 smithi059 ceph-mon[98847]: pgmap v98: 1 pgs: 1 peering; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-12-11T17:40:53.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:53 smithi059 ceph-mon[98847]: from='client.? 172.21.15.115:0/3768288896' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-11T17:40:53.532 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:53 smithi115 ceph-mon[102748]: pgmap v98: 1 pgs: 1 peering; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-12-11T17:40:53.532 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:53 smithi115 ceph-mon[102748]: from='client.? 172.21.15.115:0/3768288896' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-11T17:40:55.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:55 smithi181 ceph-mon[103627]: pgmap v99: 1 pgs: 1 peering; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 115 KiB/s, 0 objects/s recovering 2023-12-11T17:40:55.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:55 smithi059 ceph-mon[98847]: pgmap v99: 1 pgs: 1 peering; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 115 KiB/s, 0 objects/s recovering 2023-12-11T17:40:55.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:55 smithi115 ceph-mon[102748]: pgmap v99: 1 pgs: 1 peering; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 115 KiB/s, 0 objects/s recovering 2023-12-11T17:40:57.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:57 smithi115 ceph-mon[102748]: pgmap v100: 1 pgs: 1 peering; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-12-11T17:40:57.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:57 smithi181 ceph-mon[103627]: pgmap v100: 1 pgs: 1 peering; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-12-11T17:40:57.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:57 smithi059 ceph-mon[98847]: pgmap v100: 1 pgs: 1 peering; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-12-11T17:40:59.246 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:59 smithi115 ceph-mon[102748]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-12-11T17:40:59.246 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:59 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-12-11T17:40:59.246 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:59 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:40:59.247 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:40:59 smithi115 ceph-mon[102748]: Deploying daemon osd.4 on smithi115 2023-12-11T17:40:59.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:59 smithi181 ceph-mon[103627]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-12-11T17:40:59.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:59 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-12-11T17:40:59.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:59 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:40:59.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:40:59 smithi181 ceph-mon[103627]: Deploying daemon osd.4 on smithi115 2023-12-11T17:40:59.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:59 smithi059 ceph-mon[98847]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-12-11T17:40:59.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:59 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-12-11T17:40:59.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:59 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:40:59.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:40:59 smithi059 ceph-mon[98847]: Deploying daemon osd.4 on smithi115 2023-12-11T17:41:01.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:01 smithi181 ceph-mon[103627]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 64 KiB/s, 0 objects/s recovering 2023-12-11T17:41:01.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:01 smithi059 ceph-mon[98847]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 64 KiB/s, 0 objects/s recovering 2023-12-11T17:41:01.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:01 smithi115 ceph-mon[102748]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 64 KiB/s, 0 objects/s recovering 2023-12-11T17:41:03.987 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:03 smithi115 ceph-mon[102748]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 56 KiB/s, 0 objects/s recovering 2023-12-11T17:41:03.987 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:03 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:41:03.987 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:03 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:41:03.987 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:03 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:41:04.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:03 smithi181 ceph-mon[103627]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 56 KiB/s, 0 objects/s recovering 2023-12-11T17:41:04.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:03 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:41:04.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:03 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:41:04.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:03 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:41:04.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:03 smithi059 ceph-mon[98847]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 56 KiB/s, 0 objects/s recovering 2023-12-11T17:41:04.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:03 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:41:04.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:03 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:41:04.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:03 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:41:05.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:05 smithi181 ceph-mon[103627]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-12-11T17:41:05.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:05 smithi059 ceph-mon[98847]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-12-11T17:41:05.486 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:05 smithi115 ceph-mon[102748]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-12-11T17:41:05.584 INFO:teuthology.orchestra.run.smithi115.stdout:Created osd(s) 4 on host 'smithi115' 2023-12-11T17:41:05.585 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:41:05.582+0000 7fe375ffb700 1 -- 172.21.15.115:0/480438360 <== mgr.14150 v2:172.21.15.59:6800/3627728353 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fe3780ffea0 con 0x7fe3680a1480 2023-12-11T17:41:05.586 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:41:05.584+0000 7fe37f0e7700 1 -- 172.21.15.115:0/480438360 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fe3680a1480 msgr2=0x7fe3680a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:41:05.586 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:41:05.584+0000 7fe37f0e7700 1 --2- 172.21.15.115:0/480438360 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fe3680a1480 0x7fe3680a3940 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe36400fa40 tx=0x7fe364019040 comp rx=0 tx=0).stop 2023-12-11T17:41:05.586 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:41:05.584+0000 7fe37f0e7700 1 -- 172.21.15.115:0/480438360 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe3781007e0 msgr2=0x7fe378196490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:41:05.586 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:41:05.584+0000 7fe37f0e7700 1 --2- 172.21.15.115:0/480438360 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe3781007e0 0x7fe378196490 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe36c00ea80 tx=0x7fe36c00ee40 comp rx=0 tx=0).stop 2023-12-11T17:41:05.586 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:41:05.584+0000 7fe37f0e7700 1 -- 172.21.15.115:0/480438360 shutdown_connections 2023-12-11T17:41:05.586 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:41:05.584+0000 7fe37f0e7700 1 --2- 172.21.15.115:0/480438360 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fe3680a1480 0x7fe3680a3940 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:41:05.587 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:41:05.584+0000 7fe37f0e7700 1 --2- 172.21.15.115:0/480438360 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe378101aa0 0x7fe37819da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:41:05.587 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:41:05.584+0000 7fe37f0e7700 1 --2- 172.21.15.115:0/480438360 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe3781007e0 0x7fe378196490 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:41:05.587 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:41:05.584+0000 7fe37f0e7700 1 --2- 172.21.15.115:0/480438360 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe378101180 0x7fe3781969d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:41:05.587 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:41:05.584+0000 7fe37f0e7700 1 -- 172.21.15.115:0/480438360 >> 172.21.15.115:0/480438360 conn(0x7fe3780fb230 msgr2=0x7fe378104c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:41:05.587 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:41:05.584+0000 7fe37f0e7700 1 -- 172.21.15.115:0/480438360 shutdown_connections 2023-12-11T17:41:05.587 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:41:05.584+0000 7fe37f0e7700 1 -- 172.21.15.115:0/480438360 wait complete. 2023-12-11T17:41:06.016 DEBUG:teuthology.orchestra.run.smithi115:osd.4> sudo journalctl -f -n 0 -u ceph-d0a07eaa-984b-11ee-95a3-87774f69a715@osd.4.service 2023-12-11T17:41:06.018 INFO:tasks.cephadm:Deploying osd.5 on smithi181 with /dev/vg_nvme/lv_3... 2023-12-11T17:41:06.018 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d0a07eaa-984b-11ee-95a3-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-12-11T17:41:06.047 INFO:journalctl@ceph.osd.4.smithi115.stdout:-- Logs begin at Mon 2023-12-11 17:22:43 UTC. -- 2023-12-11T17:41:06.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:06 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:41:06.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:06 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:41:06.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:06 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:41:06.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:06 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:41:06.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:06 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:41:06.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:06 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:41:06.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:06 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:41:06.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:06 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:41:06.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:06 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:41:06.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:06 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:41:06.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:06 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:41:06.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:06 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:41:06.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:06 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:41:06.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:06 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:41:06.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:06 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:41:06.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:06 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:41:06.621 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/mon.c/config 2023-12-11T17:41:06.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:06 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:41:06.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:06 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:41:06.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:06 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:41:06.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:06 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:41:06.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:06 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:41:06.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:06 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:41:06.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:06 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:41:06.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:06 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:41:06.778 INFO:journalctl@ceph.osd.4.smithi115.stdout:Dec 11 17:41:06 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-4[116520]: 2023-12-11T17:41:06.719+0000 7ff71f734640 -1 osd.4 0 log_to_monitors true 2023-12-11T17:41:07.896 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:07 smithi181 ceph-mon[103627]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-12-11T17:41:07.896 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:07 smithi181 ceph-mon[103627]: from='osd.4 [v2:172.21.15.115:6808/1021473826,v1:172.21.15.115:6809/1021473826]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-12-11T17:41:07.896 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:07 smithi181 ceph-mon[103627]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-12-11T17:41:07.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:07 smithi059 ceph-mon[98847]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-12-11T17:41:07.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:07 smithi059 ceph-mon[98847]: from='osd.4 [v2:172.21.15.115:6808/1021473826,v1:172.21.15.115:6809/1021473826]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-12-11T17:41:07.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:07 smithi059 ceph-mon[98847]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-12-11T17:41:08.020 INFO:teuthology.orchestra.run.smithi181.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-12-11T17:41:08.020 INFO:teuthology.orchestra.run.smithi181.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-12-11T17:41:08.021 INFO:teuthology.orchestra.run.smithi181.stderr: stderr: 10+0 records in 2023-12-11T17:41:08.021 INFO:teuthology.orchestra.run.smithi181.stderr:10+0 records out 2023-12-11T17:41:08.021 INFO:teuthology.orchestra.run.smithi181.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0111585 s, 940 MB/s 2023-12-11T17:41:08.021 INFO:teuthology.orchestra.run.smithi181.stderr:--> Zapping successful for: 2023-12-11T17:41:08.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:07 smithi115 ceph-mon[102748]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-12-11T17:41:08.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:07 smithi115 ceph-mon[102748]: from='osd.4 [v2:172.21.15.115:6808/1021473826,v1:172.21.15.115:6809/1021473826]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-12-11T17:41:08.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:07 smithi115 ceph-mon[102748]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-12-11T17:41:08.471 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d0a07eaa-984b-11ee-95a3-87774f69a715 -- ceph orch daemon add osd smithi181:vg_nvme/lv_3 2023-12-11T17:41:08.783 INFO:journalctl@ceph.osd.4.smithi115.stdout:Dec 11 17:41:08 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-4[116520]: 2023-12-11T17:41:08.595+0000 7ff714dab700 -1 osd.4 0 waiting for initial osdmap 2023-12-11T17:41:08.783 INFO:journalctl@ceph.osd.4.smithi115.stdout:Dec 11 17:41:08 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-4[116520]: 2023-12-11T17:41:08.602+0000 7ff70eba1700 -1 osd.4 31 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-11T17:41:08.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:08 smithi115 ceph-mon[102748]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-12-11T17:41:08.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:08 smithi115 ceph-mon[102748]: osdmap e30: 5 total, 4 up, 5 in 2023-12-11T17:41:08.784 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:08 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-11T17:41:08.784 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:08 smithi115 ceph-mon[102748]: from='osd.4 [v2:172.21.15.115:6808/1021473826,v1:172.21.15.115:6809/1021473826]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2023-12-11T17:41:08.784 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:08 smithi115 ceph-mon[102748]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2023-12-11T17:41:08.784 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:08 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:41:08.784 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:08 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:41:08.784 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:08 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:41:08.784 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:08 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:41:08.784 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:08 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:41:08.784 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:08 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:41:08.784 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:08 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:41:08.784 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:08 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:41:08.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:08 smithi181 ceph-mon[103627]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-12-11T17:41:08.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:08 smithi181 ceph-mon[103627]: osdmap e30: 5 total, 4 up, 5 in 2023-12-11T17:41:08.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:08 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-11T17:41:08.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:08 smithi181 ceph-mon[103627]: from='osd.4 [v2:172.21.15.115:6808/1021473826,v1:172.21.15.115:6809/1021473826]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2023-12-11T17:41:08.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:08 smithi181 ceph-mon[103627]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2023-12-11T17:41:08.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:08 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:41:08.896 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:08 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:41:08.896 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:08 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:41:08.896 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:08 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:41:08.896 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:08 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:41:08.896 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:08 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:41:08.896 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:08 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:41:08.896 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:08 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:41:08.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:08 smithi059 ceph-mon[98847]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-12-11T17:41:08.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:08 smithi059 ceph-mon[98847]: osdmap e30: 5 total, 4 up, 5 in 2023-12-11T17:41:08.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:08 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-11T17:41:08.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:08 smithi059 ceph-mon[98847]: from='osd.4 [v2:172.21.15.115:6808/1021473826,v1:172.21.15.115:6809/1021473826]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2023-12-11T17:41:08.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:08 smithi059 ceph-mon[98847]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2023-12-11T17:41:08.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:08 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:41:08.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:08 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:41:08.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:08 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:41:08.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:08 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:41:08.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:08 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:41:08.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:08 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:41:08.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:08 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:41:08.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:08 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:41:09.064 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/mon.c/config 2023-12-11T17:41:09.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:09 smithi181 ceph-mon[103627]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-12-11T17:41:09.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:09 smithi181 ceph-mon[103627]: Detected new or changed devices on smithi115 2023-12-11T17:41:09.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:09 smithi181 ceph-mon[103627]: Adjusting osd_memory_target on smithi115 to 8584M 2023-12-11T17:41:09.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:09 smithi181 ceph-mon[103627]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2023-12-11T17:41:09.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:09 smithi181 ceph-mon[103627]: osdmap e31: 5 total, 4 up, 5 in 2023-12-11T17:41:09.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:09 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-11T17:41:09.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:09 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-11T17:41:09.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:09 smithi059 ceph-mon[98847]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-12-11T17:41:09.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:09 smithi059 ceph-mon[98847]: Detected new or changed devices on smithi115 2023-12-11T17:41:09.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:09 smithi059 ceph-mon[98847]: Adjusting osd_memory_target on smithi115 to 8584M 2023-12-11T17:41:09.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:09 smithi059 ceph-mon[98847]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2023-12-11T17:41:09.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:09 smithi059 ceph-mon[98847]: osdmap e31: 5 total, 4 up, 5 in 2023-12-11T17:41:09.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:09 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-11T17:41:09.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:09 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-11T17:41:10.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:09 smithi115 ceph-mon[102748]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-12-11T17:41:10.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:09 smithi115 ceph-mon[102748]: Detected new or changed devices on smithi115 2023-12-11T17:41:10.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:09 smithi115 ceph-mon[102748]: Adjusting osd_memory_target on smithi115 to 8584M 2023-12-11T17:41:10.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:09 smithi115 ceph-mon[102748]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2023-12-11T17:41:10.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:09 smithi115 ceph-mon[102748]: osdmap e31: 5 total, 4 up, 5 in 2023-12-11T17:41:10.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:09 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-11T17:41:10.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:09 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-11T17:41:10.162 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:41:10.159+0000 7f7fbc928700 1 -- 172.21.15.181:0/226271440 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7fb8101620 msgr2=0x7f7fb8101a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:41:10.162 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:41:10.159+0000 7f7fbc928700 1 --2- 172.21.15.181:0/226271440 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7fb8101620 0x7f7fb8101a40 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f7fac009990 tx=0x7f7fac009ca0 comp rx=0 tx=0).stop 2023-12-11T17:41:10.162 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:41:10.159+0000 7f7fbc928700 1 -- 172.21.15.181:0/226271440 shutdown_connections 2023-12-11T17:41:10.162 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:41:10.159+0000 7f7fbc928700 1 --2- 172.21.15.181:0/226271440 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7fb81031c0 0x7f7fb8109a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:41:10.162 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:41:10.159+0000 7f7fbc928700 1 --2- 172.21.15.181:0/226271440 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7fb8101620 0x7f7fb8101a40 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:41:10.162 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:41:10.159+0000 7f7fbc928700 1 --2- 172.21.15.181:0/226271440 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7fb8102820 0x7f7fb8102c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:41:10.163 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:41:10.159+0000 7f7fbc928700 1 -- 172.21.15.181:0/226271440 >> 172.21.15.181:0/226271440 conn(0x7f7fb80fcbe0 msgr2=0x7f7fb80ff000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:41:10.163 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:41:10.159+0000 7f7fbc928700 1 -- 172.21.15.181:0/226271440 shutdown_connections 2023-12-11T17:41:10.163 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:41:10.159+0000 7f7fbc928700 1 -- 172.21.15.181:0/226271440 wait complete. 2023-12-11T17:41:10.165 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:41:10.160+0000 7f7fbc928700 1 Processor -- start 2023-12-11T17:41:10.165 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:41:10.160+0000 7f7fbc928700 1 -- start start 2023-12-11T17:41:10.165 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:41:10.160+0000 7f7fbc928700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7fb8101620 0x7f7fb8197560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:41:10.165 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:41:10.160+0000 7f7fbc928700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7fb8102820 0x7f7fb8197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:41:10.165 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:41:10.160+0000 7f7fbc928700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7fb81031c0 0x7f7fb819ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:41:10.165 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:41:10.160+0000 7f7fbc928700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb819f0a0 con 0x7f7fb81031c0 2023-12-11T17:41:10.165 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:41:10.160+0000 7f7fbc928700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb819f1e0 con 0x7f7fb8102820 2023-12-11T17:41:10.165 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:41:10.160+0000 7f7fbc928700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb819f370 con 0x7f7fb8101620 2023-12-11T17:41:10.165 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:41:10.161+0000 7f7fb659c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7fb8101620 0x7f7fb8197560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:41:10.165 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:41:10.161+0000 7f7fb659c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7fb8101620 0x7f7fb8197560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34348/0 (socket says 172.21.15.181:34348) 2023-12-11T17:41:10.166 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:41:10.161+0000 7f7fb659c700 1 -- 172.21.15.181:0/4211236686 learned_addr learned my addr 172.21.15.181:0/4211236686 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-12-11T17:41:10.166 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:41:10.161+0000 7f7fb5d9b700 1 --2- 172.21.15.181:0/4211236686 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7fb8102820 0x7f7fb8197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:41:10.166 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:41:10.161+0000 7f7fb6d9d700 1 --2- 172.21.15.181:0/4211236686 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7fb81031c0 0x7f7fb819ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:41:10.166 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:41:10.161+0000 7f7fb659c700 1 -- 172.21.15.181:0/4211236686 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7fb8102820 msgr2=0x7f7fb8197aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:41:10.166 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:41:10.161+0000 7f7fb659c700 1 --2- 172.21.15.181:0/4211236686 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7fb8102820 0x7f7fb8197aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:41:10.166 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:41:10.161+0000 7f7fb659c700 1 -- 172.21.15.181:0/4211236686 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7fb81031c0 msgr2=0x7f7fb819ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:41:10.166 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:41:10.161+0000 7f7fb659c700 1 --2- 172.21.15.181:0/4211236686 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7fb81031c0 0x7f7fb819ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:41:10.166 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:41:10.161+0000 7f7fb659c700 1 -- 172.21.15.181:0/4211236686 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fac0095f0 con 0x7f7fb8101620 2023-12-11T17:41:10.166 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:41:10.161+0000 7f7fb5d9b700 1 --2- 172.21.15.181:0/4211236686 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7fb8102820 0x7f7fb8197aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T17:41:10.167 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:41:10.161+0000 7f7fb6d9d700 1 --2- 172.21.15.181:0/4211236686 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7fb81031c0 0x7f7fb819ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T17:41:10.167 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:41:10.161+0000 7f7fb659c700 1 --2- 172.21.15.181:0/4211236686 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7fb8101620 0x7f7fb8197560 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f7fac000c00 tx=0x7f7fac00f740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:41:10.167 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:41:10.161+0000 7f7fa77fe700 1 -- 172.21.15.181:0/4211236686 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fac01d070 con 0x7f7fb8101620 2023-12-11T17:41:10.167 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:41:10.161+0000 7f7fa77fe700 1 -- 172.21.15.181:0/4211236686 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7fac003e20 con 0x7f7fb8101620 2023-12-11T17:41:10.167 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:41:10.161+0000 7f7fa77fe700 1 -- 172.21.15.181:0/4211236686 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fac0056c0 con 0x7f7fb8101620 2023-12-11T17:41:10.167 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:41:10.161+0000 7f7fbc928700 1 -- 172.21.15.181:0/4211236686 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fb819f5f0 con 0x7f7fb8101620 2023-12-11T17:41:10.167 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:41:10.161+0000 7f7fbc928700 1 -- 172.21.15.181:0/4211236686 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fb819f9b0 con 0x7f7fb8101620 2023-12-11T17:41:10.167 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:41:10.162+0000 7f7fbc928700 1 -- 172.21.15.181:0/4211236686 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fb8108b00 con 0x7f7fb8101620 2023-12-11T17:41:10.168 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:41:10.163+0000 7f7fa77fe700 1 -- 172.21.15.181:0/4211236686 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f7fac00fc20 con 0x7f7fb8101620 2023-12-11T17:41:10.168 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:41:10.164+0000 7f7fa77fe700 1 --2- 172.21.15.181:0/4211236686 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f7fa00a1480 0x7f7fa00a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:41:10.168 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:41:10.164+0000 7f7fa77fe700 1 -- 172.21.15.181:0/4211236686 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 4101+0+0 (secure 0 0 0) 0x7f7fac0ccfe0 con 0x7f7fb8101620 2023-12-11T17:41:10.168 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:41:10.164+0000 7f7fb5d9b700 1 --2- 172.21.15.181:0/4211236686 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f7fa00a1480 0x7f7fa00a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:41:10.168 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:41:10.164+0000 7f7fb5d9b700 1 --2- 172.21.15.181:0/4211236686 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f7fa00a1480 0x7f7fa00a3940 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7fa8005950 tx=0x7f7fa8005f50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:41:10.170 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:41:10.167+0000 7f7fa77fe700 1 -- 172.21.15.181:0/4211236686 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f7fac0976a0 con 0x7f7fb8101620 2023-12-11T17:41:10.368 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:41:10.366+0000 7f7fbc928700 1 -- 172.21.15.181:0/4211236686 --> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f7fb802d380 con 0x7f7fa00a1480 2023-12-11T17:41:10.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:10 smithi181 ceph-mon[103627]: purged_snaps scrub starts 2023-12-11T17:41:10.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:10 smithi181 ceph-mon[103627]: purged_snaps scrub ok 2023-12-11T17:41:10.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:10 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-11T17:41:10.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:10 smithi181 ceph-mon[103627]: osd.4 [v2:172.21.15.115:6808/1021473826,v1:172.21.15.115:6809/1021473826] boot 2023-12-11T17:41:10.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:10 smithi181 ceph-mon[103627]: osdmap e32: 5 total, 5 up, 5 in 2023-12-11T17:41:10.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:10 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-11T17:41:10.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:10 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-11T17:41:10.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:10 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-11T17:41:10.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:10 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:41:10.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:10 smithi059 ceph-mon[98847]: purged_snaps scrub starts 2023-12-11T17:41:10.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:10 smithi059 ceph-mon[98847]: purged_snaps scrub ok 2023-12-11T17:41:10.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:10 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-11T17:41:10.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:10 smithi059 ceph-mon[98847]: osd.4 [v2:172.21.15.115:6808/1021473826,v1:172.21.15.115:6809/1021473826] boot 2023-12-11T17:41:10.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:10 smithi059 ceph-mon[98847]: osdmap e32: 5 total, 5 up, 5 in 2023-12-11T17:41:10.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:10 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-11T17:41:10.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:10 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-11T17:41:10.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:10 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-11T17:41:10.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:10 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:41:11.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:10 smithi115 ceph-mon[102748]: purged_snaps scrub starts 2023-12-11T17:41:11.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:10 smithi115 ceph-mon[102748]: purged_snaps scrub ok 2023-12-11T17:41:11.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:10 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-11T17:41:11.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:10 smithi115 ceph-mon[102748]: osd.4 [v2:172.21.15.115:6808/1021473826,v1:172.21.15.115:6809/1021473826] boot 2023-12-11T17:41:11.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:10 smithi115 ceph-mon[102748]: osdmap e32: 5 total, 5 up, 5 in 2023-12-11T17:41:11.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:10 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-11T17:41:11.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:10 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-11T17:41:11.034 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:10 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-11T17:41:11.034 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:10 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:41:11.896 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:11 smithi181 ceph-mon[103627]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-12-11T17:41:11.896 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:11 smithi181 ceph-mon[103627]: from='client.24232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T17:41:11.896 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:11 smithi181 ceph-mon[103627]: osdmap e33: 5 total, 5 up, 5 in 2023-12-11T17:41:11.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:11 smithi059 ceph-mon[98847]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-12-11T17:41:11.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:11 smithi059 ceph-mon[98847]: from='client.24232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T17:41:11.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:11 smithi059 ceph-mon[98847]: osdmap e33: 5 total, 5 up, 5 in 2023-12-11T17:41:12.032 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:11 smithi115 ceph-mon[102748]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-12-11T17:41:12.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:11 smithi115 ceph-mon[102748]: from='client.24232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T17:41:12.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:11 smithi115 ceph-mon[102748]: osdmap e33: 5 total, 5 up, 5 in 2023-12-11T17:41:13.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:13 smithi181 ceph-mon[103627]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-12-11T17:41:13.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:13 smithi181 ceph-mon[103627]: from='client.? 172.21.15.181:0/1009925783' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4233a512-9e8f-4db8-aed2-6c3af706e589"}]: dispatch 2023-12-11T17:41:13.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:13 smithi181 ceph-mon[103627]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4233a512-9e8f-4db8-aed2-6c3af706e589"}]: dispatch 2023-12-11T17:41:13.896 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:13 smithi181 ceph-mon[103627]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4233a512-9e8f-4db8-aed2-6c3af706e589"}]': finished 2023-12-11T17:41:13.896 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:13 smithi181 ceph-mon[103627]: osdmap e34: 6 total, 5 up, 6 in 2023-12-11T17:41:13.896 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:13 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-11T17:41:13.896 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:13 smithi181 ceph-mon[103627]: from='client.? 172.21.15.181:0/3484289244' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-11T17:41:13.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:13 smithi059 ceph-mon[98847]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-12-11T17:41:13.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:13 smithi059 ceph-mon[98847]: from='client.? 172.21.15.181:0/1009925783' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4233a512-9e8f-4db8-aed2-6c3af706e589"}]: dispatch 2023-12-11T17:41:13.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:13 smithi059 ceph-mon[98847]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4233a512-9e8f-4db8-aed2-6c3af706e589"}]: dispatch 2023-12-11T17:41:13.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:13 smithi059 ceph-mon[98847]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4233a512-9e8f-4db8-aed2-6c3af706e589"}]': finished 2023-12-11T17:41:13.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:13 smithi059 ceph-mon[98847]: osdmap e34: 6 total, 5 up, 6 in 2023-12-11T17:41:13.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:13 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-11T17:41:13.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:13 smithi059 ceph-mon[98847]: from='client.? 172.21.15.181:0/3484289244' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-11T17:41:14.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:13 smithi115 ceph-mon[102748]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-12-11T17:41:14.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:13 smithi115 ceph-mon[102748]: from='client.? 172.21.15.181:0/1009925783' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4233a512-9e8f-4db8-aed2-6c3af706e589"}]: dispatch 2023-12-11T17:41:14.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:13 smithi115 ceph-mon[102748]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4233a512-9e8f-4db8-aed2-6c3af706e589"}]: dispatch 2023-12-11T17:41:14.034 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:13 smithi115 ceph-mon[102748]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4233a512-9e8f-4db8-aed2-6c3af706e589"}]': finished 2023-12-11T17:41:14.034 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:13 smithi115 ceph-mon[102748]: osdmap e34: 6 total, 5 up, 6 in 2023-12-11T17:41:14.034 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:13 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-11T17:41:14.034 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:13 smithi115 ceph-mon[102748]: from='client.? 172.21.15.181:0/3484289244' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-11T17:41:15.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:15 smithi181 ceph-mon[103627]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-12-11T17:41:15.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:15 smithi059 ceph-mon[98847]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-12-11T17:41:16.032 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:15 smithi115 ceph-mon[102748]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-12-11T17:41:17.887 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:17 smithi181 ceph-mon[103627]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-12-11T17:41:17.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:17 smithi059 ceph-mon[98847]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-12-11T17:41:18.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:17 smithi115 ceph-mon[102748]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-12-11T17:41:19.882 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:19 smithi181 ceph-mon[103627]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-12-11T17:41:19.882 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:19 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-12-11T17:41:19.882 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:19 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:41:19.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:19 smithi059 ceph-mon[98847]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-12-11T17:41:19.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:19 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-12-11T17:41:19.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:19 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:41:20.032 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:19 smithi115 ceph-mon[102748]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-12-11T17:41:20.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:19 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-12-11T17:41:20.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:19 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:41:20.894 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:20 smithi181 ceph-mon[103627]: Deploying daemon osd.5 on smithi181 2023-12-11T17:41:20.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:20 smithi059 ceph-mon[98847]: Deploying daemon osd.5 on smithi181 2023-12-11T17:41:21.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:20 smithi115 ceph-mon[102748]: Deploying daemon osd.5 on smithi181 2023-12-11T17:41:21.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:21 smithi181 ceph-mon[103627]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-12-11T17:41:21.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:21 smithi059 ceph-mon[98847]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-12-11T17:41:22.032 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:21 smithi115 ceph-mon[102748]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-12-11T17:41:23.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:23 smithi181 ceph-mon[103627]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-12-11T17:41:23.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:23 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:41:23.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:23 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:41:23.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:23 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:41:23.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:23 smithi059 ceph-mon[98847]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-12-11T17:41:23.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:23 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:41:23.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:23 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:41:23.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:23 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:41:24.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:23 smithi115 ceph-mon[102748]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-12-11T17:41:24.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:23 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:41:24.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:23 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:41:24.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:23 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:41:25.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:25 smithi181 ceph-mon[103627]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-12-11T17:41:25.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:25 smithi059 ceph-mon[98847]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-12-11T17:41:26.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:25 smithi115 ceph-mon[102748]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-12-11T17:41:26.336 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:41:26.333+0000 7f7fa77fe700 1 -- 172.21.15.181:0/4211236686 <== mgr.14150 v2:172.21.15.59:6800/3627728353 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f7fb802d380 con 0x7f7fa00a1480 2023-12-11T17:41:26.336 INFO:teuthology.orchestra.run.smithi181.stdout:Created osd(s) 5 on host 'smithi181' 2023-12-11T17:41:26.338 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:41:26.335+0000 7f7fbc928700 1 -- 172.21.15.181:0/4211236686 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f7fa00a1480 msgr2=0x7f7fa00a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:41:26.338 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:41:26.335+0000 7f7fbc928700 1 --2- 172.21.15.181:0/4211236686 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f7fa00a1480 0x7f7fa00a3940 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7fa8005950 tx=0x7f7fa8005f50 comp rx=0 tx=0).stop 2023-12-11T17:41:26.338 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:41:26.335+0000 7f7fbc928700 1 -- 172.21.15.181:0/4211236686 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7fb8101620 msgr2=0x7f7fb8197560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:41:26.338 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:41:26.335+0000 7f7fbc928700 1 --2- 172.21.15.181:0/4211236686 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7fb8101620 0x7f7fb8197560 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f7fac000c00 tx=0x7f7fac00f740 comp rx=0 tx=0).stop 2023-12-11T17:41:26.338 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:41:26.335+0000 7f7fbc928700 1 -- 172.21.15.181:0/4211236686 shutdown_connections 2023-12-11T17:41:26.338 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:41:26.335+0000 7f7fbc928700 1 --2- 172.21.15.181:0/4211236686 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f7fa00a1480 0x7f7fa00a3940 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:41:26.338 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:41:26.335+0000 7f7fbc928700 1 --2- 172.21.15.181:0/4211236686 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7fb81031c0 0x7f7fb819ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:41:26.338 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:41:26.335+0000 7f7fbc928700 1 --2- 172.21.15.181:0/4211236686 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7fb8101620 0x7f7fb8197560 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:41:26.338 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:41:26.335+0000 7f7fbc928700 1 --2- 172.21.15.181:0/4211236686 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7fb8102820 0x7f7fb8197aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:41:26.338 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:41:26.335+0000 7f7fbc928700 1 -- 172.21.15.181:0/4211236686 >> 172.21.15.181:0/4211236686 conn(0x7f7fb80fcbe0 msgr2=0x7f7fb8105c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:41:26.339 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:41:26.335+0000 7f7fbc928700 1 -- 172.21.15.181:0/4211236686 shutdown_connections 2023-12-11T17:41:26.339 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:41:26.335+0000 7f7fbc928700 1 -- 172.21.15.181:0/4211236686 wait complete. 2023-12-11T17:41:26.788 DEBUG:teuthology.orchestra.run.smithi181:osd.5> sudo journalctl -f -n 0 -u ceph-d0a07eaa-984b-11ee-95a3-87774f69a715@osd.5.service 2023-12-11T17:41:26.791 INFO:tasks.cephadm:Deploying osd.6 on smithi059 with /dev/vg_nvme/lv_2... 2023-12-11T17:41:26.791 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d0a07eaa-984b-11ee-95a3-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-12-11T17:41:26.821 INFO:journalctl@ceph.osd.5.smithi181.stdout:-- Logs begin at Mon 2023-12-11 17:22:43 UTC. -- 2023-12-11T17:41:26.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:26 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:41:26.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:26 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:41:26.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:26 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:41:26.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:26 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:41:26.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:26 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:41:26.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:26 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:41:26.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:26 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:41:26.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:26 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:41:27.032 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:26 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:41:27.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:26 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:41:27.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:26 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:41:27.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:26 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:41:27.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:26 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:41:27.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:26 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:41:27.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:26 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:41:27.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:26 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:41:27.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:26 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:41:27.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:26 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:41:27.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:26 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:41:27.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:26 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:41:27.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:26 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:41:27.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:26 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:41:27.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:26 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:41:27.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:26 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:41:27.411 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/mon.a/config 2023-12-11T17:41:27.518 INFO:journalctl@ceph.osd.5.smithi181.stdout:Dec 11 17:41:27 smithi181 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-5[117382]: 2023-12-11T17:41:27.464+0000 7feb0a16a640 -1 osd.5 0 log_to_monitors true 2023-12-11T17:41:27.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:27 smithi181 ceph-mon[103627]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-12-11T17:41:27.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:27 smithi181 ceph-mon[103627]: from='osd.5 [v2:172.21.15.181:6808/2405172521,v1:172.21.15.181:6809/2405172521]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-12-11T17:41:27.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:27 smithi181 ceph-mon[103627]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-12-11T17:41:27.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:27 smithi059 ceph-mon[98847]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-12-11T17:41:27.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:27 smithi059 ceph-mon[98847]: from='osd.5 [v2:172.21.15.181:6808/2405172521,v1:172.21.15.181:6809/2405172521]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-12-11T17:41:27.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:27 smithi059 ceph-mon[98847]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-12-11T17:41:28.032 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:27 smithi115 ceph-mon[102748]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-12-11T17:41:28.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:27 smithi115 ceph-mon[102748]: from='osd.5 [v2:172.21.15.181:6808/2405172521,v1:172.21.15.181:6809/2405172521]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-12-11T17:41:28.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:27 smithi115 ceph-mon[102748]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-12-11T17:41:28.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:28 smithi059 ceph-mon[98847]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-12-11T17:41:28.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:28 smithi059 ceph-mon[98847]: from='osd.5 [v2:172.21.15.181:6808/2405172521,v1:172.21.15.181:6809/2405172521]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2023-12-11T17:41:28.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:28 smithi059 ceph-mon[98847]: osdmap e35: 6 total, 5 up, 6 in 2023-12-11T17:41:28.906 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:28 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-11T17:41:28.906 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:28 smithi059 ceph-mon[98847]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2023-12-11T17:41:28.906 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:28 smithi059 ceph-mon[98847]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]': finished 2023-12-11T17:41:28.906 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:28 smithi059 ceph-mon[98847]: osdmap e36: 6 total, 5 up, 6 in 2023-12-11T17:41:28.906 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:28 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-11T17:41:29.003 INFO:journalctl@ceph.osd.5.smithi181.stdout:Dec 11 17:41:28 smithi181 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-5[117382]: 2023-12-11T17:41:28.666+0000 7feaff7e1700 -1 osd.5 0 waiting for initial osdmap 2023-12-11T17:41:29.003 INFO:journalctl@ceph.osd.5.smithi181.stdout:Dec 11 17:41:28 smithi181 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-5[117382]: 2023-12-11T17:41:28.674+0000 7feafb5db700 -1 osd.5 36 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-11T17:41:29.003 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:28 smithi181 ceph-mon[103627]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-12-11T17:41:29.003 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:28 smithi181 ceph-mon[103627]: from='osd.5 [v2:172.21.15.181:6808/2405172521,v1:172.21.15.181:6809/2405172521]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2023-12-11T17:41:29.003 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:28 smithi181 ceph-mon[103627]: osdmap e35: 6 total, 5 up, 6 in 2023-12-11T17:41:29.004 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:28 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-11T17:41:29.004 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:28 smithi181 ceph-mon[103627]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2023-12-11T17:41:29.004 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:28 smithi181 ceph-mon[103627]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]': finished 2023-12-11T17:41:29.004 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:28 smithi181 ceph-mon[103627]: osdmap e36: 6 total, 5 up, 6 in 2023-12-11T17:41:29.004 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:28 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-11T17:41:29.032 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:28 smithi115 ceph-mon[102748]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-12-11T17:41:29.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:28 smithi115 ceph-mon[102748]: from='osd.5 [v2:172.21.15.181:6808/2405172521,v1:172.21.15.181:6809/2405172521]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2023-12-11T17:41:29.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:28 smithi115 ceph-mon[102748]: osdmap e35: 6 total, 5 up, 6 in 2023-12-11T17:41:29.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:28 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-11T17:41:29.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:28 smithi115 ceph-mon[102748]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2023-12-11T17:41:29.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:28 smithi115 ceph-mon[102748]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]': finished 2023-12-11T17:41:29.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:28 smithi115 ceph-mon[102748]: osdmap e36: 6 total, 5 up, 6 in 2023-12-11T17:41:29.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:28 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-11T17:41:29.103 INFO:teuthology.orchestra.run.smithi059.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-12-11T17:41:29.103 INFO:teuthology.orchestra.run.smithi059.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-12-11T17:41:29.103 INFO:teuthology.orchestra.run.smithi059.stderr: stderr: 10+0 records in 2023-12-11T17:41:29.103 INFO:teuthology.orchestra.run.smithi059.stderr:10+0 records out 2023-12-11T17:41:29.103 INFO:teuthology.orchestra.run.smithi059.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0110331 s, 950 MB/s 2023-12-11T17:41:29.103 INFO:teuthology.orchestra.run.smithi059.stderr:--> Zapping successful for: 2023-12-11T17:41:29.467 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d0a07eaa-984b-11ee-95a3-87774f69a715 -- ceph orch daemon add osd smithi059:vg_nvme/lv_2 2023-12-11T17:41:29.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:29 smithi059 ceph-mon[98847]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-12-11T17:41:29.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:29 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-11T17:41:29.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:29 smithi059 ceph-mon[98847]: osd.5 [v2:172.21.15.181:6808/2405172521,v1:172.21.15.181:6809/2405172521] boot 2023-12-11T17:41:29.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:29 smithi059 ceph-mon[98847]: osdmap e37: 6 total, 6 up, 6 in 2023-12-11T17:41:29.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:29 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-11T17:41:29.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:29 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:41:29.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:29 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:41:29.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:29 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:41:29.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:29 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:41:29.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:29 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:41:29.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:29 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:41:29.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:29 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:41:29.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:29 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:41:30.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:29 smithi115 ceph-mon[102748]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-12-11T17:41:30.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:29 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-11T17:41:30.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:29 smithi115 ceph-mon[102748]: osd.5 [v2:172.21.15.181:6808/2405172521,v1:172.21.15.181:6809/2405172521] boot 2023-12-11T17:41:30.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:29 smithi115 ceph-mon[102748]: osdmap e37: 6 total, 6 up, 6 in 2023-12-11T17:41:30.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:29 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-11T17:41:30.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:29 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:41:30.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:29 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:41:30.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:29 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:41:30.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:29 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:41:30.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:29 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:41:30.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:29 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:41:30.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:29 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:41:30.034 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:29 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:41:30.089 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/mon.a/config 2023-12-11T17:41:30.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:29 smithi181 ceph-mon[103627]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-12-11T17:41:30.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:29 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-11T17:41:30.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:29 smithi181 ceph-mon[103627]: osd.5 [v2:172.21.15.181:6808/2405172521,v1:172.21.15.181:6809/2405172521] boot 2023-12-11T17:41:30.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:29 smithi181 ceph-mon[103627]: osdmap e37: 6 total, 6 up, 6 in 2023-12-11T17:41:30.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:29 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-11T17:41:30.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:29 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:41:30.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:29 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:41:30.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:29 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:41:30.146 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:29 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:41:30.146 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:29 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:41:30.146 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:29 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:41:30.146 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:29 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:41:30.146 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:29 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:41:30.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:30 smithi059 ceph-mon[98847]: purged_snaps scrub starts 2023-12-11T17:41:30.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:30 smithi059 ceph-mon[98847]: purged_snaps scrub ok 2023-12-11T17:41:30.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:30 smithi059 ceph-mon[98847]: Detected new or changed devices on smithi181 2023-12-11T17:41:30.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:30 smithi059 ceph-mon[98847]: Adjusting osd_memory_target on smithi181 to 8584M 2023-12-11T17:41:30.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:30 smithi059 ceph-mon[98847]: osdmap e38: 6 total, 6 up, 6 in 2023-12-11T17:41:31.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:30 smithi115 ceph-mon[102748]: purged_snaps scrub starts 2023-12-11T17:41:31.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:30 smithi115 ceph-mon[102748]: purged_snaps scrub ok 2023-12-11T17:41:31.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:30 smithi115 ceph-mon[102748]: Detected new or changed devices on smithi181 2023-12-11T17:41:31.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:30 smithi115 ceph-mon[102748]: Adjusting osd_memory_target on smithi181 to 8584M 2023-12-11T17:41:31.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:30 smithi115 ceph-mon[102748]: osdmap e38: 6 total, 6 up, 6 in 2023-12-11T17:41:31.129 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:41:31.124+0000 7f0d8e60e700 1 -- 172.21.15.59:0/3792121993 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0d88101610 msgr2=0x7f0d88101a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:41:31.129 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:41:31.124+0000 7f0d8e60e700 1 --2- 172.21.15.59:0/3792121993 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0d88101610 0x7f0d88101a30 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f0d700099c0 tx=0x7f0d70009cd0 comp rx=0 tx=0).stop 2023-12-11T17:41:31.129 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:41:31.124+0000 7f0d8e60e700 1 -- 172.21.15.59:0/3792121993 shutdown_connections 2023-12-11T17:41:31.129 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:41:31.124+0000 7f0d8e60e700 1 --2- 172.21.15.59:0/3792121993 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d881031b0 0x7f0d88109a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:41:31.129 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:41:31.124+0000 7f0d8e60e700 1 --2- 172.21.15.59:0/3792121993 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0d88101610 0x7f0d88101a30 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:41:31.129 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:41:31.124+0000 7f0d8e60e700 1 --2- 172.21.15.59:0/3792121993 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0d88102810 0x7f0d88102c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:41:31.129 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:41:31.124+0000 7f0d8e60e700 1 -- 172.21.15.59:0/3792121993 >> 172.21.15.59:0/3792121993 conn(0x7f0d880fcbb0 msgr2=0x7f0d880feff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:41:31.129 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:41:31.124+0000 7f0d8e60e700 1 -- 172.21.15.59:0/3792121993 shutdown_connections 2023-12-11T17:41:31.129 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:41:31.124+0000 7f0d8e60e700 1 -- 172.21.15.59:0/3792121993 wait complete. 2023-12-11T17:41:31.129 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:41:31.125+0000 7f0d8e60e700 1 Processor -- start 2023-12-11T17:41:31.129 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:41:31.125+0000 7f0d8e60e700 1 -- start start 2023-12-11T17:41:31.130 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:41:31.125+0000 7f0d8e60e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d88101610 0x7f0d88197550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:41:31.130 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:41:31.125+0000 7f0d8e60e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0d88102810 0x7f0d88197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:41:31.130 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:41:31.125+0000 7f0d8e60e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0d881031b0 0x7f0d8819eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:41:31.130 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:41:31.125+0000 7f0d8e60e700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d8819f090 con 0x7f0d88102810 2023-12-11T17:41:31.130 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:41:31.125+0000 7f0d8e60e700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d8819f1d0 con 0x7f0d881031b0 2023-12-11T17:41:31.130 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:41:31.125+0000 7f0d8e60e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d8819f360 con 0x7f0d88101610 2023-12-11T17:41:31.130 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:41:31.125+0000 7f0d877fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0d88102810 0x7f0d88197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:41:31.130 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:41:31.125+0000 7f0d877fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0d88102810 0x7f0d88197a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60398/0 (socket says 172.21.15.59:60398) 2023-12-11T17:41:31.130 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:41:31.125+0000 7f0d877fe700 1 -- 172.21.15.59:0/4033836310 learned_addr learned my addr 172.21.15.59:0/4033836310 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:41:31.130 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:41:31.125+0000 7f0d87fff700 1 --2- 172.21.15.59:0/4033836310 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d88101610 0x7f0d88197550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:41:31.130 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:41:31.125+0000 7f0d8cbab700 1 --2- 172.21.15.59:0/4033836310 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0d881031b0 0x7f0d8819eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:41:31.130 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:41:31.126+0000 7f0d877fe700 1 -- 172.21.15.59:0/4033836310 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d88101610 msgr2=0x7f0d88197550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:41:31.130 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:41:31.126+0000 7f0d877fe700 1 --2- 172.21.15.59:0/4033836310 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d88101610 0x7f0d88197550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:41:31.130 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:41:31.126+0000 7f0d877fe700 1 -- 172.21.15.59:0/4033836310 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0d881031b0 msgr2=0x7f0d8819eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:41:31.131 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:41:31.126+0000 7f0d877fe700 1 --2- 172.21.15.59:0/4033836310 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0d881031b0 0x7f0d8819eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:41:31.131 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:41:31.126+0000 7f0d877fe700 1 -- 172.21.15.59:0/4033836310 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d700095f0 con 0x7f0d88102810 2023-12-11T17:41:31.131 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:41:31.126+0000 7f0d87fff700 1 --2- 172.21.15.59:0/4033836310 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d88101610 0x7f0d88197550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T17:41:31.131 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:41:31.126+0000 7f0d877fe700 1 --2- 172.21.15.59:0/4033836310 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0d88102810 0x7f0d88197a90 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0d7800c9e0 tx=0x7f0d7800cda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:41:31.131 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:41:31.126+0000 7f0d857fa700 1 -- 172.21.15.59:0/4033836310 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d78007ac0 con 0x7f0d88102810 2023-12-11T17:41:31.131 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:41:31.126+0000 7f0d857fa700 1 -- 172.21.15.59:0/4033836310 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0d78007c20 con 0x7f0d88102810 2023-12-11T17:41:31.131 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:41:31.126+0000 7f0d8e60e700 1 -- 172.21.15.59:0/4033836310 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d880739a0 con 0x7f0d88102810 2023-12-11T17:41:31.131 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:41:31.126+0000 7f0d857fa700 1 -- 172.21.15.59:0/4033836310 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d78018660 con 0x7f0d88102810 2023-12-11T17:41:31.131 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:41:31.126+0000 7f0d8e60e700 1 -- 172.21.15.59:0/4033836310 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d88073ea0 con 0x7f0d88102810 2023-12-11T17:41:31.132 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:41:31.127+0000 7f0d8e60e700 1 -- 172.21.15.59:0/4033836310 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d8804f290 con 0x7f0d88102810 2023-12-11T17:41:31.133 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:41:31.128+0000 7f0d857fa700 1 -- 172.21.15.59:0/4033836310 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f0d7800f450 con 0x7f0d88102810 2023-12-11T17:41:31.133 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:41:31.129+0000 7f0d857fa700 1 --2- 172.21.15.59:0/4033836310 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f0d740a1480 0x7f0d740a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:41:31.133 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:41:31.129+0000 7f0d87fff700 1 --2- 172.21.15.59:0/4033836310 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f0d740a1480 0x7f0d740a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:41:31.133 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:41:31.129+0000 7f0d857fa700 1 -- 172.21.15.59:0/4033836310 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4562+0+0 (secure 0 0 0) 0x7f0d780cbf00 con 0x7f0d88102810 2023-12-11T17:41:31.133 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:41:31.129+0000 7f0d87fff700 1 --2- 172.21.15.59:0/4033836310 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f0d740a1480 0x7f0d740a3940 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0d7000b580 tx=0x7f0d70005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:41:31.136 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:41:31.133+0000 7f0d857fa700 1 -- 172.21.15.59:0/4033836310 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f0d78096360 con 0x7f0d88102810 2023-12-11T17:41:31.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:30 smithi181 ceph-mon[103627]: purged_snaps scrub starts 2023-12-11T17:41:31.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:30 smithi181 ceph-mon[103627]: purged_snaps scrub ok 2023-12-11T17:41:31.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:30 smithi181 ceph-mon[103627]: Detected new or changed devices on smithi181 2023-12-11T17:41:31.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:30 smithi181 ceph-mon[103627]: Adjusting osd_memory_target on smithi181 to 8584M 2023-12-11T17:41:31.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:30 smithi181 ceph-mon[103627]: osdmap e38: 6 total, 6 up, 6 in 2023-12-11T17:41:31.346 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:41:31.344+0000 7f0d8e60e700 1 -- 172.21.15.59:0/4033836310 --> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi059:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f0d8802d380 con 0x7f0d740a1480 2023-12-11T17:41:32.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:31 smithi115 ceph-mon[102748]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-12-11T17:41:32.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:31 smithi115 ceph-mon[102748]: osdmap e39: 6 total, 6 up, 6 in 2023-12-11T17:41:32.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:31 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-11T17:41:32.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:31 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-11T17:41:32.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:31 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:41:32.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:31 smithi181 ceph-mon[103627]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-12-11T17:41:32.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:31 smithi181 ceph-mon[103627]: osdmap e39: 6 total, 6 up, 6 in 2023-12-11T17:41:32.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:31 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-11T17:41:32.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:31 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-11T17:41:32.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:31 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:41:32.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:31 smithi059 ceph-mon[98847]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-12-11T17:41:32.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:31 smithi059 ceph-mon[98847]: osdmap e39: 6 total, 6 up, 6 in 2023-12-11T17:41:32.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:31 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-11T17:41:32.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:31 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-11T17:41:32.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:31 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:41:32.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:32 smithi059 ceph-mon[98847]: from='client.14337 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi059:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T17:41:32.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:32 smithi059 ceph-mon[98847]: osdmap e40: 6 total, 6 up, 6 in 2023-12-11T17:41:33.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:32 smithi115 ceph-mon[102748]: from='client.14337 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi059:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T17:41:33.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:32 smithi115 ceph-mon[102748]: osdmap e40: 6 total, 6 up, 6 in 2023-12-11T17:41:33.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:32 smithi181 ceph-mon[103627]: from='client.14337 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi059:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T17:41:33.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:32 smithi181 ceph-mon[103627]: osdmap e40: 6 total, 6 up, 6 in 2023-12-11T17:41:34.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:33 smithi115 ceph-mon[102748]: pgmap v128: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-12-11T17:41:34.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:33 smithi115 ceph-mon[102748]: osdmap e41: 6 total, 6 up, 6 in 2023-12-11T17:41:34.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:33 smithi181 ceph-mon[103627]: pgmap v128: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-12-11T17:41:34.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:33 smithi181 ceph-mon[103627]: osdmap e41: 6 total, 6 up, 6 in 2023-12-11T17:41:34.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:33 smithi059 ceph-mon[98847]: pgmap v128: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-12-11T17:41:34.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:33 smithi059 ceph-mon[98847]: osdmap e41: 6 total, 6 up, 6 in 2023-12-11T17:41:35.034 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:34 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/2382496393' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8faab332-016e-4c77-8370-8bef95fc6f34"}]: dispatch 2023-12-11T17:41:35.034 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:34 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/2382496393' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8faab332-016e-4c77-8370-8bef95fc6f34"}]': finished 2023-12-11T17:41:35.034 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:34 smithi115 ceph-mon[102748]: osdmap e42: 7 total, 6 up, 7 in 2023-12-11T17:41:35.034 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:34 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-11T17:41:35.034 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:34 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/2716476074' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-11T17:41:35.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:34 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/2382496393' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8faab332-016e-4c77-8370-8bef95fc6f34"}]: dispatch 2023-12-11T17:41:35.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:34 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/2382496393' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8faab332-016e-4c77-8370-8bef95fc6f34"}]': finished 2023-12-11T17:41:35.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:34 smithi181 ceph-mon[103627]: osdmap e42: 7 total, 6 up, 7 in 2023-12-11T17:41:35.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:34 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-11T17:41:35.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:34 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/2716476074' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-11T17:41:35.155 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:34 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/2382496393' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8faab332-016e-4c77-8370-8bef95fc6f34"}]: dispatch 2023-12-11T17:41:35.155 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:34 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/2382496393' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8faab332-016e-4c77-8370-8bef95fc6f34"}]': finished 2023-12-11T17:41:35.155 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:34 smithi059 ceph-mon[98847]: osdmap e42: 7 total, 6 up, 7 in 2023-12-11T17:41:35.155 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:34 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-11T17:41:35.155 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:34 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/2716476074' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-11T17:41:36.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:35 smithi181 ceph-mon[103627]: pgmap v132: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-12-11T17:41:36.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:35 smithi059 ceph-mon[98847]: pgmap v132: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-12-11T17:41:36.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:35 smithi115 ceph-mon[102748]: pgmap v132: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-12-11T17:41:38.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:37 smithi181 ceph-mon[103627]: pgmap v133: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 115 KiB/s, 0 objects/s recovering 2023-12-11T17:41:38.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:37 smithi059 ceph-mon[98847]: pgmap v133: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 115 KiB/s, 0 objects/s recovering 2023-12-11T17:41:38.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:37 smithi115 ceph-mon[102748]: pgmap v133: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 115 KiB/s, 0 objects/s recovering 2023-12-11T17:41:40.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:39 smithi181 ceph-mon[103627]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-12-11T17:41:40.151 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:39 smithi059 ceph-mon[98847]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-12-11T17:41:40.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:39 smithi115 ceph-mon[102748]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-12-11T17:41:41.099 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:40 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-12-11T17:41:41.099 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:40 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:41:41.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:40 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-12-11T17:41:41.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:40 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:41:41.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:40 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-12-11T17:41:41.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:40 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:41:42.059 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:41 smithi059 ceph-mon[98847]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-12-11T17:41:42.059 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:41 smithi059 ceph-mon[98847]: Deploying daemon osd.6 on smithi059 2023-12-11T17:41:42.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:41 smithi181 ceph-mon[103627]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-12-11T17:41:42.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:41 smithi181 ceph-mon[103627]: Deploying daemon osd.6 on smithi059 2023-12-11T17:41:42.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:41 smithi115 ceph-mon[102748]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-12-11T17:41:42.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:41 smithi115 ceph-mon[102748]: Deploying daemon osd.6 on smithi059 2023-12-11T17:41:44.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:43 smithi181 ceph-mon[103627]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-12-11T17:41:44.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:43 smithi059 ceph-mon[98847]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-12-11T17:41:44.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:43 smithi115 ceph-mon[102748]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-12-11T17:41:45.890 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:45 smithi059 ceph-mon[98847]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-12-11T17:41:45.890 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:45 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:41:45.890 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:45 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:41:45.890 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:45 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:41:45.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:45 smithi181 ceph-mon[103627]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-12-11T17:41:45.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:45 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:41:45.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:45 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:41:45.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:45 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:41:46.032 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:45 smithi115 ceph-mon[102748]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-12-11T17:41:46.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:45 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:41:46.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:45 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:41:46.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:45 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:41:47.557 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:41:47.553+0000 7f0d857fa700 1 -- 172.21.15.59:0/4033836310 <== mgr.14150 v2:172.21.15.59:6800/3627728353 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f0d8802d380 con 0x7f0d740a1480 2023-12-11T17:41:47.557 INFO:teuthology.orchestra.run.smithi059.stdout:Created osd(s) 6 on host 'smithi059' 2023-12-11T17:41:47.558 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:41:47.555+0000 7f0d8e60e700 1 -- 172.21.15.59:0/4033836310 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f0d740a1480 msgr2=0x7f0d740a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:41:47.558 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:41:47.556+0000 7f0d8e60e700 1 --2- 172.21.15.59:0/4033836310 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f0d740a1480 0x7f0d740a3940 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0d7000b580 tx=0x7f0d70005fb0 comp rx=0 tx=0).stop 2023-12-11T17:41:47.558 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:41:47.556+0000 7f0d8e60e700 1 -- 172.21.15.59:0/4033836310 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0d88102810 msgr2=0x7f0d88197a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:41:47.558 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:41:47.556+0000 7f0d8e60e700 1 --2- 172.21.15.59:0/4033836310 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0d88102810 0x7f0d88197a90 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0d7800c9e0 tx=0x7f0d7800cda0 comp rx=0 tx=0).stop 2023-12-11T17:41:47.559 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:41:47.556+0000 7f0d8e60e700 1 -- 172.21.15.59:0/4033836310 shutdown_connections 2023-12-11T17:41:47.559 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:41:47.556+0000 7f0d8e60e700 1 --2- 172.21.15.59:0/4033836310 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f0d740a1480 0x7f0d740a3940 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:41:47.559 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:41:47.556+0000 7f0d8e60e700 1 --2- 172.21.15.59:0/4033836310 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0d881031b0 0x7f0d8819eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:41:47.559 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:41:47.556+0000 7f0d8e60e700 1 --2- 172.21.15.59:0/4033836310 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d88101610 0x7f0d88197550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:41:47.559 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:41:47.556+0000 7f0d8e60e700 1 --2- 172.21.15.59:0/4033836310 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0d88102810 0x7f0d88197a90 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:41:47.559 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:41:47.556+0000 7f0d8e60e700 1 -- 172.21.15.59:0/4033836310 >> 172.21.15.59:0/4033836310 conn(0x7f0d880fcbb0 msgr2=0x7f0d88105be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:41:47.559 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:41:47.556+0000 7f0d8e60e700 1 -- 172.21.15.59:0/4033836310 shutdown_connections 2023-12-11T17:41:47.559 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:41:47.556+0000 7f0d8e60e700 1 -- 172.21.15.59:0/4033836310 wait complete. 2023-12-11T17:41:47.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:47 smithi181 ceph-mon[103627]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-12-11T17:41:47.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:47 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:41:47.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:47 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:41:47.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:47 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:41:47.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:47 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:41:47.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:47 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:41:47.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:47 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:41:47.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:47 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:41:47.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:47 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:41:47.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:47 smithi059 ceph-mon[98847]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-12-11T17:41:47.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:47 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:41:47.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:47 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:41:47.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:47 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:41:47.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:47 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:41:47.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:47 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:41:47.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:47 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:41:47.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:47 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:41:47.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:47 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:41:47.987 DEBUG:teuthology.orchestra.run.smithi059:osd.6> sudo journalctl -f -n 0 -u ceph-d0a07eaa-984b-11ee-95a3-87774f69a715@osd.6.service 2023-12-11T17:41:47.990 INFO:tasks.cephadm:Deploying osd.7 on smithi115 with /dev/vg_nvme/lv_2... 2023-12-11T17:41:47.990 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d0a07eaa-984b-11ee-95a3-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-12-11T17:41:48.023 INFO:journalctl@ceph.osd.6.smithi059.stdout:-- Logs begin at Mon 2023-12-11 17:22:26 UTC. -- 2023-12-11T17:41:48.032 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:47 smithi115 ceph-mon[102748]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-12-11T17:41:48.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:47 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:41:48.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:47 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:41:48.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:47 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:41:48.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:47 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:41:48.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:47 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:41:48.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:47 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:41:48.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:47 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:41:48.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:47 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:41:48.605 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/mon.b/config 2023-12-11T17:41:48.764 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:48 smithi059 ceph-mon[98847]: from='osd.6 [v2:172.21.15.59:6818/815374853,v1:172.21.15.59:6819/815374853]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-12-11T17:41:48.765 INFO:journalctl@ceph.osd.6.smithi059.stdout:Dec 11 17:41:48 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-6[133999]: 2023-12-11T17:41:48.458+0000 7f6dc0eec640 -1 osd.6 0 log_to_monitors true 2023-12-11T17:41:48.816 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:48 smithi115 ceph-mon[102748]: from='osd.6 [v2:172.21.15.59:6818/815374853,v1:172.21.15.59:6819/815374853]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-12-11T17:41:48.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:48 smithi181 ceph-mon[103627]: from='osd.6 [v2:172.21.15.59:6818/815374853,v1:172.21.15.59:6819/815374853]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-12-11T17:41:49.564 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:49 smithi059 ceph-mon[98847]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-12-11T17:41:49.565 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:49 smithi059 ceph-mon[98847]: from='osd.6 [v2:172.21.15.59:6818/815374853,v1:172.21.15.59:6819/815374853]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-12-11T17:41:49.565 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:49 smithi059 ceph-mon[98847]: osdmap e43: 7 total, 6 up, 7 in 2023-12-11T17:41:49.565 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:49 smithi059 ceph-mon[98847]: from='osd.6 [v2:172.21.15.59:6818/815374853,v1:172.21.15.59:6819/815374853]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2023-12-11T17:41:49.565 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:49 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-11T17:41:49.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:49 smithi181 ceph-mon[103627]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-12-11T17:41:49.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:49 smithi181 ceph-mon[103627]: from='osd.6 [v2:172.21.15.59:6818/815374853,v1:172.21.15.59:6819/815374853]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-12-11T17:41:49.896 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:49 smithi181 ceph-mon[103627]: osdmap e43: 7 total, 6 up, 7 in 2023-12-11T17:41:49.896 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:49 smithi181 ceph-mon[103627]: from='osd.6 [v2:172.21.15.59:6818/815374853,v1:172.21.15.59:6819/815374853]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2023-12-11T17:41:49.896 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:49 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-11T17:41:49.904 INFO:journalctl@ceph.osd.6.smithi059.stdout:Dec 11 17:41:49 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-6[133999]: 2023-12-11T17:41:49.560+0000 7f6db6d64700 -1 osd.6 0 waiting for initial osdmap 2023-12-11T17:41:49.904 INFO:journalctl@ceph.osd.6.smithi059.stdout:Dec 11 17:41:49 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-6[133999]: 2023-12-11T17:41:49.569+0000 7f6dae355700 -1 osd.6 44 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-11T17:41:50.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:49 smithi115 ceph-mon[102748]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-12-11T17:41:50.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:49 smithi115 ceph-mon[102748]: from='osd.6 [v2:172.21.15.59:6818/815374853,v1:172.21.15.59:6819/815374853]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-12-11T17:41:50.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:49 smithi115 ceph-mon[102748]: osdmap e43: 7 total, 6 up, 7 in 2023-12-11T17:41:50.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:49 smithi115 ceph-mon[102748]: from='osd.6 [v2:172.21.15.59:6818/815374853,v1:172.21.15.59:6819/815374853]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2023-12-11T17:41:50.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:49 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-11T17:41:50.118 INFO:teuthology.orchestra.run.smithi115.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-12-11T17:41:50.118 INFO:teuthology.orchestra.run.smithi115.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-12-11T17:41:50.118 INFO:teuthology.orchestra.run.smithi115.stderr: stderr: 10+0 records in 2023-12-11T17:41:50.118 INFO:teuthology.orchestra.run.smithi115.stderr:10+0 records out 2023-12-11T17:41:50.118 INFO:teuthology.orchestra.run.smithi115.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0131093 s, 800 MB/s 2023-12-11T17:41:50.118 INFO:teuthology.orchestra.run.smithi115.stderr:--> Zapping successful for: 2023-12-11T17:41:50.552 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d0a07eaa-984b-11ee-95a3-87774f69a715 -- ceph orch daemon add osd smithi115:vg_nvme/lv_2 2023-12-11T17:41:50.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:50 smithi059 ceph-mon[98847]: from='osd.6 [v2:172.21.15.59:6818/815374853,v1:172.21.15.59:6819/815374853]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]': finished 2023-12-11T17:41:50.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:50 smithi059 ceph-mon[98847]: osdmap e44: 7 total, 6 up, 7 in 2023-12-11T17:41:50.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:50 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-11T17:41:50.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:50 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-11T17:41:50.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:50 smithi115 ceph-mon[102748]: from='osd.6 [v2:172.21.15.59:6818/815374853,v1:172.21.15.59:6819/815374853]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]': finished 2023-12-11T17:41:50.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:50 smithi115 ceph-mon[102748]: osdmap e44: 7 total, 6 up, 7 in 2023-12-11T17:41:50.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:50 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-11T17:41:50.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:50 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-11T17:41:50.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:50 smithi181 ceph-mon[103627]: from='osd.6 [v2:172.21.15.59:6818/815374853,v1:172.21.15.59:6819/815374853]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]': finished 2023-12-11T17:41:50.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:50 smithi181 ceph-mon[103627]: osdmap e44: 7 total, 6 up, 7 in 2023-12-11T17:41:50.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:50 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-11T17:41:50.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:50 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-11T17:41:51.189 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/mon.b/config 2023-12-11T17:41:51.693 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:51 smithi115 ceph-mon[102748]: purged_snaps scrub starts 2023-12-11T17:41:51.693 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:51 smithi115 ceph-mon[102748]: purged_snaps scrub ok 2023-12-11T17:41:51.693 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:51 smithi115 ceph-mon[102748]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-12-11T17:41:51.693 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:51 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-11T17:41:51.693 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:51 smithi115 ceph-mon[102748]: osd.6 [v2:172.21.15.59:6818/815374853,v1:172.21.15.59:6819/815374853] boot 2023-12-11T17:41:51.693 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:51 smithi115 ceph-mon[102748]: osdmap e45: 7 total, 7 up, 7 in 2023-12-11T17:41:51.693 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:51 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-11T17:41:51.694 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:51 smithi115 ceph-mon[102748]: Detected new or changed devices on smithi059 2023-12-11T17:41:51.694 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:51 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:41:51.694 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:51 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:41:51.694 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:51 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:41:51.694 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:51 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:41:51.694 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:51 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:41:51.694 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:51 smithi115 ceph-mon[102748]: Adjusting osd_memory_target on smithi059 to 5724M 2023-12-11T17:41:51.694 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:51 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:41:51.694 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:51 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:41:51.694 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:51 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:41:51.694 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:51 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:41:51.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:51 smithi181 ceph-mon[103627]: purged_snaps scrub starts 2023-12-11T17:41:51.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:51 smithi181 ceph-mon[103627]: purged_snaps scrub ok 2023-12-11T17:41:51.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:51 smithi181 ceph-mon[103627]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-12-11T17:41:51.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:51 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-11T17:41:51.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:51 smithi181 ceph-mon[103627]: osd.6 [v2:172.21.15.59:6818/815374853,v1:172.21.15.59:6819/815374853] boot 2023-12-11T17:41:51.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:51 smithi181 ceph-mon[103627]: osdmap e45: 7 total, 7 up, 7 in 2023-12-11T17:41:51.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:51 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-11T17:41:51.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:51 smithi181 ceph-mon[103627]: Detected new or changed devices on smithi059 2023-12-11T17:41:51.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:51 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:41:51.896 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:51 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:41:51.896 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:51 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:41:51.896 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:51 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:41:51.896 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:51 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:41:51.896 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:51 smithi181 ceph-mon[103627]: Adjusting osd_memory_target on smithi059 to 5724M 2023-12-11T17:41:51.896 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:51 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:41:51.896 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:51 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:41:51.896 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:51 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:41:51.896 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:51 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:41:51.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:51 smithi059 ceph-mon[98847]: purged_snaps scrub starts 2023-12-11T17:41:51.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:51 smithi059 ceph-mon[98847]: purged_snaps scrub ok 2023-12-11T17:41:51.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:51 smithi059 ceph-mon[98847]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-12-11T17:41:51.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:51 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-11T17:41:51.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:51 smithi059 ceph-mon[98847]: osd.6 [v2:172.21.15.59:6818/815374853,v1:172.21.15.59:6819/815374853] boot 2023-12-11T17:41:51.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:51 smithi059 ceph-mon[98847]: osdmap e45: 7 total, 7 up, 7 in 2023-12-11T17:41:51.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:51 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-11T17:41:51.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:51 smithi059 ceph-mon[98847]: Detected new or changed devices on smithi059 2023-12-11T17:41:51.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:51 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:41:51.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:51 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:41:51.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:51 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:41:51.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:51 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:41:51.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:51 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:41:51.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:51 smithi059 ceph-mon[98847]: Adjusting osd_memory_target on smithi059 to 5724M 2023-12-11T17:41:51.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:51 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:41:51.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:51 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:41:51.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:51 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:41:51.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:51 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:41:52.365 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:41:52.361+0000 7f656b39f700 1 -- 172.21.15.115:0/3612959355 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f65640fff10 msgr2=0x7f6564100330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:41:52.365 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:41:52.361+0000 7f656b39f700 1 --2- 172.21.15.115:0/3612959355 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f65640fff10 0x7f6564100330 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f6560009990 tx=0x7f6560009ca0 comp rx=0 tx=0).stop 2023-12-11T17:41:52.365 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:41:52.361+0000 7f656b39f700 1 -- 172.21.15.115:0/3612959355 shutdown_connections 2023-12-11T17:41:52.365 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:41:52.361+0000 7f656b39f700 1 --2- 172.21.15.115:0/3612959355 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6564101230 0x7f65641016b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:41:52.365 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:41:52.361+0000 7f656b39f700 1 --2- 172.21.15.115:0/3612959355 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f65640fff10 0x7f6564100330 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:41:52.365 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:41:52.361+0000 7f656b39f700 1 --2- 172.21.15.115:0/3612959355 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6564100870 0x7f6564100cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:41:52.365 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:41:52.361+0000 7f656b39f700 1 -- 172.21.15.115:0/3612959355 >> 172.21.15.115:0/3612959355 conn(0x7f65640fbad0 msgr2=0x7f65640fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:41:52.365 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:41:52.361+0000 7f656b39f700 1 -- 172.21.15.115:0/3612959355 shutdown_connections 2023-12-11T17:41:52.365 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:41:52.361+0000 7f656b39f700 1 -- 172.21.15.115:0/3612959355 wait complete. 2023-12-11T17:41:52.366 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:41:52.361+0000 7f656b39f700 1 Processor -- start 2023-12-11T17:41:52.366 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:41:52.361+0000 7f656b39f700 1 -- start start 2023-12-11T17:41:52.366 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:41:52.361+0000 7f656b39f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65640fff10 0x7f65641997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:41:52.366 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:41:52.361+0000 7f656b39f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6564100870 0x7f6564199ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:41:52.366 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:41:52.361+0000 7f656b39f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6564101230 0x7f65641a0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:41:52.366 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:41:52.361+0000 7f656b39f700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65641a12e0 con 0x7f6564101230 2023-12-11T17:41:52.366 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:41:52.361+0000 7f656b39f700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65641a1420 con 0x7f6564100870 2023-12-11T17:41:52.366 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:41:52.361+0000 7f656b39f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65641a15b0 con 0x7f65640fff10 2023-12-11T17:41:52.366 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:41:52.362+0000 7f656893a700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6564100870 0x7f6564199ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:41:52.366 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:41:52.362+0000 7f656893a700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6564100870 0x7f6564199ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:54848/0 (socket says 172.21.15.115:54848) 2023-12-11T17:41:52.366 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:41:52.362+0000 7f656893a700 1 -- 172.21.15.115:0/3151414763 learned_addr learned my addr 172.21.15.115:0/3151414763 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-11T17:41:52.366 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:41:52.362+0000 7f656913b700 1 --2- 172.21.15.115:0/3151414763 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65640fff10 0x7f65641997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:41:52.366 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:41:52.362+0000 7f656893a700 1 -- 172.21.15.115:0/3151414763 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65640fff10 msgr2=0x7f65641997a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:41:52.367 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:41:52.362+0000 7f656893a700 1 --2- 172.21.15.115:0/3151414763 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65640fff10 0x7f65641997a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:41:52.367 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:41:52.362+0000 7f656893a700 1 -- 172.21.15.115:0/3151414763 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6564101230 msgr2=0x7f65641a0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:41:52.367 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:41:52.362+0000 7f656993c700 1 --2- 172.21.15.115:0/3151414763 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6564101230 0x7f65641a0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:41:52.367 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:41:52.362+0000 7f656893a700 1 --2- 172.21.15.115:0/3151414763 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6564101230 0x7f65641a0d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:41:52.367 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:41:52.362+0000 7f656893a700 1 -- 172.21.15.115:0/3151414763 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65600095f0 con 0x7f6564100870 2023-12-11T17:41:52.367 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:41:52.362+0000 7f656893a700 1 --2- 172.21.15.115:0/3151414763 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6564100870 0x7f6564199ce0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f655400b530 tx=0x7f655400b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:41:52.367 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:41:52.362+0000 7f655a7fc700 1 -- 172.21.15.115:0/3151414763 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6554011830 con 0x7f6564100870 2023-12-11T17:41:52.367 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:41:52.362+0000 7f655a7fc700 1 -- 172.21.15.115:0/3151414763 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6554011e50 con 0x7f6564100870 2023-12-11T17:41:52.367 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:41:52.362+0000 7f656b39f700 1 -- 172.21.15.115:0/3151414763 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65641a1890 con 0x7f6564100870 2023-12-11T17:41:52.367 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:41:52.362+0000 7f655a7fc700 1 -- 172.21.15.115:0/3151414763 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f655400f2e0 con 0x7f6564100870 2023-12-11T17:41:52.367 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:41:52.362+0000 7f656b39f700 1 -- 172.21.15.115:0/3151414763 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65641a1d90 con 0x7f6564100870 2023-12-11T17:41:52.367 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:41:52.363+0000 7f656b39f700 1 -- 172.21.15.115:0/3151414763 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f656410acf0 con 0x7f6564100870 2023-12-11T17:41:52.368 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:41:52.364+0000 7f655a7fc700 1 -- 172.21.15.115:0/3151414763 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f6554010380 con 0x7f6564100870 2023-12-11T17:41:52.368 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:41:52.365+0000 7f655a7fc700 1 --2- 172.21.15.115:0/3151414763 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f65500a1480 0x7f65500a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:41:52.368 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:41:52.365+0000 7f655a7fc700 1 -- 172.21.15.115:0/3151414763 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 4965+0+0 (secure 0 0 0) 0x7f65540cc110 con 0x7f6564100870 2023-12-11T17:41:52.368 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:41:52.365+0000 7f656913b700 1 --2- 172.21.15.115:0/3151414763 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f65500a1480 0x7f65500a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:41:52.368 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:41:52.366+0000 7f656913b700 1 --2- 172.21.15.115:0/3151414763 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f65500a1480 0x7f65500a3940 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f656000b5c0 tx=0x7f6560003790 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:41:52.370 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:41:52.368+0000 7f655a7fc700 1 -- 172.21.15.115:0/3151414763 <== mon.2 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f65540963e0 con 0x7f6564100870 2023-12-11T17:41:52.567 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:41:52.566+0000 7f656b39f700 1 -- 172.21.15.115:0/3151414763 --> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f656402d380 con 0x7f65500a1480 2023-12-11T17:41:53.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:52 smithi181 ceph-mon[103627]: osdmap e46: 7 total, 7 up, 7 in 2023-12-11T17:41:53.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:52 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-11T17:41:53.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:52 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-11T17:41:53.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:52 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:41:53.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:52 smithi059 ceph-mon[98847]: osdmap e46: 7 total, 7 up, 7 in 2023-12-11T17:41:53.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:52 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-11T17:41:53.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:52 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-11T17:41:53.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:52 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:41:53.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:52 smithi115 ceph-mon[102748]: osdmap e46: 7 total, 7 up, 7 in 2023-12-11T17:41:53.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:52 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-11T17:41:53.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:52 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-11T17:41:53.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:52 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:41:54.032 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:53 smithi115 ceph-mon[102748]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-12-11T17:41:54.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:53 smithi115 ceph-mon[102748]: from='client.24272 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T17:41:54.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:53 smithi181 ceph-mon[103627]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-12-11T17:41:54.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:53 smithi181 ceph-mon[103627]: from='client.24272 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T17:41:54.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:53 smithi059 ceph-mon[98847]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-12-11T17:41:54.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:53 smithi059 ceph-mon[98847]: from='client.24272 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T17:41:56.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:55 smithi181 ceph-mon[103627]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-12-11T17:41:56.146 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:55 smithi181 ceph-mon[103627]: from='client.? 172.21.15.115:0/859109746' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f20f470a-0d40-4ed3-8e0e-fd58675cf143"}]: dispatch 2023-12-11T17:41:56.146 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:55 smithi181 ceph-mon[103627]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f20f470a-0d40-4ed3-8e0e-fd58675cf143"}]: dispatch 2023-12-11T17:41:56.146 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:55 smithi181 ceph-mon[103627]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f20f470a-0d40-4ed3-8e0e-fd58675cf143"}]': finished 2023-12-11T17:41:56.146 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:55 smithi181 ceph-mon[103627]: osdmap e47: 8 total, 7 up, 8 in 2023-12-11T17:41:56.146 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:55 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-11T17:41:56.146 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:55 smithi181 ceph-mon[103627]: from='client.? 172.21.15.115:0/732866332' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-11T17:41:56.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:55 smithi059 ceph-mon[98847]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-12-11T17:41:56.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:55 smithi059 ceph-mon[98847]: from='client.? 172.21.15.115:0/859109746' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f20f470a-0d40-4ed3-8e0e-fd58675cf143"}]: dispatch 2023-12-11T17:41:56.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:55 smithi059 ceph-mon[98847]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f20f470a-0d40-4ed3-8e0e-fd58675cf143"}]: dispatch 2023-12-11T17:41:56.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:55 smithi059 ceph-mon[98847]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f20f470a-0d40-4ed3-8e0e-fd58675cf143"}]': finished 2023-12-11T17:41:56.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:55 smithi059 ceph-mon[98847]: osdmap e47: 8 total, 7 up, 8 in 2023-12-11T17:41:56.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:55 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-11T17:41:56.155 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:55 smithi059 ceph-mon[98847]: from='client.? 172.21.15.115:0/732866332' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-11T17:41:56.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:55 smithi115 ceph-mon[102748]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-12-11T17:41:56.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:55 smithi115 ceph-mon[102748]: from='client.? 172.21.15.115:0/859109746' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f20f470a-0d40-4ed3-8e0e-fd58675cf143"}]: dispatch 2023-12-11T17:41:56.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:55 smithi115 ceph-mon[102748]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f20f470a-0d40-4ed3-8e0e-fd58675cf143"}]: dispatch 2023-12-11T17:41:56.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:55 smithi115 ceph-mon[102748]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f20f470a-0d40-4ed3-8e0e-fd58675cf143"}]': finished 2023-12-11T17:41:56.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:55 smithi115 ceph-mon[102748]: osdmap e47: 8 total, 7 up, 8 in 2023-12-11T17:41:56.284 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:55 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-11T17:41:56.284 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:55 smithi115 ceph-mon[102748]: from='client.? 172.21.15.115:0/732866332' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-11T17:41:58.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:57 smithi181 ceph-mon[103627]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-12-11T17:41:58.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:57 smithi059 ceph-mon[98847]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-12-11T17:41:58.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:57 smithi115 ceph-mon[102748]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-12-11T17:42:00.097 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:41:59 smithi115 ceph-mon[102748]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-12-11T17:42:00.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:41:59 smithi181 ceph-mon[103627]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-12-11T17:42:00.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:41:59 smithi059 ceph-mon[98847]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-12-11T17:42:02.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:01 smithi181 ceph-mon[103627]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-12-11T17:42:02.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:01 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-12-11T17:42:02.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:01 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:42:02.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:01 smithi059 ceph-mon[98847]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-12-11T17:42:02.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:01 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-12-11T17:42:02.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:01 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:42:02.179 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:01 smithi115 ceph-mon[102748]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-12-11T17:42:02.179 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:01 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-12-11T17:42:02.179 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:01 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:42:03.096 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:02 smithi115 ceph-mon[102748]: Deploying daemon osd.7 on smithi115 2023-12-11T17:42:03.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:02 smithi181 ceph-mon[103627]: Deploying daemon osd.7 on smithi115 2023-12-11T17:42:03.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:02 smithi059 ceph-mon[98847]: Deploying daemon osd.7 on smithi115 2023-12-11T17:42:04.005 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:03 smithi115 ceph-mon[102748]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-12-11T17:42:04.146 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:03 smithi181 ceph-mon[103627]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-12-11T17:42:04.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:03 smithi059 ceph-mon[98847]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-12-11T17:42:05.960 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:05 smithi115 ceph-mon[102748]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-12-11T17:42:06.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:05 smithi181 ceph-mon[103627]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-12-11T17:42:06.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:05 smithi059 ceph-mon[98847]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-12-11T17:42:07.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:07 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:42:07.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:07 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:42:07.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:07 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:42:07.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:07 smithi115 ceph-mon[102748]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-12-11T17:42:07.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:07 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:42:07.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:07 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:42:07.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:07 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:42:07.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:07 smithi181 ceph-mon[103627]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-12-11T17:42:07.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:07 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:42:07.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:07 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:42:07.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:07 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:42:07.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:07 smithi059 ceph-mon[98847]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-12-11T17:42:08.883 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:42:08.880+0000 7f655a7fc700 1 -- 172.21.15.115:0/3151414763 <== mgr.14150 v2:172.21.15.59:6800/3627728353 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f656402d380 con 0x7f65500a1480 2023-12-11T17:42:08.883 INFO:teuthology.orchestra.run.smithi115.stdout:Created osd(s) 7 on host 'smithi115' 2023-12-11T17:42:08.885 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:42:08.882+0000 7f656b39f700 1 -- 172.21.15.115:0/3151414763 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f65500a1480 msgr2=0x7f65500a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:42:08.885 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:42:08.882+0000 7f656b39f700 1 --2- 172.21.15.115:0/3151414763 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f65500a1480 0x7f65500a3940 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f656000b5c0 tx=0x7f6560003790 comp rx=0 tx=0).stop 2023-12-11T17:42:08.886 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:42:08.882+0000 7f656b39f700 1 -- 172.21.15.115:0/3151414763 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6564100870 msgr2=0x7f6564199ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:42:08.886 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:42:08.882+0000 7f656b39f700 1 --2- 172.21.15.115:0/3151414763 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6564100870 0x7f6564199ce0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f655400b530 tx=0x7f655400b840 comp rx=0 tx=0).stop 2023-12-11T17:42:08.886 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:42:08.882+0000 7f656b39f700 1 -- 172.21.15.115:0/3151414763 shutdown_connections 2023-12-11T17:42:08.886 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:42:08.882+0000 7f656b39f700 1 --2- 172.21.15.115:0/3151414763 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f65500a1480 0x7f65500a3940 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:42:08.886 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:42:08.882+0000 7f656b39f700 1 --2- 172.21.15.115:0/3151414763 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6564101230 0x7f65641a0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:42:08.886 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:42:08.882+0000 7f656b39f700 1 --2- 172.21.15.115:0/3151414763 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65640fff10 0x7f65641997a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:42:08.886 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:42:08.882+0000 7f656b39f700 1 --2- 172.21.15.115:0/3151414763 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6564100870 0x7f6564199ce0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:42:08.886 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:42:08.882+0000 7f656b39f700 1 -- 172.21.15.115:0/3151414763 >> 172.21.15.115:0/3151414763 conn(0x7f65640fbad0 msgr2=0x7f65640fd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:42:08.886 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:42:08.882+0000 7f656b39f700 1 -- 172.21.15.115:0/3151414763 shutdown_connections 2023-12-11T17:42:08.886 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:42:08.882+0000 7f656b39f700 1 -- 172.21.15.115:0/3151414763 wait complete. 2023-12-11T17:42:09.271 DEBUG:teuthology.orchestra.run.smithi115:osd.7> sudo journalctl -f -n 0 -u ceph-d0a07eaa-984b-11ee-95a3-87774f69a715@osd.7.service 2023-12-11T17:42:09.274 INFO:tasks.cephadm:Deploying osd.8 on smithi181 with /dev/vg_nvme/lv_2... 2023-12-11T17:42:09.274 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d0a07eaa-984b-11ee-95a3-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-12-11T17:42:09.304 INFO:journalctl@ceph.osd.7.smithi115.stdout:-- Logs begin at Mon 2023-12-11 17:22:43 UTC. -- 2023-12-11T17:42:09.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:09 smithi115 ceph-mon[102748]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-12-11T17:42:09.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:09 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:42:09.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:09 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:42:09.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:09 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:42:09.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:09 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:42:09.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:09 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:42:09.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:09 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:42:09.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:09 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:42:09.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:09 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:42:09.645 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:09 smithi181 ceph-mon[103627]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-12-11T17:42:09.645 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:09 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:42:09.645 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:09 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:42:09.645 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:09 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:42:09.645 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:09 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:42:09.645 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:09 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:42:09.645 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:09 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:42:09.645 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:09 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:42:09.645 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:09 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:42:09.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:09 smithi059 ceph-mon[98847]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-12-11T17:42:09.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:09 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:42:09.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:09 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:42:09.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:09 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:42:09.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:09 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:42:09.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:09 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:42:09.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:09 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:42:09.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:09 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:42:09.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:09 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:42:09.890 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/mon.c/config 2023-12-11T17:42:10.032 INFO:journalctl@ceph.osd.7.smithi115.stdout:Dec 11 17:42:09 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-7[124665]: 2023-12-11T17:42:09.903+0000 7f9a8083d640 -1 osd.7 0 log_to_monitors true 2023-12-11T17:42:10.645 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:10 smithi181 ceph-mon[103627]: from='osd.7 [v2:172.21.15.115:6816/1314971870,v1:172.21.15.115:6817/1314971870]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-12-11T17:42:10.645 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:10 smithi181 ceph-mon[103627]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-12-11T17:42:10.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:10 smithi059 ceph-mon[98847]: from='osd.7 [v2:172.21.15.115:6816/1314971870,v1:172.21.15.115:6817/1314971870]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-12-11T17:42:10.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:10 smithi059 ceph-mon[98847]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-12-11T17:42:10.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:10 smithi115 ceph-mon[102748]: from='osd.7 [v2:172.21.15.115:6816/1314971870,v1:172.21.15.115:6817/1314971870]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-12-11T17:42:10.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:10 smithi115 ceph-mon[102748]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-12-11T17:42:11.406 INFO:teuthology.orchestra.run.smithi181.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-12-11T17:42:11.406 INFO:teuthology.orchestra.run.smithi181.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-12-11T17:42:11.406 INFO:teuthology.orchestra.run.smithi181.stderr: stderr: 10+0 records in 2023-12-11T17:42:11.407 INFO:teuthology.orchestra.run.smithi181.stderr:10+0 records out 2023-12-11T17:42:11.407 INFO:teuthology.orchestra.run.smithi181.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0128619 s, 815 MB/s 2023-12-11T17:42:11.407 INFO:teuthology.orchestra.run.smithi181.stderr:--> Zapping successful for: 2023-12-11T17:42:11.645 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:11 smithi181 ceph-mon[103627]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-12-11T17:42:11.645 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:11 smithi181 ceph-mon[103627]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-12-11T17:42:11.645 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:11 smithi181 ceph-mon[103627]: osdmap e48: 8 total, 7 up, 8 in 2023-12-11T17:42:11.645 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:11 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-11T17:42:11.645 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:11 smithi181 ceph-mon[103627]: from='osd.7 [v2:172.21.15.115:6816/1314971870,v1:172.21.15.115:6817/1314971870]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2023-12-11T17:42:11.645 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:11 smithi181 ceph-mon[103627]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2023-12-11T17:42:11.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:11 smithi059 ceph-mon[98847]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-12-11T17:42:11.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:11 smithi059 ceph-mon[98847]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-12-11T17:42:11.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:11 smithi059 ceph-mon[98847]: osdmap e48: 8 total, 7 up, 8 in 2023-12-11T17:42:11.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:11 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-11T17:42:11.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:11 smithi059 ceph-mon[98847]: from='osd.7 [v2:172.21.15.115:6816/1314971870,v1:172.21.15.115:6817/1314971870]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2023-12-11T17:42:11.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:11 smithi059 ceph-mon[98847]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2023-12-11T17:42:11.741 INFO:journalctl@ceph.osd.7.smithi115.stdout:Dec 11 17:42:11 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-7[124665]: 2023-12-11T17:42:11.362+0000 7f9a75eb4700 -1 osd.7 0 waiting for initial osdmap 2023-12-11T17:42:11.741 INFO:journalctl@ceph.osd.7.smithi115.stdout:Dec 11 17:42:11 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-7[124665]: 2023-12-11T17:42:11.371+0000 7f9a704ab700 -1 osd.7 49 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-11T17:42:11.741 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:11 smithi115 ceph-mon[102748]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-12-11T17:42:11.741 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:11 smithi115 ceph-mon[102748]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-12-11T17:42:11.742 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:11 smithi115 ceph-mon[102748]: osdmap e48: 8 total, 7 up, 8 in 2023-12-11T17:42:11.742 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:11 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-11T17:42:11.742 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:11 smithi115 ceph-mon[102748]: from='osd.7 [v2:172.21.15.115:6816/1314971870,v1:172.21.15.115:6817/1314971870]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2023-12-11T17:42:11.742 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:11 smithi115 ceph-mon[102748]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2023-12-11T17:42:11.842 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d0a07eaa-984b-11ee-95a3-87774f69a715 -- ceph orch daemon add osd smithi181:vg_nvme/lv_2 2023-12-11T17:42:12.439 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/mon.c/config 2023-12-11T17:42:12.625 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:12 smithi181 ceph-mon[103627]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2023-12-11T17:42:12.625 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:12 smithi181 ceph-mon[103627]: osdmap e49: 8 total, 7 up, 8 in 2023-12-11T17:42:12.625 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:12 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-11T17:42:12.625 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:12 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-11T17:42:12.625 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:12 smithi181 ceph-mon[103627]: Detected new or changed devices on smithi115 2023-12-11T17:42:12.625 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:12 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:42:12.626 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:12 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:42:12.626 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:12 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:42:12.626 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:12 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:42:12.626 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:12 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:42:12.626 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:12 smithi181 ceph-mon[103627]: Adjusting osd_memory_target on smithi115 to 5722M 2023-12-11T17:42:12.626 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:12 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:42:12.626 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:12 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:42:12.626 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:12 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:42:12.626 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:12 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:42:12.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:12 smithi059 ceph-mon[98847]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2023-12-11T17:42:12.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:12 smithi059 ceph-mon[98847]: osdmap e49: 8 total, 7 up, 8 in 2023-12-11T17:42:12.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:12 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-11T17:42:12.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:12 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-11T17:42:12.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:12 smithi059 ceph-mon[98847]: Detected new or changed devices on smithi115 2023-12-11T17:42:12.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:12 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:42:12.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:12 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:42:12.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:12 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:42:12.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:12 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:42:12.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:12 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:42:12.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:12 smithi059 ceph-mon[98847]: Adjusting osd_memory_target on smithi115 to 5722M 2023-12-11T17:42:12.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:12 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:42:12.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:12 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:42:12.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:12 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:42:12.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:12 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:42:12.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:12 smithi115 ceph-mon[102748]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2023-12-11T17:42:12.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:12 smithi115 ceph-mon[102748]: osdmap e49: 8 total, 7 up, 8 in 2023-12-11T17:42:12.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:12 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-11T17:42:12.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:12 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-11T17:42:12.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:12 smithi115 ceph-mon[102748]: Detected new or changed devices on smithi115 2023-12-11T17:42:12.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:12 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:42:12.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:12 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:42:12.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:12 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:42:12.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:12 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:42:12.784 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:12 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:42:12.784 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:12 smithi115 ceph-mon[102748]: Adjusting osd_memory_target on smithi115 to 5722M 2023-12-11T17:42:12.784 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:12 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:42:12.784 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:12 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:42:12.784 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:12 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:42:12.784 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:12 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:42:13.641 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:42:13.638+0000 7fec63a36700 1 -- 172.21.15.181:0/968166266 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec5c101730 msgr2=0x7fec5c101b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:42:13.641 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:42:13.638+0000 7fec63a36700 1 --2- 172.21.15.181:0/968166266 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec5c101730 0x7fec5c101b90 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fec4c009990 tx=0x7fec4c009ca0 comp rx=0 tx=0).stop 2023-12-11T17:42:13.641 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:42:13.638+0000 7fec63a36700 1 -- 172.21.15.181:0/968166266 shutdown_connections 2023-12-11T17:42:13.641 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:42:13.638+0000 7fec63a36700 1 --2- 172.21.15.181:0/968166266 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fec5c1020d0 0x7fec5c108980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:42:13.641 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:42:13.638+0000 7fec63a36700 1 --2- 172.21.15.181:0/968166266 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fec5c100530 0x7fec5c100950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:42:13.641 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:42:13.638+0000 7fec63a36700 1 --2- 172.21.15.181:0/968166266 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec5c101730 0x7fec5c101b90 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:42:13.641 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:42:13.638+0000 7fec63a36700 1 -- 172.21.15.181:0/968166266 >> 172.21.15.181:0/968166266 conn(0x7fec5c0fbad0 msgr2=0x7fec5c0fdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:42:13.641 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:42:13.638+0000 7fec63a36700 1 -- 172.21.15.181:0/968166266 shutdown_connections 2023-12-11T17:42:13.641 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:42:13.638+0000 7fec63a36700 1 -- 172.21.15.181:0/968166266 wait complete. 2023-12-11T17:42:13.643 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:42:13.638+0000 7fec63a36700 1 Processor -- start 2023-12-11T17:42:13.643 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:42:13.638+0000 7fec63a36700 1 -- start start 2023-12-11T17:42:13.643 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:42:13.639+0000 7fec63a36700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fec5c100530 0x7fec5c06a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:42:13.643 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:42:13.639+0000 7fec63a36700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec5c101730 0x7fec5c19c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:42:13.643 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:42:13.639+0000 7fec63a36700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fec5c1020d0 0x7fec5c19e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:42:13.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:42:13.639+0000 7fec63a36700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec5c19eef0 con 0x7fec5c100530 2023-12-11T17:42:13.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:42:13.639+0000 7fec63a36700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec5c19f060 con 0x7fec5c1020d0 2023-12-11T17:42:13.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:42:13.639+0000 7fec63a36700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec5c19f220 con 0x7fec5c101730 2023-12-11T17:42:13.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:42:13.639+0000 7fec60fd1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec5c101730 0x7fec5c19c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:42:13.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:42:13.639+0000 7fec60fd1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec5c101730 0x7fec5c19c160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34462/0 (socket says 172.21.15.181:34462) 2023-12-11T17:42:13.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:42:13.639+0000 7fec60fd1700 1 -- 172.21.15.181:0/3347592743 learned_addr learned my addr 172.21.15.181:0/3347592743 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-12-11T17:42:13.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:42:13.639+0000 7fec617d2700 1 --2- 172.21.15.181:0/3347592743 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fec5c100530 0x7fec5c06a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:42:13.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:42:13.639+0000 7fec61fd3700 1 --2- 172.21.15.181:0/3347592743 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fec5c1020d0 0x7fec5c19e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:42:13.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:42:13.639+0000 7fec60fd1700 1 -- 172.21.15.181:0/3347592743 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fec5c1020d0 msgr2=0x7fec5c19e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:42:13.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:42:13.639+0000 7fec60fd1700 1 --2- 172.21.15.181:0/3347592743 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fec5c1020d0 0x7fec5c19e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:42:13.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:42:13.639+0000 7fec60fd1700 1 -- 172.21.15.181:0/3347592743 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fec5c100530 msgr2=0x7fec5c06a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:42:13.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:42:13.639+0000 7fec60fd1700 1 --2- 172.21.15.181:0/3347592743 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fec5c100530 0x7fec5c06a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:42:13.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:42:13.639+0000 7fec60fd1700 1 -- 172.21.15.181:0/3347592743 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec4c0095f0 con 0x7fec5c101730 2023-12-11T17:42:13.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:42:13.639+0000 7fec60fd1700 1 --2- 172.21.15.181:0/3347592743 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec5c101730 0x7fec5c19c160 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fec4c0036c0 tx=0x7fec4c00f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:42:13.644 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:42:13.639+0000 7fec527fc700 1 -- 172.21.15.181:0/3347592743 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec4c01d070 con 0x7fec5c101730 2023-12-11T17:42:13.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:42:13.639+0000 7fec527fc700 1 -- 172.21.15.181:0/3347592743 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fec4c022470 con 0x7fec5c101730 2023-12-11T17:42:13.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:42:13.639+0000 7fec63a36700 1 -- 172.21.15.181:0/3347592743 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec5c19f4a0 con 0x7fec5c101730 2023-12-11T17:42:13.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:42:13.639+0000 7fec527fc700 1 -- 172.21.15.181:0/3347592743 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec4c017650 con 0x7fec5c101730 2023-12-11T17:42:13.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:42:13.640+0000 7fec63a36700 1 -- 172.21.15.181:0/3347592743 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec5c19f910 con 0x7fec5c101730 2023-12-11T17:42:13.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:42:13.640+0000 7fec63a36700 1 -- 172.21.15.181:0/3347592743 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec5c1078f0 con 0x7fec5c101730 2023-12-11T17:42:13.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:42:13.641+0000 7fec527fc700 1 -- 172.21.15.181:0/3347592743 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7fec4c022850 con 0x7fec5c101730 2023-12-11T17:42:13.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:42:13.642+0000 7fec527fc700 1 --2- 172.21.15.181:0/3347592743 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fec480a1500 0x7fec480a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:42:13.646 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:13 smithi181 ceph-mon[103627]: purged_snaps scrub starts 2023-12-11T17:42:13.646 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:13 smithi181 ceph-mon[103627]: purged_snaps scrub ok 2023-12-11T17:42:13.646 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:13 smithi181 ceph-mon[103627]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-12-11T17:42:13.646 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:13 smithi181 ceph-mon[103627]: osd.7 [v2:172.21.15.115:6816/1314971870,v1:172.21.15.115:6817/1314971870] boot 2023-12-11T17:42:13.646 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:13 smithi181 ceph-mon[103627]: osdmap e50: 8 total, 8 up, 8 in 2023-12-11T17:42:13.646 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:13 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-11T17:42:13.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:42:13.642+0000 7fec527fc700 1 -- 172.21.15.181:0/3347592743 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5426+0+0 (secure 0 0 0) 0x7fec4c0cd440 con 0x7fec5c101730 2023-12-11T17:42:13.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:42:13.642+0000 7fec617d2700 1 --2- 172.21.15.181:0/3347592743 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fec480a1500 0x7fec480a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:42:13.647 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:42:13.643+0000 7fec617d2700 1 --2- 172.21.15.181:0/3347592743 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fec480a1500 0x7fec480a39c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fec5c101590 tx=0x7fec5800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:42:13.649 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:42:13.647+0000 7fec527fc700 1 -- 172.21.15.181:0/3347592743 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fec4c097670 con 0x7fec5c101730 2023-12-11T17:42:13.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:13 smithi059 ceph-mon[98847]: purged_snaps scrub starts 2023-12-11T17:42:13.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:13 smithi059 ceph-mon[98847]: purged_snaps scrub ok 2023-12-11T17:42:13.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:13 smithi059 ceph-mon[98847]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-12-11T17:42:13.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:13 smithi059 ceph-mon[98847]: osd.7 [v2:172.21.15.115:6816/1314971870,v1:172.21.15.115:6817/1314971870] boot 2023-12-11T17:42:13.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:13 smithi059 ceph-mon[98847]: osdmap e50: 8 total, 8 up, 8 in 2023-12-11T17:42:13.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:13 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-11T17:42:13.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:13 smithi115 ceph-mon[102748]: purged_snaps scrub starts 2023-12-11T17:42:13.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:13 smithi115 ceph-mon[102748]: purged_snaps scrub ok 2023-12-11T17:42:13.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:13 smithi115 ceph-mon[102748]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-12-11T17:42:13.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:13 smithi115 ceph-mon[102748]: osd.7 [v2:172.21.15.115:6816/1314971870,v1:172.21.15.115:6817/1314971870] boot 2023-12-11T17:42:13.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:13 smithi115 ceph-mon[102748]: osdmap e50: 8 total, 8 up, 8 in 2023-12-11T17:42:13.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:13 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-11T17:42:13.849 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:42:13.846+0000 7fec63a36700 1 -- 172.21.15.181:0/3347592743 --> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fec5c061a10 con 0x7fec480a1500 2023-12-11T17:42:14.645 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:14 smithi181 ceph-mon[103627]: osdmap e51: 8 total, 8 up, 8 in 2023-12-11T17:42:14.645 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:14 smithi181 ceph-mon[103627]: from='client.24289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T17:42:14.645 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:14 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-11T17:42:14.645 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:14 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-11T17:42:14.645 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:14 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:42:14.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:14 smithi059 ceph-mon[98847]: osdmap e51: 8 total, 8 up, 8 in 2023-12-11T17:42:14.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:14 smithi059 ceph-mon[98847]: from='client.24289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T17:42:14.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:14 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-11T17:42:14.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:14 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-11T17:42:14.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:14 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:42:14.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:14 smithi115 ceph-mon[102748]: osdmap e51: 8 total, 8 up, 8 in 2023-12-11T17:42:14.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:14 smithi115 ceph-mon[102748]: from='client.24289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T17:42:14.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:14 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-11T17:42:14.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:14 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-11T17:42:14.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:14 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:42:15.645 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:15 smithi181 ceph-mon[103627]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-12-11T17:42:15.645 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:15 smithi181 ceph-mon[103627]: osdmap e52: 8 total, 8 up, 8 in 2023-12-11T17:42:15.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:15 smithi059 ceph-mon[98847]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-12-11T17:42:15.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:15 smithi059 ceph-mon[98847]: osdmap e52: 8 total, 8 up, 8 in 2023-12-11T17:42:15.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:15 smithi115 ceph-mon[102748]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-12-11T17:42:15.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:15 smithi115 ceph-mon[102748]: osdmap e52: 8 total, 8 up, 8 in 2023-12-11T17:42:16.645 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:16 smithi181 ceph-mon[103627]: osdmap e53: 8 total, 8 up, 8 in 2023-12-11T17:42:16.645 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:16 smithi181 ceph-mon[103627]: from='client.? 172.21.15.181:0/1307564862' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eb657d32-1a95-4fd7-a293-d0a970837fc3"}]: dispatch 2023-12-11T17:42:16.645 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:16 smithi181 ceph-mon[103627]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eb657d32-1a95-4fd7-a293-d0a970837fc3"}]: dispatch 2023-12-11T17:42:16.645 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:16 smithi181 ceph-mon[103627]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "eb657d32-1a95-4fd7-a293-d0a970837fc3"}]': finished 2023-12-11T17:42:16.645 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:16 smithi181 ceph-mon[103627]: osdmap e54: 9 total, 8 up, 9 in 2023-12-11T17:42:16.646 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:16 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-11T17:42:16.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:16 smithi059 ceph-mon[98847]: osdmap e53: 8 total, 8 up, 8 in 2023-12-11T17:42:16.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:16 smithi059 ceph-mon[98847]: from='client.? 172.21.15.181:0/1307564862' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eb657d32-1a95-4fd7-a293-d0a970837fc3"}]: dispatch 2023-12-11T17:42:16.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:16 smithi059 ceph-mon[98847]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eb657d32-1a95-4fd7-a293-d0a970837fc3"}]: dispatch 2023-12-11T17:42:16.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:16 smithi059 ceph-mon[98847]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "eb657d32-1a95-4fd7-a293-d0a970837fc3"}]': finished 2023-12-11T17:42:16.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:16 smithi059 ceph-mon[98847]: osdmap e54: 9 total, 8 up, 9 in 2023-12-11T17:42:16.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:16 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-11T17:42:16.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:16 smithi115 ceph-mon[102748]: osdmap e53: 8 total, 8 up, 8 in 2023-12-11T17:42:16.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:16 smithi115 ceph-mon[102748]: from='client.? 172.21.15.181:0/1307564862' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eb657d32-1a95-4fd7-a293-d0a970837fc3"}]: dispatch 2023-12-11T17:42:16.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:16 smithi115 ceph-mon[102748]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eb657d32-1a95-4fd7-a293-d0a970837fc3"}]: dispatch 2023-12-11T17:42:16.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:16 smithi115 ceph-mon[102748]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "eb657d32-1a95-4fd7-a293-d0a970837fc3"}]': finished 2023-12-11T17:42:16.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:16 smithi115 ceph-mon[102748]: osdmap e54: 9 total, 8 up, 9 in 2023-12-11T17:42:16.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:16 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-11T17:42:17.645 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:17 smithi181 ceph-mon[103627]: pgmap v164: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-12-11T17:42:17.645 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:17 smithi181 ceph-mon[103627]: from='client.? 172.21.15.181:0/4242517816' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-11T17:42:17.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:17 smithi059 ceph-mon[98847]: pgmap v164: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-12-11T17:42:17.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:17 smithi059 ceph-mon[98847]: from='client.? 172.21.15.181:0/4242517816' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-11T17:42:17.782 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:17 smithi115 ceph-mon[102748]: pgmap v164: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-12-11T17:42:17.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:17 smithi115 ceph-mon[102748]: from='client.? 172.21.15.181:0/4242517816' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-11T17:42:19.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:19 smithi059 ceph-mon[98847]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-12-11T17:42:19.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:19 smithi115 ceph-mon[102748]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-12-11T17:42:19.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:19 smithi181 ceph-mon[103627]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-12-11T17:42:21.484 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:21 smithi181 ceph-mon[103627]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-12-11T17:42:21.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:21 smithi059 ceph-mon[98847]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-12-11T17:42:21.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:21 smithi115 ceph-mon[102748]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-12-11T17:42:23.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:23 smithi059 ceph-mon[98847]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-12-11T17:42:23.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:23 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-12-11T17:42:23.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:23 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:42:23.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:23 smithi059 ceph-mon[98847]: Deploying daemon osd.8 on smithi181 2023-12-11T17:42:23.688 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:23 smithi181 ceph-mon[103627]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-12-11T17:42:23.688 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:23 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-12-11T17:42:23.688 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:23 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:42:23.688 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:23 smithi181 ceph-mon[103627]: Deploying daemon osd.8 on smithi181 2023-12-11T17:42:23.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:23 smithi115 ceph-mon[102748]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-12-11T17:42:23.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:23 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-12-11T17:42:23.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:23 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:42:23.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:23 smithi115 ceph-mon[102748]: Deploying daemon osd.8 on smithi181 2023-12-11T17:42:25.645 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:25 smithi181 ceph-mon[103627]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-12-11T17:42:25.782 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:25 smithi115 ceph-mon[102748]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-12-11T17:42:25.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:25 smithi059 ceph-mon[98847]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-12-11T17:42:27.704 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:27 smithi181 ceph-mon[103627]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-12-11T17:42:27.704 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:27 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:42:27.704 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:27 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:42:27.704 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:27 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:42:27.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:27 smithi115 ceph-mon[102748]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-12-11T17:42:27.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:27 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:42:27.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:27 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:42:27.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:27 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:42:27.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:27 smithi059 ceph-mon[98847]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-12-11T17:42:27.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:27 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:42:27.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:27 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:42:27.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:27 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:42:29.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:29 smithi115 ceph-mon[102748]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-12-11T17:42:29.783 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:29 smithi181 ceph-mon[103627]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-12-11T17:42:29.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:29 smithi059 ceph-mon[98847]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-12-11T17:42:30.066 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:42:30.064+0000 7fec527fc700 1 -- 172.21.15.181:0/3347592743 <== mgr.14150 v2:172.21.15.59:6800/3627728353 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fec5c061a10 con 0x7fec480a1500 2023-12-11T17:42:30.067 INFO:teuthology.orchestra.run.smithi181.stdout:Created osd(s) 8 on host 'smithi181' 2023-12-11T17:42:30.069 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:42:30.066+0000 7fec63a36700 1 -- 172.21.15.181:0/3347592743 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fec480a1500 msgr2=0x7fec480a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:42:30.069 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:42:30.066+0000 7fec63a36700 1 --2- 172.21.15.181:0/3347592743 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fec480a1500 0x7fec480a39c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fec5c101590 tx=0x7fec5800b410 comp rx=0 tx=0).stop 2023-12-11T17:42:30.069 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:42:30.066+0000 7fec63a36700 1 -- 172.21.15.181:0/3347592743 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec5c101730 msgr2=0x7fec5c19c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:42:30.069 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:42:30.066+0000 7fec63a36700 1 --2- 172.21.15.181:0/3347592743 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec5c101730 0x7fec5c19c160 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fec4c0036c0 tx=0x7fec4c00f690 comp rx=0 tx=0).stop 2023-12-11T17:42:30.069 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:42:30.066+0000 7fec63a36700 1 -- 172.21.15.181:0/3347592743 shutdown_connections 2023-12-11T17:42:30.069 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:42:30.066+0000 7fec63a36700 1 --2- 172.21.15.181:0/3347592743 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fec480a1500 0x7fec480a39c0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:42:30.069 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:42:30.066+0000 7fec63a36700 1 --2- 172.21.15.181:0/3347592743 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fec5c1020d0 0x7fec5c19e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:42:30.069 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:42:30.066+0000 7fec63a36700 1 --2- 172.21.15.181:0/3347592743 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fec5c100530 0x7fec5c06a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:42:30.069 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:42:30.066+0000 7fec63a36700 1 --2- 172.21.15.181:0/3347592743 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec5c101730 0x7fec5c19c160 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:42:30.069 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:42:30.066+0000 7fec63a36700 1 -- 172.21.15.181:0/3347592743 >> 172.21.15.181:0/3347592743 conn(0x7fec5c0fbad0 msgr2=0x7fec5c104a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:42:30.069 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:42:30.066+0000 7fec63a36700 1 -- 172.21.15.181:0/3347592743 shutdown_connections 2023-12-11T17:42:30.069 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:42:30.066+0000 7fec63a36700 1 -- 172.21.15.181:0/3347592743 wait complete. 2023-12-11T17:42:30.544 DEBUG:teuthology.orchestra.run.smithi181:osd.8> sudo journalctl -f -n 0 -u ceph-d0a07eaa-984b-11ee-95a3-87774f69a715@osd.8.service 2023-12-11T17:42:30.546 INFO:tasks.cephadm:Deploying osd.9 on smithi059 with /dev/vg_nvme/lv_1... 2023-12-11T17:42:30.546 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d0a07eaa-984b-11ee-95a3-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-12-11T17:42:30.577 INFO:journalctl@ceph.osd.8.smithi181.stdout:-- Logs begin at Mon 2023-12-11 17:22:43 UTC. -- 2023-12-11T17:42:30.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:30 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:42:30.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:30 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:42:30.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:30 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:42:30.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:30 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:42:30.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:30 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:42:30.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:30 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:42:30.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:30 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:42:30.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:30 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:42:31.014 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:30 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:42:31.015 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:30 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:42:31.015 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:30 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:42:31.015 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:30 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:42:31.015 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:30 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:42:31.015 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:30 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:42:31.015 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:30 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:42:31.015 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:30 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:42:31.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:30 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:42:31.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:30 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:42:31.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:30 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:42:31.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:30 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:42:31.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:30 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:42:31.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:30 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:42:31.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:30 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:42:31.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:30 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:42:31.165 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/mon.a/config 2023-12-11T17:42:31.395 INFO:journalctl@ceph.osd.8.smithi181.stdout:Dec 11 17:42:31 smithi181 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-8[126177]: 2023-12-11T17:42:31.256+0000 7f87df864640 -1 osd.8 0 log_to_monitors true 2023-12-11T17:42:31.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:31 smithi059 ceph-mon[98847]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-12-11T17:42:31.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:31 smithi059 ceph-mon[98847]: from='osd.8 [v2:172.21.15.181:6816/3954509873,v1:172.21.15.181:6817/3954509873]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-12-11T17:42:31.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:31 smithi059 ceph-mon[98847]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-12-11T17:42:32.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:31 smithi115 ceph-mon[102748]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-12-11T17:42:32.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:31 smithi115 ceph-mon[102748]: from='osd.8 [v2:172.21.15.181:6816/3954509873,v1:172.21.15.181:6817/3954509873]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-12-11T17:42:32.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:31 smithi115 ceph-mon[102748]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-12-11T17:42:32.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:31 smithi181 ceph-mon[103627]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-12-11T17:42:32.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:31 smithi181 ceph-mon[103627]: from='osd.8 [v2:172.21.15.181:6816/3954509873,v1:172.21.15.181:6817/3954509873]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-12-11T17:42:32.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:31 smithi181 ceph-mon[103627]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-12-11T17:42:32.692 INFO:teuthology.orchestra.run.smithi059.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-12-11T17:42:32.692 INFO:teuthology.orchestra.run.smithi059.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-12-11T17:42:32.692 INFO:teuthology.orchestra.run.smithi059.stderr: stderr: 10+0 records in 2023-12-11T17:42:32.692 INFO:teuthology.orchestra.run.smithi059.stderr:10+0 records out 2023-12-11T17:42:32.692 INFO:teuthology.orchestra.run.smithi059.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0123213 s, 851 MB/s 2023-12-11T17:42:32.693 INFO:teuthology.orchestra.run.smithi059.stderr:--> Zapping successful for: 2023-12-11T17:42:33.133 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d0a07eaa-984b-11ee-95a3-87774f69a715 -- ceph orch daemon add osd smithi059:vg_nvme/lv_1 2023-12-11T17:42:33.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:33 smithi115 ceph-mon[102748]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-12-11T17:42:33.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:33 smithi115 ceph-mon[102748]: osdmap e55: 9 total, 8 up, 9 in 2023-12-11T17:42:33.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:33 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-11T17:42:33.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:33 smithi115 ceph-mon[102748]: from='osd.8 [v2:172.21.15.181:6816/3954509873,v1:172.21.15.181:6817/3954509873]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2023-12-11T17:42:33.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:33 smithi115 ceph-mon[102748]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2023-12-11T17:42:33.284 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:33 smithi115 ceph-mon[102748]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-12-11T17:42:33.395 INFO:journalctl@ceph.osd.8.smithi181.stdout:Dec 11 17:42:33 smithi181 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-8[126177]: 2023-12-11T17:42:33.025+0000 7f87d56dc700 -1 osd.8 0 waiting for initial osdmap 2023-12-11T17:42:33.395 INFO:journalctl@ceph.osd.8.smithi181.stdout:Dec 11 17:42:33 smithi181 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-8[126177]: 2023-12-11T17:42:33.033+0000 7f87cf4d2700 -1 osd.8 56 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-11T17:42:33.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:33 smithi181 ceph-mon[103627]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-12-11T17:42:33.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:33 smithi181 ceph-mon[103627]: osdmap e55: 9 total, 8 up, 9 in 2023-12-11T17:42:33.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:33 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-11T17:42:33.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:33 smithi181 ceph-mon[103627]: from='osd.8 [v2:172.21.15.181:6816/3954509873,v1:172.21.15.181:6817/3954509873]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2023-12-11T17:42:33.396 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:33 smithi181 ceph-mon[103627]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2023-12-11T17:42:33.396 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:33 smithi181 ceph-mon[103627]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-12-11T17:42:33.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:33 smithi059 ceph-mon[98847]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-12-11T17:42:33.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:33 smithi059 ceph-mon[98847]: osdmap e55: 9 total, 8 up, 9 in 2023-12-11T17:42:33.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:33 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-11T17:42:33.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:33 smithi059 ceph-mon[98847]: from='osd.8 [v2:172.21.15.181:6816/3954509873,v1:172.21.15.181:6817/3954509873]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2023-12-11T17:42:33.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:33 smithi059 ceph-mon[98847]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2023-12-11T17:42:33.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:33 smithi059 ceph-mon[98847]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-12-11T17:42:33.748 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/mon.a/config 2023-12-11T17:42:34.275 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:34 smithi059 ceph-mon[98847]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]': finished 2023-12-11T17:42:34.276 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:34 smithi059 ceph-mon[98847]: osdmap e56: 9 total, 8 up, 9 in 2023-12-11T17:42:34.276 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:34 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-11T17:42:34.276 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:34 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-11T17:42:34.276 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:34 smithi059 ceph-mon[98847]: Detected new or changed devices on smithi181 2023-12-11T17:42:34.276 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:34 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:42:34.276 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:34 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:42:34.276 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:34 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:42:34.276 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:34 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:42:34.276 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:34 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:42:34.276 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:34 smithi059 ceph-mon[98847]: Adjusting osd_memory_target on smithi181 to 5722M 2023-12-11T17:42:34.277 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:34 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:42:34.277 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:34 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:42:34.277 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:34 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:42:34.277 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:34 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:42:34.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:34 smithi115 ceph-mon[102748]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]': finished 2023-12-11T17:42:34.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:34 smithi115 ceph-mon[102748]: osdmap e56: 9 total, 8 up, 9 in 2023-12-11T17:42:34.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:34 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-11T17:42:34.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:34 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-11T17:42:34.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:34 smithi115 ceph-mon[102748]: Detected new or changed devices on smithi181 2023-12-11T17:42:34.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:34 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:42:34.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:34 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:42:34.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:34 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:42:34.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:34 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:42:34.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:34 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:42:34.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:34 smithi115 ceph-mon[102748]: Adjusting osd_memory_target on smithi181 to 5722M 2023-12-11T17:42:34.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:34 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:42:34.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:34 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:42:34.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:34 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:42:34.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:34 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:42:34.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:34 smithi181 ceph-mon[103627]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]': finished 2023-12-11T17:42:34.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:34 smithi181 ceph-mon[103627]: osdmap e56: 9 total, 8 up, 9 in 2023-12-11T17:42:34.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:34 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-11T17:42:34.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:34 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-11T17:42:34.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:34 smithi181 ceph-mon[103627]: Detected new or changed devices on smithi181 2023-12-11T17:42:34.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:34 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:42:34.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:34 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:42:34.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:34 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:42:34.396 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:34 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:42:34.396 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:34 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:42:34.396 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:34 smithi181 ceph-mon[103627]: Adjusting osd_memory_target on smithi181 to 5722M 2023-12-11T17:42:34.396 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:34 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:42:34.396 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:34 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:42:34.396 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:34 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:42:34.396 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:34 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:42:34.968 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:42:34.965+0000 7fbe3199a700 1 -- 172.21.15.59:0/3107189751 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbe2c102140 msgr2=0x7fbe2c104530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:42:34.968 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:42:34.965+0000 7fbe3199a700 1 --2- 172.21.15.59:0/3107189751 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbe2c102140 0x7fbe2c104530 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fbe1c009990 tx=0x7fbe1c009ca0 comp rx=0 tx=0).stop 2023-12-11T17:42:34.968 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:42:34.965+0000 7fbe3199a700 1 -- 172.21.15.59:0/3107189751 shutdown_connections 2023-12-11T17:42:34.968 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:42:34.965+0000 7fbe3199a700 1 --2- 172.21.15.59:0/3107189751 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe2c1073a0 0x7fbe2c109790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:42:34.969 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:42:34.965+0000 7fbe3199a700 1 --2- 172.21.15.59:0/3107189751 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbe2c102140 0x7fbe2c104530 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:42:34.969 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:42:34.965+0000 7fbe3199a700 1 --2- 172.21.15.59:0/3107189751 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbe2c104a70 0x7fbe2c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:42:34.969 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:42:34.966+0000 7fbe3199a700 1 -- 172.21.15.59:0/3107189751 >> 172.21.15.59:0/3107189751 conn(0x7fbe2c0fbad0 msgr2=0x7fbe2c0fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:42:34.969 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:42:34.966+0000 7fbe3199a700 1 -- 172.21.15.59:0/3107189751 shutdown_connections 2023-12-11T17:42:34.969 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:42:34.966+0000 7fbe3199a700 1 -- 172.21.15.59:0/3107189751 wait complete. 2023-12-11T17:42:34.971 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:42:34.966+0000 7fbe3199a700 1 Processor -- start 2023-12-11T17:42:34.971 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:42:34.966+0000 7fbe3199a700 1 -- start start 2023-12-11T17:42:34.971 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:42:34.967+0000 7fbe3199a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe2c102140 0x7fbe2c101a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:42:34.971 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:42:34.967+0000 7fbe3199a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbe2c104a70 0x7fbe2c1000f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:42:34.971 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:42:34.967+0000 7fbe3199a700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbe2c1073a0 0x7fbe2c100630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:42:34.971 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:42:34.967+0000 7fbe3199a700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe2c100dd0 con 0x7fbe2c104a70 2023-12-11T17:42:34.972 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:42:34.967+0000 7fbe3199a700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe2c100f40 con 0x7fbe2c1073a0 2023-12-11T17:42:34.972 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:42:34.967+0000 7fbe3199a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe2c1a1580 con 0x7fbe2c102140 2023-12-11T17:42:34.972 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:42:34.967+0000 7fbe2a7fc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbe2c104a70 0x7fbe2c1000f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:42:34.972 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:42:34.967+0000 7fbe2a7fc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbe2c104a70 0x7fbe2c1000f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60542/0 (socket says 172.21.15.59:60542) 2023-12-11T17:42:34.972 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:42:34.967+0000 7fbe2a7fc700 1 -- 172.21.15.59:0/2940778815 learned_addr learned my addr 172.21.15.59:0/2940778815 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:42:34.972 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:42:34.967+0000 7fbe2affd700 1 --2- 172.21.15.59:0/2940778815 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe2c102140 0x7fbe2c101a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:42:34.972 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:42:34.967+0000 7fbe2b7fe700 1 --2- 172.21.15.59:0/2940778815 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbe2c1073a0 0x7fbe2c100630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:42:34.972 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:42:34.967+0000 7fbe2a7fc700 1 -- 172.21.15.59:0/2940778815 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe2c102140 msgr2=0x7fbe2c101a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:42:34.972 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:42:34.967+0000 7fbe2a7fc700 1 --2- 172.21.15.59:0/2940778815 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe2c102140 0x7fbe2c101a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:42:34.972 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:42:34.967+0000 7fbe2a7fc700 1 -- 172.21.15.59:0/2940778815 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbe2c1073a0 msgr2=0x7fbe2c100630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:42:34.972 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:42:34.967+0000 7fbe2a7fc700 1 --2- 172.21.15.59:0/2940778815 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbe2c1073a0 0x7fbe2c100630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:42:34.972 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:42:34.967+0000 7fbe2a7fc700 1 -- 172.21.15.59:0/2940778815 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe20009520 con 0x7fbe2c104a70 2023-12-11T17:42:34.972 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:42:34.967+0000 7fbe2affd700 1 --2- 172.21.15.59:0/2940778815 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe2c102140 0x7fbe2c101a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T17:42:34.972 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:42:34.967+0000 7fbe2a7fc700 1 --2- 172.21.15.59:0/2940778815 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbe2c104a70 0x7fbe2c1000f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fbe2000eb30 tx=0x7fbe2000ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:42:34.973 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:42:34.967+0000 7fbe30998700 1 -- 172.21.15.59:0/2940778815 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe20009880 con 0x7fbe2c104a70 2023-12-11T17:42:34.973 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:42:34.967+0000 7fbe30998700 1 -- 172.21.15.59:0/2940778815 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbe200044e0 con 0x7fbe2c104a70 2023-12-11T17:42:34.973 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:42:34.967+0000 7fbe3199a700 1 -- 172.21.15.59:0/2940778815 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe1c0095f0 con 0x7fbe2c104a70 2023-12-11T17:42:34.973 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:42:34.968+0000 7fbe30998700 1 -- 172.21.15.59:0/2940778815 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe20003d80 con 0x7fbe2c104a70 2023-12-11T17:42:34.973 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:42:34.968+0000 7fbe3199a700 1 -- 172.21.15.59:0/2940778815 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe2c1a1bd0 con 0x7fbe2c104a70 2023-12-11T17:42:34.973 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:42:34.968+0000 7fbe3199a700 1 -- 172.21.15.59:0/2940778815 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe2c0fd650 con 0x7fbe2c104a70 2023-12-11T17:42:34.974 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:42:34.970+0000 7fbe30998700 1 -- 172.21.15.59:0/2940778815 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7fbe20004000 con 0x7fbe2c104a70 2023-12-11T17:42:34.974 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:42:34.970+0000 7fbe30998700 1 --2- 172.21.15.59:0/2940778815 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fbe140a1500 0x7fbe140a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:42:34.974 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:42:34.970+0000 7fbe2affd700 1 --2- 172.21.15.59:0/2940778815 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fbe140a1500 0x7fbe140a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:42:34.974 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:42:34.970+0000 7fbe30998700 1 -- 172.21.15.59:0/2940778815 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5858+0+0 (secure 0 0 0) 0x7fbe20014070 con 0x7fbe2c104a70 2023-12-11T17:42:34.974 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:42:34.971+0000 7fbe2affd700 1 --2- 172.21.15.59:0/2940778815 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fbe140a1500 0x7fbe140a39c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fbe1c00b2e0 tx=0x7fbe1c005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:42:34.976 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:42:34.974+0000 7fbe30998700 1 -- 172.21.15.59:0/2940778815 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fbe20096960 con 0x7fbe2c104a70 2023-12-11T17:42:35.189 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:42:35.187+0000 7fbe3199a700 1 -- 172.21.15.59:0/2940778815 --> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi059:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fbe2c061a10 con 0x7fbe140a1500 2023-12-11T17:42:35.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:35 smithi181 ceph-mon[103627]: purged_snaps scrub starts 2023-12-11T17:42:35.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:35 smithi181 ceph-mon[103627]: purged_snaps scrub ok 2023-12-11T17:42:35.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:35 smithi181 ceph-mon[103627]: osd.8 [v2:172.21.15.181:6816/3954509873,v1:172.21.15.181:6817/3954509873] boot 2023-12-11T17:42:35.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:35 smithi181 ceph-mon[103627]: osdmap e57: 9 total, 9 up, 9 in 2023-12-11T17:42:35.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:35 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-11T17:42:35.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:35 smithi181 ceph-mon[103627]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-12-11T17:42:35.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:35 smithi181 ceph-mon[103627]: osdmap e58: 9 total, 9 up, 9 in 2023-12-11T17:42:35.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:35 smithi059 ceph-mon[98847]: purged_snaps scrub starts 2023-12-11T17:42:35.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:35 smithi059 ceph-mon[98847]: purged_snaps scrub ok 2023-12-11T17:42:35.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:35 smithi059 ceph-mon[98847]: osd.8 [v2:172.21.15.181:6816/3954509873,v1:172.21.15.181:6817/3954509873] boot 2023-12-11T17:42:35.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:35 smithi059 ceph-mon[98847]: osdmap e57: 9 total, 9 up, 9 in 2023-12-11T17:42:35.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:35 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-11T17:42:35.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:35 smithi059 ceph-mon[98847]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-12-11T17:42:35.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:35 smithi059 ceph-mon[98847]: osdmap e58: 9 total, 9 up, 9 in 2023-12-11T17:42:35.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:35 smithi115 ceph-mon[102748]: purged_snaps scrub starts 2023-12-11T17:42:35.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:35 smithi115 ceph-mon[102748]: purged_snaps scrub ok 2023-12-11T17:42:35.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:35 smithi115 ceph-mon[102748]: osd.8 [v2:172.21.15.181:6816/3954509873,v1:172.21.15.181:6817/3954509873] boot 2023-12-11T17:42:35.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:35 smithi115 ceph-mon[102748]: osdmap e57: 9 total, 9 up, 9 in 2023-12-11T17:42:35.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:35 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-11T17:42:35.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:35 smithi115 ceph-mon[102748]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-12-11T17:42:35.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:35 smithi115 ceph-mon[102748]: osdmap e58: 9 total, 9 up, 9 in 2023-12-11T17:42:36.342 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:36 smithi059 ceph-mon[98847]: osdmap e59: 9 total, 9 up, 9 in 2023-12-11T17:42:36.342 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:36 smithi059 ceph-mon[98847]: from='client.14397 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi059:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T17:42:36.342 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:36 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-11T17:42:36.342 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:36 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-11T17:42:36.342 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:36 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:42:36.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:36 smithi181 ceph-mon[103627]: osdmap e59: 9 total, 9 up, 9 in 2023-12-11T17:42:36.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:36 smithi181 ceph-mon[103627]: from='client.14397 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi059:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T17:42:36.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:36 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-11T17:42:36.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:36 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-11T17:42:36.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:36 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:42:36.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:36 smithi115 ceph-mon[102748]: osdmap e59: 9 total, 9 up, 9 in 2023-12-11T17:42:36.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:36 smithi115 ceph-mon[102748]: from='client.14397 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi059:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T17:42:36.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:36 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-11T17:42:36.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:36 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-11T17:42:36.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:36 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:42:37.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:37 smithi181 ceph-mon[103627]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-12-11T17:42:37.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:37 smithi181 ceph-mon[103627]: osdmap e60: 9 total, 9 up, 9 in 2023-12-11T17:42:37.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:37 smithi059 ceph-mon[98847]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-12-11T17:42:37.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:37 smithi059 ceph-mon[98847]: osdmap e60: 9 total, 9 up, 9 in 2023-12-11T17:42:37.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:37 smithi115 ceph-mon[102748]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-12-11T17:42:37.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:37 smithi115 ceph-mon[102748]: osdmap e60: 9 total, 9 up, 9 in 2023-12-11T17:42:38.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:38 smithi181 ceph-mon[103627]: osdmap e61: 9 total, 9 up, 9 in 2023-12-11T17:42:38.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:38 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/2739445168' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8f7717d4-b6f2-4ec1-9b58-62feb0c9a146"}]: dispatch 2023-12-11T17:42:38.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:38 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/2739445168' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8f7717d4-b6f2-4ec1-9b58-62feb0c9a146"}]': finished 2023-12-11T17:42:38.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:38 smithi181 ceph-mon[103627]: osdmap e62: 10 total, 9 up, 10 in 2023-12-11T17:42:38.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:38 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-11T17:42:38.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:38 smithi059 ceph-mon[98847]: osdmap e61: 9 total, 9 up, 9 in 2023-12-11T17:42:38.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:38 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/2739445168' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8f7717d4-b6f2-4ec1-9b58-62feb0c9a146"}]: dispatch 2023-12-11T17:42:38.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:38 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/2739445168' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8f7717d4-b6f2-4ec1-9b58-62feb0c9a146"}]': finished 2023-12-11T17:42:38.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:38 smithi059 ceph-mon[98847]: osdmap e62: 10 total, 9 up, 10 in 2023-12-11T17:42:38.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:38 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-11T17:42:38.532 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:38 smithi115 ceph-mon[102748]: osdmap e61: 9 total, 9 up, 9 in 2023-12-11T17:42:38.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:38 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/2739445168' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8f7717d4-b6f2-4ec1-9b58-62feb0c9a146"}]: dispatch 2023-12-11T17:42:38.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:38 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/2739445168' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8f7717d4-b6f2-4ec1-9b58-62feb0c9a146"}]': finished 2023-12-11T17:42:38.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:38 smithi115 ceph-mon[102748]: osdmap e62: 10 total, 9 up, 10 in 2023-12-11T17:42:38.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:38 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-11T17:42:39.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:39 smithi181 ceph-mon[103627]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-12-11T17:42:39.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:39 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/3822906044' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-11T17:42:39.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:39 smithi059 ceph-mon[98847]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-12-11T17:42:39.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:39 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/3822906044' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-11T17:42:39.532 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:39 smithi115 ceph-mon[102748]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-12-11T17:42:39.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:39 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/3822906044' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-11T17:42:41.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:41 smithi181 ceph-mon[103627]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 116 KiB/s, 0 objects/s recovering 2023-12-11T17:42:41.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:41 smithi059 ceph-mon[98847]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 116 KiB/s, 0 objects/s recovering 2023-12-11T17:42:41.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:41 smithi115 ceph-mon[102748]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 116 KiB/s, 0 objects/s recovering 2023-12-11T17:42:43.266 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:43 smithi059 ceph-mon[98847]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-12-11T17:42:43.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:43 smithi181 ceph-mon[103627]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-12-11T17:42:43.532 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:43 smithi115 ceph-mon[102748]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-12-11T17:42:45.389 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:45 smithi059 ceph-mon[98847]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-12-11T17:42:45.389 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:45 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-12-11T17:42:45.389 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:45 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:42:45.389 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:45 smithi059 ceph-mon[98847]: Deploying daemon osd.9 on smithi059 2023-12-11T17:42:45.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:45 smithi181 ceph-mon[103627]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-12-11T17:42:45.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:45 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-12-11T17:42:45.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:45 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:42:45.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:45 smithi181 ceph-mon[103627]: Deploying daemon osd.9 on smithi059 2023-12-11T17:42:45.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:45 smithi115 ceph-mon[102748]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-12-11T17:42:45.534 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:45 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-12-11T17:42:45.534 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:45 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:42:45.534 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:45 smithi115 ceph-mon[102748]: Deploying daemon osd.9 on smithi059 2023-12-11T17:42:47.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:47 smithi181 ceph-mon[103627]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 64 KiB/s, 0 objects/s recovering 2023-12-11T17:42:47.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:47 smithi059 ceph-mon[98847]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 64 KiB/s, 0 objects/s recovering 2023-12-11T17:42:47.532 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:47 smithi115 ceph-mon[102748]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 64 KiB/s, 0 objects/s recovering 2023-12-11T17:42:49.128 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:49 smithi059 ceph-mon[98847]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-12-11T17:42:49.128 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:49 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:42:49.128 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:49 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:42:49.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:49 smithi181 ceph-mon[103627]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-12-11T17:42:49.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:49 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:42:49.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:49 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:42:49.396 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:49 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:42:49.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:49 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:42:49.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:49 smithi115 ceph-mon[102748]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-12-11T17:42:49.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:49 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:42:49.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:49 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:42:49.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:49 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:42:51.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:51 smithi181 ceph-mon[103627]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-12-11T17:42:51.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:51 smithi059 ceph-mon[98847]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-12-11T17:42:51.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:51 smithi115 ceph-mon[102748]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-12-11T17:42:51.949 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:42:51.946+0000 7fbe30998700 1 -- 172.21.15.59:0/2940778815 <== mgr.14150 v2:172.21.15.59:6800/3627728353 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fbe2c061a10 con 0x7fbe140a1500 2023-12-11T17:42:51.949 INFO:teuthology.orchestra.run.smithi059.stdout:Created osd(s) 9 on host 'smithi059' 2023-12-11T17:42:51.952 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:42:51.949+0000 7fbe3199a700 1 -- 172.21.15.59:0/2940778815 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fbe140a1500 msgr2=0x7fbe140a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:42:51.952 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:42:51.949+0000 7fbe3199a700 1 --2- 172.21.15.59:0/2940778815 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fbe140a1500 0x7fbe140a39c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fbe1c00b2e0 tx=0x7fbe1c005fd0 comp rx=0 tx=0).stop 2023-12-11T17:42:51.952 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:42:51.949+0000 7fbe3199a700 1 -- 172.21.15.59:0/2940778815 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbe2c104a70 msgr2=0x7fbe2c1000f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:42:51.952 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:42:51.949+0000 7fbe3199a700 1 --2- 172.21.15.59:0/2940778815 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbe2c104a70 0x7fbe2c1000f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fbe2000eb30 tx=0x7fbe2000ee40 comp rx=0 tx=0).stop 2023-12-11T17:42:51.952 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:42:51.949+0000 7fbe3199a700 1 -- 172.21.15.59:0/2940778815 shutdown_connections 2023-12-11T17:42:51.952 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:42:51.949+0000 7fbe3199a700 1 --2- 172.21.15.59:0/2940778815 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fbe140a1500 0x7fbe140a39c0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:42:51.952 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:42:51.949+0000 7fbe3199a700 1 --2- 172.21.15.59:0/2940778815 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbe2c1073a0 0x7fbe2c100630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:42:51.952 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:42:51.949+0000 7fbe3199a700 1 --2- 172.21.15.59:0/2940778815 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe2c102140 0x7fbe2c101a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:42:51.952 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:42:51.949+0000 7fbe3199a700 1 --2- 172.21.15.59:0/2940778815 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbe2c104a70 0x7fbe2c1000f0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:42:51.952 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:42:51.949+0000 7fbe3199a700 1 -- 172.21.15.59:0/2940778815 >> 172.21.15.59:0/2940778815 conn(0x7fbe2c0fbad0 msgr2=0x7fbe2c102c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:42:51.952 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:42:51.949+0000 7fbe3199a700 1 -- 172.21.15.59:0/2940778815 shutdown_connections 2023-12-11T17:42:51.952 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:42:51.949+0000 7fbe3199a700 1 -- 172.21.15.59:0/2940778815 wait complete. 2023-12-11T17:42:52.436 DEBUG:teuthology.orchestra.run.smithi059:osd.9> sudo journalctl -f -n 0 -u ceph-d0a07eaa-984b-11ee-95a3-87774f69a715@osd.9.service 2023-12-11T17:42:52.438 INFO:tasks.cephadm:Deploying osd.10 on smithi115 with /dev/vg_nvme/lv_1... 2023-12-11T17:42:52.439 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d0a07eaa-984b-11ee-95a3-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-12-11T17:42:52.468 INFO:journalctl@ceph.osd.9.smithi059.stdout:-- Logs begin at Mon 2023-12-11 17:22:26 UTC. -- 2023-12-11T17:42:52.782 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:52 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:42:52.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:52 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:42:52.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:52 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:42:52.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:52 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:42:52.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:52 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:42:52.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:52 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:42:52.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:52 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:42:52.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:52 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:42:52.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:52 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:42:52.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:52 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:42:52.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:52 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:42:52.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:52 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:42:52.906 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:52 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:42:52.906 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:52 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:42:52.906 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:52 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:42:52.906 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:52 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:42:53.042 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/mon.b/config 2023-12-11T17:42:53.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:52 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:42:53.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:52 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:42:53.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:52 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:42:53.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:52 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:42:53.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:52 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:42:53.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:52 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:42:53.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:52 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:42:53.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:52 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:42:53.173 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 17:42:53 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T17:42:53.101+0000 7f8631465640 -1 osd.9 0 log_to_monitors true 2023-12-11T17:42:54.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:53 smithi115 ceph-mon[102748]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-12-11T17:42:54.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:53 smithi115 ceph-mon[102748]: from='osd.9 [v2:172.21.15.59:6826/1163095677,v1:172.21.15.59:6827/1163095677]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-12-11T17:42:54.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:53 smithi181 ceph-mon[103627]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-12-11T17:42:54.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:53 smithi181 ceph-mon[103627]: from='osd.9 [v2:172.21.15.59:6826/1163095677,v1:172.21.15.59:6827/1163095677]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-12-11T17:42:54.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:53 smithi059 ceph-mon[98847]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-12-11T17:42:54.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:53 smithi059 ceph-mon[98847]: from='osd.9 [v2:172.21.15.59:6826/1163095677,v1:172.21.15.59:6827/1163095677]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-12-11T17:42:54.583 INFO:teuthology.orchestra.run.smithi115.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-12-11T17:42:54.584 INFO:teuthology.orchestra.run.smithi115.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-12-11T17:42:54.584 INFO:teuthology.orchestra.run.smithi115.stderr: stderr: 10+0 records in 2023-12-11T17:42:54.584 INFO:teuthology.orchestra.run.smithi115.stderr:10+0 records out 2023-12-11T17:42:54.584 INFO:teuthology.orchestra.run.smithi115.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0128349 s, 817 MB/s 2023-12-11T17:42:54.584 INFO:teuthology.orchestra.run.smithi115.stderr:--> Zapping successful for: 2023-12-11T17:42:55.002 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d0a07eaa-984b-11ee-95a3-87774f69a715 -- ceph orch daemon add osd smithi115:vg_nvme/lv_1 2023-12-11T17:42:55.046 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 17:42:54 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T17:42:54.910+0000 7f8626adc700 -1 osd.9 0 waiting for initial osdmap 2023-12-11T17:42:55.047 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 17:42:54 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T17:42:54.923+0000 7f86208d2700 -1 osd.9 64 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-11T17:42:55.047 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:54 smithi059 ceph-mon[98847]: from='osd.9 [v2:172.21.15.59:6826/1163095677,v1:172.21.15.59:6827/1163095677]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-12-11T17:42:55.047 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:54 smithi059 ceph-mon[98847]: osdmap e63: 10 total, 9 up, 10 in 2023-12-11T17:42:55.047 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:54 smithi059 ceph-mon[98847]: from='osd.9 [v2:172.21.15.59:6826/1163095677,v1:172.21.15.59:6827/1163095677]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2023-12-11T17:42:55.047 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:54 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-11T17:42:55.047 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:54 smithi059 ceph-mon[98847]: from='osd.9 [v2:172.21.15.59:6826/1163095677,v1:172.21.15.59:6827/1163095677]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]': finished 2023-12-11T17:42:55.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:54 smithi115 ceph-mon[102748]: from='osd.9 [v2:172.21.15.59:6826/1163095677,v1:172.21.15.59:6827/1163095677]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-12-11T17:42:55.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:54 smithi115 ceph-mon[102748]: osdmap e63: 10 total, 9 up, 10 in 2023-12-11T17:42:55.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:54 smithi115 ceph-mon[102748]: from='osd.9 [v2:172.21.15.59:6826/1163095677,v1:172.21.15.59:6827/1163095677]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2023-12-11T17:42:55.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:54 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-11T17:42:55.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:54 smithi115 ceph-mon[102748]: from='osd.9 [v2:172.21.15.59:6826/1163095677,v1:172.21.15.59:6827/1163095677]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]': finished 2023-12-11T17:42:55.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:54 smithi181 ceph-mon[103627]: from='osd.9 [v2:172.21.15.59:6826/1163095677,v1:172.21.15.59:6827/1163095677]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-12-11T17:42:55.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:54 smithi181 ceph-mon[103627]: osdmap e63: 10 total, 9 up, 10 in 2023-12-11T17:42:55.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:54 smithi181 ceph-mon[103627]: from='osd.9 [v2:172.21.15.59:6826/1163095677,v1:172.21.15.59:6827/1163095677]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2023-12-11T17:42:55.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:54 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-11T17:42:55.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:54 smithi181 ceph-mon[103627]: from='osd.9 [v2:172.21.15.59:6826/1163095677,v1:172.21.15.59:6827/1163095677]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]': finished 2023-12-11T17:42:55.597 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/mon.b/config 2023-12-11T17:42:56.190 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:55 smithi115 ceph-mon[102748]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-12-11T17:42:56.191 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:55 smithi115 ceph-mon[102748]: osdmap e64: 10 total, 9 up, 10 in 2023-12-11T17:42:56.191 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:55 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-11T17:42:56.191 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:55 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-11T17:42:56.191 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:55 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:42:56.191 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:55 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:42:56.191 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:55 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:42:56.191 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:55 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:42:56.191 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:55 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:42:56.191 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:55 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:42:56.192 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:55 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:42:56.192 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:55 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:42:56.192 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:55 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:42:56.192 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:55 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:42:56.192 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:55 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-11T17:42:56.396 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:55 smithi181 ceph-mon[103627]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-12-11T17:42:56.396 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:55 smithi181 ceph-mon[103627]: osdmap e64: 10 total, 9 up, 10 in 2023-12-11T17:42:56.396 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:55 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-11T17:42:56.397 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:55 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-11T17:42:56.397 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:55 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:42:56.397 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:55 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:42:56.397 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:55 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:42:56.397 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:55 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:42:56.397 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:55 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:42:56.397 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:55 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:42:56.397 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:55 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:42:56.397 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:55 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:42:56.397 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:55 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:42:56.397 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:55 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:42:56.397 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:55 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-11T17:42:56.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:55 smithi059 ceph-mon[98847]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-12-11T17:42:56.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:55 smithi059 ceph-mon[98847]: osdmap e64: 10 total, 9 up, 10 in 2023-12-11T17:42:56.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:55 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-11T17:42:56.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:55 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-11T17:42:56.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:55 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:42:56.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:55 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:42:56.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:55 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:42:56.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:55 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:42:56.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:55 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:42:56.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:55 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:42:56.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:55 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:42:56.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:55 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:42:56.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:55 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:42:56.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:55 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:42:56.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:55 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-11T17:42:56.899 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:42:56.896+0000 7f010533c700 1 -- 172.21.15.115:0/3714446667 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0100100760 msgr2=0x7f0100100be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:42:56.899 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:42:56.896+0000 7f010533c700 1 --2- 172.21.15.115:0/3714446667 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0100100760 0x7f0100100be0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f00f0009990 tx=0x7f00f0009ca0 comp rx=0 tx=0).stop 2023-12-11T17:42:56.899 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:42:56.896+0000 7f010533c700 1 -- 172.21.15.115:0/3714446667 shutdown_connections 2023-12-11T17:42:56.899 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:42:56.896+0000 7f010533c700 1 --2- 172.21.15.115:0/3714446667 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0100101120 0x7f01001089c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:42:56.899 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:42:56.896+0000 7f010533c700 1 --2- 172.21.15.115:0/3714446667 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0100101c00 0x7f0100102020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:42:56.899 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:42:56.896+0000 7f010533c700 1 --2- 172.21.15.115:0/3714446667 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0100100760 0x7f0100100be0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:42:56.899 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:42:56.896+0000 7f010533c700 1 -- 172.21.15.115:0/3714446667 >> 172.21.15.115:0/3714446667 conn(0x7f01000fb270 msgr2=0x7f01000fd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:42:56.900 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:42:56.896+0000 7f010533c700 1 -- 172.21.15.115:0/3714446667 shutdown_connections 2023-12-11T17:42:56.900 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:42:56.896+0000 7f010533c700 1 -- 172.21.15.115:0/3714446667 wait complete. 2023-12-11T17:42:56.902 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:42:56.896+0000 7f010533c700 1 Processor -- start 2023-12-11T17:42:56.902 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:42:56.897+0000 7f010533c700 1 -- start start 2023-12-11T17:42:56.902 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:42:56.897+0000 7f010533c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0100100760 0x7f0100100180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:42:56.902 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:42:56.897+0000 7f010533c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0100101120 0x7f010019c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:42:56.902 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:42:56.897+0000 7f010533c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0100101c00 0x7f010019e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:42:56.902 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:42:56.897+0000 7f010533c700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f010019f0f0 con 0x7f0100101120 2023-12-11T17:42:56.902 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:42:56.897+0000 7f010533c700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f010019f260 con 0x7f0100100760 2023-12-11T17:42:56.902 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:42:56.897+0000 7f010533c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f010019f420 con 0x7f0100101c00 2023-12-11T17:42:56.902 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:42:56.897+0000 7f00feffd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0100100760 0x7f0100100180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:42:56.903 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:42:56.897+0000 7f00feffd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0100100760 0x7f0100100180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:54974/0 (socket says 172.21.15.115:54974) 2023-12-11T17:42:56.903 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:42:56.897+0000 7f00feffd700 1 -- 172.21.15.115:0/83213005 learned_addr learned my addr 172.21.15.115:0/83213005 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-11T17:42:56.903 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:42:56.897+0000 7f00fe7fc700 1 --2- 172.21.15.115:0/83213005 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0100101120 0x7f010019c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:42:56.903 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:42:56.897+0000 7f00ff7fe700 1 --2- 172.21.15.115:0/83213005 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0100101c00 0x7f010019e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:42:56.903 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:42:56.897+0000 7f00feffd700 1 -- 172.21.15.115:0/83213005 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0100101c00 msgr2=0x7f010019e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:42:56.903 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:42:56.897+0000 7f00feffd700 1 --2- 172.21.15.115:0/83213005 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0100101c00 0x7f010019e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:42:56.903 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:42:56.897+0000 7f00feffd700 1 -- 172.21.15.115:0/83213005 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0100101120 msgr2=0x7f010019c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:42:56.903 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:42:56.897+0000 7f00feffd700 1 --2- 172.21.15.115:0/83213005 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0100101120 0x7f010019c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:42:56.903 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:42:56.897+0000 7f00feffd700 1 -- 172.21.15.115:0/83213005 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00f00095f0 con 0x7f0100100760 2023-12-11T17:42:56.903 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:42:56.897+0000 7f00ff7fe700 1 --2- 172.21.15.115:0/83213005 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0100101c00 0x7f010019e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-11T17:42:56.903 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:42:56.897+0000 7f00feffd700 1 --2- 172.21.15.115:0/83213005 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0100100760 0x7f0100100180 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f00f400d980 tx=0x7f00f400dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:42:56.903 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:42:56.897+0000 7f00e7fff700 1 -- 172.21.15.115:0/83213005 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f4004220 con 0x7f0100100760 2023-12-11T17:42:56.903 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:42:56.897+0000 7f00e7fff700 1 -- 172.21.15.115:0/83213005 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f00f4007640 con 0x7f0100100760 2023-12-11T17:42:56.903 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:42:56.897+0000 7f010533c700 1 -- 172.21.15.115:0/83213005 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0100196250 con 0x7f0100100760 2023-12-11T17:42:56.904 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:42:56.897+0000 7f00e7fff700 1 -- 172.21.15.115:0/83213005 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f4005610 con 0x7f0100100760 2023-12-11T17:42:56.904 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:42:56.897+0000 7f010533c700 1 -- 172.21.15.115:0/83213005 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0100196750 con 0x7f0100100760 2023-12-11T17:42:56.904 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:42:56.898+0000 7f010533c700 1 -- 172.21.15.115:0/83213005 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01001079f0 con 0x7f0100100760 2023-12-11T17:42:56.904 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:42:56.900+0000 7f00e7fff700 1 -- 172.21.15.115:0/83213005 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f00f4004750 con 0x7f0100100760 2023-12-11T17:42:56.905 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:42:56.900+0000 7f00e7fff700 1 --2- 172.21.15.115:0/83213005 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f00e80a1480 0x7f00e80a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:42:56.905 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:42:56.900+0000 7f00e7fff700 1 -- 172.21.15.115:0/83213005 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f00f40cc370 con 0x7f0100100760 2023-12-11T17:42:56.905 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:42:56.900+0000 7f00fe7fc700 1 --2- 172.21.15.115:0/83213005 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f00e80a1480 0x7f00e80a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:42:56.905 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:42:56.901+0000 7f00fe7fc700 1 --2- 172.21.15.115:0/83213005 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f00e80a1480 0x7f00e80a3940 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f00f000b2e0 tx=0x7f00f0005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:42:56.908 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:42:56.904+0000 7f00e7fff700 1 -- 172.21.15.115:0/83213005 <== mon.2 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f00f40961d0 con 0x7f0100100760 2023-12-11T17:42:57.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:56 smithi115 ceph-mon[102748]: purged_snaps scrub starts 2023-12-11T17:42:57.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:56 smithi115 ceph-mon[102748]: purged_snaps scrub ok 2023-12-11T17:42:57.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:56 smithi115 ceph-mon[102748]: Detected new or changed devices on smithi059 2023-12-11T17:42:57.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:56 smithi115 ceph-mon[102748]: Adjusting osd_memory_target on smithi059 to 4293M 2023-12-11T17:42:57.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:56 smithi115 ceph-mon[102748]: osd.9 [v2:172.21.15.59:6826/1163095677,v1:172.21.15.59:6827/1163095677] boot 2023-12-11T17:42:57.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:56 smithi115 ceph-mon[102748]: osdmap e65: 10 total, 10 up, 10 in 2023-12-11T17:42:57.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:56 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-11T17:42:57.109 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:42:57.106+0000 7f010533c700 1 -- 172.21.15.115:0/83213005 --> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f0100061a10 con 0x7f00e80a1480 2023-12-11T17:42:57.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:56 smithi181 ceph-mon[103627]: purged_snaps scrub starts 2023-12-11T17:42:57.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:56 smithi181 ceph-mon[103627]: purged_snaps scrub ok 2023-12-11T17:42:57.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:56 smithi181 ceph-mon[103627]: Detected new or changed devices on smithi059 2023-12-11T17:42:57.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:56 smithi181 ceph-mon[103627]: Adjusting osd_memory_target on smithi059 to 4293M 2023-12-11T17:42:57.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:56 smithi181 ceph-mon[103627]: osd.9 [v2:172.21.15.59:6826/1163095677,v1:172.21.15.59:6827/1163095677] boot 2023-12-11T17:42:57.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:56 smithi181 ceph-mon[103627]: osdmap e65: 10 total, 10 up, 10 in 2023-12-11T17:42:57.396 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:56 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-11T17:42:57.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:56 smithi059 ceph-mon[98847]: purged_snaps scrub starts 2023-12-11T17:42:57.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:56 smithi059 ceph-mon[98847]: purged_snaps scrub ok 2023-12-11T17:42:57.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:56 smithi059 ceph-mon[98847]: Detected new or changed devices on smithi059 2023-12-11T17:42:57.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:56 smithi059 ceph-mon[98847]: Adjusting osd_memory_target on smithi059 to 4293M 2023-12-11T17:42:57.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:56 smithi059 ceph-mon[98847]: osd.9 [v2:172.21.15.59:6826/1163095677,v1:172.21.15.59:6827/1163095677] boot 2023-12-11T17:42:57.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:56 smithi059 ceph-mon[98847]: osdmap e65: 10 total, 10 up, 10 in 2023-12-11T17:42:57.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:56 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-11T17:42:58.254 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:57 smithi115 ceph-mon[102748]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-12-11T17:42:58.254 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:57 smithi115 ceph-mon[102748]: osdmap e66: 10 total, 10 up, 10 in 2023-12-11T17:42:58.255 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:57 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-11T17:42:58.255 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:57 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-11T17:42:58.255 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:57 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:42:58.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:57 smithi181 ceph-mon[103627]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-12-11T17:42:58.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:57 smithi181 ceph-mon[103627]: osdmap e66: 10 total, 10 up, 10 in 2023-12-11T17:42:58.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:57 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-11T17:42:58.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:57 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-11T17:42:58.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:57 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:42:58.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:57 smithi059 ceph-mon[98847]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-12-11T17:42:58.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:57 smithi059 ceph-mon[98847]: osdmap e66: 10 total, 10 up, 10 in 2023-12-11T17:42:58.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:57 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-11T17:42:58.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:57 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-11T17:42:58.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:57 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:42:59.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:58 smithi115 ceph-mon[102748]: from='client.24338 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T17:42:59.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:58 smithi181 ceph-mon[103627]: from='client.24338 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T17:42:59.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:58 smithi059 ceph-mon[98847]: from='client.24338 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T17:43:00.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:59 smithi115 ceph-mon[102748]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-12-11T17:43:00.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:59 smithi115 ceph-mon[102748]: from='client.? 172.21.15.115:0/3651003868' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b75c179a-875d-43bc-b4c1-e8eb80fb64fa"}]: dispatch 2023-12-11T17:43:00.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:59 smithi115 ceph-mon[102748]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b75c179a-875d-43bc-b4c1-e8eb80fb64fa"}]: dispatch 2023-12-11T17:43:00.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:59 smithi115 ceph-mon[102748]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b75c179a-875d-43bc-b4c1-e8eb80fb64fa"}]': finished 2023-12-11T17:43:00.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:59 smithi115 ceph-mon[102748]: osdmap e67: 11 total, 10 up, 11 in 2023-12-11T17:43:00.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:42:59 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-11T17:43:00.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:59 smithi181 ceph-mon[103627]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-12-11T17:43:00.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:59 smithi181 ceph-mon[103627]: from='client.? 172.21.15.115:0/3651003868' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b75c179a-875d-43bc-b4c1-e8eb80fb64fa"}]: dispatch 2023-12-11T17:43:00.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:59 smithi181 ceph-mon[103627]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b75c179a-875d-43bc-b4c1-e8eb80fb64fa"}]: dispatch 2023-12-11T17:43:00.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:59 smithi181 ceph-mon[103627]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b75c179a-875d-43bc-b4c1-e8eb80fb64fa"}]': finished 2023-12-11T17:43:00.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:59 smithi181 ceph-mon[103627]: osdmap e67: 11 total, 10 up, 11 in 2023-12-11T17:43:00.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:42:59 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-11T17:43:00.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:59 smithi059 ceph-mon[98847]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-12-11T17:43:00.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:59 smithi059 ceph-mon[98847]: from='client.? 172.21.15.115:0/3651003868' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b75c179a-875d-43bc-b4c1-e8eb80fb64fa"}]: dispatch 2023-12-11T17:43:00.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:59 smithi059 ceph-mon[98847]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b75c179a-875d-43bc-b4c1-e8eb80fb64fa"}]: dispatch 2023-12-11T17:43:00.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:59 smithi059 ceph-mon[98847]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b75c179a-875d-43bc-b4c1-e8eb80fb64fa"}]': finished 2023-12-11T17:43:00.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:59 smithi059 ceph-mon[98847]: osdmap e67: 11 total, 10 up, 11 in 2023-12-11T17:43:00.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:42:59 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-11T17:43:01.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:00 smithi115 ceph-mon[102748]: from='client.? 172.21.15.115:0/2945523261' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-11T17:43:01.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:00 smithi181 ceph-mon[103627]: from='client.? 172.21.15.115:0/2945523261' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-11T17:43:01.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:00 smithi059 ceph-mon[98847]: from='client.? 172.21.15.115:0/2945523261' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-11T17:43:02.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:01 smithi115 ceph-mon[102748]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-12-11T17:43:02.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:01 smithi181 ceph-mon[103627]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-12-11T17:43:02.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:01 smithi059 ceph-mon[98847]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-12-11T17:43:04.032 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:03 smithi115 ceph-mon[102748]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-12-11T17:43:04.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:03 smithi181 ceph-mon[103627]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-12-11T17:43:04.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:03 smithi059 ceph-mon[98847]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-12-11T17:43:06.168 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:05 smithi115 ceph-mon[102748]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-12-11T17:43:06.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:05 smithi181 ceph-mon[103627]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-12-11T17:43:06.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:05 smithi059 ceph-mon[98847]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-12-11T17:43:07.184 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:06 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-12-11T17:43:07.184 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:06 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:43:07.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:06 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-12-11T17:43:07.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:06 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:43:07.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:06 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-12-11T17:43:07.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:06 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:43:08.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:07 smithi115 ceph-mon[102748]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-12-11T17:43:08.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:07 smithi115 ceph-mon[102748]: Deploying daemon osd.10 on smithi115 2023-12-11T17:43:08.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:07 smithi181 ceph-mon[103627]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-12-11T17:43:08.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:07 smithi181 ceph-mon[103627]: Deploying daemon osd.10 on smithi115 2023-12-11T17:43:08.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:07 smithi059 ceph-mon[98847]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-12-11T17:43:08.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:07 smithi059 ceph-mon[98847]: Deploying daemon osd.10 on smithi115 2023-12-11T17:43:10.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:09 smithi115 ceph-mon[102748]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-12-11T17:43:10.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:09 smithi181 ceph-mon[103627]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-12-11T17:43:10.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:09 smithi059 ceph-mon[98847]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-12-11T17:43:11.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:11 smithi181 ceph-mon[103627]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-12-11T17:43:11.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:11 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:43:11.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:11 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:43:11.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:11 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:43:11.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:11 smithi059 ceph-mon[98847]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-12-11T17:43:11.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:11 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:43:11.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:11 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:43:11.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:11 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:43:12.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:11 smithi115 ceph-mon[102748]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-12-11T17:43:12.034 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:11 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:43:12.034 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:11 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:43:12.034 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:11 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:43:13.671 INFO:teuthology.orchestra.run.smithi115.stdout:Created osd(s) 10 on host 'smithi115' 2023-12-11T17:43:13.672 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:43:13.668+0000 7f00e7fff700 1 -- 172.21.15.115:0/83213005 <== mgr.14150 v2:172.21.15.59:6800/3627728353 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f0100061a10 con 0x7f00e80a1480 2023-12-11T17:43:13.674 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:43:13.671+0000 7f010533c700 1 -- 172.21.15.115:0/83213005 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f00e80a1480 msgr2=0x7f00e80a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:43:13.674 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:43:13.671+0000 7f010533c700 1 --2- 172.21.15.115:0/83213005 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f00e80a1480 0x7f00e80a3940 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f00f000b2e0 tx=0x7f00f0005fd0 comp rx=0 tx=0).stop 2023-12-11T17:43:13.674 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:43:13.671+0000 7f010533c700 1 -- 172.21.15.115:0/83213005 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0100100760 msgr2=0x7f0100100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:43:13.674 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:43:13.671+0000 7f010533c700 1 --2- 172.21.15.115:0/83213005 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0100100760 0x7f0100100180 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f00f400d980 tx=0x7f00f400dc90 comp rx=0 tx=0).stop 2023-12-11T17:43:13.674 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:43:13.671+0000 7f010533c700 1 -- 172.21.15.115:0/83213005 shutdown_connections 2023-12-11T17:43:13.674 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:43:13.671+0000 7f010533c700 1 --2- 172.21.15.115:0/83213005 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f00e80a1480 0x7f00e80a3940 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:43:13.674 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:43:13.671+0000 7f010533c700 1 --2- 172.21.15.115:0/83213005 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0100101c00 0x7f010019e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:43:13.674 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:43:13.671+0000 7f010533c700 1 --2- 172.21.15.115:0/83213005 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0100100760 0x7f0100100180 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:43:13.674 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:43:13.671+0000 7f010533c700 1 --2- 172.21.15.115:0/83213005 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0100101120 0x7f010019c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:43:13.675 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:43:13.671+0000 7f010533c700 1 -- 172.21.15.115:0/83213005 >> 172.21.15.115:0/83213005 conn(0x7f01000fb270 msgr2=0x7f0100104b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:43:13.675 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:43:13.671+0000 7f010533c700 1 -- 172.21.15.115:0/83213005 shutdown_connections 2023-12-11T17:43:13.675 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-11T17:43:13.671+0000 7f010533c700 1 -- 172.21.15.115:0/83213005 wait complete. 2023-12-11T17:43:13.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:13 smithi181 ceph-mon[103627]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-12-11T17:43:13.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:13 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:43:13.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:13 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:43:13.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:13 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:43:13.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:13 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:43:13.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:13 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:43:13.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:13 smithi059 ceph-mon[98847]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-12-11T17:43:13.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:13 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:43:13.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:13 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:43:13.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:13 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:43:13.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:13 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:43:13.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:13 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:43:14.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:13 smithi115 ceph-mon[102748]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-12-11T17:43:14.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:13 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:43:14.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:13 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:43:14.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:13 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:43:14.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:13 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:43:14.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:13 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:43:14.100 DEBUG:teuthology.orchestra.run.smithi115:osd.10> sudo journalctl -f -n 0 -u ceph-d0a07eaa-984b-11ee-95a3-87774f69a715@osd.10.service 2023-12-11T17:43:14.102 INFO:tasks.cephadm:Deploying osd.11 on smithi181 with /dev/vg_nvme/lv_1... 2023-12-11T17:43:14.102 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d0a07eaa-984b-11ee-95a3-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-12-11T17:43:14.134 INFO:journalctl@ceph.osd.10.smithi115.stdout:-- Logs begin at Mon 2023-12-11 17:22:43 UTC. -- 2023-12-11T17:43:14.708 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/mon.c/config 2023-12-11T17:43:14.858 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:14 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:43:14.858 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:14 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:43:14.858 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:14 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:43:14.858 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:14 smithi115 ceph-mon[102748]: from='osd.10 [v2:172.21.15.115:6824/2199610269,v1:172.21.15.115:6825/2199610269]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-12-11T17:43:14.859 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:14 smithi115 ceph-mon[102748]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-12-11T17:43:14.859 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 17:43:14 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T17:43:14.595+0000 7f1eca60b640 -1 osd.10 0 log_to_monitors true 2023-12-11T17:43:14.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:14 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:43:14.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:14 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:43:14.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:14 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:43:14.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:14 smithi059 ceph-mon[98847]: from='osd.10 [v2:172.21.15.115:6824/2199610269,v1:172.21.15.115:6825/2199610269]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-12-11T17:43:14.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:14 smithi059 ceph-mon[98847]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-12-11T17:43:14.941 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:14 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:43:14.941 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:14 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:43:14.941 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:14 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:43:14.941 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:14 smithi181 ceph-mon[103627]: from='osd.10 [v2:172.21.15.115:6824/2199610269,v1:172.21.15.115:6825/2199610269]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-12-11T17:43:14.941 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:14 smithi181 ceph-mon[103627]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-12-11T17:43:15.807 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:15 smithi181 ceph-mon[103627]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-12-11T17:43:15.807 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:15 smithi181 ceph-mon[103627]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-12-11T17:43:15.807 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:15 smithi181 ceph-mon[103627]: from='osd.10 [v2:172.21.15.115:6824/2199610269,v1:172.21.15.115:6825/2199610269]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2023-12-11T17:43:15.807 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:15 smithi181 ceph-mon[103627]: osdmap e68: 11 total, 10 up, 11 in 2023-12-11T17:43:15.807 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:15 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-11T17:43:15.807 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:15 smithi181 ceph-mon[103627]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2023-12-11T17:43:16.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:15 smithi115 ceph-mon[102748]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-12-11T17:43:16.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:15 smithi115 ceph-mon[102748]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-12-11T17:43:16.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:15 smithi115 ceph-mon[102748]: from='osd.10 [v2:172.21.15.115:6824/2199610269,v1:172.21.15.115:6825/2199610269]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2023-12-11T17:43:16.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:15 smithi115 ceph-mon[102748]: osdmap e68: 11 total, 10 up, 11 in 2023-12-11T17:43:16.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:15 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-11T17:43:16.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:15 smithi115 ceph-mon[102748]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2023-12-11T17:43:16.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:15 smithi059 ceph-mon[98847]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-12-11T17:43:16.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:15 smithi059 ceph-mon[98847]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-12-11T17:43:16.155 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:15 smithi059 ceph-mon[98847]: from='osd.10 [v2:172.21.15.115:6824/2199610269,v1:172.21.15.115:6825/2199610269]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2023-12-11T17:43:16.155 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:15 smithi059 ceph-mon[98847]: osdmap e68: 11 total, 10 up, 11 in 2023-12-11T17:43:16.155 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:15 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-11T17:43:16.155 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:15 smithi059 ceph-mon[98847]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2023-12-11T17:43:16.285 INFO:teuthology.orchestra.run.smithi181.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-12-11T17:43:16.285 INFO:teuthology.orchestra.run.smithi181.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-12-11T17:43:16.285 INFO:teuthology.orchestra.run.smithi181.stderr: stderr: 10+0 records in 2023-12-11T17:43:16.285 INFO:teuthology.orchestra.run.smithi181.stderr:10+0 records out 2023-12-11T17:43:16.285 INFO:teuthology.orchestra.run.smithi181.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0151619 s, 692 MB/s 2023-12-11T17:43:16.285 INFO:teuthology.orchestra.run.smithi181.stderr:--> Zapping successful for: 2023-12-11T17:43:16.432 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 17:43:16 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T17:43:16.113+0000 7f1ebfc82700 -1 osd.10 0 waiting for initial osdmap 2023-12-11T17:43:16.433 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 17:43:16 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T17:43:16.126+0000 7f1eb9a78700 -1 osd.10 69 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-11T17:43:16.658 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d0a07eaa-984b-11ee-95a3-87774f69a715 -- ceph orch daemon add osd smithi181:vg_nvme/lv_1 2023-12-11T17:43:17.256 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/mon.c/config 2023-12-11T17:43:17.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:17 smithi181 ceph-mon[103627]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-12-11T17:43:17.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:17 smithi181 ceph-mon[103627]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2023-12-11T17:43:17.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:17 smithi181 ceph-mon[103627]: osdmap e69: 11 total, 10 up, 11 in 2023-12-11T17:43:17.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:17 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-11T17:43:17.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:17 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-11T17:43:17.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:17 smithi181 ceph-mon[103627]: Detected new or changed devices on smithi115 2023-12-11T17:43:17.396 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:17 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:43:17.396 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:17 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:43:17.396 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:17 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:43:17.396 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:17 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:43:17.396 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:17 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:43:17.396 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:17 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:43:17.396 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:17 smithi181 ceph-mon[103627]: Adjusting osd_memory_target on smithi115 to 4292M 2023-12-11T17:43:17.396 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:17 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:43:17.396 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:17 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:43:17.396 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:17 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:43:17.396 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:17 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:43:17.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:17 smithi059 ceph-mon[98847]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-12-11T17:43:17.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:17 smithi059 ceph-mon[98847]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2023-12-11T17:43:17.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:17 smithi059 ceph-mon[98847]: osdmap e69: 11 total, 10 up, 11 in 2023-12-11T17:43:17.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:17 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-11T17:43:17.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:17 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-11T17:43:17.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:17 smithi059 ceph-mon[98847]: Detected new or changed devices on smithi115 2023-12-11T17:43:17.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:17 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:43:17.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:17 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:43:17.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:17 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:43:17.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:17 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:43:17.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:17 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:43:17.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:17 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:43:17.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:17 smithi059 ceph-mon[98847]: Adjusting osd_memory_target on smithi115 to 4292M 2023-12-11T17:43:17.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:17 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:43:17.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:17 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:43:17.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:17 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:43:17.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:17 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:43:17.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:17 smithi115 ceph-mon[102748]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-12-11T17:43:17.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:17 smithi115 ceph-mon[102748]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2023-12-11T17:43:17.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:17 smithi115 ceph-mon[102748]: osdmap e69: 11 total, 10 up, 11 in 2023-12-11T17:43:17.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:17 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-11T17:43:17.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:17 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-11T17:43:17.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:17 smithi115 ceph-mon[102748]: Detected new or changed devices on smithi115 2023-12-11T17:43:17.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:17 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:43:17.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:17 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:43:17.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:17 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:43:17.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:17 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:43:17.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:17 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:43:17.534 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:17 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:43:17.534 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:17 smithi115 ceph-mon[102748]: Adjusting osd_memory_target on smithi115 to 4292M 2023-12-11T17:43:17.534 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:17 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:43:17.534 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:17 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:43:17.534 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:17 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:43:17.534 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:17 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:43:18.270 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:18 smithi181 ceph-mon[103627]: purged_snaps scrub starts 2023-12-11T17:43:18.270 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:18 smithi181 ceph-mon[103627]: purged_snaps scrub ok 2023-12-11T17:43:18.270 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:18 smithi181 ceph-mon[103627]: osd.10 [v2:172.21.15.115:6824/2199610269,v1:172.21.15.115:6825/2199610269] boot 2023-12-11T17:43:18.270 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:18 smithi181 ceph-mon[103627]: osdmap e70: 11 total, 11 up, 11 in 2023-12-11T17:43:18.270 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:18 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-11T17:43:18.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:18 smithi059 ceph-mon[98847]: purged_snaps scrub starts 2023-12-11T17:43:18.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:18 smithi059 ceph-mon[98847]: purged_snaps scrub ok 2023-12-11T17:43:18.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:18 smithi059 ceph-mon[98847]: osd.10 [v2:172.21.15.115:6824/2199610269,v1:172.21.15.115:6825/2199610269] boot 2023-12-11T17:43:18.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:18 smithi059 ceph-mon[98847]: osdmap e70: 11 total, 11 up, 11 in 2023-12-11T17:43:18.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:18 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-11T17:43:18.407 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:43:18.402+0000 7f9c62f61700 1 -- 172.21.15.181:0/3843550202 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c5c1020b0 msgr2=0x7f9c5c108960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:43:18.407 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:43:18.402+0000 7f9c62f61700 1 --2- 172.21.15.181:0/3843550202 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c5c1020b0 0x7f9c5c108960 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f9c5400b0a0 tx=0x7f9c5400b3b0 comp rx=0 tx=0).stop 2023-12-11T17:43:18.407 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:43:18.402+0000 7f9c62f61700 1 -- 172.21.15.181:0/3843550202 shutdown_connections 2023-12-11T17:43:18.407 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:43:18.402+0000 7f9c62f61700 1 --2- 172.21.15.181:0/3843550202 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c5c1020b0 0x7f9c5c108960 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:43:18.407 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:43:18.402+0000 7f9c62f61700 1 --2- 172.21.15.181:0/3843550202 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9c5c100510 0x7f9c5c100930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:43:18.407 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:43:18.402+0000 7f9c62f61700 1 --2- 172.21.15.181:0/3843550202 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c5c101710 0x7f9c5c101b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:43:18.407 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:43:18.402+0000 7f9c62f61700 1 -- 172.21.15.181:0/3843550202 >> 172.21.15.181:0/3843550202 conn(0x7f9c5c0fbad0 msgr2=0x7f9c5c0fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:43:18.407 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:43:18.402+0000 7f9c62f61700 1 -- 172.21.15.181:0/3843550202 shutdown_connections 2023-12-11T17:43:18.407 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:43:18.402+0000 7f9c62f61700 1 -- 172.21.15.181:0/3843550202 wait complete. 2023-12-11T17:43:18.408 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:43:18.403+0000 7f9c62f61700 1 Processor -- start 2023-12-11T17:43:18.408 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:43:18.403+0000 7f9c62f61700 1 -- start start 2023-12-11T17:43:18.408 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:43:18.403+0000 7f9c62f61700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c5c100510 0x7f9c5c1964f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:43:18.408 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:43:18.403+0000 7f9c62f61700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c5c101710 0x7f9c5c196a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:43:18.408 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:43:18.403+0000 7f9c62f61700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9c5c1020b0 0x7f9c5c19da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:43:18.408 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:43:18.403+0000 7f9c62f61700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c5c19e030 con 0x7f9c5c1020b0 2023-12-11T17:43:18.408 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:43:18.403+0000 7f9c62f61700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c5c19e170 con 0x7f9c5c100510 2023-12-11T17:43:18.408 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:43:18.403+0000 7f9c62f61700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c5c19e300 con 0x7f9c5c101710 2023-12-11T17:43:18.408 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:43:18.403+0000 7f9c5bfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c5c101710 0x7f9c5c196a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:43:18.408 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:43:18.403+0000 7f9c5bfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c5c101710 0x7f9c5c196a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34614/0 (socket says 172.21.15.181:34614) 2023-12-11T17:43:18.408 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:43:18.403+0000 7f9c5bfff700 1 -- 172.21.15.181:0/1573190169 learned_addr learned my addr 172.21.15.181:0/1573190169 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-12-11T17:43:18.408 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:43:18.403+0000 7f9c60cfd700 1 --2- 172.21.15.181:0/1573190169 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c5c100510 0x7f9c5c1964f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:43:18.408 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:43:18.403+0000 7f9c614fe700 1 --2- 172.21.15.181:0/1573190169 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9c5c1020b0 0x7f9c5c19da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:43:18.408 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:43:18.403+0000 7f9c5bfff700 1 -- 172.21.15.181:0/1573190169 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c5c100510 msgr2=0x7f9c5c1964f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:43:18.408 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:43:18.403+0000 7f9c5bfff700 1 --2- 172.21.15.181:0/1573190169 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c5c100510 0x7f9c5c1964f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:43:18.409 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:43:18.403+0000 7f9c5bfff700 1 -- 172.21.15.181:0/1573190169 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9c5c1020b0 msgr2=0x7f9c5c19da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:43:18.409 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:43:18.403+0000 7f9c5bfff700 1 --2- 172.21.15.181:0/1573190169 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9c5c1020b0 0x7f9c5c19da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:43:18.409 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:43:18.403+0000 7f9c5bfff700 1 -- 172.21.15.181:0/1573190169 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c54009d00 con 0x7f9c5c101710 2023-12-11T17:43:18.409 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:43:18.403+0000 7f9c614fe700 1 --2- 172.21.15.181:0/1573190169 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9c5c1020b0 0x7f9c5c19da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-11T17:43:18.409 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:43:18.403+0000 7f9c5bfff700 1 --2- 172.21.15.181:0/1573190169 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c5c101710 0x7f9c5c196a30 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f9c4800d980 tx=0x7f9c4800dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:43:18.409 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:43:18.404+0000 7f9c59ffb700 1 -- 172.21.15.181:0/1573190169 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c48009b40 con 0x7f9c5c101710 2023-12-11T17:43:18.409 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:43:18.404+0000 7f9c62f61700 1 -- 172.21.15.181:0/1573190169 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c5c19e5e0 con 0x7f9c5c101710 2023-12-11T17:43:18.409 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:43:18.404+0000 7f9c59ffb700 1 -- 172.21.15.181:0/1573190169 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9c4800b6e0 con 0x7f9c5c101710 2023-12-11T17:43:18.409 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:43:18.404+0000 7f9c59ffb700 1 -- 172.21.15.181:0/1573190169 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c4800f620 con 0x7f9c5c101710 2023-12-11T17:43:18.409 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:43:18.404+0000 7f9c62f61700 1 -- 172.21.15.181:0/1573190169 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c5c19eae0 con 0x7f9c5c101710 2023-12-11T17:43:18.410 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:43:18.405+0000 7f9c62f61700 1 -- 172.21.15.181:0/1573190169 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c5c107950 con 0x7f9c5c101710 2023-12-11T17:43:18.410 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:43:18.406+0000 7f9c59ffb700 1 -- 172.21.15.181:0/1573190169 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f9c48010460 con 0x7f9c5c101710 2023-12-11T17:43:18.410 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:43:18.406+0000 7f9c59ffb700 1 --2- 172.21.15.181:0/1573190169 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f9c4c0a1480 0x7f9c4c0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:43:18.410 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:43:18.406+0000 7f9c59ffb700 1 -- 172.21.15.181:0/1573190169 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6723+0+0 (secure 0 0 0) 0x7f9c480cc960 con 0x7f9c5c101710 2023-12-11T17:43:18.410 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:43:18.407+0000 7f9c60cfd700 1 --2- 172.21.15.181:0/1573190169 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f9c4c0a1480 0x7f9c4c0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:43:18.410 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:43:18.407+0000 7f9c60cfd700 1 --2- 172.21.15.181:0/1573190169 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f9c4c0a1480 0x7f9c4c0a3940 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c101570 tx=0x7f9c5000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:43:18.413 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:43:18.411+0000 7f9c59ffb700 1 -- 172.21.15.181:0/1573190169 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f9c480965f0 con 0x7f9c5c101710 2023-12-11T17:43:18.532 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:18 smithi115 ceph-mon[102748]: purged_snaps scrub starts 2023-12-11T17:43:18.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:18 smithi115 ceph-mon[102748]: purged_snaps scrub ok 2023-12-11T17:43:18.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:18 smithi115 ceph-mon[102748]: osd.10 [v2:172.21.15.115:6824/2199610269,v1:172.21.15.115:6825/2199610269] boot 2023-12-11T17:43:18.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:18 smithi115 ceph-mon[102748]: osdmap e70: 11 total, 11 up, 11 in 2023-12-11T17:43:18.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:18 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-11T17:43:18.610 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:43:18.608+0000 7f9c62f61700 1 -- 172.21.15.181:0/1573190169 --> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f9c5c061a10 con 0x7f9c4c0a1480 2023-12-11T17:43:19.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:19 smithi181 ceph-mon[103627]: pgmap v212: 1 pgs: 1 remapped+peering; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-12-11T17:43:19.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:19 smithi181 ceph-mon[103627]: osdmap e71: 11 total, 11 up, 11 in 2023-12-11T17:43:19.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:19 smithi181 ceph-mon[103627]: from='client.24352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T17:43:19.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:19 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-11T17:43:19.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:19 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-11T17:43:19.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:19 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:43:19.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:19 smithi059 ceph-mon[98847]: pgmap v212: 1 pgs: 1 remapped+peering; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-12-11T17:43:19.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:19 smithi059 ceph-mon[98847]: osdmap e71: 11 total, 11 up, 11 in 2023-12-11T17:43:19.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:19 smithi059 ceph-mon[98847]: from='client.24352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T17:43:19.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:19 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-11T17:43:19.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:19 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-11T17:43:19.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:19 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:43:19.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:19 smithi115 ceph-mon[102748]: pgmap v212: 1 pgs: 1 remapped+peering; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-12-11T17:43:19.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:19 smithi115 ceph-mon[102748]: osdmap e71: 11 total, 11 up, 11 in 2023-12-11T17:43:19.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:19 smithi115 ceph-mon[102748]: from='client.24352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T17:43:19.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:19 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-11T17:43:19.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:19 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-11T17:43:19.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:19 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:43:20.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:20 smithi181 ceph-mon[103627]: osdmap e72: 11 total, 11 up, 11 in 2023-12-11T17:43:20.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:20 smithi059 ceph-mon[98847]: osdmap e72: 11 total, 11 up, 11 in 2023-12-11T17:43:20.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:20 smithi115 ceph-mon[102748]: osdmap e72: 11 total, 11 up, 11 in 2023-12-11T17:43:21.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:21 smithi181 ceph-mon[103627]: pgmap v215: 1 pgs: 1 remapped+peering; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-12-11T17:43:21.396 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:21 smithi181 ceph-mon[103627]: osdmap e73: 11 total, 11 up, 11 in 2023-12-11T17:43:21.396 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:21 smithi181 ceph-mon[103627]: from='client.? 172.21.15.181:0/789747277' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5e141dfe-8a00-4bbc-8204-2dc9be24e18d"}]: dispatch 2023-12-11T17:43:21.396 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:21 smithi181 ceph-mon[103627]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5e141dfe-8a00-4bbc-8204-2dc9be24e18d"}]: dispatch 2023-12-11T17:43:21.396 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:21 smithi181 ceph-mon[103627]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5e141dfe-8a00-4bbc-8204-2dc9be24e18d"}]': finished 2023-12-11T17:43:21.396 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:21 smithi181 ceph-mon[103627]: osdmap e74: 12 total, 11 up, 12 in 2023-12-11T17:43:21.396 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:21 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-11T17:43:21.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:21 smithi059 ceph-mon[98847]: pgmap v215: 1 pgs: 1 remapped+peering; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-12-11T17:43:21.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:21 smithi059 ceph-mon[98847]: osdmap e73: 11 total, 11 up, 11 in 2023-12-11T17:43:21.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:21 smithi059 ceph-mon[98847]: from='client.? 172.21.15.181:0/789747277' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5e141dfe-8a00-4bbc-8204-2dc9be24e18d"}]: dispatch 2023-12-11T17:43:21.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:21 smithi059 ceph-mon[98847]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5e141dfe-8a00-4bbc-8204-2dc9be24e18d"}]: dispatch 2023-12-11T17:43:21.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:21 smithi059 ceph-mon[98847]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5e141dfe-8a00-4bbc-8204-2dc9be24e18d"}]': finished 2023-12-11T17:43:21.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:21 smithi059 ceph-mon[98847]: osdmap e74: 12 total, 11 up, 12 in 2023-12-11T17:43:21.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:21 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-11T17:43:21.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:21 smithi115 ceph-mon[102748]: pgmap v215: 1 pgs: 1 remapped+peering; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-12-11T17:43:21.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:21 smithi115 ceph-mon[102748]: osdmap e73: 11 total, 11 up, 11 in 2023-12-11T17:43:21.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:21 smithi115 ceph-mon[102748]: from='client.? 172.21.15.181:0/789747277' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5e141dfe-8a00-4bbc-8204-2dc9be24e18d"}]: dispatch 2023-12-11T17:43:21.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:21 smithi115 ceph-mon[102748]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5e141dfe-8a00-4bbc-8204-2dc9be24e18d"}]: dispatch 2023-12-11T17:43:21.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:21 smithi115 ceph-mon[102748]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5e141dfe-8a00-4bbc-8204-2dc9be24e18d"}]': finished 2023-12-11T17:43:21.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:21 smithi115 ceph-mon[102748]: osdmap e74: 12 total, 11 up, 12 in 2023-12-11T17:43:21.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:21 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-11T17:43:22.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:22 smithi059 ceph-mon[98847]: from='client.? 172.21.15.181:0/683674933' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-11T17:43:22.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:22 smithi115 ceph-mon[102748]: from='client.? 172.21.15.181:0/683674933' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-11T17:43:22.645 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:22 smithi181 ceph-mon[103627]: from='client.? 172.21.15.181:0/683674933' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-11T17:43:23.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:23 smithi059 ceph-mon[98847]: pgmap v218: 1 pgs: 1 remapped+peering; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-12-11T17:43:23.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:23 smithi115 ceph-mon[102748]: pgmap v218: 1 pgs: 1 remapped+peering; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-12-11T17:43:23.645 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:23 smithi181 ceph-mon[103627]: pgmap v218: 1 pgs: 1 remapped+peering; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-12-11T17:43:25.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:25 smithi181 ceph-mon[103627]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 97 KiB/s, 0 objects/s recovering 2023-12-11T17:43:25.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:25 smithi059 ceph-mon[98847]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 97 KiB/s, 0 objects/s recovering 2023-12-11T17:43:25.532 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:25 smithi115 ceph-mon[102748]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 97 KiB/s, 0 objects/s recovering 2023-12-11T17:43:27.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:27 smithi181 ceph-mon[103627]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 83 KiB/s, 0 objects/s recovering 2023-12-11T17:43:27.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:27 smithi115 ceph-mon[102748]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 83 KiB/s, 0 objects/s recovering 2023-12-11T17:43:27.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:27 smithi059 ceph-mon[98847]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 83 KiB/s, 0 objects/s recovering 2023-12-11T17:43:28.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:28 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-12-11T17:43:28.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:28 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:43:28.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:28 smithi181 ceph-mon[103627]: Deploying daemon osd.11 on smithi181 2023-12-11T17:43:28.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:28 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-12-11T17:43:28.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:28 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:43:28.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:28 smithi115 ceph-mon[102748]: Deploying daemon osd.11 on smithi181 2023-12-11T17:43:28.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:28 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-12-11T17:43:28.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:28 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:43:28.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:28 smithi059 ceph-mon[98847]: Deploying daemon osd.11 on smithi181 2023-12-11T17:43:29.294 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:29 smithi181 ceph-mon[103627]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-12-11T17:43:29.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:29 smithi115 ceph-mon[102748]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-12-11T17:43:29.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:29 smithi059 ceph-mon[98847]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-12-11T17:43:31.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:31 smithi181 ceph-mon[103627]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-12-11T17:43:31.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:31 smithi115 ceph-mon[102748]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-12-11T17:43:31.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:31 smithi059 ceph-mon[98847]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-12-11T17:43:33.265 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:33 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:43:33.265 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:33 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:43:33.265 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:33 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:43:33.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:33 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:43:33.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:33 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:43:33.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:33 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:43:33.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:33 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:43:33.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:33 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:43:33.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:33 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:43:34.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:34 smithi115 ceph-mon[102748]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 52 KiB/s, 0 objects/s recovering 2023-12-11T17:43:34.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:34 smithi181 ceph-mon[103627]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 52 KiB/s, 0 objects/s recovering 2023-12-11T17:43:34.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:34 smithi059 ceph-mon[98847]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 52 KiB/s, 0 objects/s recovering 2023-12-11T17:43:34.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:43:34.976+0000 7f9c59ffb700 1 -- 172.21.15.181:0/1573190169 <== mgr.14150 v2:172.21.15.59:6800/3627728353 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f9c5c061a10 con 0x7f9c4c0a1480 2023-12-11T17:43:34.979 INFO:teuthology.orchestra.run.smithi181.stdout:Created osd(s) 11 on host 'smithi181' 2023-12-11T17:43:34.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:43:34.978+0000 7f9c62f61700 1 -- 172.21.15.181:0/1573190169 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f9c4c0a1480 msgr2=0x7f9c4c0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:43:34.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:43:34.978+0000 7f9c62f61700 1 --2- 172.21.15.181:0/1573190169 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f9c4c0a1480 0x7f9c4c0a3940 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c101570 tx=0x7f9c5000b410 comp rx=0 tx=0).stop 2023-12-11T17:43:34.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:43:34.978+0000 7f9c62f61700 1 -- 172.21.15.181:0/1573190169 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c5c101710 msgr2=0x7f9c5c196a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:43:34.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:43:34.978+0000 7f9c62f61700 1 --2- 172.21.15.181:0/1573190169 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c5c101710 0x7f9c5c196a30 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f9c4800d980 tx=0x7f9c4800dc90 comp rx=0 tx=0).stop 2023-12-11T17:43:34.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:43:34.978+0000 7f9c62f61700 1 -- 172.21.15.181:0/1573190169 shutdown_connections 2023-12-11T17:43:34.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:43:34.978+0000 7f9c62f61700 1 --2- 172.21.15.181:0/1573190169 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f9c4c0a1480 0x7f9c4c0a3940 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:43:34.981 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:43:34.978+0000 7f9c62f61700 1 --2- 172.21.15.181:0/1573190169 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9c5c1020b0 0x7f9c5c19da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:43:34.982 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:43:34.978+0000 7f9c62f61700 1 --2- 172.21.15.181:0/1573190169 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c5c100510 0x7f9c5c1964f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:43:34.982 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:43:34.978+0000 7f9c62f61700 1 --2- 172.21.15.181:0/1573190169 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c5c101710 0x7f9c5c196a30 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:43:34.982 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:43:34.978+0000 7f9c62f61700 1 -- 172.21.15.181:0/1573190169 >> 172.21.15.181:0/1573190169 conn(0x7f9c5c0fbad0 msgr2=0x7f9c5c104a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:43:34.982 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:43:34.978+0000 7f9c62f61700 1 -- 172.21.15.181:0/1573190169 shutdown_connections 2023-12-11T17:43:34.982 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:43:34.978+0000 7f9c62f61700 1 -- 172.21.15.181:0/1573190169 wait complete. 2023-12-11T17:43:35.357 DEBUG:teuthology.orchestra.run.smithi181:osd.11> sudo journalctl -f -n 0 -u ceph-d0a07eaa-984b-11ee-95a3-87774f69a715@osd.11.service 2023-12-11T17:43:35.359 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2023-12-11T17:43:35.360 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d0a07eaa-984b-11ee-95a3-87774f69a715 -- ceph osd stat -f json 2023-12-11T17:43:35.388 INFO:journalctl@ceph.osd.11.smithi181.stdout:-- Logs begin at Mon 2023-12-11 17:22:43 UTC. -- 2023-12-11T17:43:35.645 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:35 smithi181 ceph-mon[103627]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-12-11T17:43:35.645 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:35 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:43:35.645 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:35 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:43:35.645 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:35 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:43:35.645 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:35 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:43:35.645 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:35 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:43:35.645 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:35 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:43:35.645 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:35 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:43:35.645 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:35 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:43:35.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:35 smithi059 ceph-mon[98847]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-12-11T17:43:35.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:35 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:43:35.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:35 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:43:35.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:35 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:43:35.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:35 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:43:35.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:35 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:43:35.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:35 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:43:35.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:35 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:43:35.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:35 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:43:35.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:35 smithi115 ceph-mon[102748]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-12-11T17:43:35.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:35 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:43:35.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:35 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:43:35.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:35 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:43:35.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:35 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:43:35.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:35 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:43:35.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:35 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:43:35.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:35 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:43:35.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:35 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:43:35.978 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/mon.a/config 2023-12-11T17:43:36.173 INFO:journalctl@ceph.osd.11.smithi181.stdout:Dec 11 17:43:35 smithi181 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-11[134974]: 2023-12-11T17:43:35.922+0000 7f8908065640 -1 osd.11 0 log_to_monitors true 2023-12-11T17:43:36.772 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:36 smithi059 ceph-mon[98847]: from='osd.11 [v2:172.21.15.181:6824/226506989,v1:172.21.15.181:6825/226506989]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-12-11T17:43:36.772 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:36 smithi059 ceph-mon[98847]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-12-11T17:43:36.782 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:36 smithi115 ceph-mon[102748]: from='osd.11 [v2:172.21.15.181:6824/226506989,v1:172.21.15.181:6825/226506989]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-12-11T17:43:36.782 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:36 smithi115 ceph-mon[102748]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-12-11T17:43:36.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:36 smithi181 ceph-mon[103627]: from='osd.11 [v2:172.21.15.181:6824/226506989,v1:172.21.15.181:6825/226506989]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-12-11T17:43:36.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:36 smithi181 ceph-mon[103627]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-12-11T17:43:37.149 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:37.144+0000 7f7cb0692700 1 -- 172.21.15.59:0/2388073001 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7ca8100530 msgr2=0x7f7ca8100950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:43:37.150 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:37.144+0000 7f7cb0692700 1 --2- 172.21.15.59:0/2388073001 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7ca8100530 0x7f7ca8100950 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7c980099c0 tx=0x7f7c98009cd0 comp rx=0 tx=0).stop 2023-12-11T17:43:37.150 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:37.144+0000 7f7cb0692700 1 -- 172.21.15.59:0/2388073001 shutdown_connections 2023-12-11T17:43:37.150 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:37.144+0000 7f7cb0692700 1 --2- 172.21.15.59:0/2388073001 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ca81020d0 0x7f7ca8108980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:43:37.150 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:37.144+0000 7f7cb0692700 1 --2- 172.21.15.59:0/2388073001 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7ca8100530 0x7f7ca8100950 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:43:37.150 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:37.144+0000 7f7cb0692700 1 --2- 172.21.15.59:0/2388073001 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ca8101730 0x7f7ca8101b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:43:37.150 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:37.144+0000 7f7cb0692700 1 -- 172.21.15.59:0/2388073001 >> 172.21.15.59:0/2388073001 conn(0x7f7ca80fbad0 msgr2=0x7f7ca80fdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:43:37.150 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:37.144+0000 7f7cb0692700 1 -- 172.21.15.59:0/2388073001 shutdown_connections 2023-12-11T17:43:37.150 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:37.144+0000 7f7cb0692700 1 -- 172.21.15.59:0/2388073001 wait complete. 2023-12-11T17:43:37.150 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:37.145+0000 7f7cb0692700 1 Processor -- start 2023-12-11T17:43:37.150 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:37.145+0000 7f7cb0692700 1 -- start start 2023-12-11T17:43:37.150 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:37.145+0000 7f7cb0692700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7ca8100530 0x7f7ca8197550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:43:37.150 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:37.145+0000 7f7cb0692700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ca8101730 0x7f7ca8197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:43:37.151 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:37.145+0000 7f7cb0692700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ca81020d0 0x7f7ca819eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:43:37.151 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:37.145+0000 7f7cb0692700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca819f090 con 0x7f7ca8100530 2023-12-11T17:43:37.151 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:37.145+0000 7f7cb0692700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca819f1d0 con 0x7f7ca81020d0 2023-12-11T17:43:37.151 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:37.145+0000 7f7cb0692700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca819f360 con 0x7f7ca8101730 2023-12-11T17:43:37.151 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:37.145+0000 7f7cae42e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7ca8100530 0x7f7ca8197550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:43:37.151 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:37.145+0000 7f7cae42e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7ca8100530 0x7f7ca8197550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60700/0 (socket says 172.21.15.59:60700) 2023-12-11T17:43:37.151 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:37.145+0000 7f7cae42e700 1 -- 172.21.15.59:0/1647958328 learned_addr learned my addr 172.21.15.59:0/1647958328 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:43:37.151 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:37.145+0000 7f7cadc2d700 1 --2- 172.21.15.59:0/1647958328 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ca8101730 0x7f7ca8197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:43:37.151 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:37.145+0000 7f7caec2f700 1 --2- 172.21.15.59:0/1647958328 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ca81020d0 0x7f7ca819eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:43:37.151 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:37.145+0000 7f7cae42e700 1 -- 172.21.15.59:0/1647958328 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ca8101730 msgr2=0x7f7ca8197a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:43:37.151 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:37.145+0000 7f7cae42e700 1 --2- 172.21.15.59:0/1647958328 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ca8101730 0x7f7ca8197a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:43:37.151 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:37.145+0000 7f7cae42e700 1 -- 172.21.15.59:0/1647958328 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ca81020d0 msgr2=0x7f7ca819eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:43:37.151 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:37.145+0000 7f7cae42e700 1 --2- 172.21.15.59:0/1647958328 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ca81020d0 0x7f7ca819eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:43:37.151 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:37.145+0000 7f7cae42e700 1 -- 172.21.15.59:0/1647958328 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c980095f0 con 0x7f7ca8100530 2023-12-11T17:43:37.151 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:37.146+0000 7f7cae42e700 1 --2- 172.21.15.59:0/1647958328 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7ca8100530 0x7f7ca8197550 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7c980098e0 tx=0x7f7c98005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:43:37.152 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:37.146+0000 7f7c9f7fe700 1 -- 172.21.15.59:0/1647958328 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c9801d070 con 0x7f7ca8100530 2023-12-11T17:43:37.152 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:37.146+0000 7f7c9f7fe700 1 -- 172.21.15.59:0/1647958328 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7c980059a0 con 0x7f7ca8100530 2023-12-11T17:43:37.152 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:37.146+0000 7f7c9f7fe700 1 -- 172.21.15.59:0/1647958328 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c980173f0 con 0x7f7ca8100530 2023-12-11T17:43:37.152 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:37.146+0000 7f7cb0692700 1 -- 172.21.15.59:0/1647958328 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ca819f5e0 con 0x7f7ca8100530 2023-12-11T17:43:37.152 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:37.146+0000 7f7cb0692700 1 -- 172.21.15.59:0/1647958328 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ca819fa80 con 0x7f7ca8100530 2023-12-11T17:43:37.153 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:37.147+0000 7f7cb0692700 1 -- 172.21.15.59:0/1647958328 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ca804f290 con 0x7f7ca8100530 2023-12-11T17:43:37.153 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:37.148+0000 7f7c9f7fe700 1 -- 172.21.15.59:0/1647958328 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f7c980226e0 con 0x7f7ca8100530 2023-12-11T17:43:37.153 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:37.149+0000 7f7c9f7fe700 1 --2- 172.21.15.59:0/1647958328 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f7c940a1500 0x7f7c940a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:43:37.153 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:37.149+0000 7f7cadc2d700 1 --2- 172.21.15.59:0/1647958328 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f7c940a1500 0x7f7c940a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:43:37.153 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:37.149+0000 7f7c9f7fe700 1 -- 172.21.15.59:0/1647958328 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6831+0+0 (secure 0 0 0) 0x7f7c980cd8b0 con 0x7f7ca8100530 2023-12-11T17:43:37.153 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:37.149+0000 7f7cadc2d700 1 --2- 172.21.15.59:0/1647958328 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f7c940a1500 0x7f7c940a39c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f7ca4009c60 tx=0x7f7ca4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:43:37.156 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:37.152+0000 7f7c9f7fe700 1 -- 172.21.15.59:0/1647958328 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f7c98097570 con 0x7f7ca8100530 2023-12-11T17:43:37.354 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:37.350+0000 7f7cb0692700 1 -- 172.21.15.59:0/1647958328 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f7ca810bd40 con 0x7f7ca8100530 2023-12-11T17:43:37.354 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:37.351+0000 7f7c9f7fe700 1 -- 172.21.15.59:0/1647958328 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v75) v1 ==== 74+0+133 (secure 0 0 0) 0x7f7c9809ab90 con 0x7f7ca8100530 2023-12-11T17:43:37.354 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-12-11T17:43:37.356 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:37.353+0000 7f7cb0692700 1 -- 172.21.15.59:0/1647958328 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f7c940a1500 msgr2=0x7f7c940a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:43:37.356 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:37.353+0000 7f7cb0692700 1 --2- 172.21.15.59:0/1647958328 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f7c940a1500 0x7f7c940a39c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f7ca4009c60 tx=0x7f7ca4009260 comp rx=0 tx=0).stop 2023-12-11T17:43:37.357 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:37.353+0000 7f7cb0692700 1 -- 172.21.15.59:0/1647958328 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7ca8100530 msgr2=0x7f7ca8197550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:43:37.357 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:37.353+0000 7f7cb0692700 1 --2- 172.21.15.59:0/1647958328 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7ca8100530 0x7f7ca8197550 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7c980098e0 tx=0x7f7c98005f10 comp rx=0 tx=0).stop 2023-12-11T17:43:37.357 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:37.353+0000 7f7cb0692700 1 -- 172.21.15.59:0/1647958328 shutdown_connections 2023-12-11T17:43:37.357 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:37.353+0000 7f7cb0692700 1 --2- 172.21.15.59:0/1647958328 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f7c940a1500 0x7f7c940a39c0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:43:37.357 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:37.353+0000 7f7cb0692700 1 --2- 172.21.15.59:0/1647958328 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ca81020d0 0x7f7ca819eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:43:37.357 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:37.353+0000 7f7cb0692700 1 --2- 172.21.15.59:0/1647958328 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7ca8100530 0x7f7ca8197550 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:43:37.357 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:37.353+0000 7f7cb0692700 1 --2- 172.21.15.59:0/1647958328 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ca8101730 0x7f7ca8197a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:43:37.357 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:37.353+0000 7f7cb0692700 1 -- 172.21.15.59:0/1647958328 >> 172.21.15.59:0/1647958328 conn(0x7f7ca80fbad0 msgr2=0x7f7ca80fc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:43:37.357 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:37.353+0000 7f7cb0692700 1 -- 172.21.15.59:0/1647958328 shutdown_connections 2023-12-11T17:43:37.357 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:37.353+0000 7f7cb0692700 1 -- 172.21.15.59:0/1647958328 wait complete. 2023-12-11T17:43:37.657 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:37 smithi059 ceph-mon[98847]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-12-11T17:43:37.657 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:37 smithi059 ceph-mon[98847]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-12-11T17:43:37.657 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:37 smithi059 ceph-mon[98847]: osdmap e75: 12 total, 11 up, 12 in 2023-12-11T17:43:37.657 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:37 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-11T17:43:37.657 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:37 smithi059 ceph-mon[98847]: from='osd.11 [v2:172.21.15.181:6824/226506989,v1:172.21.15.181:6825/226506989]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2023-12-11T17:43:37.657 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:37 smithi059 ceph-mon[98847]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2023-12-11T17:43:37.658 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:37 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/1647958328' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-12-11T17:43:37.833 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":75,"num_osds":12,"num_up_osds":11,"osd_up_since":1702316597,"num_in_osds":12,"osd_in_since":1702316601,"num_remapped_pgs":0} 2023-12-11T17:43:37.895 INFO:journalctl@ceph.osd.11.smithi181.stdout:Dec 11 17:43:37 smithi181 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-11[134974]: 2023-12-11T17:43:37.542+0000 7f88fd6dc700 -1 osd.11 0 waiting for initial osdmap 2023-12-11T17:43:37.895 INFO:journalctl@ceph.osd.11.smithi181.stdout:Dec 11 17:43:37 smithi181 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-11[134974]: 2023-12-11T17:43:37.558+0000 7f88f8cd5700 -1 osd.11 76 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-11T17:43:37.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:37 smithi181 ceph-mon[103627]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-12-11T17:43:37.896 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:37 smithi181 ceph-mon[103627]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-12-11T17:43:37.896 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:37 smithi181 ceph-mon[103627]: osdmap e75: 12 total, 11 up, 12 in 2023-12-11T17:43:37.896 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:37 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-11T17:43:37.896 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:37 smithi181 ceph-mon[103627]: from='osd.11 [v2:172.21.15.181:6824/226506989,v1:172.21.15.181:6825/226506989]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2023-12-11T17:43:37.896 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:37 smithi181 ceph-mon[103627]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2023-12-11T17:43:37.896 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:37 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/1647958328' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-12-11T17:43:38.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:37 smithi115 ceph-mon[102748]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-12-11T17:43:38.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:37 smithi115 ceph-mon[102748]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-12-11T17:43:38.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:37 smithi115 ceph-mon[102748]: osdmap e75: 12 total, 11 up, 12 in 2023-12-11T17:43:38.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:37 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-11T17:43:38.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:37 smithi115 ceph-mon[102748]: from='osd.11 [v2:172.21.15.181:6824/226506989,v1:172.21.15.181:6825/226506989]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2023-12-11T17:43:38.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:37 smithi115 ceph-mon[102748]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2023-12-11T17:43:38.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:37 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/1647958328' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-12-11T17:43:38.834 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d0a07eaa-984b-11ee-95a3-87774f69a715 -- ceph osd stat -f json 2023-12-11T17:43:38.867 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:38 smithi059 ceph-mon[98847]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]': finished 2023-12-11T17:43:38.867 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:38 smithi059 ceph-mon[98847]: osdmap e76: 12 total, 11 up, 12 in 2023-12-11T17:43:38.867 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:38 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-11T17:43:38.867 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:38 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-11T17:43:38.867 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:38 smithi059 ceph-mon[98847]: Detected new or changed devices on smithi181 2023-12-11T17:43:38.867 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:38 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:43:38.867 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:38 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:43:38.867 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:38 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:43:38.867 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:38 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:43:38.867 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:38 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:43:38.867 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:38 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:43:38.867 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:38 smithi059 ceph-mon[98847]: Adjusting osd_memory_target on smithi181 to 4292M 2023-12-11T17:43:38.868 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:38 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:43:38.868 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:38 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:43:38.868 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:38 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:43:38.868 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:38 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:43:38.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:38 smithi181 ceph-mon[103627]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]': finished 2023-12-11T17:43:38.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:38 smithi181 ceph-mon[103627]: osdmap e76: 12 total, 11 up, 12 in 2023-12-11T17:43:38.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:38 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-11T17:43:38.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:38 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-11T17:43:38.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:38 smithi181 ceph-mon[103627]: Detected new or changed devices on smithi181 2023-12-11T17:43:38.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:38 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:43:38.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:38 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:43:38.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:38 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:43:38.896 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:38 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:43:38.896 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:38 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:43:38.896 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:38 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:43:38.896 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:38 smithi181 ceph-mon[103627]: Adjusting osd_memory_target on smithi181 to 4292M 2023-12-11T17:43:38.896 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:38 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:43:38.896 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:38 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:43:38.896 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:38 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:43:38.896 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:38 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:43:39.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:38 smithi115 ceph-mon[102748]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]': finished 2023-12-11T17:43:39.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:38 smithi115 ceph-mon[102748]: osdmap e76: 12 total, 11 up, 12 in 2023-12-11T17:43:39.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:38 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-11T17:43:39.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:38 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-11T17:43:39.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:38 smithi115 ceph-mon[102748]: Detected new or changed devices on smithi181 2023-12-11T17:43:39.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:38 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:43:39.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:38 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:43:39.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:38 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:43:39.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:38 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:43:39.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:38 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:43:39.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:38 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:43:39.034 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:38 smithi115 ceph-mon[102748]: Adjusting osd_memory_target on smithi181 to 4292M 2023-12-11T17:43:39.034 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:38 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:43:39.034 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:38 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:43:39.034 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:38 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:43:39.034 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:38 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:43:39.452 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/mon.a/config 2023-12-11T17:43:39.877 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:39 smithi059 ceph-mon[98847]: purged_snaps scrub starts 2023-12-11T17:43:39.877 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:39 smithi059 ceph-mon[98847]: purged_snaps scrub ok 2023-12-11T17:43:39.878 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:39 smithi059 ceph-mon[98847]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-12-11T17:43:39.878 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:39 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-11T17:43:39.878 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:39 smithi059 ceph-mon[98847]: osd.11 [v2:172.21.15.181:6824/226506989,v1:172.21.15.181:6825/226506989] boot 2023-12-11T17:43:39.878 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:39 smithi059 ceph-mon[98847]: osdmap e77: 12 total, 12 up, 12 in 2023-12-11T17:43:39.878 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:39 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-11T17:43:39.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:39 smithi181 ceph-mon[103627]: purged_snaps scrub starts 2023-12-11T17:43:39.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:39 smithi181 ceph-mon[103627]: purged_snaps scrub ok 2023-12-11T17:43:39.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:39 smithi181 ceph-mon[103627]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-12-11T17:43:39.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:39 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-11T17:43:39.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:39 smithi181 ceph-mon[103627]: osd.11 [v2:172.21.15.181:6824/226506989,v1:172.21.15.181:6825/226506989] boot 2023-12-11T17:43:39.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:39 smithi181 ceph-mon[103627]: osdmap e77: 12 total, 12 up, 12 in 2023-12-11T17:43:39.896 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:39 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-11T17:43:40.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:39 smithi115 ceph-mon[102748]: purged_snaps scrub starts 2023-12-11T17:43:40.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:39 smithi115 ceph-mon[102748]: purged_snaps scrub ok 2023-12-11T17:43:40.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:39 smithi115 ceph-mon[102748]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-12-11T17:43:40.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:39 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-11T17:43:40.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:39 smithi115 ceph-mon[102748]: osd.11 [v2:172.21.15.181:6824/226506989,v1:172.21.15.181:6825/226506989] boot 2023-12-11T17:43:40.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:39 smithi115 ceph-mon[102748]: osdmap e77: 12 total, 12 up, 12 in 2023-12-11T17:43:40.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:39 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-11T17:43:40.675 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:40.670+0000 7f4024acc700 1 -- 172.21.15.59:0/1471828205 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f40201020b0 msgr2=0x7f4020108960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:43:40.676 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:40.670+0000 7f4024acc700 1 --2- 172.21.15.59:0/1471828205 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f40201020b0 0x7f4020108960 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f401800b0d0 tx=0x7f401800b3e0 comp rx=0 tx=0).stop 2023-12-11T17:43:40.676 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:40.670+0000 7f4024acc700 1 -- 172.21.15.59:0/1471828205 shutdown_connections 2023-12-11T17:43:40.676 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:40.670+0000 7f4024acc700 1 --2- 172.21.15.59:0/1471828205 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f40201020b0 0x7f4020108960 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:43:40.676 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:40.670+0000 7f4024acc700 1 --2- 172.21.15.59:0/1471828205 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4020100510 0x7f4020100930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:43:40.676 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:40.670+0000 7f4024acc700 1 --2- 172.21.15.59:0/1471828205 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4020101710 0x7f4020101b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:43:40.676 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:40.670+0000 7f4024acc700 1 -- 172.21.15.59:0/1471828205 >> 172.21.15.59:0/1471828205 conn(0x7f40200fbad0 msgr2=0x7f40200fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:43:40.676 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:40.670+0000 7f4024acc700 1 -- 172.21.15.59:0/1471828205 shutdown_connections 2023-12-11T17:43:40.676 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:40.670+0000 7f4024acc700 1 -- 172.21.15.59:0/1471828205 wait complete. 2023-12-11T17:43:40.676 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:40.671+0000 7f4024acc700 1 Processor -- start 2023-12-11T17:43:40.676 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:40.671+0000 7f4024acc700 1 -- start start 2023-12-11T17:43:40.676 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:40.671+0000 7f4024acc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4020100510 0x7f4020104450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:43:40.676 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:40.671+0000 7f4024acc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4020101710 0x7f4020102af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:43:40.676 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:40.671+0000 7f4024acc700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f40201020b0 0x7f4020103030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:43:40.676 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:40.671+0000 7f4024acc700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40201037d0 con 0x7f4020100510 2023-12-11T17:43:40.677 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:40.671+0000 7f4024acc700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4020103940 con 0x7f40201020b0 2023-12-11T17:43:40.677 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:40.671+0000 7f4024acc700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40201a15e0 con 0x7f4020101710 2023-12-11T17:43:40.677 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:40.671+0000 7f401e59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4020100510 0x7f4020104450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:43:40.677 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:40.671+0000 7f401e59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4020100510 0x7f4020104450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60726/0 (socket says 172.21.15.59:60726) 2023-12-11T17:43:40.677 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:40.671+0000 7f401e59c700 1 -- 172.21.15.59:0/3314752201 learned_addr learned my addr 172.21.15.59:0/3314752201 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:43:40.677 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:40.671+0000 7f401dd9b700 1 --2- 172.21.15.59:0/3314752201 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4020101710 0x7f4020102af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:43:40.677 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:40.671+0000 7f401ed9d700 1 --2- 172.21.15.59:0/3314752201 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f40201020b0 0x7f4020103030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:43:40.677 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:40.671+0000 7f401e59c700 1 -- 172.21.15.59:0/3314752201 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4020101710 msgr2=0x7f4020102af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:43:40.677 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:40.671+0000 7f401e59c700 1 --2- 172.21.15.59:0/3314752201 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4020101710 0x7f4020102af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:43:40.677 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:40.671+0000 7f401e59c700 1 -- 172.21.15.59:0/3314752201 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f40201020b0 msgr2=0x7f4020103030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:43:40.677 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:40.671+0000 7f401e59c700 1 --2- 172.21.15.59:0/3314752201 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f40201020b0 0x7f4020103030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:43:40.677 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:40.672+0000 7f401e59c700 1 -- 172.21.15.59:0/3314752201 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4018009d00 con 0x7f4020100510 2023-12-11T17:43:40.677 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:40.672+0000 7f401e59c700 1 --2- 172.21.15.59:0/3314752201 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4020100510 0x7f4020104450 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f401000d980 tx=0x7f401000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:43:40.677 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:40.672+0000 7f400f7fe700 1 -- 172.21.15.59:0/3314752201 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4010004020 con 0x7f4020100510 2023-12-11T17:43:40.677 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:40.672+0000 7f4024acc700 1 -- 172.21.15.59:0/3314752201 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40201a1870 con 0x7f4020100510 2023-12-11T17:43:40.678 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:40.672+0000 7f400f7fe700 1 -- 172.21.15.59:0/3314752201 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f401000f690 con 0x7f4020100510 2023-12-11T17:43:40.678 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:40.672+0000 7f400f7fe700 1 -- 172.21.15.59:0/3314752201 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4010003a60 con 0x7f4020100510 2023-12-11T17:43:40.678 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:40.672+0000 7f4024acc700 1 -- 172.21.15.59:0/3314752201 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40201a1d70 con 0x7f4020100510 2023-12-11T17:43:40.678 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:40.673+0000 7f4024acc700 1 -- 172.21.15.59:0/3314752201 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f402004f290 con 0x7f4020100510 2023-12-11T17:43:40.679 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:40.674+0000 7f400f7fe700 1 -- 172.21.15.59:0/3314752201 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f401000fb00 con 0x7f4020100510 2023-12-11T17:43:40.679 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:40.674+0000 7f400f7fe700 1 --2- 172.21.15.59:0/3314752201 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f40080a1480 0x7f40080a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:43:40.679 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:40.675+0000 7f401dd9b700 1 --2- 172.21.15.59:0/3314752201 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f40080a1480 0x7f40080a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:43:40.679 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:40.675+0000 7f400f7fe700 1 -- 172.21.15.59:0/3314752201 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f40100cc4a0 con 0x7f4020100510 2023-12-11T17:43:40.679 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:40.675+0000 7f401dd9b700 1 --2- 172.21.15.59:0/3314752201 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f40080a1480 0x7f40080a3940 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f401400ac50 tx=0x7f401400a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:43:40.682 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:40.679+0000 7f400f7fe700 1 -- 172.21.15.59:0/3314752201 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f4010095f00 con 0x7f4020100510 2023-12-11T17:43:40.880 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:40.876+0000 7f4024acc700 1 -- 172.21.15.59:0/3314752201 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f402010bd30 con 0x7f4020100510 2023-12-11T17:43:40.880 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:40.876+0000 7f400f7fe700 1 -- 172.21.15.59:0/3314752201 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v78) v1 ==== 74+0+133 (secure 0 0 0) 0x7f4010099520 con 0x7f4020100510 2023-12-11T17:43:40.880 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-12-11T17:43:40.882 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:40.878+0000 7f4024acc700 1 -- 172.21.15.59:0/3314752201 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f40080a1480 msgr2=0x7f40080a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:43:40.882 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:40.878+0000 7f4024acc700 1 --2- 172.21.15.59:0/3314752201 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f40080a1480 0x7f40080a3940 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f401400ac50 tx=0x7f401400a380 comp rx=0 tx=0).stop 2023-12-11T17:43:40.882 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:40.878+0000 7f4024acc700 1 -- 172.21.15.59:0/3314752201 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4020100510 msgr2=0x7f4020104450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:43:40.882 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:40.878+0000 7f4024acc700 1 --2- 172.21.15.59:0/3314752201 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4020100510 0x7f4020104450 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f401000d980 tx=0x7f401000dc90 comp rx=0 tx=0).stop 2023-12-11T17:43:40.882 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:40.878+0000 7f4024acc700 1 -- 172.21.15.59:0/3314752201 shutdown_connections 2023-12-11T17:43:40.882 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:40.878+0000 7f4024acc700 1 --2- 172.21.15.59:0/3314752201 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f40080a1480 0x7f40080a3940 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:43:40.882 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:40.878+0000 7f4024acc700 1 --2- 172.21.15.59:0/3314752201 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f40201020b0 0x7f4020103030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:43:40.882 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:40.878+0000 7f4024acc700 1 --2- 172.21.15.59:0/3314752201 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4020100510 0x7f4020104450 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:43:40.883 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:40.878+0000 7f4024acc700 1 --2- 172.21.15.59:0/3314752201 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4020101710 0x7f4020102af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:43:40.883 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:40.878+0000 7f4024acc700 1 -- 172.21.15.59:0/3314752201 >> 172.21.15.59:0/3314752201 conn(0x7f40200fbad0 msgr2=0x7f40200fd500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:43:40.883 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:40.879+0000 7f4024acc700 1 -- 172.21.15.59:0/3314752201 shutdown_connections 2023-12-11T17:43:40.883 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:40.879+0000 7f4024acc700 1 -- 172.21.15.59:0/3314752201 wait complete. 2023-12-11T17:43:40.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:40 smithi181 ceph-mon[103627]: osdmap e78: 12 total, 12 up, 12 in 2023-12-11T17:43:40.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:40 smithi059 ceph-mon[98847]: osdmap e78: 12 total, 12 up, 12 in 2023-12-11T17:43:41.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:40 smithi115 ceph-mon[102748]: osdmap e78: 12 total, 12 up, 12 in 2023-12-11T17:43:41.320 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":78,"num_osds":12,"num_up_osds":12,"osd_up_since":1702316618,"num_in_osds":12,"osd_in_since":1702316601,"num_remapped_pgs":0} 2023-12-11T17:43:41.321 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell --fsid d0a07eaa-984b-11ee-95a3-87774f69a715 -- ceph osd dump --format=json 2023-12-11T17:43:41.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:41 smithi181 ceph-mon[103627]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T17:43:41.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:41 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/3314752201' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-12-11T17:43:41.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:41 smithi059 ceph-mon[98847]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T17:43:41.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:41 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/3314752201' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-12-11T17:43:41.962 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/mon.a/config 2023-12-11T17:43:42.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:41 smithi115 ceph-mon[102748]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T17:43:42.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:41 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/3314752201' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-12-11T17:43:43.191 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:43.186+0000 7fc029c66700 1 -- 172.21.15.59:0/2154553283 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc024106570 msgr2=0x7fc024106950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:43:43.191 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:43.186+0000 7fc029c66700 1 --2- 172.21.15.59:0/2154553283 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc024106570 0x7fc024106950 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc0140099c0 tx=0x7fc014009cd0 comp rx=0 tx=0).stop 2023-12-11T17:43:43.191 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:43.186+0000 7fc029c66700 1 -- 172.21.15.59:0/2154553283 shutdown_connections 2023-12-11T17:43:43.191 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:43.186+0000 7fc029c66700 1 --2- 172.21.15.59:0/2154553283 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc024100e80 0x7fc02410d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:43:43.191 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:43.186+0000 7fc029c66700 1 --2- 172.21.15.59:0/2154553283 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc024106570 0x7fc024106950 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:43:43.191 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:43.186+0000 7fc029c66700 1 --2- 172.21.15.59:0/2154553283 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc0241004c0 0x7fc024100940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:43:43.191 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:43.186+0000 7fc029c66700 1 -- 172.21.15.59:0/2154553283 >> 172.21.15.59:0/2154553283 conn(0x7fc0240774d0 msgr2=0x7fc0240778e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:43:43.192 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:43.186+0000 7fc029c66700 1 -- 172.21.15.59:0/2154553283 shutdown_connections 2023-12-11T17:43:43.192 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:43.186+0000 7fc029c66700 1 -- 172.21.15.59:0/2154553283 wait complete. 2023-12-11T17:43:43.192 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:43.187+0000 7fc029c66700 1 Processor -- start 2023-12-11T17:43:43.192 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:43.187+0000 7fc029c66700 1 -- start start 2023-12-11T17:43:43.192 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:43.187+0000 7fc029c66700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc0241004c0 0x7fc02419d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:43:43.192 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:43.187+0000 7fc029c66700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc024100e80 0x7fc02419db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:43:43.192 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:43.187+0000 7fc029c66700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc024106570 0x7fc024197790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:43:43.192 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:43.187+0000 7fc029c66700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc024197e00 con 0x7fc0241004c0 2023-12-11T17:43:43.192 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:43.187+0000 7fc029c66700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc024197f70 con 0x7fc024100e80 2023-12-11T17:43:43.192 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:43.187+0000 7fc029c66700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc024198130 con 0x7fc024106570 2023-12-11T17:43:43.192 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:43.187+0000 7fc0237fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc0241004c0 0x7fc02419d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:43:43.192 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:43.187+0000 7fc0237fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc0241004c0 0x7fc02419d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60738/0 (socket says 172.21.15.59:60738) 2023-12-11T17:43:43.192 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:43.187+0000 7fc0237fe700 1 -- 172.21.15.59:0/2616294117 learned_addr learned my addr 172.21.15.59:0/2616294117 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:43:43.192 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:43.188+0000 7fc022ffd700 1 --2- 172.21.15.59:0/2616294117 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc024100e80 0x7fc02419db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:43:43.193 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:43.188+0000 7fc023fff700 1 --2- 172.21.15.59:0/2616294117 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc024106570 0x7fc024197790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:43:43.193 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:43.188+0000 7fc0237fe700 1 -- 172.21.15.59:0/2616294117 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc024106570 msgr2=0x7fc024197790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:43:43.193 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:43.188+0000 7fc0237fe700 1 --2- 172.21.15.59:0/2616294117 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc024106570 0x7fc024197790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:43:43.193 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:43.188+0000 7fc0237fe700 1 -- 172.21.15.59:0/2616294117 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc024100e80 msgr2=0x7fc02419db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:43:43.193 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:43.188+0000 7fc0237fe700 1 --2- 172.21.15.59:0/2616294117 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc024100e80 0x7fc02419db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:43:43.193 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:43.188+0000 7fc0237fe700 1 -- 172.21.15.59:0/2616294117 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0140095f0 con 0x7fc0241004c0 2023-12-11T17:43:43.193 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:43.188+0000 7fc023fff700 1 --2- 172.21.15.59:0/2616294117 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc024106570 0x7fc024197790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T17:43:43.193 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:43.188+0000 7fc0237fe700 1 --2- 172.21.15.59:0/2616294117 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc0241004c0 0x7fc02419d650 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc0140098e0 tx=0x7fc014005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:43:43.193 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:43.188+0000 7fc020ff9700 1 -- 172.21.15.59:0/2616294117 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc01401d070 con 0x7fc0241004c0 2023-12-11T17:43:43.193 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:43.188+0000 7fc020ff9700 1 -- 172.21.15.59:0/2616294117 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc01400fb30 con 0x7fc0241004c0 2023-12-11T17:43:43.193 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:43.188+0000 7fc029c66700 1 -- 172.21.15.59:0/2616294117 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0241983b0 con 0x7fc0241004c0 2023-12-11T17:43:43.193 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:43.188+0000 7fc020ff9700 1 -- 172.21.15.59:0/2616294117 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc014022c90 con 0x7fc0241004c0 2023-12-11T17:43:43.193 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:43.188+0000 7fc029c66700 1 -- 172.21.15.59:0/2616294117 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0241a4640 con 0x7fc0241004c0 2023-12-11T17:43:43.194 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:43.189+0000 7fc029c66700 1 -- 172.21.15.59:0/2616294117 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc024101be0 con 0x7fc0241004c0 2023-12-11T17:43:43.194 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:43.190+0000 7fc020ff9700 1 -- 172.21.15.59:0/2616294117 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7fc01400fca0 con 0x7fc0241004c0 2023-12-11T17:43:43.194 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:43.191+0000 7fc020ff9700 1 --2- 172.21.15.59:0/2616294117 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fc00c0a1480 0x7fc00c0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:43:43.194 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:43.191+0000 7fc022ffd700 1 --2- 172.21.15.59:0/2616294117 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fc00c0a1480 0x7fc00c0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:43:43.194 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:43.191+0000 7fc020ff9700 1 -- 172.21.15.59:0/2616294117 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fc0140ce3a0 con 0x7fc0241004c0 2023-12-11T17:43:43.194 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:43.191+0000 7fc022ffd700 1 --2- 172.21.15.59:0/2616294117 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fc00c0a1480 0x7fc00c0a3940 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc024198bb0 tx=0x7fc0180073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:43:43.196 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:43.194+0000 7fc020ff9700 1 -- 172.21.15.59:0/2616294117 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fc014097f30 con 0x7fc0241004c0 2023-12-11T17:43:43.395 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:43.392+0000 7fc029c66700 1 -- 172.21.15.59:0/2616294117 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc02404f290 con 0x7fc0241004c0 2023-12-11T17:43:43.396 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:43.394+0000 7fc020ff9700 1 -- 172.21.15.59:0/2616294117 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) v1 ==== 74+0+18422 (secure 0 0 0) 0x7fc01409b550 con 0x7fc0241004c0 2023-12-11T17:43:43.396 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-12-11T17:43:43.397 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":78,"fsid":"d0a07eaa-984b-11ee-95a3-87774f69a715","created":"2023-12-11T17:37:14.526539+0000","modified":"2023-12-11T17:43:39.541636+0000","last_up_change":"2023-12-11T17:43:38.535503+0000","last_in_change":"2023-12-11T17:43:21.014888+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-11T17:40:26.596029+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3c0b3218-4b2b-40f8-8bef-ed8cd5633607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":1040759398},{"type":"v1","addr":"172.21.15.59:6803","nonce":1040759398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":1040759398},{"type":"v1","addr":"172.21.15.59:6805","nonce":1040759398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":1040759398},{"type":"v1","addr":"172.21.15.59:6809","nonce":1040759398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":1040759398},{"type":"v1","addr":"172.21.15.59:6807","nonce":1040759398}]},"public_addr":"172.21.15.59:6803/1040759398","cluster_addr":"172.21.15.59:6805/1040759398","heartbeat_back_addr":"172.21.15.59:6809/1040759398","heartbeat_front_addr":"172.21.15.59:6807/1040759398","state":["exists","up"]},{"osd":1,"uuid":"2ead7578-765b-4f5a-a72c-863731ed69e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":2776354245},{"type":"v1","addr":"172.21.15.115:6801","nonce":2776354245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":2776354245},{"type":"v1","addr":"172.21.15.115:6803","nonce":2776354245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":2776354245},{"type":"v1","addr":"172.21.15.115:6807","nonce":2776354245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":2776354245},{"type":"v1","addr":"172.21.15.115:6805","nonce":2776354245}]},"public_addr":"172.21.15.115:6801/2776354245","cluster_addr":"172.21.15.115:6803/2776354245","heartbeat_back_addr":"172.21.15.115:6807/2776354245","heartbeat_front_addr":"172.21.15.115:6805/2776354245","state":["exists","up"]},{"osd":2,"uuid":"8cb1d7ec-53cc-4672-b0b6-70d73797c44f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4287675481},{"type":"v1","addr":"172.21.15.181:6801","nonce":4287675481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4287675481},{"type":"v1","addr":"172.21.15.181:6803","nonce":4287675481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4287675481},{"type":"v1","addr":"172.21.15.181:6807","nonce":4287675481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4287675481},{"type":"v1","addr":"172.21.15.181:6805","nonce":4287675481}]},"public_addr":"172.21.15.181:6801/4287675481","cluster_addr":"172.21.15.181:6803/4287675481","heartbeat_back_addr":"172.21.15.181:6807/4287675481","heartbeat_front_addr":"172.21.15.181:6805/4287675481","state":["exists","up"]},{"osd":3,"uuid":"5d64d9af-831b-4d0a-8ace-a452169351d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":219909995},{"type":"v1","addr":"172.21.15.59:6811","nonce":219909995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":219909995},{"type":"v1","addr":"172.21.15.59:6813","nonce":219909995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":219909995},{"type":"v1","addr":"172.21.15.59:6817","nonce":219909995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":219909995},{"type":"v1","addr":"172.21.15.59:6815","nonce":219909995}]},"public_addr":"172.21.15.59:6811/219909995","cluster_addr":"172.21.15.59:6813/219909995","heartbeat_back_addr":"172.21.15.59:6817/219909995","heartbeat_front_addr":"172.21.15.59:6815/219909995","state":["exists","up"]},{"osd":4,"uuid":"ef8fad28-1db5-4cba-b5f2-10e54e8e1549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":1021473826},{"type":"v1","addr":"172.21.15.115:6809","nonce":1021473826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":1021473826},{"type":"v1","addr":"172.21.15.115:6811","nonce":1021473826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":1021473826},{"type":"v1","addr":"172.21.15.115:6815","nonce":1021473826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":1021473826},{"type":"v1","addr":"172.21.15.115:6813","nonce":1021473826}]},"public_addr":"172.21.15.115:6809/1021473826","cluster_addr":"172.21.15.115:6811/1021473826","heartbeat_back_addr":"172.21.15.115:6815/1021473826","heartbeat_front_addr":"172.21.15.115:6813/1021473826","state":["exists","up"]},{"osd":5,"uuid":"4233a512-9e8f-4db8-aed2-6c3af706e589","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":2405172521},{"type":"v1","addr":"172.21.15.181:6809","nonce":2405172521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":2405172521},{"type":"v1","addr":"172.21.15.181:6811","nonce":2405172521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2405172521},{"type":"v1","addr":"172.21.15.181:6815","nonce":2405172521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":2405172521},{"type":"v1","addr":"172.21.15.181:6813","nonce":2405172521}]},"public_addr":"172.21.15.181:6809/2405172521","cluster_addr":"172.21.15.181:6811/2405172521","heartbeat_back_addr":"172.21.15.181:6815/2405172521","heartbeat_front_addr":"172.21.15.181:6813/2405172521","state":["exists","up"]},{"osd":6,"uuid":"8faab332-016e-4c77-8370-8bef95fc6f34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":815374853},{"type":"v1","addr":"172.21.15.59:6819","nonce":815374853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":815374853},{"type":"v1","addr":"172.21.15.59:6821","nonce":815374853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":815374853},{"type":"v1","addr":"172.21.15.59:6825","nonce":815374853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":815374853},{"type":"v1","addr":"172.21.15.59:6823","nonce":815374853}]},"public_addr":"172.21.15.59:6819/815374853","cluster_addr":"172.21.15.59:6821/815374853","heartbeat_back_addr":"172.21.15.59:6825/815374853","heartbeat_front_addr":"172.21.15.59:6823/815374853","state":["exists","up"]},{"osd":7,"uuid":"f20f470a-0d40-4ed3-8e0e-fd58675cf143","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":1314971870},{"type":"v1","addr":"172.21.15.115:6817","nonce":1314971870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":1314971870},{"type":"v1","addr":"172.21.15.115:6819","nonce":1314971870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":1314971870},{"type":"v1","addr":"172.21.15.115:6823","nonce":1314971870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":1314971870},{"type":"v1","addr":"172.21.15.115:6821","nonce":1314971870}]},"public_addr":"172.21.15.115:6817/1314971870","cluster_addr":"172.21.15.115:6819/1314971870","heartbeat_back_addr":"172.21.15.115:6823/1314971870","heartbeat_front_addr":"172.21.15.115:6821/1314971870","state":["exists","up"]},{"osd":8,"uuid":"eb657d32-1a95-4fd7-a293-d0a970837fc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":3954509873},{"type":"v1","addr":"172.21.15.181:6817","nonce":3954509873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":3954509873},{"type":"v1","addr":"172.21.15.181:6819","nonce":3954509873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":3954509873},{"type":"v1","addr":"172.21.15.181:6823","nonce":3954509873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":3954509873},{"type":"v1","addr":"172.21.15.181:6821","nonce":3954509873}]},"public_addr":"172.21.15.181:6817/3954509873","cluster_addr":"172.21.15.181:6819/3954509873","heartbeat_back_addr":"172.21.15.181:6823/3954509873","heartbeat_front_addr":"172.21.15.181:6821/3954509873","state":["exists","up"]},{"osd":9,"uuid":"8f7717d4-b6f2-4ec1-9b58-62feb0c9a146","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":1163095677},{"type":"v1","addr":"172.21.15.59:6827","nonce":1163095677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":1163095677},{"type":"v1","addr":"172.21.15.59:6829","nonce":1163095677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":1163095677},{"type":"v1","addr":"172.21.15.59:6833","nonce":1163095677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":1163095677},{"type":"v1","addr":"172.21.15.59:6831","nonce":1163095677}]},"public_addr":"172.21.15.59:6827/1163095677","cluster_addr":"172.21.15.59:6829/1163095677","heartbeat_back_addr":"172.21.15.59:6833/1163095677","heartbeat_front_addr":"172.21.15.59:6831/1163095677","state":["exists","up"]},{"osd":10,"uuid":"b75c179a-875d-43bc-b4c1-e8eb80fb64fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":2199610269},{"type":"v1","addr":"172.21.15.115:6825","nonce":2199610269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":2199610269},{"type":"v1","addr":"172.21.15.115:6827","nonce":2199610269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":2199610269},{"type":"v1","addr":"172.21.15.115:6831","nonce":2199610269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":2199610269},{"type":"v1","addr":"172.21.15.115:6829","nonce":2199610269}]},"public_addr":"172.21.15.115:6825/2199610269","cluster_addr":"172.21.15.115:6827/2199610269","heartbeat_back_addr":"172.21.15.115:6831/2199610269","heartbeat_front_addr":"172.21.15.115:6829/2199610269","state":["exists","up"]},{"osd":11,"uuid":"5e141dfe-8a00-4bbc-8204-2dc9be24e18d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":226506989},{"type":"v1","addr":"172.21.15.181:6825","nonce":226506989}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":226506989},{"type":"v1","addr":"172.21.15.181:6827","nonce":226506989}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":226506989},{"type":"v1","addr":"172.21.15.181:6831","nonce":226506989}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":226506989},{"type":"v1","addr":"172.21.15.181:6829","nonce":226506989}]},"public_addr":"172.21.15.181:6825/226506989","cluster_addr":"172.21.15.181:6827/226506989","heartbeat_back_addr":"172.21.15.181:6831/226506989","heartbeat_front_addr":"172.21.15.181:6829/226506989","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:39:42.687084+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:40:03.903448+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:40:24.899908+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:40:46.600251+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:41:07.690037+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:41:28.488855+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:41:49.459274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:42:10.863881+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:42:32.248391+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:42:54.112265+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:43:15.634872+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:43:36.922534+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.59:6800/1259916482":"2023-12-12T17:38:14.037922+0000","172.21.15.59:0/17457774":"2023-12-12T17:38:14.037922+0000","172.21.15.59:6801/1672530462":"2023-12-12T17:37:44.773925+0000","172.21.15.59:0/1053944547":"2023-12-12T17:38:14.037922+0000","172.21.15.59:6801/1259916482":"2023-12-12T17:38:14.037922+0000","172.21.15.59:6800/1672530462":"2023-12-12T17:37:44.773925+0000","172.21.15.59:0/3570767650":"2023-12-12T17:37:44.773925+0000","172.21.15.59:0/3853462899":"2023-12-12T17:37:44.773925+0000","172.21.15.59:0/2859415380":"2023-12-12T17:38:14.037922+0000","172.21.15.59:0/2192742819":"2023-12-12T17:37:44.773925+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-11T17:43:43.398 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:43.396+0000 7fc029c66700 1 -- 172.21.15.59:0/2616294117 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fc00c0a1480 msgr2=0x7fc00c0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:43:43.398 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:43.396+0000 7fc029c66700 1 --2- 172.21.15.59:0/2616294117 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fc00c0a1480 0x7fc00c0a3940 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc024198bb0 tx=0x7fc0180073d0 comp rx=0 tx=0).stop 2023-12-11T17:43:43.399 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:43.396+0000 7fc029c66700 1 -- 172.21.15.59:0/2616294117 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc0241004c0 msgr2=0x7fc02419d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:43:43.399 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:43.396+0000 7fc029c66700 1 --2- 172.21.15.59:0/2616294117 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc0241004c0 0x7fc02419d650 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc0140098e0 tx=0x7fc014005740 comp rx=0 tx=0).stop 2023-12-11T17:43:43.399 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:43.396+0000 7fc029c66700 1 -- 172.21.15.59:0/2616294117 shutdown_connections 2023-12-11T17:43:43.399 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:43.396+0000 7fc029c66700 1 --2- 172.21.15.59:0/2616294117 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fc00c0a1480 0x7fc00c0a3940 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:43:43.399 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:43.396+0000 7fc029c66700 1 --2- 172.21.15.59:0/2616294117 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc024106570 0x7fc024197790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:43:43.399 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:43.396+0000 7fc029c66700 1 --2- 172.21.15.59:0/2616294117 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc0241004c0 0x7fc02419d650 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:43:43.399 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:43.396+0000 7fc029c66700 1 --2- 172.21.15.59:0/2616294117 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc024100e80 0x7fc02419db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:43:43.399 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:43.396+0000 7fc029c66700 1 -- 172.21.15.59:0/2616294117 >> 172.21.15.59:0/2616294117 conn(0x7fc0240774d0 msgr2=0x7fc0240ff850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:43:43.399 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:43.396+0000 7fc029c66700 1 -- 172.21.15.59:0/2616294117 shutdown_connections 2023-12-11T17:43:43.399 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:43.396+0000 7fc029c66700 1 -- 172.21.15.59:0/2616294117 wait complete. 2023-12-11T17:43:43.784 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:43 smithi059 ceph-mon[98847]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T17:43:43.784 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:43 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/2616294117' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-11T17:43:43.829 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-11T17:40:26.596029+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-11T17:43:43.830 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell --fsid d0a07eaa-984b-11ee-95a3-87774f69a715 -- ceph osd pool get .mgr pg_num 2023-12-11T17:43:43.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:43 smithi181 ceph-mon[103627]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T17:43:43.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:43 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/2616294117' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-11T17:43:44.032 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:43 smithi115 ceph-mon[102748]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T17:43:44.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:43 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/2616294117' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-11T17:43:44.430 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/mon.a/config 2023-12-11T17:43:45.736 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:45.731+0000 7fcd2091f700 1 -- 172.21.15.59:0/1836719947 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcd1c100530 msgr2=0x7fcd1c1009b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:43:45.736 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:45.731+0000 7fcd2091f700 1 --2- 172.21.15.59:0/1836719947 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcd1c100530 0x7fcd1c1009b0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c0099c0 tx=0x7fcd0c009cd0 comp rx=0 tx=0).stop 2023-12-11T17:43:45.736 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:45.731+0000 7fcd2091f700 1 -- 172.21.15.59:0/1836719947 shutdown_connections 2023-12-11T17:43:45.736 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:45.731+0000 7fcd2091f700 1 --2- 172.21.15.59:0/1836719947 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd1c100ef0 0x7fcd1c10d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:43:45.736 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:45.731+0000 7fcd2091f700 1 --2- 172.21.15.59:0/1836719947 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd1c106560 0x7fcd1c106940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:43:45.736 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:45.731+0000 7fcd2091f700 1 --2- 172.21.15.59:0/1836719947 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcd1c100530 0x7fcd1c1009b0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:43:45.736 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:45.731+0000 7fcd2091f700 1 -- 172.21.15.59:0/1836719947 >> 172.21.15.59:0/1836719947 conn(0x7fcd1c076570 msgr2=0x7fcd1c076980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:43:45.736 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:45.731+0000 7fcd2091f700 1 -- 172.21.15.59:0/1836719947 shutdown_connections 2023-12-11T17:43:45.737 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:45.731+0000 7fcd2091f700 1 -- 172.21.15.59:0/1836719947 wait complete. 2023-12-11T17:43:45.737 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:45.732+0000 7fcd2091f700 1 Processor -- start 2023-12-11T17:43:45.737 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:45.732+0000 7fcd2091f700 1 -- start start 2023-12-11T17:43:45.737 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:45.732+0000 7fcd2091f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcd1c100530 0x7fcd1c19c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:43:45.737 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:45.732+0000 7fcd2091f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd1c100ef0 0x7fcd1c19ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:43:45.737 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:45.732+0000 7fcd2091f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd1c106560 0x7fcd1c1966d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:43:45.737 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:45.732+0000 7fcd2091f700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd1c196d30 con 0x7fcd1c100530 2023-12-11T17:43:45.737 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:45.732+0000 7fcd2091f700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd1c196ea0 con 0x7fcd1c100ef0 2023-12-11T17:43:45.737 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:45.732+0000 7fcd2091f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd1c197060 con 0x7fcd1c106560 2023-12-11T17:43:45.737 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:45.732+0000 7fcd1a59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcd1c100530 0x7fcd1c19c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:43:45.737 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:45.732+0000 7fcd1a59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcd1c100530 0x7fcd1c19c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60750/0 (socket says 172.21.15.59:60750) 2023-12-11T17:43:45.737 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:45.732+0000 7fcd1a59c700 1 -- 172.21.15.59:0/50081065 learned_addr learned my addr 172.21.15.59:0/50081065 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:43:45.738 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:45.733+0000 7fcd1ad9d700 1 --2- 172.21.15.59:0/50081065 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd1c106560 0x7fcd1c1966d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:43:45.738 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:45.733+0000 7fcd19d9b700 1 --2- 172.21.15.59:0/50081065 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd1c100ef0 0x7fcd1c19ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:43:45.738 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:45.733+0000 7fcd1a59c700 1 -- 172.21.15.59:0/50081065 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd1c106560 msgr2=0x7fcd1c1966d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:43:45.738 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:45.733+0000 7fcd1a59c700 1 --2- 172.21.15.59:0/50081065 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd1c106560 0x7fcd1c1966d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:43:45.738 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:45.733+0000 7fcd1a59c700 1 -- 172.21.15.59:0/50081065 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd1c100ef0 msgr2=0x7fcd1c19ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:43:45.738 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:45.733+0000 7fcd1a59c700 1 --2- 172.21.15.59:0/50081065 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd1c100ef0 0x7fcd1c19ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:43:45.738 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:45.733+0000 7fcd1a59c700 1 -- 172.21.15.59:0/50081065 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd0c0095f0 con 0x7fcd1c100530 2023-12-11T17:43:45.738 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:45.733+0000 7fcd1a59c700 1 --2- 172.21.15.59:0/50081065 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcd1c100530 0x7fcd1c19c540 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fcd10007b50 tx=0x7fcd10007e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:43:45.738 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:45.733+0000 7fcd0b7fe700 1 -- 172.21.15.59:0/50081065 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd10011070 con 0x7fcd1c100530 2023-12-11T17:43:45.738 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:45.733+0000 7fcd2091f700 1 -- 172.21.15.59:0/50081065 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd1c197340 con 0x7fcd1c100530 2023-12-11T17:43:45.738 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:45.733+0000 7fcd0b7fe700 1 -- 172.21.15.59:0/50081065 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcd1000c2e0 con 0x7fcd1c100530 2023-12-11T17:43:45.738 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:45.733+0000 7fcd0b7fe700 1 -- 172.21.15.59:0/50081065 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd1000e500 con 0x7fcd1c100530 2023-12-11T17:43:45.738 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:45.733+0000 7fcd2091f700 1 -- 172.21.15.59:0/50081065 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd1c069140 con 0x7fcd1c100530 2023-12-11T17:43:45.739 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:45.734+0000 7fcd2091f700 1 -- 172.21.15.59:0/50081065 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd1c101bd0 con 0x7fcd1c100530 2023-12-11T17:43:45.739 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:45.735+0000 7fcd0b7fe700 1 -- 172.21.15.59:0/50081065 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7fcd10004750 con 0x7fcd1c100530 2023-12-11T17:43:45.739 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:45.736+0000 7fcd0b7fe700 1 --2- 172.21.15.59:0/50081065 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fcd040a1480 0x7fcd040a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:43:45.740 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:45.736+0000 7fcd19d9b700 1 --2- 172.21.15.59:0/50081065 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fcd040a1480 0x7fcd040a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:43:45.740 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:45.736+0000 7fcd0b7fe700 1 -- 172.21.15.59:0/50081065 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fcd100cdbe0 con 0x7fcd1c100530 2023-12-11T17:43:45.740 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:45.736+0000 7fcd19d9b700 1 --2- 172.21.15.59:0/50081065 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fcd040a1480 0x7fcd040a3940 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c197d90 tx=0x7fcd0c005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:43:45.743 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:45.740+0000 7fcd0b7fe700 1 -- 172.21.15.59:0/50081065 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fcd100976d0 con 0x7fcd1c100530 2023-12-11T17:43:45.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:45 smithi181 ceph-mon[103627]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T17:43:45.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:45 smithi059 ceph-mon[98847]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T17:43:45.940 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:45.937+0000 7fcd2091f700 1 -- 172.21.15.59:0/50081065 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcd1c1103d0 con 0x7fcd1c100530 2023-12-11T17:43:45.940 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:45.938+0000 7fcd0b7fe700 1 -- 172.21.15.59:0/50081065 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v78) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcd1009acf0 con 0x7fcd1c100530 2023-12-11T17:43:45.940 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2023-12-11T17:43:45.942 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:45.940+0000 7fcd2091f700 1 -- 172.21.15.59:0/50081065 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fcd040a1480 msgr2=0x7fcd040a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:43:45.942 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:45.940+0000 7fcd2091f700 1 --2- 172.21.15.59:0/50081065 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fcd040a1480 0x7fcd040a3940 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c197d90 tx=0x7fcd0c005fd0 comp rx=0 tx=0).stop 2023-12-11T17:43:45.943 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:45.940+0000 7fcd2091f700 1 -- 172.21.15.59:0/50081065 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcd1c100530 msgr2=0x7fcd1c19c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:43:45.943 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:45.940+0000 7fcd2091f700 1 --2- 172.21.15.59:0/50081065 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcd1c100530 0x7fcd1c19c540 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fcd10007b50 tx=0x7fcd10007e60 comp rx=0 tx=0).stop 2023-12-11T17:43:45.943 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:45.940+0000 7fcd2091f700 1 -- 172.21.15.59:0/50081065 shutdown_connections 2023-12-11T17:43:45.943 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:45.940+0000 7fcd2091f700 1 --2- 172.21.15.59:0/50081065 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fcd040a1480 0x7fcd040a3940 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:43:45.943 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:45.940+0000 7fcd2091f700 1 --2- 172.21.15.59:0/50081065 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd1c106560 0x7fcd1c1966d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:43:45.943 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:45.940+0000 7fcd2091f700 1 --2- 172.21.15.59:0/50081065 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcd1c100530 0x7fcd1c19c540 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:43:45.943 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:45.940+0000 7fcd2091f700 1 --2- 172.21.15.59:0/50081065 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd1c100ef0 0x7fcd1c19ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:43:45.943 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:45.940+0000 7fcd2091f700 1 -- 172.21.15.59:0/50081065 >> 172.21.15.59:0/50081065 conn(0x7fcd1c076570 msgr2=0x7fcd1c0ff280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:43:45.943 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:45.940+0000 7fcd2091f700 1 -- 172.21.15.59:0/50081065 shutdown_connections 2023-12-11T17:43:45.943 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:45.940+0000 7fcd2091f700 1 -- 172.21.15.59:0/50081065 wait complete. 2023-12-11T17:43:46.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:45 smithi115 ceph-mon[102748]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T17:43:46.299 INFO:tasks.cephadm:Adding mds.a on smithi059 2023-12-11T17:43:46.299 INFO:tasks.cephadm:Adding mds.d on smithi059 2023-12-11T17:43:46.299 INFO:tasks.cephadm:Adding mds.g on smithi059 2023-12-11T17:43:46.299 INFO:tasks.cephadm:Adding mds.j on smithi059 2023-12-11T17:43:46.299 INFO:tasks.cephadm:Adding mds.b on smithi115 2023-12-11T17:43:46.299 INFO:tasks.cephadm:Adding mds.e on smithi115 2023-12-11T17:43:46.299 INFO:tasks.cephadm:Adding mds.h on smithi115 2023-12-11T17:43:46.299 INFO:tasks.cephadm:Adding mds.k on smithi115 2023-12-11T17:43:46.299 INFO:tasks.cephadm:Adding mds.c on smithi181 2023-12-11T17:43:46.299 INFO:tasks.cephadm:Adding mds.f on smithi181 2023-12-11T17:43:46.299 INFO:tasks.cephadm:Adding mds.i on smithi181 2023-12-11T17:43:46.299 INFO:tasks.cephadm:Adding mds.l on smithi181 2023-12-11T17:43:46.300 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d0a07eaa-984b-11ee-95a3-87774f69a715 -- ceph orch apply mds all '12;smithi059=a;smithi059=d;smithi059=g;smithi059=j;smithi115=b;smithi115=e;smithi115=h;smithi115=k;smithi181=c;smithi181=f;smithi181=i;smithi181=l' 2023-12-11T17:43:46.645 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:46 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/50081065' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-11T17:43:46.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:46 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/50081065' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-11T17:43:46.906 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/mon.c/config 2023-12-11T17:43:47.032 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:46 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/50081065' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-11T17:43:47.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:47 smithi181 ceph-mon[103627]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T17:43:47.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:47 smithi059 ceph-mon[98847]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T17:43:48.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:47 smithi115 ceph-mon[102748]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T17:43:48.131 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:43:48.126+0000 7f2c38727700 1 -- 172.21.15.181:0/4011030623 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c301007a0 msgr2=0x7f2c30100c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:43:48.131 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:43:48.126+0000 7f2c38727700 1 --2- 172.21.15.181:0/4011030623 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c301007a0 0x7f2c30100c20 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f2c20009990 tx=0x7f2c20009ca0 comp rx=0 tx=0).stop 2023-12-11T17:43:48.131 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:43:48.126+0000 7f2c38727700 1 -- 172.21.15.181:0/4011030623 shutdown_connections 2023-12-11T17:43:48.131 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:43:48.126+0000 7f2c38727700 1 --2- 172.21.15.181:0/4011030623 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c301011b0 0x7f2c30108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:43:48.132 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:43:48.126+0000 7f2c38727700 1 --2- 172.21.15.181:0/4011030623 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c30101cd0 0x7f2c301020f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:43:48.132 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:43:48.126+0000 7f2c38727700 1 --2- 172.21.15.181:0/4011030623 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c301007a0 0x7f2c30100c20 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:43:48.132 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:43:48.126+0000 7f2c38727700 1 -- 172.21.15.181:0/4011030623 >> 172.21.15.181:0/4011030623 conn(0x7f2c300fb270 msgr2=0x7f2c300fd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:43:48.132 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:43:48.126+0000 7f2c38727700 1 -- 172.21.15.181:0/4011030623 shutdown_connections 2023-12-11T17:43:48.132 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:43:48.126+0000 7f2c38727700 1 -- 172.21.15.181:0/4011030623 wait complete. 2023-12-11T17:43:48.132 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:43:48.126+0000 7f2c38727700 1 Processor -- start 2023-12-11T17:43:48.132 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:43:48.126+0000 7f2c38727700 1 -- start start 2023-12-11T17:43:48.132 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:43:48.127+0000 7f2c38727700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c301007a0 0x7f2c301044a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:43:48.132 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:43:48.127+0000 7f2c38727700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c301011b0 0x7f2c30102b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:43:48.132 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:43:48.127+0000 7f2c38727700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c30101cd0 0x7f2c301030d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:43:48.132 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:43:48.127+0000 7f2c38727700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c301037b0 con 0x7f2c301007a0 2023-12-11T17:43:48.132 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:43:48.127+0000 7f2c38727700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c30103920 con 0x7f2c301011b0 2023-12-11T17:43:48.133 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:43:48.127+0000 7f2c38727700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c301a1740 con 0x7f2c30101cd0 2023-12-11T17:43:48.133 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:43:48.127+0000 7f2c36cc4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c30101cd0 0x7f2c301030d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:43:48.133 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:43:48.127+0000 7f2c36cc4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c30101cd0 0x7f2c301030d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34742/0 (socket says 172.21.15.181:34742) 2023-12-11T17:43:48.133 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:43:48.127+0000 7f2c36cc4700 1 -- 172.21.15.181:0/2440346083 learned_addr learned my addr 172.21.15.181:0/2440346083 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-12-11T17:43:48.133 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:43:48.127+0000 7f2c35cc2700 1 --2- 172.21.15.181:0/2440346083 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c301011b0 0x7f2c30102b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:43:48.133 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:43:48.127+0000 7f2c364c3700 1 --2- 172.21.15.181:0/2440346083 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c301007a0 0x7f2c301044a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:43:48.133 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:43:48.127+0000 7f2c36cc4700 1 -- 172.21.15.181:0/2440346083 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c301011b0 msgr2=0x7f2c30102b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:43:48.133 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:43:48.127+0000 7f2c36cc4700 1 --2- 172.21.15.181:0/2440346083 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c301011b0 0x7f2c30102b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:43:48.133 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:43:48.127+0000 7f2c36cc4700 1 -- 172.21.15.181:0/2440346083 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c301007a0 msgr2=0x7f2c301044a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:43:48.133 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:43:48.127+0000 7f2c36cc4700 1 --2- 172.21.15.181:0/2440346083 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c301007a0 0x7f2c301044a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:43:48.133 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:43:48.127+0000 7f2c36cc4700 1 -- 172.21.15.181:0/2440346083 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c200095f0 con 0x7f2c30101cd0 2023-12-11T17:43:48.133 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:43:48.127+0000 7f2c364c3700 1 --2- 172.21.15.181:0/2440346083 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c301007a0 0x7f2c301044a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-11T17:43:48.133 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:43:48.127+0000 7f2c36cc4700 1 --2- 172.21.15.181:0/2440346083 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c30101cd0 0x7f2c301030d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f2c2800b970 tx=0x7f2c2800bd30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:43:48.134 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:43:48.127+0000 7f2c277fe700 1 -- 172.21.15.181:0/2440346083 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c2800c9b0 con 0x7f2c30101cd0 2023-12-11T17:43:48.134 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:43:48.127+0000 7f2c277fe700 1 -- 172.21.15.181:0/2440346083 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2c28014440 con 0x7f2c30101cd0 2023-12-11T17:43:48.134 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:43:48.127+0000 7f2c38727700 1 -- 172.21.15.181:0/2440346083 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c301a19a0 con 0x7f2c30101cd0 2023-12-11T17:43:48.134 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:43:48.127+0000 7f2c277fe700 1 -- 172.21.15.181:0/2440346083 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c280075d0 con 0x7f2c30101cd0 2023-12-11T17:43:48.134 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:43:48.127+0000 7f2c38727700 1 -- 172.21.15.181:0/2440346083 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c301a1e70 con 0x7f2c30101cd0 2023-12-11T17:43:48.134 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:43:48.128+0000 7f2c38727700 1 -- 172.21.15.181:0/2440346083 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c3004f290 con 0x7f2c30101cd0 2023-12-11T17:43:48.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:43:48.130+0000 7f2c277fe700 1 -- 172.21.15.181:0/2440346083 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f2c280145b0 con 0x7f2c30101cd0 2023-12-11T17:43:48.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:43:48.130+0000 7f2c277fe700 1 --2- 172.21.15.181:0/2440346083 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f2c1c0a1500 0x7f2c1c0a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:43:48.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:43:48.130+0000 7f2c277fe700 1 -- 172.21.15.181:0/2440346083 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f2c28006e40 con 0x7f2c30101cd0 2023-12-11T17:43:48.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:43:48.130+0000 7f2c364c3700 1 --2- 172.21.15.181:0/2440346083 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f2c1c0a1500 0x7f2c1c0a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:43:48.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:43:48.131+0000 7f2c364c3700 1 --2- 172.21.15.181:0/2440346083 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f2c1c0a1500 0x7f2c1c0a39c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c005fd0 tx=0x7f2c2c005dc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:43:48.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:43:48.134+0000 7f2c277fe700 1 -- 172.21.15.181:0/2440346083 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f2c28095e30 con 0x7f2c30101cd0 2023-12-11T17:43:48.334 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:43:48.331+0000 7f2c38727700 1 -- 172.21.15.181:0/2440346083 --> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi059=a;smithi059=d;smithi059=g;smithi059=j;smithi115=b;smithi115=e;smithi115=h;smithi115=k;smithi181=c;smithi181=f;smithi181=i;smithi181=l", "target": ["mon-mgr", ""]}) v1 -- 0x7f2c300fd480 con 0x7f2c1c0a1500 2023-12-11T17:43:48.343 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:43:48.340+0000 7f2c277fe700 1 -- 172.21.15.181:0/2440346083 <== mgr.14150 v2:172.21.15.59:6800/3627728353 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7f2c300fd480 con 0x7f2c1c0a1500 2023-12-11T17:43:48.343 INFO:teuthology.orchestra.run.smithi181.stdout:Scheduled mds.all update... 2023-12-11T17:43:48.345 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:43:48.342+0000 7f2c38727700 1 -- 172.21.15.181:0/2440346083 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f2c1c0a1500 msgr2=0x7f2c1c0a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:43:48.345 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:43:48.342+0000 7f2c38727700 1 --2- 172.21.15.181:0/2440346083 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f2c1c0a1500 0x7f2c1c0a39c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c005fd0 tx=0x7f2c2c005dc0 comp rx=0 tx=0).stop 2023-12-11T17:43:48.345 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:43:48.342+0000 7f2c38727700 1 -- 172.21.15.181:0/2440346083 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c30101cd0 msgr2=0x7f2c301030d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:43:48.345 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:43:48.342+0000 7f2c38727700 1 --2- 172.21.15.181:0/2440346083 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c30101cd0 0x7f2c301030d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f2c2800b970 tx=0x7f2c2800bd30 comp rx=0 tx=0).stop 2023-12-11T17:43:48.345 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:43:48.342+0000 7f2c38727700 1 -- 172.21.15.181:0/2440346083 shutdown_connections 2023-12-11T17:43:48.346 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:43:48.342+0000 7f2c38727700 1 --2- 172.21.15.181:0/2440346083 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f2c1c0a1500 0x7f2c1c0a39c0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:43:48.346 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:43:48.342+0000 7f2c38727700 1 --2- 172.21.15.181:0/2440346083 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c30101cd0 0x7f2c301030d0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:43:48.346 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:43:48.342+0000 7f2c38727700 1 --2- 172.21.15.181:0/2440346083 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c301007a0 0x7f2c301044a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:43:48.346 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:43:48.342+0000 7f2c38727700 1 --2- 172.21.15.181:0/2440346083 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c301011b0 0x7f2c30102b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:43:48.346 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:43:48.342+0000 7f2c38727700 1 -- 172.21.15.181:0/2440346083 >> 172.21.15.181:0/2440346083 conn(0x7f2c300fb270 msgr2=0x7f2c300fcd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:43:48.346 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:43:48.343+0000 7f2c38727700 1 -- 172.21.15.181:0/2440346083 shutdown_connections 2023-12-11T17:43:48.346 INFO:teuthology.orchestra.run.smithi181.stderr:2023-12-11T17:43:48.343+0000 7f2c38727700 1 -- 172.21.15.181:0/2440346083 wait complete. 2023-12-11T17:43:48.661 DEBUG:teuthology.orchestra.run.smithi059:mds.a> sudo journalctl -f -n 0 -u ceph-d0a07eaa-984b-11ee-95a3-87774f69a715@mds.a.service 2023-12-11T17:43:48.664 DEBUG:teuthology.orchestra.run.smithi059:mds.d> sudo journalctl -f -n 0 -u ceph-d0a07eaa-984b-11ee-95a3-87774f69a715@mds.d.service 2023-12-11T17:43:48.702 INFO:journalctl@ceph.mds.a.smithi059.stdout:-- Logs begin at Mon 2023-12-11 17:22:26 UTC. -- 2023-12-11T17:43:48.703 DEBUG:teuthology.orchestra.run.smithi059:mds.g> sudo journalctl -f -n 0 -u ceph-d0a07eaa-984b-11ee-95a3-87774f69a715@mds.g.service 2023-12-11T17:43:48.733 INFO:journalctl@ceph.mds.d.smithi059.stdout:-- Logs begin at Mon 2023-12-11 17:22:26 UTC. -- 2023-12-11T17:43:48.735 DEBUG:teuthology.orchestra.run.smithi059:mds.j> sudo journalctl -f -n 0 -u ceph-d0a07eaa-984b-11ee-95a3-87774f69a715@mds.j.service 2023-12-11T17:43:48.762 INFO:journalctl@ceph.mds.g.smithi059.stdout:-- Logs begin at Mon 2023-12-11 17:22:26 UTC. -- 2023-12-11T17:43:48.763 DEBUG:teuthology.orchestra.run.smithi115:mds.b> sudo journalctl -f -n 0 -u ceph-d0a07eaa-984b-11ee-95a3-87774f69a715@mds.b.service 2023-12-11T17:43:48.766 DEBUG:teuthology.orchestra.run.smithi115:mds.e> sudo journalctl -f -n 0 -u ceph-d0a07eaa-984b-11ee-95a3-87774f69a715@mds.e.service 2023-12-11T17:43:48.791 INFO:journalctl@ceph.mds.j.smithi059.stdout:-- Logs begin at Mon 2023-12-11 17:22:26 UTC. -- 2023-12-11T17:43:48.803 INFO:journalctl@ceph.mds.b.smithi115.stdout:-- Logs begin at Mon 2023-12-11 17:22:43 UTC. -- 2023-12-11T17:43:48.805 DEBUG:teuthology.orchestra.run.smithi115:mds.h> sudo journalctl -f -n 0 -u ceph-d0a07eaa-984b-11ee-95a3-87774f69a715@mds.h.service 2023-12-11T17:43:48.833 INFO:journalctl@ceph.mds.e.smithi115.stdout:-- Logs begin at Mon 2023-12-11 17:22:43 UTC. -- 2023-12-11T17:43:48.834 DEBUG:teuthology.orchestra.run.smithi115:mds.k> sudo journalctl -f -n 0 -u ceph-d0a07eaa-984b-11ee-95a3-87774f69a715@mds.k.service 2023-12-11T17:43:48.863 INFO:journalctl@ceph.mds.h.smithi115.stdout:-- Logs begin at Mon 2023-12-11 17:22:43 UTC. -- 2023-12-11T17:43:48.864 DEBUG:teuthology.orchestra.run.smithi181:mds.c> sudo journalctl -f -n 0 -u ceph-d0a07eaa-984b-11ee-95a3-87774f69a715@mds.c.service 2023-12-11T17:43:48.866 DEBUG:teuthology.orchestra.run.smithi181:mds.f> sudo journalctl -f -n 0 -u ceph-d0a07eaa-984b-11ee-95a3-87774f69a715@mds.f.service 2023-12-11T17:43:48.891 INFO:journalctl@ceph.mds.k.smithi115.stdout:-- Logs begin at Mon 2023-12-11 17:22:43 UTC. -- 2023-12-11T17:43:48.895 INFO:journalctl@ceph.mds.c.smithi181.stdout:-- Logs begin at Mon 2023-12-11 17:22:43 UTC. -- 2023-12-11T17:43:48.896 DEBUG:teuthology.orchestra.run.smithi181:mds.i> sudo journalctl -f -n 0 -u ceph-d0a07eaa-984b-11ee-95a3-87774f69a715@mds.i.service 2023-12-11T17:43:48.921 INFO:journalctl@ceph.mds.f.smithi181.stdout:-- Logs begin at Mon 2023-12-11 17:22:43 UTC. -- 2023-12-11T17:43:48.922 DEBUG:teuthology.orchestra.run.smithi181:mds.l> sudo journalctl -f -n 0 -u ceph-d0a07eaa-984b-11ee-95a3-87774f69a715@mds.l.service 2023-12-11T17:43:48.947 INFO:journalctl@ceph.mds.i.smithi181.stdout:-- Logs begin at Mon 2023-12-11 17:22:43 UTC. -- 2023-12-11T17:43:48.948 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2023-12-11T17:43:48.949 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-11T17:43:48.975 INFO:journalctl@ceph.mds.l.smithi181.stdout:-- Logs begin at Mon 2023-12-11 17:22:43 UTC. -- 2023-12-11T17:43:49.098 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.093+0000 7f610625d700 1 -- 172.21.15.59:0/3651707192 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61001014b0 msgr2=0x7f61001018d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:43:49.098 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.093+0000 7f610625d700 1 --2- 172.21.15.59:0/3651707192 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61001014b0 0x7f61001018d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f60f00099c0 tx=0x7f60f0009cd0 comp rx=0 tx=0).stop 2023-12-11T17:43:49.098 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.093+0000 7f610625d700 1 -- 172.21.15.59:0/3651707192 shutdown_connections 2023-12-11T17:43:49.098 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.093+0000 7f610625d700 1 --2- 172.21.15.59:0/3651707192 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6100103040 0x7f61001098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:43:49.098 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.093+0000 7f610625d700 1 --2- 172.21.15.59:0/3651707192 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61001014b0 0x7f61001018d0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:43:49.098 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.093+0000 7f610625d700 1 --2- 172.21.15.59:0/3651707192 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f61001026a0 0x7f6100102b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:43:49.099 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.093+0000 7f610625d700 1 -- 172.21.15.59:0/3651707192 >> 172.21.15.59:0/3651707192 conn(0x7f61000fca20 msgr2=0x7f61000fee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:43:49.099 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.093+0000 7f610625d700 1 -- 172.21.15.59:0/3651707192 shutdown_connections 2023-12-11T17:43:49.099 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.093+0000 7f610625d700 1 -- 172.21.15.59:0/3651707192 wait complete. 2023-12-11T17:43:49.099 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.094+0000 7f610625d700 1 Processor -- start 2023-12-11T17:43:49.099 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.094+0000 7f610625d700 1 -- start start 2023-12-11T17:43:49.099 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.094+0000 7f610625d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61001014b0 0x7f6100197480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:43:49.099 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.094+0000 7f610625d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61001026a0 0x7f61001979c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:43:49.099 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.094+0000 7f610625d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6100103040 0x7f610019e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:43:49.099 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.094+0000 7f610625d700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f610019efb0 con 0x7f61001014b0 2023-12-11T17:43:49.099 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.094+0000 7f610625d700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f610019f0f0 con 0x7f6100103040 2023-12-11T17:43:49.099 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.094+0000 7f610625d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f610019f280 con 0x7f61001026a0 2023-12-11T17:43:49.099 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.094+0000 7f60ff7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61001014b0 0x7f6100197480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:43:49.099 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.094+0000 7f60ff7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61001014b0 0x7f6100197480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60764/0 (socket says 172.21.15.59:60764) 2023-12-11T17:43:49.100 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.094+0000 7f60ff7fe700 1 -- 172.21.15.59:0/2650392013 learned_addr learned my addr 172.21.15.59:0/2650392013 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:43:49.100 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.094+0000 7f60feffd700 1 --2- 172.21.15.59:0/2650392013 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61001026a0 0x7f61001979c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:43:49.100 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.094+0000 7f60fffff700 1 --2- 172.21.15.59:0/2650392013 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6100103040 0x7f610019e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:43:49.100 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.094+0000 7f60ff7fe700 1 -- 172.21.15.59:0/2650392013 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61001026a0 msgr2=0x7f61001979c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:43:49.100 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.094+0000 7f60ff7fe700 1 --2- 172.21.15.59:0/2650392013 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61001026a0 0x7f61001979c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:43:49.100 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.094+0000 7f60ff7fe700 1 -- 172.21.15.59:0/2650392013 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6100103040 msgr2=0x7f610019e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:43:49.100 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.094+0000 7f60ff7fe700 1 --2- 172.21.15.59:0/2650392013 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6100103040 0x7f610019e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:43:49.100 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.094+0000 7f60ff7fe700 1 -- 172.21.15.59:0/2650392013 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60f00095f0 con 0x7f61001014b0 2023-12-11T17:43:49.100 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.094+0000 7f60ff7fe700 1 --2- 172.21.15.59:0/2650392013 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61001014b0 0x7f6100197480 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f60f0003730 tx=0x7f60f000fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:43:49.100 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.095+0000 7f60fcff9700 1 -- 172.21.15.59:0/2650392013 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60f001d070 con 0x7f61001014b0 2023-12-11T17:43:49.100 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.095+0000 7f610625d700 1 -- 172.21.15.59:0/2650392013 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f610019f500 con 0x7f61001014b0 2023-12-11T17:43:49.100 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.095+0000 7f60fcff9700 1 -- 172.21.15.59:0/2650392013 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f60f000bda0 con 0x7f61001014b0 2023-12-11T17:43:49.100 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.095+0000 7f60fcff9700 1 -- 172.21.15.59:0/2650392013 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60f00173f0 con 0x7f61001014b0 2023-12-11T17:43:49.101 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.095+0000 7f610625d700 1 -- 172.21.15.59:0/2650392013 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6100073800 con 0x7f61001014b0 2023-12-11T17:43:49.101 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.096+0000 7f610625d700 1 -- 172.21.15.59:0/2650392013 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6100108980 con 0x7f61001014b0 2023-12-11T17:43:49.101 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.097+0000 7f60fcff9700 1 -- 172.21.15.59:0/2650392013 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f60f0022470 con 0x7f61001014b0 2023-12-11T17:43:49.101 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.097+0000 7f60fcff9700 1 --2- 172.21.15.59:0/2650392013 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f60e80a5890 0x7f60e80a7d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:43:49.101 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.097+0000 7f60fcff9700 1 -- 172.21.15.59:0/2650392013 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f60f00cd8e0 con 0x7f61001014b0 2023-12-11T17:43:49.101 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.097+0000 7f60feffd700 1 --2- 172.21.15.59:0/2650392013 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f60e80a5890 0x7f60e80a7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:43:49.102 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.098+0000 7f60feffd700 1 --2- 172.21.15.59:0/2650392013 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f60e80a5890 0x7f60e80a7d50 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f60f4005950 tx=0x7f60f400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:43:49.104 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.102+0000 7f60fcff9700 1 -- 172.21.15.59:0/2650392013 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f60f0097470 con 0x7f61001014b0 2023-12-11T17:43:49.311 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.308+0000 7f610625d700 1 -- 172.21.15.59:0/2650392013 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f610010cd10 con 0x7f61001014b0 2023-12-11T17:43:49.312 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.309+0000 7f60fcff9700 1 -- 172.21.15.59:0/2650392013 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) v1 ==== 74+0+18422 (secure 0 0 0) 0x7f60f009aa90 con 0x7f61001014b0 2023-12-11T17:43:49.312 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-12-11T17:43:49.313 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":78,"fsid":"d0a07eaa-984b-11ee-95a3-87774f69a715","created":"2023-12-11T17:37:14.526539+0000","modified":"2023-12-11T17:43:39.541636+0000","last_up_change":"2023-12-11T17:43:38.535503+0000","last_in_change":"2023-12-11T17:43:21.014888+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-11T17:40:26.596029+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3c0b3218-4b2b-40f8-8bef-ed8cd5633607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":1040759398},{"type":"v1","addr":"172.21.15.59:6803","nonce":1040759398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":1040759398},{"type":"v1","addr":"172.21.15.59:6805","nonce":1040759398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":1040759398},{"type":"v1","addr":"172.21.15.59:6809","nonce":1040759398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":1040759398},{"type":"v1","addr":"172.21.15.59:6807","nonce":1040759398}]},"public_addr":"172.21.15.59:6803/1040759398","cluster_addr":"172.21.15.59:6805/1040759398","heartbeat_back_addr":"172.21.15.59:6809/1040759398","heartbeat_front_addr":"172.21.15.59:6807/1040759398","state":["exists","up"]},{"osd":1,"uuid":"2ead7578-765b-4f5a-a72c-863731ed69e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":2776354245},{"type":"v1","addr":"172.21.15.115:6801","nonce":2776354245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":2776354245},{"type":"v1","addr":"172.21.15.115:6803","nonce":2776354245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":2776354245},{"type":"v1","addr":"172.21.15.115:6807","nonce":2776354245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":2776354245},{"type":"v1","addr":"172.21.15.115:6805","nonce":2776354245}]},"public_addr":"172.21.15.115:6801/2776354245","cluster_addr":"172.21.15.115:6803/2776354245","heartbeat_back_addr":"172.21.15.115:6807/2776354245","heartbeat_front_addr":"172.21.15.115:6805/2776354245","state":["exists","up"]},{"osd":2,"uuid":"8cb1d7ec-53cc-4672-b0b6-70d73797c44f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4287675481},{"type":"v1","addr":"172.21.15.181:6801","nonce":4287675481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4287675481},{"type":"v1","addr":"172.21.15.181:6803","nonce":4287675481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4287675481},{"type":"v1","addr":"172.21.15.181:6807","nonce":4287675481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4287675481},{"type":"v1","addr":"172.21.15.181:6805","nonce":4287675481}]},"public_addr":"172.21.15.181:6801/4287675481","cluster_addr":"172.21.15.181:6803/4287675481","heartbeat_back_addr":"172.21.15.181:6807/4287675481","heartbeat_front_addr":"172.21.15.181:6805/4287675481","state":["exists","up"]},{"osd":3,"uuid":"5d64d9af-831b-4d0a-8ace-a452169351d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":219909995},{"type":"v1","addr":"172.21.15.59:6811","nonce":219909995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":219909995},{"type":"v1","addr":"172.21.15.59:6813","nonce":219909995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":219909995},{"type":"v1","addr":"172.21.15.59:6817","nonce":219909995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":219909995},{"type":"v1","addr":"172.21.15.59:6815","nonce":219909995}]},"public_addr":"172.21.15.59:6811/219909995","cluster_addr":"172.21.15.59:6813/219909995","heartbeat_back_addr":"172.21.15.59:6817/219909995","heartbeat_front_addr":"172.21.15.59:6815/219909995","state":["exists","up"]},{"osd":4,"uuid":"ef8fad28-1db5-4cba-b5f2-10e54e8e1549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":1021473826},{"type":"v1","addr":"172.21.15.115:6809","nonce":1021473826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":1021473826},{"type":"v1","addr":"172.21.15.115:6811","nonce":1021473826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":1021473826},{"type":"v1","addr":"172.21.15.115:6815","nonce":1021473826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":1021473826},{"type":"v1","addr":"172.21.15.115:6813","nonce":1021473826}]},"public_addr":"172.21.15.115:6809/1021473826","cluster_addr":"172.21.15.115:6811/1021473826","heartbeat_back_addr":"172.21.15.115:6815/1021473826","heartbeat_front_addr":"172.21.15.115:6813/1021473826","state":["exists","up"]},{"osd":5,"uuid":"4233a512-9e8f-4db8-aed2-6c3af706e589","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":2405172521},{"type":"v1","addr":"172.21.15.181:6809","nonce":2405172521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":2405172521},{"type":"v1","addr":"172.21.15.181:6811","nonce":2405172521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2405172521},{"type":"v1","addr":"172.21.15.181:6815","nonce":2405172521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":2405172521},{"type":"v1","addr":"172.21.15.181:6813","nonce":2405172521}]},"public_addr":"172.21.15.181:6809/2405172521","cluster_addr":"172.21.15.181:6811/2405172521","heartbeat_back_addr":"172.21.15.181:6815/2405172521","heartbeat_front_addr":"172.21.15.181:6813/2405172521","state":["exists","up"]},{"osd":6,"uuid":"8faab332-016e-4c77-8370-8bef95fc6f34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":815374853},{"type":"v1","addr":"172.21.15.59:6819","nonce":815374853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":815374853},{"type":"v1","addr":"172.21.15.59:6821","nonce":815374853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":815374853},{"type":"v1","addr":"172.21.15.59:6825","nonce":815374853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":815374853},{"type":"v1","addr":"172.21.15.59:6823","nonce":815374853}]},"public_addr":"172.21.15.59:6819/815374853","cluster_addr":"172.21.15.59:6821/815374853","heartbeat_back_addr":"172.21.15.59:6825/815374853","heartbeat_front_addr":"172.21.15.59:6823/815374853","state":["exists","up"]},{"osd":7,"uuid":"f20f470a-0d40-4ed3-8e0e-fd58675cf143","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":1314971870},{"type":"v1","addr":"172.21.15.115:6817","nonce":1314971870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":1314971870},{"type":"v1","addr":"172.21.15.115:6819","nonce":1314971870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":1314971870},{"type":"v1","addr":"172.21.15.115:6823","nonce":1314971870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":1314971870},{"type":"v1","addr":"172.21.15.115:6821","nonce":1314971870}]},"public_addr":"172.21.15.115:6817/1314971870","cluster_addr":"172.21.15.115:6819/1314971870","heartbeat_back_addr":"172.21.15.115:6823/1314971870","heartbeat_front_addr":"172.21.15.115:6821/1314971870","state":["exists","up"]},{"osd":8,"uuid":"eb657d32-1a95-4fd7-a293-d0a970837fc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":3954509873},{"type":"v1","addr":"172.21.15.181:6817","nonce":3954509873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":3954509873},{"type":"v1","addr":"172.21.15.181:6819","nonce":3954509873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":3954509873},{"type":"v1","addr":"172.21.15.181:6823","nonce":3954509873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":3954509873},{"type":"v1","addr":"172.21.15.181:6821","nonce":3954509873}]},"public_addr":"172.21.15.181:6817/3954509873","cluster_addr":"172.21.15.181:6819/3954509873","heartbeat_back_addr":"172.21.15.181:6823/3954509873","heartbeat_front_addr":"172.21.15.181:6821/3954509873","state":["exists","up"]},{"osd":9,"uuid":"8f7717d4-b6f2-4ec1-9b58-62feb0c9a146","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":1163095677},{"type":"v1","addr":"172.21.15.59:6827","nonce":1163095677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":1163095677},{"type":"v1","addr":"172.21.15.59:6829","nonce":1163095677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":1163095677},{"type":"v1","addr":"172.21.15.59:6833","nonce":1163095677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":1163095677},{"type":"v1","addr":"172.21.15.59:6831","nonce":1163095677}]},"public_addr":"172.21.15.59:6827/1163095677","cluster_addr":"172.21.15.59:6829/1163095677","heartbeat_back_addr":"172.21.15.59:6833/1163095677","heartbeat_front_addr":"172.21.15.59:6831/1163095677","state":["exists","up"]},{"osd":10,"uuid":"b75c179a-875d-43bc-b4c1-e8eb80fb64fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":2199610269},{"type":"v1","addr":"172.21.15.115:6825","nonce":2199610269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":2199610269},{"type":"v1","addr":"172.21.15.115:6827","nonce":2199610269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":2199610269},{"type":"v1","addr":"172.21.15.115:6831","nonce":2199610269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":2199610269},{"type":"v1","addr":"172.21.15.115:6829","nonce":2199610269}]},"public_addr":"172.21.15.115:6825/2199610269","cluster_addr":"172.21.15.115:6827/2199610269","heartbeat_back_addr":"172.21.15.115:6831/2199610269","heartbeat_front_addr":"172.21.15.115:6829/2199610269","state":["exists","up"]},{"osd":11,"uuid":"5e141dfe-8a00-4bbc-8204-2dc9be24e18d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":226506989},{"type":"v1","addr":"172.21.15.181:6825","nonce":226506989}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":226506989},{"type":"v1","addr":"172.21.15.181:6827","nonce":226506989}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":226506989},{"type":"v1","addr":"172.21.15.181:6831","nonce":226506989}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":226506989},{"type":"v1","addr":"172.21.15.181:6829","nonce":226506989}]},"public_addr":"172.21.15.181:6825/226506989","cluster_addr":"172.21.15.181:6827/226506989","heartbeat_back_addr":"172.21.15.181:6831/226506989","heartbeat_front_addr":"172.21.15.181:6829/226506989","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:39:42.687084+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:40:03.903448+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:40:24.899908+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:40:46.600251+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:41:07.690037+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:41:28.488855+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:41:49.459274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:42:10.863881+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:42:32.248391+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:42:54.112265+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:43:15.634872+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:43:36.922534+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.59:6800/1259916482":"2023-12-12T17:38:14.037922+0000","172.21.15.59:0/17457774":"2023-12-12T17:38:14.037922+0000","172.21.15.59:6801/1672530462":"2023-12-12T17:37:44.773925+0000","172.21.15.59:0/1053944547":"2023-12-12T17:38:14.037922+0000","172.21.15.59:6801/1259916482":"2023-12-12T17:38:14.037922+0000","172.21.15.59:6800/1672530462":"2023-12-12T17:37:44.773925+0000","172.21.15.59:0/3570767650":"2023-12-12T17:37:44.773925+0000","172.21.15.59:0/3853462899":"2023-12-12T17:37:44.773925+0000","172.21.15.59:0/2859415380":"2023-12-12T17:38:14.037922+0000","172.21.15.59:0/2192742819":"2023-12-12T17:37:44.773925+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-11T17:43:49.314 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.311+0000 7f610625d700 1 -- 172.21.15.59:0/2650392013 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f60e80a5890 msgr2=0x7f60e80a7d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:43:49.314 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.311+0000 7f610625d700 1 --2- 172.21.15.59:0/2650392013 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f60e80a5890 0x7f60e80a7d50 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f60f4005950 tx=0x7f60f400b410 comp rx=0 tx=0).stop 2023-12-11T17:43:49.314 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.311+0000 7f610625d700 1 -- 172.21.15.59:0/2650392013 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61001014b0 msgr2=0x7f6100197480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:43:49.314 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.311+0000 7f610625d700 1 --2- 172.21.15.59:0/2650392013 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61001014b0 0x7f6100197480 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f60f0003730 tx=0x7f60f000fa00 comp rx=0 tx=0).stop 2023-12-11T17:43:49.314 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.311+0000 7f610625d700 1 -- 172.21.15.59:0/2650392013 shutdown_connections 2023-12-11T17:43:49.314 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.311+0000 7f610625d700 1 --2- 172.21.15.59:0/2650392013 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f60e80a5890 0x7f60e80a7d50 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:43:49.315 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.311+0000 7f610625d700 1 --2- 172.21.15.59:0/2650392013 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6100103040 0x7f610019e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:43:49.315 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.311+0000 7f610625d700 1 --2- 172.21.15.59:0/2650392013 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61001014b0 0x7f6100197480 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:43:49.315 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.311+0000 7f610625d700 1 --2- 172.21.15.59:0/2650392013 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61001026a0 0x7f61001979c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:43:49.315 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.311+0000 7f610625d700 1 -- 172.21.15.59:0/2650392013 >> 172.21.15.59:0/2650392013 conn(0x7f61000fca20 msgr2=0x7f6100105ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:43:49.315 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.312+0000 7f610625d700 1 -- 172.21.15.59:0/2650392013 shutdown_connections 2023-12-11T17:43:49.315 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.312+0000 7f610625d700 1 -- 172.21.15.59:0/2650392013 wait complete. 2023-12-11T17:43:49.327 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-11T17:40:26.596029+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-11T17:43:49.328 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-11T17:43:49.458 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.455+0000 7fa0f8cb4700 1 -- 172.21.15.59:0/2865921062 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0f4102790 msgr2=0x7fa0f4102bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:43:49.458 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.455+0000 7fa0f8cb4700 1 --2- 172.21.15.59:0/2865921062 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0f4102790 0x7fa0f4102bf0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa0e40099c0 tx=0x7fa0e4009cd0 comp rx=0 tx=0).stop 2023-12-11T17:43:49.458 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.455+0000 7fa0f8cb4700 1 -- 172.21.15.59:0/2865921062 shutdown_connections 2023-12-11T17:43:49.458 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.455+0000 7fa0f8cb4700 1 --2- 172.21.15.59:0/2865921062 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa0f4103130 0x7fa0f41099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:43:49.458 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.455+0000 7fa0f8cb4700 1 --2- 172.21.15.59:0/2865921062 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa0f41015a0 0x7fa0f41019c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:43:49.458 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.455+0000 7fa0f8cb4700 1 --2- 172.21.15.59:0/2865921062 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0f4102790 0x7fa0f4102bf0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:43:49.458 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.455+0000 7fa0f8cb4700 1 -- 172.21.15.59:0/2865921062 >> 172.21.15.59:0/2865921062 conn(0x7fa0f40fcb10 msgr2=0x7fa0f40fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:43:49.459 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.455+0000 7fa0f8cb4700 1 -- 172.21.15.59:0/2865921062 shutdown_connections 2023-12-11T17:43:49.459 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.455+0000 7fa0f8cb4700 1 -- 172.21.15.59:0/2865921062 wait complete. 2023-12-11T17:43:49.461 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.456+0000 7fa0f8cb4700 1 Processor -- start 2023-12-11T17:43:49.461 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.456+0000 7fa0f8cb4700 1 -- start start 2023-12-11T17:43:49.461 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.457+0000 7fa0f8cb4700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0f41015a0 0x7fa0f41975f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:43:49.461 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.457+0000 7fa0f8cb4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa0f4102790 0x7fa0f4197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:43:49.461 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.457+0000 7fa0f8cb4700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa0f4103130 0x7fa0f419eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:43:49.461 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.457+0000 7fa0f8cb4700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0f419f120 con 0x7fa0f41015a0 2023-12-11T17:43:49.461 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.457+0000 7fa0f8cb4700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0f419f260 con 0x7fa0f4103130 2023-12-11T17:43:49.461 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.457+0000 7fa0f8cb4700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0f419f3f0 con 0x7fa0f4102790 2023-12-11T17:43:49.461 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.457+0000 7fa0f259c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0f41015a0 0x7fa0f41975f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:43:49.461 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.457+0000 7fa0f259c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0f41015a0 0x7fa0f41975f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60778/0 (socket says 172.21.15.59:60778) 2023-12-11T17:43:49.461 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.457+0000 7fa0f259c700 1 -- 172.21.15.59:0/367704681 learned_addr learned my addr 172.21.15.59:0/367704681 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:43:49.462 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.457+0000 7fa0f1d9b700 1 --2- 172.21.15.59:0/367704681 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa0f4102790 0x7fa0f4197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:43:49.462 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.457+0000 7fa0f2d9d700 1 --2- 172.21.15.59:0/367704681 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa0f4103130 0x7fa0f419eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:43:49.462 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.457+0000 7fa0f259c700 1 -- 172.21.15.59:0/367704681 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa0f4102790 msgr2=0x7fa0f4197b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:43:49.462 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.457+0000 7fa0f259c700 1 --2- 172.21.15.59:0/367704681 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa0f4102790 0x7fa0f4197b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:43:49.462 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.457+0000 7fa0f259c700 1 -- 172.21.15.59:0/367704681 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa0f4103130 msgr2=0x7fa0f419eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:43:49.462 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.457+0000 7fa0f259c700 1 --2- 172.21.15.59:0/367704681 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa0f4103130 0x7fa0f419eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:43:49.462 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.457+0000 7fa0f259c700 1 -- 172.21.15.59:0/367704681 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0e40095f0 con 0x7fa0f41015a0 2023-12-11T17:43:49.462 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.457+0000 7fa0f259c700 1 --2- 172.21.15.59:0/367704681 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0f41015a0 0x7fa0f41975f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa0dc00b500 tx=0x7fa0dc00b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:43:49.462 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.457+0000 7fa0eb7fe700 1 -- 172.21.15.59:0/367704681 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0dc011830 con 0x7fa0f41015a0 2023-12-11T17:43:49.462 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.457+0000 7fa0eb7fe700 1 -- 172.21.15.59:0/367704681 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa0dc011e50 con 0x7fa0f41015a0 2023-12-11T17:43:49.462 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.457+0000 7fa0f8cb4700 1 -- 172.21.15.59:0/367704681 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0f419f6d0 con 0x7fa0f41015a0 2023-12-11T17:43:49.462 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.457+0000 7fa0eb7fe700 1 -- 172.21.15.59:0/367704681 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0dc00f2e0 con 0x7fa0f41015a0 2023-12-11T17:43:49.462 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.458+0000 7fa0f8cb4700 1 -- 172.21.15.59:0/367704681 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0f419fbe0 con 0x7fa0f41015a0 2023-12-11T17:43:49.462 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.459+0000 7fa0f8cb4700 1 -- 172.21.15.59:0/367704681 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0f4108a70 con 0x7fa0f41015a0 2023-12-11T17:43:49.463 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.459+0000 7fa0eb7fe700 1 -- 172.21.15.59:0/367704681 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7fa0dc010580 con 0x7fa0f41015a0 2023-12-11T17:43:49.463 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.460+0000 7fa0eb7fe700 1 --2- 172.21.15.59:0/367704681 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fa0e00a1480 0x7fa0e00a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:43:49.463 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.460+0000 7fa0f1d9b700 1 --2- 172.21.15.59:0/367704681 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fa0e00a1480 0x7fa0e00a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:43:49.463 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.460+0000 7fa0eb7fe700 1 -- 172.21.15.59:0/367704681 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fa0dc0cc2e0 con 0x7fa0f41015a0 2023-12-11T17:43:49.463 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.461+0000 7fa0f1d9b700 1 --2- 172.21.15.59:0/367704681 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fa0e00a1480 0x7fa0e00a3940 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa0e4009990 tx=0x7fa0e4005ee0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:43:49.467 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.465+0000 7fa0eb7fe700 1 -- 172.21.15.59:0/367704681 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fa0dc095de0 con 0x7fa0f41015a0 2023-12-11T17:43:49.540 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:49 smithi181 ceph-mon[103627]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T17:43:49.541 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:49 smithi181 ceph-mon[103627]: from='client.24400 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi059=a;smithi059=d;smithi059=g;smithi059=j;smithi115=b;smithi115=e;smithi115=h;smithi115=k;smithi181=c;smithi181=f;smithi181=i;smithi181=l", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T17:43:49.541 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:49 smithi181 ceph-mon[103627]: Saving service mds.all spec with placement smithi059=a;smithi059=d;smithi059=g;smithi059=j;smithi115=b;smithi115=e;smithi115=h;smithi115=k;smithi181=c;smithi181=f;smithi181=i;smithi181=l;count:12 2023-12-11T17:43:49.541 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:49 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:43:49.541 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:49 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:43:49.541 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:49 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:43:49.541 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:49 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:43:49.541 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:49 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:43:49.541 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:49 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-11T17:43:49.541 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:49 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-11T17:43:49.541 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:49 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:43:49.541 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:49 smithi181 ceph-mon[103627]: Deploying daemon mds.l on smithi181 2023-12-11T17:43:49.541 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:49 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/2650392013' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-11T17:43:49.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:49 smithi059 ceph-mon[98847]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T17:43:49.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:49 smithi059 ceph-mon[98847]: from='client.24400 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi059=a;smithi059=d;smithi059=g;smithi059=j;smithi115=b;smithi115=e;smithi115=h;smithi115=k;smithi181=c;smithi181=f;smithi181=i;smithi181=l", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T17:43:49.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:49 smithi059 ceph-mon[98847]: Saving service mds.all spec with placement smithi059=a;smithi059=d;smithi059=g;smithi059=j;smithi115=b;smithi115=e;smithi115=h;smithi115=k;smithi181=c;smithi181=f;smithi181=i;smithi181=l;count:12 2023-12-11T17:43:49.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:49 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:43:49.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:49 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:43:49.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:49 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:43:49.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:49 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:43:49.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:49 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:43:49.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:49 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-11T17:43:49.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:49 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-11T17:43:49.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:49 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:43:49.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:49 smithi059 ceph-mon[98847]: Deploying daemon mds.l on smithi181 2023-12-11T17:43:49.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:49 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/2650392013' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-11T17:43:49.667 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.665+0000 7fa0f8cb4700 1 -- 172.21.15.59:0/367704681 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa0f410ced0 con 0x7fa0f41015a0 2023-12-11T17:43:49.668 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.665+0000 7fa0eb7fe700 1 -- 172.21.15.59:0/367704681 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v78) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa0dc099400 con 0x7fa0f41015a0 2023-12-11T17:43:49.668 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2023-12-11T17:43:49.670 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.667+0000 7fa0f8cb4700 1 -- 172.21.15.59:0/367704681 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fa0e00a1480 msgr2=0x7fa0e00a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:43:49.670 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.667+0000 7fa0f8cb4700 1 --2- 172.21.15.59:0/367704681 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fa0e00a1480 0x7fa0e00a3940 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa0e4009990 tx=0x7fa0e4005ee0 comp rx=0 tx=0).stop 2023-12-11T17:43:49.670 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.667+0000 7fa0f8cb4700 1 -- 172.21.15.59:0/367704681 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0f41015a0 msgr2=0x7fa0f41975f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:43:49.670 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.667+0000 7fa0f8cb4700 1 --2- 172.21.15.59:0/367704681 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0f41015a0 0x7fa0f41975f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa0dc00b500 tx=0x7fa0dc00b810 comp rx=0 tx=0).stop 2023-12-11T17:43:49.671 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.668+0000 7fa0f8cb4700 1 -- 172.21.15.59:0/367704681 shutdown_connections 2023-12-11T17:43:49.671 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.668+0000 7fa0f8cb4700 1 --2- 172.21.15.59:0/367704681 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fa0e00a1480 0x7fa0e00a3940 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:43:49.671 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.668+0000 7fa0f8cb4700 1 --2- 172.21.15.59:0/367704681 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa0f4103130 0x7fa0f419eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:43:49.671 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.668+0000 7fa0f8cb4700 1 --2- 172.21.15.59:0/367704681 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0f41015a0 0x7fa0f41975f0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:43:49.671 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.668+0000 7fa0f8cb4700 1 --2- 172.21.15.59:0/367704681 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa0f4102790 0x7fa0f4197b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:43:49.671 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.668+0000 7fa0f8cb4700 1 -- 172.21.15.59:0/367704681 >> 172.21.15.59:0/367704681 conn(0x7fa0f40fcb10 msgr2=0x7fa0f4105bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:43:49.671 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.668+0000 7fa0f8cb4700 1 -- 172.21.15.59:0/367704681 shutdown_connections 2023-12-11T17:43:49.671 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.668+0000 7fa0f8cb4700 1 -- 172.21.15.59:0/367704681 wait complete. 2023-12-11T17:43:49.683 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-11T17:43:49.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:49 smithi115 ceph-mon[102748]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T17:43:49.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:49 smithi115 ceph-mon[102748]: from='client.24400 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi059=a;smithi059=d;smithi059=g;smithi059=j;smithi115=b;smithi115=e;smithi115=h;smithi115=k;smithi181=c;smithi181=f;smithi181=i;smithi181=l", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T17:43:49.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:49 smithi115 ceph-mon[102748]: Saving service mds.all spec with placement smithi059=a;smithi059=d;smithi059=g;smithi059=j;smithi115=b;smithi115=e;smithi115=h;smithi115=k;smithi181=c;smithi181=f;smithi181=i;smithi181=l;count:12 2023-12-11T17:43:49.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:49 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:43:49.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:49 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:43:49.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:49 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:43:49.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:49 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:43:49.784 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:49 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:43:49.784 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:49 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-11T17:43:49.784 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:49 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-11T17:43:49.784 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:49 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:43:49.784 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:49 smithi115 ceph-mon[102748]: Deploying daemon mds.l on smithi181 2023-12-11T17:43:49.784 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:49 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/2650392013' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-11T17:43:49.812 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.807+0000 7f898ab5a700 1 -- 172.21.15.59:0/1241093588 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f89841014a0 msgr2=0x7f89841018c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:43:49.812 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.807+0000 7f898ab5a700 1 --2- 172.21.15.59:0/1241093588 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f89841014a0 0x7f89841018c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f89780099c0 tx=0x7f8978009cd0 comp rx=0 tx=0).stop 2023-12-11T17:43:49.812 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.807+0000 7f898ab5a700 1 -- 172.21.15.59:0/1241093588 shutdown_connections 2023-12-11T17:43:49.812 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.807+0000 7f898ab5a700 1 --2- 172.21.15.59:0/1241093588 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8984103030 0x7f89841098e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:43:49.812 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.807+0000 7f898ab5a700 1 --2- 172.21.15.59:0/1241093588 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f89841014a0 0x7f89841018c0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:43:49.813 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.807+0000 7f898ab5a700 1 --2- 172.21.15.59:0/1241093588 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8984102690 0x7f8984102af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:43:49.813 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.807+0000 7f898ab5a700 1 -- 172.21.15.59:0/1241093588 >> 172.21.15.59:0/1241093588 conn(0x7f89840fca10 msgr2=0x7f89840fee50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:43:49.813 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.807+0000 7f898ab5a700 1 -- 172.21.15.59:0/1241093588 shutdown_connections 2023-12-11T17:43:49.813 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.807+0000 7f898ab5a700 1 -- 172.21.15.59:0/1241093588 wait complete. 2023-12-11T17:43:49.813 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.808+0000 7f898ab5a700 1 Processor -- start 2023-12-11T17:43:49.813 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.808+0000 7f898ab5a700 1 -- start start 2023-12-11T17:43:49.813 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.808+0000 7f898ab5a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f89841014a0 0x7f8984197420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:43:49.813 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.808+0000 7f898ab5a700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8984102690 0x7f8984197960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:43:49.813 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.808+0000 7f898ab5a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8984103030 0x7f898419e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:43:49.813 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.808+0000 7f898ab5a700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f898419ef50 con 0x7f89841014a0 2023-12-11T17:43:49.813 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.808+0000 7f898ab5a700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f898419f090 con 0x7f8984102690 2023-12-11T17:43:49.813 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.808+0000 7f898ab5a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f898419f220 con 0x7f8984103030 2023-12-11T17:43:49.813 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.808+0000 7f89888f6700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f89841014a0 0x7f8984197420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:43:49.814 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.808+0000 7f89888f6700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f89841014a0 0x7f8984197420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60792/0 (socket says 172.21.15.59:60792) 2023-12-11T17:43:49.814 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.808+0000 7f89888f6700 1 -- 172.21.15.59:0/4083006132 learned_addr learned my addr 172.21.15.59:0/4083006132 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:43:49.814 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.808+0000 7f8983fff700 1 --2- 172.21.15.59:0/4083006132 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8984102690 0x7f8984197960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:43:49.814 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.808+0000 7f89890f7700 1 --2- 172.21.15.59:0/4083006132 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8984103030 0x7f898419e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:43:49.814 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.808+0000 7f89888f6700 1 -- 172.21.15.59:0/4083006132 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8984103030 msgr2=0x7f898419e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:43:49.814 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.808+0000 7f89888f6700 1 --2- 172.21.15.59:0/4083006132 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8984103030 0x7f898419e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:43:49.814 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.808+0000 7f89888f6700 1 -- 172.21.15.59:0/4083006132 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8984102690 msgr2=0x7f8984197960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:43:49.814 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.808+0000 7f89888f6700 1 --2- 172.21.15.59:0/4083006132 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8984102690 0x7f8984197960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:43:49.814 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.809+0000 7f89888f6700 1 -- 172.21.15.59:0/4083006132 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89780095f0 con 0x7f89841014a0 2023-12-11T17:43:49.814 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.809+0000 7f89888f6700 1 --2- 172.21.15.59:0/4083006132 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f89841014a0 0x7f8984197420 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8978009f50 tx=0x7f8978003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:43:49.814 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.809+0000 7f8981ffb700 1 -- 172.21.15.59:0/4083006132 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f897801d070 con 0x7f89841014a0 2023-12-11T17:43:49.814 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.809+0000 7f898ab5a700 1 -- 172.21.15.59:0/4083006132 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f898419f4a0 con 0x7f89841014a0 2023-12-11T17:43:49.814 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.809+0000 7f8981ffb700 1 -- 172.21.15.59:0/4083006132 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8978022470 con 0x7f89841014a0 2023-12-11T17:43:49.815 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.809+0000 7f8981ffb700 1 -- 172.21.15.59:0/4083006132 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8978022ea0 con 0x7f89841014a0 2023-12-11T17:43:49.815 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.809+0000 7f898ab5a700 1 -- 172.21.15.59:0/4083006132 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f898419f950 con 0x7f89841014a0 2023-12-11T17:43:49.815 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.810+0000 7f898ab5a700 1 -- 172.21.15.59:0/4083006132 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8984108920 con 0x7f89841014a0 2023-12-11T17:43:49.815 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.811+0000 7f8981ffb700 1 -- 172.21.15.59:0/4083006132 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f8978022800 con 0x7f89841014a0 2023-12-11T17:43:49.816 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.812+0000 7f8981ffb700 1 --2- 172.21.15.59:0/4083006132 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f89740a1480 0x7f89740a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:43:49.816 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.812+0000 7f8983fff700 1 --2- 172.21.15.59:0/4083006132 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f89740a1480 0x7f89740a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:43:49.816 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.812+0000 7f8981ffb700 1 -- 172.21.15.59:0/4083006132 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f89780ce150 con 0x7f89841014a0 2023-12-11T17:43:49.816 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.812+0000 7f8983fff700 1 --2- 172.21.15.59:0/4083006132 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f89740a1480 0x7f89740a3940 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8970006fd0 tx=0x7f8970008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:43:49.817 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:49.815+0000 7f8981ffb700 1 -- 172.21.15.59:0/4083006132 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f8978097ce0 con 0x7f89841014a0 2023-12-11T17:43:50.064 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:50.062+0000 7f898ab5a700 1 -- 172.21.15.59:0/4083006132 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8984068230 con 0x7f89841014a0 2023-12-11T17:43:50.064 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:50.062+0000 7f8981ffb700 1 -- 172.21.15.59:0/4083006132 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) v1 ==== 93+0+535 (secure 0 0 0) 0x7f897809b300 con 0x7f89841014a0 2023-12-11T17:43:50.065 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-12-11T17:43:50.065 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2023-12-11T17:43:50.067 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:50.064+0000 7f898ab5a700 1 -- 172.21.15.59:0/4083006132 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f89740a1480 msgr2=0x7f89740a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:43:50.067 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:50.064+0000 7f898ab5a700 1 --2- 172.21.15.59:0/4083006132 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f89740a1480 0x7f89740a3940 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8970006fd0 tx=0x7f8970008040 comp rx=0 tx=0).stop 2023-12-11T17:43:50.067 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:50.064+0000 7f898ab5a700 1 -- 172.21.15.59:0/4083006132 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f89841014a0 msgr2=0x7f8984197420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:43:50.067 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:50.064+0000 7f898ab5a700 1 --2- 172.21.15.59:0/4083006132 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f89841014a0 0x7f8984197420 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8978009f50 tx=0x7f8978003970 comp rx=0 tx=0).stop 2023-12-11T17:43:50.067 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:50.064+0000 7f898ab5a700 1 -- 172.21.15.59:0/4083006132 shutdown_connections 2023-12-11T17:43:50.067 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:50.064+0000 7f898ab5a700 1 --2- 172.21.15.59:0/4083006132 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f89740a1480 0x7f89740a3940 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:43:50.068 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:50.064+0000 7f898ab5a700 1 --2- 172.21.15.59:0/4083006132 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8984103030 0x7f898419e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:43:50.068 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:50.064+0000 7f898ab5a700 1 --2- 172.21.15.59:0/4083006132 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f89841014a0 0x7f8984197420 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:43:50.068 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:50.064+0000 7f898ab5a700 1 --2- 172.21.15.59:0/4083006132 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8984102690 0x7f8984197960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:43:50.068 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:50.064+0000 7f898ab5a700 1 -- 172.21.15.59:0/4083006132 >> 172.21.15.59:0/4083006132 conn(0x7f89840fca10 msgr2=0x7f8984105a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:43:50.068 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:50.064+0000 7f898ab5a700 1 -- 172.21.15.59:0/4083006132 shutdown_connections 2023-12-11T17:43:50.068 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:50.064+0000 7f898ab5a700 1 -- 172.21.15.59:0/4083006132 wait complete. 2023-12-11T17:43:50.068 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 1 2023-12-11T17:43:50.145 INFO:journalctl@ceph.mds.l.smithi181.stdout:Dec 11 17:43:49 smithi181 systemd[1]: Starting Ceph mds.l for d0a07eaa-984b-11ee-95a3-87774f69a715... 2023-12-11T17:43:50.482 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:50 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/367704681' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-11T17:43:50.482 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:50 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/4083006132' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T17:43:50.483 INFO:journalctl@ceph.mds.l.smithi181.stdout:Dec 11 17:43:50 smithi181 podman[140495]: 2023-12-11T17:43:50.483 INFO:journalctl@ceph.mds.l.smithi181.stdout:Dec 11 17:43:50 smithi181 bash[140495]: 256fa9247ebd56b149607c91238b749d03fc8a6bb0294dfadf3cd21bc8096c0a 2023-12-11T17:43:50.483 INFO:journalctl@ceph.mds.l.smithi181.stdout:Dec 11 17:43:50 smithi181 ceph-mds[140535]: -- 172.21.15.181:0/607460922 <== mon.1 v2:172.21.15.181:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x558f2fa19520 con 0x558f2f9ef000 2023-12-11T17:43:50.483 INFO:journalctl@ceph.mds.l.smithi181.stdout:Dec 11 17:43:50 smithi181 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mds-l[140511]: starting mds.l at 2023-12-11T17:43:50.483 INFO:journalctl@ceph.mds.l.smithi181.stdout:Dec 11 17:43:50 smithi181 systemd[1]: Started Ceph mds.l for d0a07eaa-984b-11ee-95a3-87774f69a715. 2023-12-11T17:43:50.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:50 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/367704681' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-11T17:43:50.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:50 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/4083006132' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T17:43:50.782 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:50 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/367704681' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-11T17:43:50.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:50 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/4083006132' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T17:43:51.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:51 smithi115 ceph-mon[102748]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T17:43:51.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:51 smithi115 ceph-mon[102748]: mds.? [v2:172.21.15.181:6832/395684042,v1:172.21.15.181:6833/395684042] up:boot 2023-12-11T17:43:51.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:51 smithi115 ceph-mon[102748]: fsmap 1 up:standby 2023-12-11T17:43:51.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:51 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-12-11T17:43:51.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:51 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:43:51.534 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:51 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:43:51.534 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:51 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:43:51.534 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:51 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-11T17:43:51.534 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:51 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-11T17:43:51.534 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:51 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:43:51.534 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:51 smithi115 ceph-mon[102748]: Deploying daemon mds.k on smithi115 2023-12-11T17:43:51.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:51 smithi181 ceph-mon[103627]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T17:43:51.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:51 smithi181 ceph-mon[103627]: mds.? [v2:172.21.15.181:6832/395684042,v1:172.21.15.181:6833/395684042] up:boot 2023-12-11T17:43:51.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:51 smithi181 ceph-mon[103627]: fsmap 1 up:standby 2023-12-11T17:43:51.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:51 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-12-11T17:43:51.896 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:51 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:43:51.896 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:51 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:43:51.896 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:51 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:43:51.896 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:51 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-11T17:43:51.896 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:51 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-11T17:43:51.896 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:51 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:43:51.896 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:51 smithi181 ceph-mon[103627]: Deploying daemon mds.k on smithi115 2023-12-11T17:43:51.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:51 smithi059 ceph-mon[98847]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T17:43:51.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:51 smithi059 ceph-mon[98847]: mds.? [v2:172.21.15.181:6832/395684042,v1:172.21.15.181:6833/395684042] up:boot 2023-12-11T17:43:51.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:51 smithi059 ceph-mon[98847]: fsmap 1 up:standby 2023-12-11T17:43:51.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:51 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-12-11T17:43:51.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:51 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:43:51.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:51 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:43:51.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:51 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:43:51.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:51 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-11T17:43:51.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:51 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-11T17:43:51.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:51 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:43:51.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:51 smithi059 ceph-mon[98847]: Deploying daemon mds.k on smithi115 2023-12-11T17:43:52.080 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-11T17:43:52.223 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:52.217+0000 7fb906cf4700 1 -- 172.21.15.59:0/4233046382 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb900100380 msgr2=0x7fb9001007a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:43:52.223 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:52.217+0000 7fb906cf4700 1 --2- 172.21.15.59:0/4233046382 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb900100380 0x7fb9001007a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb8f40099c0 tx=0x7fb8f4009cd0 comp rx=0 tx=0).stop 2023-12-11T17:43:52.223 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:52.217+0000 7fb906cf4700 1 -- 172.21.15.59:0/4233046382 shutdown_connections 2023-12-11T17:43:52.223 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:52.217+0000 7fb906cf4700 1 --2- 172.21.15.59:0/4233046382 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb900101f10 0x7fb9001087c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:43:52.223 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:52.217+0000 7fb906cf4700 1 --2- 172.21.15.59:0/4233046382 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb900100380 0x7fb9001007a0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:43:52.223 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:52.217+0000 7fb906cf4700 1 --2- 172.21.15.59:0/4233046382 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb900101570 0x7fb9001019d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:43:52.223 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:52.217+0000 7fb906cf4700 1 -- 172.21.15.59:0/4233046382 >> 172.21.15.59:0/4233046382 conn(0x7fb9000fb8f0 msgr2=0x7fb9000fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:43:52.223 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:52.217+0000 7fb906cf4700 1 -- 172.21.15.59:0/4233046382 shutdown_connections 2023-12-11T17:43:52.223 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:52.218+0000 7fb906cf4700 1 -- 172.21.15.59:0/4233046382 wait complete. 2023-12-11T17:43:52.223 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:52.218+0000 7fb906cf4700 1 Processor -- start 2023-12-11T17:43:52.223 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:52.218+0000 7fb906cf4700 1 -- start start 2023-12-11T17:43:52.223 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:52.219+0000 7fb906cf4700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb900100380 0x7fb900069190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:43:52.224 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:52.219+0000 7fb906cf4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb900101570 0x7fb90019c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:43:52.224 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:52.219+0000 7fb906cf4700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb900101f10 0x7fb90019e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:43:52.224 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:52.219+0000 7fb906cf4700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb90019ef20 con 0x7fb900100380 2023-12-11T17:43:52.224 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:52.219+0000 7fb906cf4700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb90019f090 con 0x7fb900101f10 2023-12-11T17:43:52.224 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:52.219+0000 7fb906cf4700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb90019f250 con 0x7fb900101570 2023-12-11T17:43:52.224 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:52.219+0000 7fb904a90700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb900100380 0x7fb900069190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:43:52.224 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:52.219+0000 7fb904a90700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb900100380 0x7fb900069190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60806/0 (socket says 172.21.15.59:60806) 2023-12-11T17:43:52.224 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:52.219+0000 7fb904a90700 1 -- 172.21.15.59:0/2203256284 learned_addr learned my addr 172.21.15.59:0/2203256284 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:43:52.224 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:52.219+0000 7fb905291700 1 --2- 172.21.15.59:0/2203256284 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb900101f10 0x7fb90019e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:43:52.224 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:52.219+0000 7fb8fffff700 1 --2- 172.21.15.59:0/2203256284 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb900101570 0x7fb90019c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:43:52.224 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:52.219+0000 7fb904a90700 1 -- 172.21.15.59:0/2203256284 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb900101570 msgr2=0x7fb90019c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:43:52.224 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:52.219+0000 7fb904a90700 1 --2- 172.21.15.59:0/2203256284 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb900101570 0x7fb90019c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:43:52.224 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:52.219+0000 7fb904a90700 1 -- 172.21.15.59:0/2203256284 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb900101f10 msgr2=0x7fb90019e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:43:52.225 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:52.219+0000 7fb904a90700 1 --2- 172.21.15.59:0/2203256284 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb900101f10 0x7fb90019e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:43:52.225 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:52.219+0000 7fb904a90700 1 -- 172.21.15.59:0/2203256284 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8f40095f0 con 0x7fb900100380 2023-12-11T17:43:52.225 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:52.219+0000 7fb8fffff700 1 --2- 172.21.15.59:0/2203256284 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb900101570 0x7fb90019c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-11T17:43:52.225 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:52.219+0000 7fb904a90700 1 --2- 172.21.15.59:0/2203256284 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb900100380 0x7fb900069190 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb8f4003730 tx=0x7fb8f400fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:43:52.225 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:52.219+0000 7fb8fdffb700 1 -- 172.21.15.59:0/2203256284 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8f401d070 con 0x7fb900100380 2023-12-11T17:43:52.225 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:52.219+0000 7fb8fdffb700 1 -- 172.21.15.59:0/2203256284 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb8f400fe70 con 0x7fb900100380 2023-12-11T17:43:52.225 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:52.219+0000 7fb906cf4700 1 -- 172.21.15.59:0/2203256284 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb90019f4d0 con 0x7fb900100380 2023-12-11T17:43:52.225 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:52.219+0000 7fb8fdffb700 1 -- 172.21.15.59:0/2203256284 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8f40173f0 con 0x7fb900100380 2023-12-11T17:43:52.225 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:52.220+0000 7fb906cf4700 1 -- 172.21.15.59:0/2203256284 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb90019f950 con 0x7fb900100380 2023-12-11T17:43:52.226 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:52.221+0000 7fb906cf4700 1 -- 172.21.15.59:0/2203256284 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb900107850 con 0x7fb900100380 2023-12-11T17:43:52.226 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:52.222+0000 7fb8fdffb700 1 -- 172.21.15.59:0/2203256284 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7fb8f4022470 con 0x7fb900100380 2023-12-11T17:43:52.226 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:52.223+0000 7fb8fdffb700 1 --2- 172.21.15.59:0/2203256284 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fb8f00a1500 0x7fb8f00a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:43:52.226 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:52.223+0000 7fb8fffff700 1 --2- 172.21.15.59:0/2203256284 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fb8f00a1500 0x7fb8f00a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:43:52.226 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:52.223+0000 7fb8fdffb700 1 -- 172.21.15.59:0/2203256284 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fb8f40cd9c0 con 0x7fb900100380 2023-12-11T17:43:52.226 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:52.223+0000 7fb8fffff700 1 --2- 172.21.15.59:0/2203256284 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fb8f00a1500 0x7fb8f00a39c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec005fd0 tx=0x7fb8ec005ee0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:43:52.230 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:52.227+0000 7fb8fdffb700 1 -- 172.21.15.59:0/2203256284 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fb8f4097550 con 0x7fb900100380 2023-12-11T17:43:52.283 INFO:journalctl@ceph.mds.k.smithi115.stdout:Dec 11 17:43:52 smithi115 systemd[1]: Starting Ceph mds.k for d0a07eaa-984b-11ee-95a3-87774f69a715... 2023-12-11T17:43:52.477 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:52.474+0000 7fb906cf4700 1 -- 172.21.15.59:0/2203256284 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb900068230 con 0x7fb900100380 2023-12-11T17:43:52.477 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:52.475+0000 7fb8fdffb700 1 -- 172.21.15.59:0/2203256284 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1325 (secure 0 0 0) 0x7fb8f409ab70 con 0x7fb900100380 2023-12-11T17:43:52.477 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-12-11T17:43:52.478 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/395684042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":395684042},{"type":"v1","addr":"172.21.15.181:6833","nonce":395684042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2023-12-11T17:43:52.480 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:52.477+0000 7fb906cf4700 1 -- 172.21.15.59:0/2203256284 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fb8f00a1500 msgr2=0x7fb8f00a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:43:52.480 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:52.477+0000 7fb906cf4700 1 --2- 172.21.15.59:0/2203256284 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fb8f00a1500 0x7fb8f00a39c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec005fd0 tx=0x7fb8ec005ee0 comp rx=0 tx=0).stop 2023-12-11T17:43:52.480 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:52.477+0000 7fb906cf4700 1 -- 172.21.15.59:0/2203256284 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb900100380 msgr2=0x7fb900069190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:43:52.480 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:52.477+0000 7fb906cf4700 1 --2- 172.21.15.59:0/2203256284 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb900100380 0x7fb900069190 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb8f4003730 tx=0x7fb8f400fa00 comp rx=0 tx=0).stop 2023-12-11T17:43:52.480 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:52.477+0000 7fb906cf4700 1 -- 172.21.15.59:0/2203256284 shutdown_connections 2023-12-11T17:43:52.480 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:52.477+0000 7fb906cf4700 1 --2- 172.21.15.59:0/2203256284 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fb8f00a1500 0x7fb8f00a39c0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:43:52.480 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:52.477+0000 7fb906cf4700 1 --2- 172.21.15.59:0/2203256284 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb900101f10 0x7fb90019e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:43:52.480 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:52.477+0000 7fb906cf4700 1 --2- 172.21.15.59:0/2203256284 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb900100380 0x7fb900069190 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:43:52.480 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:52.477+0000 7fb906cf4700 1 --2- 172.21.15.59:0/2203256284 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb900101570 0x7fb90019c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:43:52.480 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:52.477+0000 7fb906cf4700 1 -- 172.21.15.59:0/2203256284 >> 172.21.15.59:0/2203256284 conn(0x7fb9000fb8f0 msgr2=0x7fb900104990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:43:52.480 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:52.477+0000 7fb906cf4700 1 -- 172.21.15.59:0/2203256284 shutdown_connections 2023-12-11T17:43:52.481 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:52.477+0000 7fb906cf4700 1 -- 172.21.15.59:0/2203256284 wait complete. 2023-12-11T17:43:52.481 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 2 2023-12-11T17:43:52.695 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:52 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/2203256284' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T17:43:52.695 INFO:journalctl@ceph.mds.k.smithi115.stdout:Dec 11 17:43:52 smithi115 podman[138664]: 2023-12-11T17:43:52.695 INFO:journalctl@ceph.mds.k.smithi115.stdout:Dec 11 17:43:52 smithi115 ceph-mds[138704]: -- 172.21.15.115:0/280003449 <== mon.2 v2:172.21.15.115:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x556977d7d520 con 0x556977d52c00 2023-12-11T17:43:52.695 INFO:journalctl@ceph.mds.k.smithi115.stdout:Dec 11 17:43:52 smithi115 bash[138664]: 7b2f668a080eedb5fbd026e85773980d73800089c8e23066bb439989c28db62e 2023-12-11T17:43:52.696 INFO:journalctl@ceph.mds.k.smithi115.stdout:Dec 11 17:43:52 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mds-k[138680]: starting mds.k at 2023-12-11T17:43:52.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:52 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/2203256284' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T17:43:52.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:52 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/2203256284' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T17:43:53.033 INFO:journalctl@ceph.mds.k.smithi115.stdout:Dec 11 17:43:52 smithi115 systemd[1]: Started Ceph mds.k for d0a07eaa-984b-11ee-95a3-87774f69a715. 2023-12-11T17:43:54.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:53 smithi115 ceph-mon[102748]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T17:43:54.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:53 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:43:54.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:53 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:43:54.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:53 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:43:54.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:53 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-11T17:43:54.034 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:53 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-11T17:43:54.034 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:53 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:43:54.034 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:53 smithi115 ceph-mon[102748]: Deploying daemon mds.j on smithi059 2023-12-11T17:43:54.034 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:53 smithi115 ceph-mon[102748]: mds.? [v2:172.21.15.115:6832/3214606191,v1:172.21.15.115:6833/3214606191] up:boot 2023-12-11T17:43:54.034 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:53 smithi115 ceph-mon[102748]: fsmap 2 up:standby 2023-12-11T17:43:54.034 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:53 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-12-11T17:43:54.046 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:53 smithi059 ceph-mon[98847]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T17:43:54.046 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:53 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:43:54.046 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:53 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:43:54.046 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:53 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:43:54.046 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:53 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-11T17:43:54.046 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:53 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-11T17:43:54.047 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:53 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:43:54.047 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:53 smithi059 ceph-mon[98847]: Deploying daemon mds.j on smithi059 2023-12-11T17:43:54.047 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:53 smithi059 ceph-mon[98847]: mds.? [v2:172.21.15.115:6832/3214606191,v1:172.21.15.115:6833/3214606191] up:boot 2023-12-11T17:43:54.047 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:53 smithi059 ceph-mon[98847]: fsmap 2 up:standby 2023-12-11T17:43:54.047 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:53 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-12-11T17:43:54.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:53 smithi181 ceph-mon[103627]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T17:43:54.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:53 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:43:54.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:53 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:43:54.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:53 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:43:54.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:53 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-11T17:43:54.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:53 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-11T17:43:54.146 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:53 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:43:54.146 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:53 smithi181 ceph-mon[103627]: Deploying daemon mds.j on smithi059 2023-12-11T17:43:54.146 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:53 smithi181 ceph-mon[103627]: mds.? [v2:172.21.15.115:6832/3214606191,v1:172.21.15.115:6833/3214606191] up:boot 2023-12-11T17:43:54.146 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:53 smithi181 ceph-mon[103627]: fsmap 2 up:standby 2023-12-11T17:43:54.146 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:53 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-12-11T17:43:54.404 INFO:journalctl@ceph.mds.j.smithi059.stdout:Dec 11 17:43:54 smithi059 systemd[1]: Starting Ceph mds.j for d0a07eaa-984b-11ee-95a3-87774f69a715... 2023-12-11T17:43:54.493 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-11T17:43:54.627 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:54.624+0000 7f429269b700 1 -- 172.21.15.59:0/2066409278 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f428c0ff510 msgr2=0x7f428c0ff970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:43:54.627 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:54.624+0000 7f429269b700 1 --2- 172.21.15.59:0/2066409278 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f428c0ff510 0x7f428c0ff970 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f42740099c0 tx=0x7f4274009cd0 comp rx=0 tx=0).stop 2023-12-11T17:43:54.627 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:54.624+0000 7f429269b700 1 -- 172.21.15.59:0/2066409278 shutdown_connections 2023-12-11T17:43:54.628 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:54.624+0000 7f429269b700 1 --2- 172.21.15.59:0/2066409278 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f428c0ffeb0 0x7f428c109a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:43:54.628 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:54.624+0000 7f429269b700 1 --2- 172.21.15.59:0/2066409278 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f428c102aa0 0x7f428c102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:43:54.628 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:54.624+0000 7f429269b700 1 --2- 172.21.15.59:0/2066409278 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f428c0ff510 0x7f428c0ff970 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:43:54.628 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:54.624+0000 7f429269b700 1 -- 172.21.15.59:0/2066409278 >> 172.21.15.59:0/2066409278 conn(0x7f428c0fb0b0 msgr2=0x7f428c0fd4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:43:54.628 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:54.625+0000 7f429269b700 1 -- 172.21.15.59:0/2066409278 shutdown_connections 2023-12-11T17:43:54.628 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:54.625+0000 7f429269b700 1 -- 172.21.15.59:0/2066409278 wait complete. 2023-12-11T17:43:54.631 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:54.625+0000 7f429269b700 1 Processor -- start 2023-12-11T17:43:54.631 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:54.625+0000 7f429269b700 1 -- start start 2023-12-11T17:43:54.631 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:54.626+0000 7f429269b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f428c0ff510 0x7f428c197410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:43:54.631 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:54.626+0000 7f429269b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f428c0ffeb0 0x7f428c197950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:43:54.631 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:54.626+0000 7f429269b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f428c102aa0 0x7f428c19e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:43:54.631 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:54.626+0000 7f429269b700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f428c19ef40 con 0x7f428c102aa0 2023-12-11T17:43:54.631 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:54.626+0000 7f429269b700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f428c19f080 con 0x7f428c0ff510 2023-12-11T17:43:54.631 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:54.626+0000 7f429269b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f428c19f210 con 0x7f428c0ffeb0 2023-12-11T17:43:54.631 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:54.626+0000 7f4290c38700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f428c102aa0 0x7f428c19e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:43:54.631 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:54.626+0000 7f4290c38700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f428c102aa0 0x7f428c19e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60822/0 (socket says 172.21.15.59:60822) 2023-12-11T17:43:54.631 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:54.626+0000 7f4290c38700 1 -- 172.21.15.59:0/1576130086 learned_addr learned my addr 172.21.15.59:0/1576130086 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:43:54.632 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:54.626+0000 7f428bfff700 1 --2- 172.21.15.59:0/1576130086 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f428c0ff510 0x7f428c197410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:43:54.632 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:54.626+0000 7f4283fff700 1 --2- 172.21.15.59:0/1576130086 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f428c0ffeb0 0x7f428c197950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:43:54.632 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:54.626+0000 7f4290c38700 1 -- 172.21.15.59:0/1576130086 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f428c0ffeb0 msgr2=0x7f428c197950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:43:54.632 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:54.626+0000 7f4290c38700 1 --2- 172.21.15.59:0/1576130086 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f428c0ffeb0 0x7f428c197950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:43:54.632 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:54.626+0000 7f4290c38700 1 -- 172.21.15.59:0/1576130086 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f428c0ff510 msgr2=0x7f428c197410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:43:54.632 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:54.626+0000 7f4290c38700 1 --2- 172.21.15.59:0/1576130086 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f428c0ff510 0x7f428c197410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:43:54.632 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:54.626+0000 7f4290c38700 1 -- 172.21.15.59:0/1576130086 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42740095f0 con 0x7f428c102aa0 2023-12-11T17:43:54.632 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:54.626+0000 7f4283fff700 1 --2- 172.21.15.59:0/1576130086 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f428c0ffeb0 0x7f428c197950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-11T17:43:54.632 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:54.626+0000 7f428bfff700 1 --2- 172.21.15.59:0/1576130086 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f428c0ff510 0x7f428c197410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T17:43:54.632 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:54.626+0000 7f4290c38700 1 --2- 172.21.15.59:0/1576130086 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f428c102aa0 0x7f428c19e970 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f4284007ee0 tx=0x7f428400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:43:54.632 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:54.626+0000 7f4289ffb700 1 -- 172.21.15.59:0/1576130086 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4284017070 con 0x7f428c102aa0 2023-12-11T17:43:54.632 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:54.626+0000 7f429269b700 1 -- 172.21.15.59:0/1576130086 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f428c19f4f0 con 0x7f428c102aa0 2023-12-11T17:43:54.632 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:54.626+0000 7f4289ffb700 1 -- 172.21.15.59:0/1576130086 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f428400f040 con 0x7f428c102aa0 2023-12-11T17:43:54.632 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:54.626+0000 7f4289ffb700 1 -- 172.21.15.59:0/1576130086 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4284005740 con 0x7f428c102aa0 2023-12-11T17:43:54.633 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:54.627+0000 7f429269b700 1 -- 172.21.15.59:0/1576130086 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f428c19fa00 con 0x7f428c102aa0 2023-12-11T17:43:54.633 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:54.627+0000 7f429269b700 1 -- 172.21.15.59:0/1576130086 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f428c108ae0 con 0x7f428c102aa0 2023-12-11T17:43:54.633 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:54.629+0000 7f4289ffb700 1 -- 172.21.15.59:0/1576130086 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f42840058a0 con 0x7f428c102aa0 2023-12-11T17:43:54.633 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:54.629+0000 7f4289ffb700 1 --2- 172.21.15.59:0/1576130086 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f42780a1480 0x7f42780a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:43:54.634 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:54.629+0000 7f428bfff700 1 --2- 172.21.15.59:0/1576130086 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f42780a1480 0x7f42780a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:43:54.634 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:54.629+0000 7f4289ffb700 1 -- 172.21.15.59:0/1576130086 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f42840cc760 con 0x7f428c102aa0 2023-12-11T17:43:54.634 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:54.630+0000 7f428bfff700 1 --2- 172.21.15.59:0/1576130086 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f42780a1480 0x7f42780a3940 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f428c069790 tx=0x7f427c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:43:54.635 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:54.633+0000 7f4289ffb700 1 -- 172.21.15.59:0/1576130086 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f4284096260 con 0x7f428c102aa0 2023-12-11T17:43:54.865 INFO:journalctl@ceph.mds.j.smithi059.stdout:Dec 11 17:43:54 smithi059 podman[150077]: 2023-12-11T17:43:54.897 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:54.895+0000 7f429269b700 1 -- 172.21.15.59:0/1576130086 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f428c068680 con 0x7f428c102aa0 2023-12-11T17:43:54.898 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:54.895+0000 7f4289ffb700 1 -- 172.21.15.59:0/1576130086 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2119 (secure 0 0 0) 0x7f4284099880 con 0x7f428c102aa0 2023-12-11T17:43:54.898 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-12-11T17:43:54.898 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/395684042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":395684042},{"type":"v1","addr":"172.21.15.181:6833","nonce":395684042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6833/3214606191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3214606191},{"type":"v1","addr":"172.21.15.115:6833","nonce":3214606191}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2023-12-11T17:43:54.901 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:54.899+0000 7f429269b700 1 -- 172.21.15.59:0/1576130086 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f42780a1480 msgr2=0x7f42780a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:43:54.902 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:54.899+0000 7f429269b700 1 --2- 172.21.15.59:0/1576130086 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f42780a1480 0x7f42780a3940 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f428c069790 tx=0x7f427c005f90 comp rx=0 tx=0).stop 2023-12-11T17:43:54.902 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:54.899+0000 7f429269b700 1 -- 172.21.15.59:0/1576130086 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f428c102aa0 msgr2=0x7f428c19e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:43:54.902 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:54.899+0000 7f429269b700 1 --2- 172.21.15.59:0/1576130086 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f428c102aa0 0x7f428c19e970 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f4284007ee0 tx=0x7f428400d3b0 comp rx=0 tx=0).stop 2023-12-11T17:43:54.902 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:54.899+0000 7f429269b700 1 -- 172.21.15.59:0/1576130086 shutdown_connections 2023-12-11T17:43:54.902 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:54.899+0000 7f429269b700 1 --2- 172.21.15.59:0/1576130086 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f42780a1480 0x7f42780a3940 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:43:54.902 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:54.899+0000 7f429269b700 1 --2- 172.21.15.59:0/1576130086 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f428c102aa0 0x7f428c19e970 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:43:54.902 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:54.899+0000 7f429269b700 1 --2- 172.21.15.59:0/1576130086 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f428c0ff510 0x7f428c197410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:43:54.902 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:54.899+0000 7f429269b700 1 --2- 172.21.15.59:0/1576130086 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f428c0ffeb0 0x7f428c197950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:43:54.902 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:54.899+0000 7f429269b700 1 -- 172.21.15.59:0/1576130086 >> 172.21.15.59:0/1576130086 conn(0x7f428c0fb0b0 msgr2=0x7f428c105c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:43:54.902 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:54.899+0000 7f429269b700 1 -- 172.21.15.59:0/1576130086 shutdown_connections 2023-12-11T17:43:54.902 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:54.899+0000 7f429269b700 1 -- 172.21.15.59:0/1576130086 wait complete. 2023-12-11T17:43:54.903 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 3 2023-12-11T17:43:55.154 INFO:journalctl@ceph.mds.j.smithi059.stdout:Dec 11 17:43:54 smithi059 bash[150077]: 934c2b8b45fe6675d4cc30e551ff8e3eaf6e1ad33222a2c2a0559548b6da4132 2023-12-11T17:43:55.154 INFO:journalctl@ceph.mds.j.smithi059.stdout:Dec 11 17:43:54 smithi059 ceph-mds[150160]: -- 172.21.15.59:0/640788384 <== mon.0 v2:172.21.15.59:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x560a9a425520 con 0x560a9a3fac00 2023-12-11T17:43:55.154 INFO:journalctl@ceph.mds.j.smithi059.stdout:Dec 11 17:43:54 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mds-j[150136]: starting mds.j at 2023-12-11T17:43:55.154 INFO:journalctl@ceph.mds.j.smithi059.stdout:Dec 11 17:43:54 smithi059 systemd[1]: Started Ceph mds.j for d0a07eaa-984b-11ee-95a3-87774f69a715. 2023-12-11T17:43:55.882 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:55 smithi059 ceph-mon[98847]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T17:43:55.882 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:55 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/1576130086' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T17:43:55.882 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:55 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:43:55.882 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:55 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:43:55.882 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:55 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:43:55.882 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:55 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-11T17:43:55.882 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:55 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-11T17:43:55.882 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:55 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:43:55.882 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:55 smithi059 ceph-mon[98847]: Deploying daemon mds.d on smithi059 2023-12-11T17:43:56.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:55 smithi115 ceph-mon[102748]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T17:43:56.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:55 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/1576130086' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T17:43:56.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:55 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:43:56.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:55 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:43:56.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:55 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:43:56.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:55 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-11T17:43:56.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:55 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-11T17:43:56.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:55 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:43:56.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:55 smithi115 ceph-mon[102748]: Deploying daemon mds.d on smithi059 2023-12-11T17:43:56.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:55 smithi181 ceph-mon[103627]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T17:43:56.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:55 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/1576130086' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T17:43:56.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:55 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:43:56.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:55 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:43:56.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:55 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:43:56.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:55 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-11T17:43:56.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:55 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-11T17:43:56.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:55 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:43:56.146 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:55 smithi181 ceph-mon[103627]: Deploying daemon mds.d on smithi059 2023-12-11T17:43:56.451 INFO:journalctl@ceph.mds.d.smithi059.stdout:Dec 11 17:43:56 smithi059 systemd[1]: Starting Ceph mds.d for d0a07eaa-984b-11ee-95a3-87774f69a715... 2023-12-11T17:43:56.920 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-11T17:43:56.966 INFO:journalctl@ceph.mds.d.smithi059.stdout:Dec 11 17:43:56 smithi059 podman[150462]: 2023-12-11T17:43:56.966 INFO:journalctl@ceph.mds.d.smithi059.stdout:Dec 11 17:43:56 smithi059 bash[150462]: 12731884662c8b8208cc097dcb74370b0c32c927e55e346a578d6f6bb4f78edd 2023-12-11T17:43:56.967 INFO:journalctl@ceph.mds.d.smithi059.stdout:Dec 11 17:43:56 smithi059 ceph-mds[150521]: -- 172.21.15.59:0/118087618 <== mon.0 v2:172.21.15.59:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x556d4d6fd520 con 0x556d4d6d2800 2023-12-11T17:43:56.967 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:56 smithi059 ceph-mon[98847]: mds.? [v2:172.21.15.59:6834/2184151593,v1:172.21.15.59:6835/2184151593] up:boot 2023-12-11T17:43:56.967 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:56 smithi059 ceph-mon[98847]: fsmap 3 up:standby 2023-12-11T17:43:56.967 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:56 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-12-11T17:43:57.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:56 smithi115 ceph-mon[102748]: mds.? [v2:172.21.15.59:6834/2184151593,v1:172.21.15.59:6835/2184151593] up:boot 2023-12-11T17:43:57.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:56 smithi115 ceph-mon[102748]: fsmap 3 up:standby 2023-12-11T17:43:57.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:56 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-12-11T17:43:57.059 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:57.054+0000 7f7c6b2e7700 1 -- 172.21.15.59:0/2949342181 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7c64104830 msgr2=0x7f7c64106c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:43:57.059 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:57.054+0000 7f7c6b2e7700 1 --2- 172.21.15.59:0/2949342181 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7c64104830 0x7f7c64106c20 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f7c54009990 tx=0x7f7c54009ca0 comp rx=0 tx=0).stop 2023-12-11T17:43:57.059 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:57.055+0000 7f7c6b2e7700 1 -- 172.21.15.59:0/2949342181 shutdown_connections 2023-12-11T17:43:57.059 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:57.055+0000 7f7c6b2e7700 1 --2- 172.21.15.59:0/2949342181 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7c64107160 0x7f7c641095d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:43:57.059 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:57.055+0000 7f7c6b2e7700 1 --2- 172.21.15.59:0/2949342181 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7c64101f00 0x7f7c641042f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:43:57.060 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:57.055+0000 7f7c6b2e7700 1 --2- 172.21.15.59:0/2949342181 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7c64104830 0x7f7c64106c20 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:43:57.060 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:57.055+0000 7f7c6b2e7700 1 -- 172.21.15.59:0/2949342181 >> 172.21.15.59:0/2949342181 conn(0x7f7c640fb8f0 msgr2=0x7f7c640fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:43:57.060 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:57.055+0000 7f7c6b2e7700 1 -- 172.21.15.59:0/2949342181 shutdown_connections 2023-12-11T17:43:57.060 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:57.055+0000 7f7c6b2e7700 1 -- 172.21.15.59:0/2949342181 wait complete. 2023-12-11T17:43:57.060 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:57.055+0000 7f7c6b2e7700 1 Processor -- start 2023-12-11T17:43:57.060 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:57.055+0000 7f7c6b2e7700 1 -- start start 2023-12-11T17:43:57.060 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:57.055+0000 7f7c6b2e7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7c64101f00 0x7f7c641973d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:43:57.060 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:57.055+0000 7f7c6b2e7700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7c64104830 0x7f7c64197910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:43:57.060 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:57.055+0000 7f7c6b2e7700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7c64107160 0x7f7c6419e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:43:57.060 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:57.055+0000 7f7c6b2e7700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c6419ef00 con 0x7f7c64104830 2023-12-11T17:43:57.060 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:57.056+0000 7f7c6b2e7700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c6419f040 con 0x7f7c64107160 2023-12-11T17:43:57.060 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:57.056+0000 7f7c6b2e7700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c6419f1d0 con 0x7f7c64101f00 2023-12-11T17:43:57.060 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:57.056+0000 7f7c68882700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7c64104830 0x7f7c64197910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:43:57.061 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:57.056+0000 7f7c69083700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7c64101f00 0x7f7c641973d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:43:57.061 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:57.056+0000 7f7c68882700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7c64104830 0x7f7c64197910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60860/0 (socket says 172.21.15.59:60860) 2023-12-11T17:43:57.061 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:57.056+0000 7f7c68882700 1 -- 172.21.15.59:0/2967508417 learned_addr learned my addr 172.21.15.59:0/2967508417 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:43:57.061 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:57.056+0000 7f7c69884700 1 --2- 172.21.15.59:0/2967508417 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7c64107160 0x7f7c6419e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:43:57.061 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:57.056+0000 7f7c68882700 1 -- 172.21.15.59:0/2967508417 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7c64101f00 msgr2=0x7f7c641973d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:43:57.061 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:57.056+0000 7f7c68882700 1 --2- 172.21.15.59:0/2967508417 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7c64101f00 0x7f7c641973d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:43:57.061 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:57.056+0000 7f7c68882700 1 -- 172.21.15.59:0/2967508417 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7c64107160 msgr2=0x7f7c6419e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:43:57.061 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:57.056+0000 7f7c68882700 1 --2- 172.21.15.59:0/2967508417 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7c64107160 0x7f7c6419e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:43:57.061 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:57.056+0000 7f7c68882700 1 -- 172.21.15.59:0/2967508417 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c540095f0 con 0x7f7c64104830 2023-12-11T17:43:57.061 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:57.056+0000 7f7c68882700 1 --2- 172.21.15.59:0/2967508417 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7c64104830 0x7f7c64197910 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7c540095c0 tx=0x7f7c5400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:43:57.061 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:57.056+0000 7f7c5a7fc700 1 -- 172.21.15.59:0/2967508417 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c5401d070 con 0x7f7c64104830 2023-12-11T17:43:57.061 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:57.056+0000 7f7c5a7fc700 1 -- 172.21.15.59:0/2967508417 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7c54005510 con 0x7f7c64104830 2023-12-11T17:43:57.061 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:57.056+0000 7f7c6b2e7700 1 -- 172.21.15.59:0/2967508417 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c6419f450 con 0x7f7c64104830 2023-12-11T17:43:57.062 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:57.056+0000 7f7c5a7fc700 1 -- 172.21.15.59:0/2967508417 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c54004b70 con 0x7f7c64104830 2023-12-11T17:43:57.062 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:57.056+0000 7f7c6b2e7700 1 -- 172.21.15.59:0/2967508417 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c6419f870 con 0x7f7c64104830 2023-12-11T17:43:57.062 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:57.057+0000 7f7c6b2e7700 1 -- 172.21.15.59:0/2967508417 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c640fd470 con 0x7f7c64104830 2023-12-11T17:43:57.063 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:57.059+0000 7f7c5a7fc700 1 -- 172.21.15.59:0/2967508417 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f7c5400fb90 con 0x7f7c64104830 2023-12-11T17:43:57.063 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:57.059+0000 7f7c5a7fc700 1 --2- 172.21.15.59:0/2967508417 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f7c500a1500 0x7f7c500a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:43:57.063 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:57.059+0000 7f7c5a7fc700 1 -- 172.21.15.59:0/2967508417 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f7c540cd7d0 con 0x7f7c64104830 2023-12-11T17:43:57.063 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:57.059+0000 7f7c69083700 1 --2- 172.21.15.59:0/2967508417 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f7c500a1500 0x7f7c500a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:43:57.063 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:57.060+0000 7f7c69083700 1 --2- 172.21.15.59:0/2967508417 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f7c500a1500 0x7f7c500a39c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f7c60007c60 tx=0x7f7c600073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:43:57.066 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:57.064+0000 7f7c5a7fc700 1 -- 172.21.15.59:0/2967508417 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f7c540972c0 con 0x7f7c64104830 2023-12-11T17:43:57.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:56 smithi181 ceph-mon[103627]: mds.? [v2:172.21.15.59:6834/2184151593,v1:172.21.15.59:6835/2184151593] up:boot 2023-12-11T17:43:57.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:56 smithi181 ceph-mon[103627]: fsmap 3 up:standby 2023-12-11T17:43:57.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:56 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-12-11T17:43:57.336 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:57.334+0000 7f7c6b2e7700 1 -- 172.21.15.59:0/2967508417 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7c64068230 con 0x7f7c64104830 2023-12-11T17:43:57.336 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:57.334+0000 7f7c5a7fc700 1 -- 172.21.15.59:0/2967508417 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) v1 ==== 93+0+2910 (secure 0 0 0) 0x7f7c5409a8e0 con 0x7f7c64104830 2023-12-11T17:43:57.337 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-12-11T17:43:57.337 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/2184151593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":2184151593},{"type":"v1","addr":"172.21.15.59:6835","nonce":2184151593}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/395684042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":395684042},{"type":"v1","addr":"172.21.15.181:6833","nonce":395684042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6833/3214606191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3214606191},{"type":"v1","addr":"172.21.15.115:6833","nonce":3214606191}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2023-12-11T17:43:57.340 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:57.336+0000 7f7c6b2e7700 1 -- 172.21.15.59:0/2967508417 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f7c500a1500 msgr2=0x7f7c500a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:43:57.340 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:57.336+0000 7f7c6b2e7700 1 --2- 172.21.15.59:0/2967508417 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f7c500a1500 0x7f7c500a39c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f7c60007c60 tx=0x7f7c600073d0 comp rx=0 tx=0).stop 2023-12-11T17:43:57.340 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:57.336+0000 7f7c6b2e7700 1 -- 172.21.15.59:0/2967508417 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7c64104830 msgr2=0x7f7c64197910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:43:57.340 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:57.336+0000 7f7c6b2e7700 1 --2- 172.21.15.59:0/2967508417 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7c64104830 0x7f7c64197910 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7c540095c0 tx=0x7f7c5400f690 comp rx=0 tx=0).stop 2023-12-11T17:43:57.340 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:57.336+0000 7f7c6b2e7700 1 -- 172.21.15.59:0/2967508417 shutdown_connections 2023-12-11T17:43:57.341 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:57.336+0000 7f7c6b2e7700 1 --2- 172.21.15.59:0/2967508417 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f7c500a1500 0x7f7c500a39c0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:43:57.341 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:57.336+0000 7f7c6b2e7700 1 --2- 172.21.15.59:0/2967508417 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7c64107160 0x7f7c6419e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:43:57.341 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:57.336+0000 7f7c6b2e7700 1 --2- 172.21.15.59:0/2967508417 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7c64101f00 0x7f7c641973d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:43:57.341 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:57.336+0000 7f7c6b2e7700 1 --2- 172.21.15.59:0/2967508417 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7c64104830 0x7f7c64197910 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:43:57.341 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:57.336+0000 7f7c6b2e7700 1 -- 172.21.15.59:0/2967508417 >> 172.21.15.59:0/2967508417 conn(0x7f7c640fb8f0 msgr2=0x7f7c64103db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:43:57.341 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:57.336+0000 7f7c6b2e7700 1 -- 172.21.15.59:0/2967508417 shutdown_connections 2023-12-11T17:43:57.341 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:57.336+0000 7f7c6b2e7700 1 -- 172.21.15.59:0/2967508417 wait complete. 2023-12-11T17:43:57.341 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 4 2023-12-11T17:43:57.350 INFO:journalctl@ceph.mds.d.smithi059.stdout:Dec 11 17:43:56 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mds-d[150478]: starting mds.d at 2023-12-11T17:43:57.350 INFO:journalctl@ceph.mds.d.smithi059.stdout:Dec 11 17:43:57 smithi059 systemd[1]: Started Ceph mds.d for d0a07eaa-984b-11ee-95a3-87774f69a715. 2023-12-11T17:43:58.026 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:57 smithi115 ceph-mon[102748]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T17:43:58.027 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:57 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:43:58.027 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:57 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:43:58.027 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:57 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:43:58.027 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:57 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-11T17:43:58.027 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:57 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-11T17:43:58.027 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:57 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:43:58.027 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:57 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/2967508417' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T17:43:58.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:57 smithi181 ceph-mon[103627]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T17:43:58.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:57 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:43:58.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:57 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:43:58.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:57 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:43:58.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:57 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-11T17:43:58.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:57 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-11T17:43:58.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:57 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:43:58.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:57 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/2967508417' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T17:43:58.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:57 smithi059 ceph-mon[98847]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T17:43:58.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:57 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:43:58.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:57 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:43:58.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:57 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:43:58.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:57 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-11T17:43:58.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:57 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-11T17:43:58.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:57 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:43:58.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:57 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/2967508417' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T17:43:58.676 INFO:journalctl@ceph.mds.e.smithi115.stdout:Dec 11 17:43:58 smithi115 systemd[1]: Starting Ceph mds.e for d0a07eaa-984b-11ee-95a3-87774f69a715... 2023-12-11T17:43:58.970 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:58 smithi115 ceph-mon[102748]: Deploying daemon mds.e on smithi115 2023-12-11T17:43:58.970 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:58 smithi115 ceph-mon[102748]: mds.? [v2:172.21.15.59:6836/2817614886,v1:172.21.15.59:6837/2817614886] up:boot 2023-12-11T17:43:58.970 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:58 smithi115 ceph-mon[102748]: fsmap 4 up:standby 2023-12-11T17:43:58.970 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:58 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-12-11T17:43:59.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:58 smithi181 ceph-mon[103627]: Deploying daemon mds.e on smithi115 2023-12-11T17:43:59.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:58 smithi181 ceph-mon[103627]: mds.? [v2:172.21.15.59:6836/2817614886,v1:172.21.15.59:6837/2817614886] up:boot 2023-12-11T17:43:59.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:58 smithi181 ceph-mon[103627]: fsmap 4 up:standby 2023-12-11T17:43:59.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:58 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-12-11T17:43:59.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:58 smithi059 ceph-mon[98847]: Deploying daemon mds.e on smithi115 2023-12-11T17:43:59.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:58 smithi059 ceph-mon[98847]: mds.? [v2:172.21.15.59:6836/2817614886,v1:172.21.15.59:6837/2817614886] up:boot 2023-12-11T17:43:59.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:58 smithi059 ceph-mon[98847]: fsmap 4 up:standby 2023-12-11T17:43:59.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:58 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-12-11T17:43:59.236 INFO:journalctl@ceph.mds.e.smithi115.stdout:Dec 11 17:43:58 smithi115 podman[139002]: 2023-12-11T17:43:59.353 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-11T17:43:59.499 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:59.494+0000 7f2a3f84f700 1 -- 172.21.15.59:0/2508588142 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2a38104840 msgr2=0x7f2a38106c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:43:59.499 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:59.494+0000 7f2a3f84f700 1 --2- 172.21.15.59:0/2508588142 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2a38104840 0x7f2a38106c30 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f2a34009990 tx=0x7f2a34009ca0 comp rx=0 tx=0).stop 2023-12-11T17:43:59.499 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:59.494+0000 7f2a3f84f700 1 -- 172.21.15.59:0/2508588142 shutdown_connections 2023-12-11T17:43:59.499 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:59.494+0000 7f2a3f84f700 1 --2- 172.21.15.59:0/2508588142 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a38107170 0x7f2a381095e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:43:59.500 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:59.494+0000 7f2a3f84f700 1 --2- 172.21.15.59:0/2508588142 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2a38101f10 0x7f2a38104300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:43:59.500 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:59.494+0000 7f2a3f84f700 1 --2- 172.21.15.59:0/2508588142 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2a38104840 0x7f2a38106c30 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:43:59.500 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:59.494+0000 7f2a3f84f700 1 -- 172.21.15.59:0/2508588142 >> 172.21.15.59:0/2508588142 conn(0x7f2a380fb920 msgr2=0x7f2a380fdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:43:59.500 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:59.494+0000 7f2a3f84f700 1 -- 172.21.15.59:0/2508588142 shutdown_connections 2023-12-11T17:43:59.500 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:59.495+0000 7f2a3f84f700 1 -- 172.21.15.59:0/2508588142 wait complete. 2023-12-11T17:43:59.500 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:59.495+0000 7f2a3f84f700 1 Processor -- start 2023-12-11T17:43:59.500 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:59.495+0000 7f2a3f84f700 1 -- start start 2023-12-11T17:43:59.500 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:59.495+0000 7f2a3f84f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2a38101f10 0x7f2a38199630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:43:59.500 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:59.495+0000 7f2a3f84f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a38104840 0x7f2a38199b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:43:59.500 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:59.495+0000 7f2a3f84f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2a38107170 0x7f2a381a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:43:59.500 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:59.495+0000 7f2a3f84f700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a381a1160 con 0x7f2a38101f10 2023-12-11T17:43:59.500 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:59.495+0000 7f2a3f84f700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a381a12a0 con 0x7f2a38104840 2023-12-11T17:43:59.500 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:59.495+0000 7f2a3f84f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a381a1430 con 0x7f2a38107170 2023-12-11T17:43:59.500 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:59.495+0000 7f2a3d5eb700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2a38101f10 0x7f2a38199630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:43:59.501 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:59.495+0000 7f2a3d5eb700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2a38101f10 0x7f2a38199630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60874/0 (socket says 172.21.15.59:60874) 2023-12-11T17:43:59.501 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:59.495+0000 7f2a3d5eb700 1 -- 172.21.15.59:0/3772952317 learned_addr learned my addr 172.21.15.59:0/3772952317 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:43:59.501 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:59.495+0000 7f2a3cdea700 1 --2- 172.21.15.59:0/3772952317 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a38104840 0x7f2a38199b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:43:59.501 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:59.496+0000 7f2a3ddec700 1 --2- 172.21.15.59:0/3772952317 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2a38107170 0x7f2a381a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:43:59.501 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:59.496+0000 7f2a3d5eb700 1 -- 172.21.15.59:0/3772952317 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2a38107170 msgr2=0x7f2a381a0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:43:59.501 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:59.496+0000 7f2a3d5eb700 1 --2- 172.21.15.59:0/3772952317 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2a38107170 0x7f2a381a0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:43:59.501 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:59.496+0000 7f2a3d5eb700 1 -- 172.21.15.59:0/3772952317 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a38104840 msgr2=0x7f2a38199b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:43:59.501 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:59.496+0000 7f2a3d5eb700 1 --2- 172.21.15.59:0/3772952317 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a38104840 0x7f2a38199b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:43:59.501 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:59.496+0000 7f2a3d5eb700 1 -- 172.21.15.59:0/3772952317 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a340095f0 con 0x7f2a38101f10 2023-12-11T17:43:59.501 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:59.496+0000 7f2a3cdea700 1 --2- 172.21.15.59:0/3772952317 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a38104840 0x7f2a38199b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T17:43:59.501 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:59.496+0000 7f2a3d5eb700 1 --2- 172.21.15.59:0/3772952317 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2a38101f10 0x7f2a38199630 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2a2c00e9a0 tx=0x7f2a2c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:43:59.502 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:59.496+0000 7f2a2a7fc700 1 -- 172.21.15.59:0/3772952317 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a2c009790 con 0x7f2a38101f10 2023-12-11T17:43:59.502 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:59.496+0000 7f2a3f84f700 1 -- 172.21.15.59:0/3772952317 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a381a1710 con 0x7f2a38101f10 2023-12-11T17:43:59.502 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:59.496+0000 7f2a2a7fc700 1 -- 172.21.15.59:0/3772952317 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2a2c010430 con 0x7f2a38101f10 2023-12-11T17:43:59.502 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:59.496+0000 7f2a2a7fc700 1 -- 172.21.15.59:0/3772952317 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a2c018680 con 0x7f2a38101f10 2023-12-11T17:43:59.502 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:59.496+0000 7f2a3f84f700 1 -- 172.21.15.59:0/3772952317 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a381a1c70 con 0x7f2a38101f10 2023-12-11T17:43:59.503 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:59.497+0000 7f2a3f84f700 1 -- 172.21.15.59:0/3772952317 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a38108d70 con 0x7f2a38101f10 2023-12-11T17:43:59.503 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:59.498+0000 7f2a2a7fc700 1 -- 172.21.15.59:0/3772952317 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f2a2c0105a0 con 0x7f2a38101f10 2023-12-11T17:43:59.503 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:59.499+0000 7f2a2a7fc700 1 --2- 172.21.15.59:0/3772952317 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f2a240a1480 0x7f2a240a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:43:59.503 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:59.499+0000 7f2a2a7fc700 1 -- 172.21.15.59:0/3772952317 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f2a2c014070 con 0x7f2a38101f10 2023-12-11T17:43:59.503 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:59.499+0000 7f2a3cdea700 1 --2- 172.21.15.59:0/3772952317 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f2a240a1480 0x7f2a240a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:43:59.503 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:59.500+0000 7f2a3cdea700 1 --2- 172.21.15.59:0/3772952317 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f2a240a1480 0x7f2a240a3940 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f2a3400b2e0 tx=0x7f2a34005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:43:59.505 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:59.502+0000 7f2a2a7fc700 1 -- 172.21.15.59:0/3772952317 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f2a2c096700 con 0x7f2a38101f10 2023-12-11T17:43:59.533 INFO:journalctl@ceph.mds.e.smithi115.stdout:Dec 11 17:43:59 smithi115 ceph-mds[139042]: -- 172.21.15.115:0/3576811188 <== mon.2 v2:172.21.15.115:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55a2beb21520 con 0x55a2beaf7000 2023-12-11T17:43:59.533 INFO:journalctl@ceph.mds.e.smithi115.stdout:Dec 11 17:43:59 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mds-e[139018]: starting mds.e at 2023-12-11T17:43:59.533 INFO:journalctl@ceph.mds.e.smithi115.stdout:Dec 11 17:43:59 smithi115 bash[139002]: 3613dbeb01a5af53e75476d185b1bc2f37cf784faad9f366f646f9483f8fc861 2023-12-11T17:43:59.533 INFO:journalctl@ceph.mds.e.smithi115.stdout:Dec 11 17:43:59 smithi115 systemd[1]: Started Ceph mds.e for d0a07eaa-984b-11ee-95a3-87774f69a715. 2023-12-11T17:43:59.752 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:59.750+0000 7f2a3f84f700 1 -- 172.21.15.59:0/3772952317 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2a380679a0 con 0x7f2a38101f10 2023-12-11T17:43:59.753 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:59.751+0000 7f2a2a7fc700 1 -- 172.21.15.59:0/3772952317 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+3701 (secure 0 0 0) 0x7f2a2c099d20 con 0x7f2a38101f10 2023-12-11T17:43:59.753 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-12-11T17:43:59.753 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/2184151593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":2184151593},{"type":"v1","addr":"172.21.15.59:6835","nonce":2184151593}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/2817614886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2817614886},{"type":"v1","addr":"172.21.15.59:6837","nonce":2817614886}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/395684042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":395684042},{"type":"v1","addr":"172.21.15.181:6833","nonce":395684042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6833/3214606191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3214606191},{"type":"v1","addr":"172.21.15.115:6833","nonce":3214606191}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2023-12-11T17:43:59.755 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:59.753+0000 7f2a3f84f700 1 -- 172.21.15.59:0/3772952317 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f2a240a1480 msgr2=0x7f2a240a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:43:59.756 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:59.753+0000 7f2a3f84f700 1 --2- 172.21.15.59:0/3772952317 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f2a240a1480 0x7f2a240a3940 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f2a3400b2e0 tx=0x7f2a34005fb0 comp rx=0 tx=0).stop 2023-12-11T17:43:59.756 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:59.753+0000 7f2a3f84f700 1 -- 172.21.15.59:0/3772952317 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2a38101f10 msgr2=0x7f2a38199630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:43:59.756 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:59.753+0000 7f2a3f84f700 1 --2- 172.21.15.59:0/3772952317 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2a38101f10 0x7f2a38199630 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2a2c00e9a0 tx=0x7f2a2c00ecb0 comp rx=0 tx=0).stop 2023-12-11T17:43:59.756 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:59.753+0000 7f2a3f84f700 1 -- 172.21.15.59:0/3772952317 shutdown_connections 2023-12-11T17:43:59.756 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:59.753+0000 7f2a3f84f700 1 --2- 172.21.15.59:0/3772952317 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f2a240a1480 0x7f2a240a3940 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:43:59.756 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:59.753+0000 7f2a3f84f700 1 --2- 172.21.15.59:0/3772952317 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2a38107170 0x7f2a381a0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:43:59.756 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:59.753+0000 7f2a3f84f700 1 --2- 172.21.15.59:0/3772952317 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2a38101f10 0x7f2a38199630 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:43:59.756 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:59.753+0000 7f2a3f84f700 1 --2- 172.21.15.59:0/3772952317 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a38104840 0x7f2a38199b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:43:59.756 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:59.753+0000 7f2a3f84f700 1 -- 172.21.15.59:0/3772952317 >> 172.21.15.59:0/3772952317 conn(0x7f2a380fb920 msgr2=0x7f2a381017a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:43:59.756 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:59.753+0000 7f2a3f84f700 1 -- 172.21.15.59:0/3772952317 shutdown_connections 2023-12-11T17:43:59.756 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:43:59.753+0000 7f2a3f84f700 1 -- 172.21.15.59:0/3772952317 wait complete. 2023-12-11T17:43:59.757 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 5 2023-12-11T17:44:00.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:59 smithi115 ceph-mon[102748]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T17:44:00.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:59 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:44:00.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:59 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:44:00.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:59 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:44:00.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:59 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-11T17:44:00.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:59 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-11T17:44:00.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:59 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:44:00.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:43:59 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/3772952317' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T17:44:00.034 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:59 smithi181 ceph-mon[103627]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T17:44:00.034 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:59 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:44:00.034 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:59 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:44:00.034 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:59 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:44:00.034 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:59 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-11T17:44:00.034 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:59 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-11T17:44:00.034 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:59 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:44:00.034 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:43:59 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/3772952317' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T17:44:00.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:59 smithi059 ceph-mon[98847]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T17:44:00.155 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:59 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:44:00.155 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:59 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:44:00.155 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:59 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:44:00.155 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:59 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-11T17:44:00.155 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:59 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-11T17:44:00.155 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:59 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:44:00.155 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:43:59 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/3772952317' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T17:44:01.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:00 smithi115 ceph-mon[102748]: Deploying daemon mds.f on smithi181 2023-12-11T17:44:01.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:00 smithi115 ceph-mon[102748]: mds.? [v2:172.21.15.115:6834/1874159657,v1:172.21.15.115:6835/1874159657] up:boot 2023-12-11T17:44:01.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:00 smithi115 ceph-mon[102748]: fsmap 5 up:standby 2023-12-11T17:44:01.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:00 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-12-11T17:44:01.145 INFO:journalctl@ceph.mds.f.smithi181.stdout:Dec 11 17:44:00 smithi181 systemd[1]: Starting Ceph mds.f for d0a07eaa-984b-11ee-95a3-87774f69a715... 2023-12-11T17:44:01.146 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:00 smithi181 ceph-mon[103627]: Deploying daemon mds.f on smithi181 2023-12-11T17:44:01.146 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:00 smithi181 ceph-mon[103627]: mds.? [v2:172.21.15.115:6834/1874159657,v1:172.21.15.115:6835/1874159657] up:boot 2023-12-11T17:44:01.146 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:00 smithi181 ceph-mon[103627]: fsmap 5 up:standby 2023-12-11T17:44:01.146 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:00 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-12-11T17:44:01.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:00 smithi059 ceph-mon[98847]: Deploying daemon mds.f on smithi181 2023-12-11T17:44:01.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:00 smithi059 ceph-mon[98847]: mds.? [v2:172.21.15.115:6834/1874159657,v1:172.21.15.115:6835/1874159657] up:boot 2023-12-11T17:44:01.155 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:00 smithi059 ceph-mon[98847]: fsmap 5 up:standby 2023-12-11T17:44:01.155 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:00 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-12-11T17:44:01.475 INFO:journalctl@ceph.mds.f.smithi181.stdout:Dec 11 17:44:01 smithi181 podman[140838]: 2023-12-11T17:44:01.768 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-11T17:44:01.781 INFO:journalctl@ceph.mds.f.smithi181.stdout:Dec 11 17:44:01 smithi181 ceph-mds[140878]: -- 172.21.15.181:0/773404520 <== mon.1 v2:172.21.15.181:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55d9a7b7b520 con 0x55d9a7b51000 2023-12-11T17:44:01.782 INFO:journalctl@ceph.mds.f.smithi181.stdout:Dec 11 17:44:01 smithi181 bash[140838]: 500dc1c7c0a89746a71438fbf5ab3d7cbdfa870bf211fadfc4b023c65e4f593d 2023-12-11T17:44:01.782 INFO:journalctl@ceph.mds.f.smithi181.stdout:Dec 11 17:44:01 smithi181 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mds-f[140854]: starting mds.f at 2023-12-11T17:44:01.782 INFO:journalctl@ceph.mds.f.smithi181.stdout:Dec 11 17:44:01 smithi181 systemd[1]: Started Ceph mds.f for d0a07eaa-984b-11ee-95a3-87774f69a715. 2023-12-11T17:44:01.911 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:01.906+0000 7fc74ee49700 1 -- 172.21.15.59:0/268207286 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc748100380 msgr2=0x7fc7481007a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:01.911 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:01.906+0000 7fc74ee49700 1 --2- 172.21.15.59:0/268207286 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc748100380 0x7fc7481007a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc73c0099c0 tx=0x7fc73c009cd0 comp rx=0 tx=0).stop 2023-12-11T17:44:01.911 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:01.906+0000 7fc74ee49700 1 -- 172.21.15.59:0/268207286 shutdown_connections 2023-12-11T17:44:01.911 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:01.906+0000 7fc74ee49700 1 --2- 172.21.15.59:0/268207286 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc748101f10 0x7fc7481087c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:01.911 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:01.906+0000 7fc74ee49700 1 --2- 172.21.15.59:0/268207286 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc748100380 0x7fc7481007a0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:01.911 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:01.906+0000 7fc74ee49700 1 --2- 172.21.15.59:0/268207286 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc748101570 0x7fc7481019d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:01.911 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:01.906+0000 7fc74ee49700 1 -- 172.21.15.59:0/268207286 >> 172.21.15.59:0/268207286 conn(0x7fc7480fb8f0 msgr2=0x7fc7480fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:44:01.911 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:01.906+0000 7fc74ee49700 1 -- 172.21.15.59:0/268207286 shutdown_connections 2023-12-11T17:44:01.912 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:01.906+0000 7fc74ee49700 1 -- 172.21.15.59:0/268207286 wait complete. 2023-12-11T17:44:01.912 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:01.907+0000 7fc74ee49700 1 Processor -- start 2023-12-11T17:44:01.912 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:01.907+0000 7fc74ee49700 1 -- start start 2023-12-11T17:44:01.912 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:01.907+0000 7fc74ee49700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc748100380 0x7fc748197420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:01.912 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:01.907+0000 7fc74ee49700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc748101570 0x7fc748197960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:01.912 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:01.907+0000 7fc74ee49700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc748101f10 0x7fc74819e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:01.912 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:01.907+0000 7fc74ee49700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc74819ef50 con 0x7fc748100380 2023-12-11T17:44:01.912 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:01.907+0000 7fc74ee49700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc74819f090 con 0x7fc748101f10 2023-12-11T17:44:01.912 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:01.907+0000 7fc74ee49700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc74819f220 con 0x7fc748101570 2023-12-11T17:44:01.912 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:01.907+0000 7fc74cbe5700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc748100380 0x7fc748197420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:01.912 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:01.907+0000 7fc74cbe5700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc748100380 0x7fc748197420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60888/0 (socket says 172.21.15.59:60888) 2023-12-11T17:44:01.912 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:01.907+0000 7fc74cbe5700 1 -- 172.21.15.59:0/2142072223 learned_addr learned my addr 172.21.15.59:0/2142072223 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:44:01.912 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:01.908+0000 7fc74d3e6700 1 --2- 172.21.15.59:0/2142072223 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc748101f10 0x7fc74819e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:01.912 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:01.908+0000 7fc747fff700 1 --2- 172.21.15.59:0/2142072223 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc748101570 0x7fc748197960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:01.913 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:01.908+0000 7fc74cbe5700 1 -- 172.21.15.59:0/2142072223 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc748101570 msgr2=0x7fc748197960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:01.913 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:01.908+0000 7fc74cbe5700 1 --2- 172.21.15.59:0/2142072223 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc748101570 0x7fc748197960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:01.913 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:01.908+0000 7fc74cbe5700 1 -- 172.21.15.59:0/2142072223 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc748101f10 msgr2=0x7fc74819e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:01.913 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:01.908+0000 7fc74cbe5700 1 --2- 172.21.15.59:0/2142072223 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc748101f10 0x7fc74819e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:01.913 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:01.908+0000 7fc74cbe5700 1 -- 172.21.15.59:0/2142072223 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc73c0095f0 con 0x7fc748100380 2023-12-11T17:44:01.913 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:01.908+0000 7fc74d3e6700 1 --2- 172.21.15.59:0/2142072223 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc748101f10 0x7fc74819e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T17:44:01.913 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:01.908+0000 7fc74cbe5700 1 --2- 172.21.15.59:0/2142072223 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc748100380 0x7fc748197420 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc73c003820 tx=0x7fc73c005ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:44:01.913 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:01.908+0000 7fc745ffb700 1 -- 172.21.15.59:0/2142072223 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc73c01d070 con 0x7fc748100380 2023-12-11T17:44:01.913 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:01.908+0000 7fc745ffb700 1 -- 172.21.15.59:0/2142072223 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc73c004110 con 0x7fc748100380 2023-12-11T17:44:01.913 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:01.908+0000 7fc74ee49700 1 -- 172.21.15.59:0/2142072223 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc74819f4a0 con 0x7fc748100380 2023-12-11T17:44:01.913 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:01.908+0000 7fc745ffb700 1 -- 172.21.15.59:0/2142072223 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc73c022e40 con 0x7fc748100380 2023-12-11T17:44:01.913 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:01.908+0000 7fc74ee49700 1 -- 172.21.15.59:0/2142072223 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc74819f950 con 0x7fc748100380 2023-12-11T17:44:01.914 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:01.909+0000 7fc74ee49700 1 -- 172.21.15.59:0/2142072223 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc748068230 con 0x7fc748100380 2023-12-11T17:44:01.914 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:01.910+0000 7fc745ffb700 1 -- 172.21.15.59:0/2142072223 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7fc73c022570 con 0x7fc748100380 2023-12-11T17:44:01.914 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:01.911+0000 7fc745ffb700 1 --2- 172.21.15.59:0/2142072223 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fc7380a1480 0x7fc7380a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:01.914 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:01.911+0000 7fc747fff700 1 --2- 172.21.15.59:0/2142072223 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fc7380a1480 0x7fc7380a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:01.915 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:01.911+0000 7fc745ffb700 1 -- 172.21.15.59:0/2142072223 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fc73c0ce150 con 0x7fc748100380 2023-12-11T17:44:01.915 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:01.912+0000 7fc747fff700 1 --2- 172.21.15.59:0/2142072223 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fc7380a1480 0x7fc7380a3940 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fc734005950 tx=0x7fc7340058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:44:01.917 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:01.915+0000 7fc745ffb700 1 -- 172.21.15.59:0/2142072223 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fc73c097c40 con 0x7fc748100380 2023-12-11T17:44:02.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:01 smithi115 ceph-mon[102748]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T17:44:02.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:01 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:44:02.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:01 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:44:02.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:01 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:44:02.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:01 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-11T17:44:02.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:01 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-11T17:44:02.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:01 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:44:02.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:01 smithi181 ceph-mon[103627]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T17:44:02.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:01 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:44:02.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:01 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:44:02.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:01 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:44:02.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:01 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-11T17:44:02.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:01 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-11T17:44:02.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:01 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:44:02.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:01 smithi059 ceph-mon[98847]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T17:44:02.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:01 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:44:02.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:01 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:44:02.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:01 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:44:02.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:01 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-11T17:44:02.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:01 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-11T17:44:02.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:01 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:44:02.166 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:02.164+0000 7fc74ee49700 1 -- 172.21.15.59:0/2142072223 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc74810bb90 con 0x7fc748100380 2023-12-11T17:44:02.167 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:02.165+0000 7fc745ffb700 1 -- 172.21.15.59:0/2142072223 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+5289 (secure 0 0 0) 0x7fc73c09b260 con 0x7fc748100380 2023-12-11T17:44:02.167 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-12-11T17:44:02.167 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/2184151593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":2184151593},{"type":"v1","addr":"172.21.15.59:6835","nonce":2184151593}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/2817614886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2817614886},{"type":"v1","addr":"172.21.15.59:6837","nonce":2817614886}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/395684042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":395684042},{"type":"v1","addr":"172.21.15.181:6833","nonce":395684042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6833/3214606191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3214606191},{"type":"v1","addr":"172.21.15.115:6833","nonce":3214606191}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24449,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/1874159657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1874159657},{"type":"v1","addr":"172.21.15.115:6835","nonce":1874159657}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/4221183620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":4221183620},{"type":"v1","addr":"172.21.15.181:6835","nonce":4221183620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7}],"filesystems":[]} 2023-12-11T17:44:02.169 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:02.167+0000 7fc74ee49700 1 -- 172.21.15.59:0/2142072223 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fc7380a1480 msgr2=0x7fc7380a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:02.170 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:02.167+0000 7fc74ee49700 1 --2- 172.21.15.59:0/2142072223 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fc7380a1480 0x7fc7380a3940 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fc734005950 tx=0x7fc7340058e0 comp rx=0 tx=0).stop 2023-12-11T17:44:02.170 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:02.167+0000 7fc74ee49700 1 -- 172.21.15.59:0/2142072223 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc748100380 msgr2=0x7fc748197420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:02.170 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:02.167+0000 7fc74ee49700 1 --2- 172.21.15.59:0/2142072223 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc748100380 0x7fc748197420 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc73c003820 tx=0x7fc73c005ad0 comp rx=0 tx=0).stop 2023-12-11T17:44:02.170 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:02.167+0000 7fc74ee49700 1 -- 172.21.15.59:0/2142072223 shutdown_connections 2023-12-11T17:44:02.170 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:02.167+0000 7fc74ee49700 1 --2- 172.21.15.59:0/2142072223 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fc7380a1480 0x7fc7380a3940 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:02.170 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:02.167+0000 7fc74ee49700 1 --2- 172.21.15.59:0/2142072223 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc748101f10 0x7fc74819e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:02.170 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:02.167+0000 7fc74ee49700 1 --2- 172.21.15.59:0/2142072223 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc748100380 0x7fc748197420 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:02.170 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:02.167+0000 7fc74ee49700 1 --2- 172.21.15.59:0/2142072223 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc748101570 0x7fc748197960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:02.170 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:02.167+0000 7fc74ee49700 1 -- 172.21.15.59:0/2142072223 >> 172.21.15.59:0/2142072223 conn(0x7fc7480fb8f0 msgr2=0x7fc748068640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:44:02.170 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:02.167+0000 7fc74ee49700 1 -- 172.21.15.59:0/2142072223 shutdown_connections 2023-12-11T17:44:02.170 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:02.167+0000 7fc74ee49700 1 -- 172.21.15.59:0/2142072223 wait complete. 2023-12-11T17:44:02.171 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 7 2023-12-11T17:44:02.850 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:02 smithi181 ceph-mon[103627]: Deploying daemon mds.c on smithi181 2023-12-11T17:44:02.850 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:02 smithi181 ceph-mon[103627]: mds.? [v2:172.21.15.181:6834/4221183620,v1:172.21.15.181:6835/4221183620] up:boot 2023-12-11T17:44:02.850 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:02 smithi181 ceph-mon[103627]: fsmap 6 up:standby 2023-12-11T17:44:02.850 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:02 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-12-11T17:44:02.850 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:02 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/2142072223' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T17:44:03.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:02 smithi059 ceph-mon[98847]: Deploying daemon mds.c on smithi181 2023-12-11T17:44:03.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:02 smithi059 ceph-mon[98847]: mds.? [v2:172.21.15.181:6834/4221183620,v1:172.21.15.181:6835/4221183620] up:boot 2023-12-11T17:44:03.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:02 smithi059 ceph-mon[98847]: fsmap 6 up:standby 2023-12-11T17:44:03.155 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:02 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-12-11T17:44:03.155 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:02 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/2142072223' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T17:44:03.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:02 smithi115 ceph-mon[102748]: Deploying daemon mds.c on smithi181 2023-12-11T17:44:03.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:02 smithi115 ceph-mon[102748]: mds.? [v2:172.21.15.181:6834/4221183620,v1:172.21.15.181:6835/4221183620] up:boot 2023-12-11T17:44:03.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:02 smithi115 ceph-mon[102748]: fsmap 6 up:standby 2023-12-11T17:44:03.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:02 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-12-11T17:44:03.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:02 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/2142072223' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T17:44:03.395 INFO:journalctl@ceph.mds.c.smithi181.stdout:Dec 11 17:44:03 smithi181 systemd[1]: Starting Ceph mds.c for d0a07eaa-984b-11ee-95a3-87774f69a715... 2023-12-11T17:44:03.659 INFO:journalctl@ceph.mds.c.smithi181.stdout:Dec 11 17:44:03 smithi181 podman[141176]: 2023-12-11T17:44:04.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:03 smithi181 ceph-mon[103627]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T17:44:04.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:03 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:44:04.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:03 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:44:04.145 INFO:journalctl@ceph.mds.c.smithi181.stdout:Dec 11 17:44:03 smithi181 ceph-mds[141216]: -- 172.21.15.181:0/2980205271 <== mon.1 v2:172.21.15.181:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x564c9928b520 con 0x564c99260c00 2023-12-11T17:44:04.145 INFO:journalctl@ceph.mds.c.smithi181.stdout:Dec 11 17:44:03 smithi181 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mds-c[141192]: starting mds.c at 2023-12-11T17:44:04.146 INFO:journalctl@ceph.mds.c.smithi181.stdout:Dec 11 17:44:03 smithi181 bash[141176]: 9a72aa6edc69bec0c9229781c8400e6bff8e7dd55889323c3a3e49c3ae2eabac 2023-12-11T17:44:04.146 INFO:journalctl@ceph.mds.c.smithi181.stdout:Dec 11 17:44:03 smithi181 systemd[1]: Started Ceph mds.c for d0a07eaa-984b-11ee-95a3-87774f69a715. 2023-12-11T17:44:04.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:03 smithi059 ceph-mon[98847]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T17:44:04.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:03 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:44:04.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:03 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:44:04.183 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-11T17:44:04.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:03 smithi115 ceph-mon[102748]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T17:44:04.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:03 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:44:04.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:03 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:44:04.323 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:04.320+0000 7f3bca8fa700 1 -- 172.21.15.59:0/2785958159 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3bc4107160 msgr2=0x7f3bc41095d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:04.323 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:04.320+0000 7f3bca8fa700 1 --2- 172.21.15.59:0/2785958159 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3bc4107160 0x7f3bc41095d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc00b0a0 tx=0x7f3bbc00b3b0 comp rx=0 tx=0).stop 2023-12-11T17:44:04.323 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:04.320+0000 7f3bca8fa700 1 -- 172.21.15.59:0/2785958159 shutdown_connections 2023-12-11T17:44:04.323 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:04.320+0000 7f3bca8fa700 1 --2- 172.21.15.59:0/2785958159 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3bc4107160 0x7f3bc41095d0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:04.323 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:04.320+0000 7f3bca8fa700 1 --2- 172.21.15.59:0/2785958159 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3bc4101f00 0x7f3bc41042f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:04.323 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:04.320+0000 7f3bca8fa700 1 --2- 172.21.15.59:0/2785958159 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3bc4104830 0x7f3bc4106c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:04.324 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:04.320+0000 7f3bca8fa700 1 -- 172.21.15.59:0/2785958159 >> 172.21.15.59:0/2785958159 conn(0x7f3bc40fb8f0 msgr2=0x7f3bc40fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:44:04.324 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:04.320+0000 7f3bca8fa700 1 -- 172.21.15.59:0/2785958159 shutdown_connections 2023-12-11T17:44:04.324 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:04.320+0000 7f3bca8fa700 1 -- 172.21.15.59:0/2785958159 wait complete. 2023-12-11T17:44:04.324 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:04.321+0000 7f3bca8fa700 1 Processor -- start 2023-12-11T17:44:04.324 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:04.321+0000 7f3bca8fa700 1 -- start start 2023-12-11T17:44:04.326 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:04.321+0000 7f3bca8fa700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3bc4101f00 0x7f3bc41974a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:04.326 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:04.321+0000 7f3bca8fa700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3bc4104830 0x7f3bc41979e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:04.326 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:04.321+0000 7f3bca8fa700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3bc4107160 0x7f3bc419ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:04.326 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:04.321+0000 7f3bca8fa700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bc419efd0 con 0x7f3bc4104830 2023-12-11T17:44:04.326 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:04.321+0000 7f3bca8fa700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bc419f110 con 0x7f3bc4101f00 2023-12-11T17:44:04.326 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:04.321+0000 7f3bca8fa700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bc419f2a0 con 0x7f3bc4107160 2023-12-11T17:44:04.326 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:04.321+0000 7f3bc37fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3bc4104830 0x7f3bc41979e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:04.326 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:04.321+0000 7f3bc37fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3bc4104830 0x7f3bc41979e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60904/0 (socket says 172.21.15.59:60904) 2023-12-11T17:44:04.326 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:04.321+0000 7f3bc37fe700 1 -- 172.21.15.59:0/2809411190 learned_addr learned my addr 172.21.15.59:0/2809411190 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:44:04.327 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:04.321+0000 7f3bc3fff700 1 --2- 172.21.15.59:0/2809411190 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3bc4101f00 0x7f3bc41974a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:04.327 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:04.322+0000 7f3bc8e97700 1 --2- 172.21.15.59:0/2809411190 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3bc4107160 0x7f3bc419ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:04.327 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:04.322+0000 7f3bc37fe700 1 -- 172.21.15.59:0/2809411190 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3bc4107160 msgr2=0x7f3bc419ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:04.327 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:04.322+0000 7f3bc37fe700 1 --2- 172.21.15.59:0/2809411190 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3bc4107160 0x7f3bc419ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:04.327 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:04.322+0000 7f3bc37fe700 1 -- 172.21.15.59:0/2809411190 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3bc4101f00 msgr2=0x7f3bc41974a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:04.327 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:04.322+0000 7f3bc37fe700 1 --2- 172.21.15.59:0/2809411190 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3bc4101f00 0x7f3bc41974a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:04.327 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:04.322+0000 7f3bc37fe700 1 -- 172.21.15.59:0/2809411190 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bbc009d00 con 0x7f3bc4104830 2023-12-11T17:44:04.327 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:04.322+0000 7f3bc8e97700 1 --2- 172.21.15.59:0/2809411190 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3bc4107160 0x7f3bc419ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T17:44:04.327 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:04.322+0000 7f3bc37fe700 1 --2- 172.21.15.59:0/2809411190 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3bc4104830 0x7f3bc41979e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f3bb800d950 tx=0x7f3bb800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:44:04.327 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:04.322+0000 7f3bc17fa700 1 -- 172.21.15.59:0/2809411190 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bb8009940 con 0x7f3bc4104830 2023-12-11T17:44:04.327 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:04.322+0000 7f3bc17fa700 1 -- 172.21.15.59:0/2809411190 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3bb8010460 con 0x7f3bc4104830 2023-12-11T17:44:04.327 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:04.322+0000 7f3bca8fa700 1 -- 172.21.15.59:0/2809411190 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bc419f580 con 0x7f3bc4104830 2023-12-11T17:44:04.327 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:04.322+0000 7f3bc17fa700 1 -- 172.21.15.59:0/2809411190 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bb800f620 con 0x7f3bc4104830 2023-12-11T17:44:04.327 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:04.322+0000 7f3bca8fa700 1 -- 172.21.15.59:0/2809411190 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bc419fa90 con 0x7f3bc4104830 2023-12-11T17:44:04.327 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:04.323+0000 7f3bca8fa700 1 -- 172.21.15.59:0/2809411190 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bc40fd470 con 0x7f3bc4104830 2023-12-11T17:44:04.329 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:04.324+0000 7f3bc17fa700 1 -- 172.21.15.59:0/2809411190 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f3bb80108d0 con 0x7f3bc4104830 2023-12-11T17:44:04.329 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:04.325+0000 7f3bc17fa700 1 --2- 172.21.15.59:0/2809411190 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f3bac0a1480 0x7f3bac0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:04.329 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:04.325+0000 7f3bc17fa700 1 -- 172.21.15.59:0/2809411190 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f3bb80cc6d0 con 0x7f3bc4104830 2023-12-11T17:44:04.330 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:04.325+0000 7f3bc3fff700 1 --2- 172.21.15.59:0/2809411190 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f3bac0a1480 0x7f3bac0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:04.330 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:04.325+0000 7f3bc3fff700 1 --2- 172.21.15.59:0/2809411190 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f3bac0a1480 0x7f3bac0a3940 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f3bb4009c60 tx=0x7f3bb4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:44:04.332 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:04.330+0000 7f3bc17fa700 1 -- 172.21.15.59:0/2809411190 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f3bb80961d0 con 0x7f3bc4104830 2023-12-11T17:44:04.584 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:04.581+0000 7f3bca8fa700 1 -- 172.21.15.59:0/2809411190 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3bc4068230 con 0x7f3bc4104830 2023-12-11T17:44:04.584 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:04.581+0000 7f3bc17fa700 1 -- 172.21.15.59:0/2809411190 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+6083 (secure 0 0 0) 0x7f3bb80997f0 con 0x7f3bc4104830 2023-12-11T17:44:04.584 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-12-11T17:44:04.585 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/2184151593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":2184151593},{"type":"v1","addr":"172.21.15.59:6835","nonce":2184151593}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/2817614886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2817614886},{"type":"v1","addr":"172.21.15.59:6837","nonce":2817614886}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/395684042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":395684042},{"type":"v1","addr":"172.21.15.181:6833","nonce":395684042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6833/3214606191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3214606191},{"type":"v1","addr":"172.21.15.115:6833","nonce":3214606191}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24449,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/1874159657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1874159657},{"type":"v1","addr":"172.21.15.115:6835","nonce":1874159657}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/4221183620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":4221183620},{"type":"v1","addr":"172.21.15.181:6835","nonce":4221183620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24466,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/3580309407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":3580309407},{"type":"v1","addr":"172.21.15.181:6837","nonce":3580309407}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8}],"filesystems":[]} 2023-12-11T17:44:04.587 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:04.584+0000 7f3bca8fa700 1 -- 172.21.15.59:0/2809411190 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f3bac0a1480 msgr2=0x7f3bac0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:04.587 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:04.584+0000 7f3bca8fa700 1 --2- 172.21.15.59:0/2809411190 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f3bac0a1480 0x7f3bac0a3940 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f3bb4009c60 tx=0x7f3bb4009260 comp rx=0 tx=0).stop 2023-12-11T17:44:04.587 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:04.584+0000 7f3bca8fa700 1 -- 172.21.15.59:0/2809411190 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3bc4104830 msgr2=0x7f3bc41979e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:04.587 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:04.584+0000 7f3bca8fa700 1 --2- 172.21.15.59:0/2809411190 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3bc4104830 0x7f3bc41979e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f3bb800d950 tx=0x7f3bb800dc60 comp rx=0 tx=0).stop 2023-12-11T17:44:04.588 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:04.584+0000 7f3bca8fa700 1 -- 172.21.15.59:0/2809411190 shutdown_connections 2023-12-11T17:44:04.588 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:04.584+0000 7f3bca8fa700 1 --2- 172.21.15.59:0/2809411190 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f3bac0a1480 0x7f3bac0a3940 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:04.588 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:04.584+0000 7f3bca8fa700 1 --2- 172.21.15.59:0/2809411190 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3bc4107160 0x7f3bc419ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:04.588 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:04.584+0000 7f3bca8fa700 1 --2- 172.21.15.59:0/2809411190 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3bc4101f00 0x7f3bc41974a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:04.588 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:04.584+0000 7f3bca8fa700 1 --2- 172.21.15.59:0/2809411190 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3bc4104830 0x7f3bc41979e0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:04.588 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:04.584+0000 7f3bca8fa700 1 -- 172.21.15.59:0/2809411190 >> 172.21.15.59:0/2809411190 conn(0x7f3bc40fb8f0 msgr2=0x7f3bc4102860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:44:04.589 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:04.584+0000 7f3bca8fa700 1 -- 172.21.15.59:0/2809411190 shutdown_connections 2023-12-11T17:44:04.589 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:04.584+0000 7f3bca8fa700 1 -- 172.21.15.59:0/2809411190 wait complete. 2023-12-11T17:44:04.589 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 8 2023-12-11T17:44:04.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:04 smithi059 ceph-mon[98847]: mds.? [v2:172.21.15.181:6836/3580309407,v1:172.21.15.181:6837/3580309407] up:boot 2023-12-11T17:44:04.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:04 smithi059 ceph-mon[98847]: fsmap 7 up:standby 2023-12-11T17:44:04.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:04 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-12-11T17:44:04.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:04 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:44:04.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:04 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-11T17:44:04.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:04 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-11T17:44:04.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:04 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:44:04.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:04 smithi059 ceph-mon[98847]: Deploying daemon mds.h on smithi115 2023-12-11T17:44:04.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:04 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/2809411190' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T17:44:05.139 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:04 smithi115 ceph-mon[102748]: mds.? [v2:172.21.15.181:6836/3580309407,v1:172.21.15.181:6837/3580309407] up:boot 2023-12-11T17:44:05.140 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:04 smithi115 ceph-mon[102748]: fsmap 7 up:standby 2023-12-11T17:44:05.140 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:04 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-12-11T17:44:05.140 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:04 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:44:05.140 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:04 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-11T17:44:05.140 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:04 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-11T17:44:05.140 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:04 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:44:05.140 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:04 smithi115 ceph-mon[102748]: Deploying daemon mds.h on smithi115 2023-12-11T17:44:05.140 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:04 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/2809411190' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T17:44:05.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:04 smithi181 ceph-mon[103627]: mds.? [v2:172.21.15.181:6836/3580309407,v1:172.21.15.181:6837/3580309407] up:boot 2023-12-11T17:44:05.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:04 smithi181 ceph-mon[103627]: fsmap 7 up:standby 2023-12-11T17:44:05.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:04 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-12-11T17:44:05.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:04 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:44:05.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:04 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-11T17:44:05.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:04 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-11T17:44:05.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:04 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:44:05.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:04 smithi181 ceph-mon[103627]: Deploying daemon mds.h on smithi115 2023-12-11T17:44:05.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:04 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/2809411190' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T17:44:05.533 INFO:journalctl@ceph.mds.h.smithi115.stdout:Dec 11 17:44:05 smithi115 systemd[1]: Starting Ceph mds.h for d0a07eaa-984b-11ee-95a3-87774f69a715... 2023-12-11T17:44:05.961 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:05 smithi115 ceph-mon[102748]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T17:44:05.961 INFO:journalctl@ceph.mds.h.smithi115.stdout:Dec 11 17:44:05 smithi115 podman[139338]: 2023-12-11T17:44:05.961 INFO:journalctl@ceph.mds.h.smithi115.stdout:Dec 11 17:44:05 smithi115 ceph-mds[139378]: -- 172.21.15.115:0/4052803000 <== mon.2 v2:172.21.15.115:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55dcb8229520 con 0x55dcb81ff000 2023-12-11T17:44:06.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:05 smithi181 ceph-mon[103627]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T17:44:06.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:05 smithi059 ceph-mon[98847]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T17:44:06.283 INFO:journalctl@ceph.mds.h.smithi115.stdout:Dec 11 17:44:05 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mds-h[139354]: starting mds.h at 2023-12-11T17:44:06.283 INFO:journalctl@ceph.mds.h.smithi115.stdout:Dec 11 17:44:05 smithi115 bash[139338]: 4e1cbc7bbb6f89a924c2b1ba3d0dab7cf9a9b54915e3a7fb538cd05654ae08fc 2023-12-11T17:44:06.283 INFO:journalctl@ceph.mds.h.smithi115.stdout:Dec 11 17:44:06 smithi115 systemd[1]: Started Ceph mds.h for d0a07eaa-984b-11ee-95a3-87774f69a715. 2023-12-11T17:44:06.601 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-11T17:44:06.741 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:06.738+0000 7fa465545700 1 -- 172.21.15.59:0/1641172666 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa460101f30 msgr2=0x7fa460104320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:06.741 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:06.738+0000 7fa465545700 1 --2- 172.21.15.59:0/1641172666 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa460101f30 0x7fa460104320 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa448009990 tx=0x7fa448009ca0 comp rx=0 tx=0).stop 2023-12-11T17:44:06.741 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:06.738+0000 7fa465545700 1 -- 172.21.15.59:0/1641172666 shutdown_connections 2023-12-11T17:44:06.741 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:06.738+0000 7fa465545700 1 --2- 172.21.15.59:0/1641172666 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa460107190 0x7fa460109600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:06.741 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:06.738+0000 7fa465545700 1 --2- 172.21.15.59:0/1641172666 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa460101f30 0x7fa460104320 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:06.741 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:06.738+0000 7fa465545700 1 --2- 172.21.15.59:0/1641172666 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa460104860 0x7fa460106c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:06.742 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:06.738+0000 7fa465545700 1 -- 172.21.15.59:0/1641172666 >> 172.21.15.59:0/1641172666 conn(0x7fa4600fb920 msgr2=0x7fa4600fdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:44:06.742 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:06.738+0000 7fa465545700 1 -- 172.21.15.59:0/1641172666 shutdown_connections 2023-12-11T17:44:06.742 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:06.738+0000 7fa465545700 1 -- 172.21.15.59:0/1641172666 wait complete. 2023-12-11T17:44:06.742 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:06.739+0000 7fa465545700 1 Processor -- start 2023-12-11T17:44:06.744 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:06.739+0000 7fa465545700 1 -- start start 2023-12-11T17:44:06.744 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:06.739+0000 7fa465545700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa460101f30 0x7fa460197420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:06.744 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:06.739+0000 7fa465545700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa460104860 0x7fa460197960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:06.744 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:06.739+0000 7fa465545700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa460107190 0x7fa46019e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:06.745 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:06.739+0000 7fa465545700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa46019ef50 con 0x7fa460107190 2023-12-11T17:44:06.745 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:06.739+0000 7fa465545700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa46019f090 con 0x7fa460101f30 2023-12-11T17:44:06.745 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:06.739+0000 7fa465545700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa46019f220 con 0x7fa460104860 2023-12-11T17:44:06.745 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:06.739+0000 7fa45f7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa460107190 0x7fa46019e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:06.745 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:06.739+0000 7fa45e7fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa460104860 0x7fa460197960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:06.745 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:06.739+0000 7fa45f7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa460107190 0x7fa46019e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60920/0 (socket says 172.21.15.59:60920) 2023-12-11T17:44:06.745 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:06.739+0000 7fa45f7fe700 1 -- 172.21.15.59:0/3873621743 learned_addr learned my addr 172.21.15.59:0/3873621743 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:44:06.745 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:06.739+0000 7fa45effd700 1 --2- 172.21.15.59:0/3873621743 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa460101f30 0x7fa460197420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:06.745 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:06.740+0000 7fa45f7fe700 1 -- 172.21.15.59:0/3873621743 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa460104860 msgr2=0x7fa460197960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:06.745 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:06.740+0000 7fa45f7fe700 1 --2- 172.21.15.59:0/3873621743 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa460104860 0x7fa460197960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:06.745 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:06.740+0000 7fa45f7fe700 1 -- 172.21.15.59:0/3873621743 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa460101f30 msgr2=0x7fa460197420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:06.745 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:06.740+0000 7fa45f7fe700 1 --2- 172.21.15.59:0/3873621743 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa460101f30 0x7fa460197420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:06.746 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:06.740+0000 7fa45f7fe700 1 -- 172.21.15.59:0/3873621743 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4480095f0 con 0x7fa460107190 2023-12-11T17:44:06.746 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:06.740+0000 7fa45e7fc700 1 --2- 172.21.15.59:0/3873621743 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa460104860 0x7fa460197960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T17:44:06.746 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:06.740+0000 7fa45f7fe700 1 --2- 172.21.15.59:0/3873621743 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa460107190 0x7fa46019e980 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa458007ee0 tx=0x7fa45800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:44:06.746 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:06.740+0000 7fa457fff700 1 -- 172.21.15.59:0/3873621743 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa45800dd20 con 0x7fa460107190 2023-12-11T17:44:06.746 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:06.740+0000 7fa465545700 1 -- 172.21.15.59:0/3873621743 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4600ffd80 con 0x7fa460107190 2023-12-11T17:44:06.746 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:06.740+0000 7fa457fff700 1 -- 172.21.15.59:0/3873621743 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa45800f040 con 0x7fa460107190 2023-12-11T17:44:06.746 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:06.740+0000 7fa457fff700 1 -- 172.21.15.59:0/3873621743 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4580048a0 con 0x7fa460107190 2023-12-11T17:44:06.746 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:06.740+0000 7fa465545700 1 -- 172.21.15.59:0/3873621743 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa460100240 con 0x7fa460107190 2023-12-11T17:44:06.746 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:06.741+0000 7fa465545700 1 -- 172.21.15.59:0/3873621743 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4600fd4a0 con 0x7fa460107190 2023-12-11T17:44:06.747 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:06.742+0000 7fa457fff700 1 -- 172.21.15.59:0/3873621743 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7fa458003a00 con 0x7fa460107190 2023-12-11T17:44:06.747 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:06.743+0000 7fa457fff700 1 --2- 172.21.15.59:0/3873621743 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fa44c0a1480 0x7fa44c0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:06.747 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:06.743+0000 7fa457fff700 1 -- 172.21.15.59:0/3873621743 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fa4580cc8d0 con 0x7fa460107190 2023-12-11T17:44:06.747 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:06.743+0000 7fa45effd700 1 --2- 172.21.15.59:0/3873621743 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fa44c0a1480 0x7fa44c0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:06.747 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:06.743+0000 7fa45effd700 1 --2- 172.21.15.59:0/3873621743 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fa44c0a1480 0x7fa44c0a3940 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa44800b2e0 tx=0x7fa448005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:44:06.749 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:06.747+0000 7fa457fff700 1 -- 172.21.15.59:0/3873621743 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fa4580963d0 con 0x7fa460107190 2023-12-11T17:44:06.999 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:06.996+0000 7fa465545700 1 -- 172.21.15.59:0/3873621743 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa460068680 con 0x7fa460107190 2023-12-11T17:44:06.999 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:06.997+0000 7fa457fff700 1 -- 172.21.15.59:0/3873621743 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6877 (secure 0 0 0) 0x7fa4580999f0 con 0x7fa460107190 2023-12-11T17:44:07.000 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-12-11T17:44:07.000 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/2184151593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":2184151593},{"type":"v1","addr":"172.21.15.59:6835","nonce":2184151593}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/2817614886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2817614886},{"type":"v1","addr":"172.21.15.59:6837","nonce":2817614886}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/395684042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":395684042},{"type":"v1","addr":"172.21.15.181:6833","nonce":395684042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6833/3214606191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3214606191},{"type":"v1","addr":"172.21.15.115:6833","nonce":3214606191}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24449,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/1874159657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1874159657},{"type":"v1","addr":"172.21.15.115:6835","nonce":1874159657}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/4221183620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":4221183620},{"type":"v1","addr":"172.21.15.181:6835","nonce":4221183620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24466,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/3580309407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":3580309407},{"type":"v1","addr":"172.21.15.181:6837","nonce":3580309407}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24482,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/1209365460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":1209365460},{"type":"v1","addr":"172.21.15.115:6837","nonce":1209365460}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[]} 2023-12-11T17:44:07.002 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:06.999+0000 7fa465545700 1 -- 172.21.15.59:0/3873621743 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fa44c0a1480 msgr2=0x7fa44c0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:07.002 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:06.999+0000 7fa465545700 1 --2- 172.21.15.59:0/3873621743 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fa44c0a1480 0x7fa44c0a3940 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa44800b2e0 tx=0x7fa448005fb0 comp rx=0 tx=0).stop 2023-12-11T17:44:07.002 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:06.999+0000 7fa465545700 1 -- 172.21.15.59:0/3873621743 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa460107190 msgr2=0x7fa46019e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:07.002 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:06.999+0000 7fa465545700 1 --2- 172.21.15.59:0/3873621743 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa460107190 0x7fa46019e980 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa458007ee0 tx=0x7fa45800d3b0 comp rx=0 tx=0).stop 2023-12-11T17:44:07.002 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:06.999+0000 7fa465545700 1 -- 172.21.15.59:0/3873621743 shutdown_connections 2023-12-11T17:44:07.002 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:06.999+0000 7fa465545700 1 --2- 172.21.15.59:0/3873621743 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fa44c0a1480 0x7fa44c0a3940 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:07.002 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:06.999+0000 7fa465545700 1 --2- 172.21.15.59:0/3873621743 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa460107190 0x7fa46019e980 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:07.002 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:06.999+0000 7fa465545700 1 --2- 172.21.15.59:0/3873621743 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa460101f30 0x7fa460197420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:07.002 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:06.999+0000 7fa465545700 1 --2- 172.21.15.59:0/3873621743 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa460104860 0x7fa460197960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:07.003 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:06.999+0000 7fa465545700 1 -- 172.21.15.59:0/3873621743 >> 172.21.15.59:0/3873621743 conn(0x7fa4600fb920 msgr2=0x7fa460102870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:44:07.003 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:06.999+0000 7fa465545700 1 -- 172.21.15.59:0/3873621743 shutdown_connections 2023-12-11T17:44:07.003 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:06.999+0000 7fa465545700 1 -- 172.21.15.59:0/3873621743 wait complete. 2023-12-11T17:44:07.003 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 9 2023-12-11T17:44:07.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:07 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:44:07.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:07 smithi181 ceph-mon[103627]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T17:44:07.396 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:07 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:44:07.396 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:07 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:44:07.396 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:07 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-11T17:44:07.396 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:07 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-11T17:44:07.396 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:07 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:44:07.396 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:07 smithi181 ceph-mon[103627]: Deploying daemon mds.i on smithi181 2023-12-11T17:44:07.397 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:07 smithi181 ceph-mon[103627]: mds.? [v2:172.21.15.115:6836/1209365460,v1:172.21.15.115:6837/1209365460] up:boot 2023-12-11T17:44:07.397 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:07 smithi181 ceph-mon[103627]: fsmap 8 up:standby 2023-12-11T17:44:07.397 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:07 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-12-11T17:44:07.397 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:07 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/3873621743' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T17:44:07.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:07 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:44:07.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:07 smithi059 ceph-mon[98847]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T17:44:07.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:07 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:44:07.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:07 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:44:07.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:07 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-11T17:44:07.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:07 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-11T17:44:07.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:07 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:44:07.406 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:07 smithi059 ceph-mon[98847]: Deploying daemon mds.i on smithi181 2023-12-11T17:44:07.406 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:07 smithi059 ceph-mon[98847]: mds.? [v2:172.21.15.115:6836/1209365460,v1:172.21.15.115:6837/1209365460] up:boot 2023-12-11T17:44:07.406 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:07 smithi059 ceph-mon[98847]: fsmap 8 up:standby 2023-12-11T17:44:07.406 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:07 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-12-11T17:44:07.406 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:07 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/3873621743' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T17:44:07.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:07 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:44:07.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:07 smithi115 ceph-mon[102748]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T17:44:07.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:07 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:44:07.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:07 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:44:07.534 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:07 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-11T17:44:07.534 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:07 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-11T17:44:07.534 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:07 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:44:07.534 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:07 smithi115 ceph-mon[102748]: Deploying daemon mds.i on smithi181 2023-12-11T17:44:07.534 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:07 smithi115 ceph-mon[102748]: mds.? [v2:172.21.15.115:6836/1209365460,v1:172.21.15.115:6837/1209365460] up:boot 2023-12-11T17:44:07.534 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:07 smithi115 ceph-mon[102748]: fsmap 8 up:standby 2023-12-11T17:44:07.534 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:07 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-12-11T17:44:07.534 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:07 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/3873621743' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T17:44:08.013 INFO:journalctl@ceph.mds.i.smithi181.stdout:Dec 11 17:44:07 smithi181 systemd[1]: Starting Ceph mds.i for d0a07eaa-984b-11ee-95a3-87774f69a715... 2023-12-11T17:44:08.302 INFO:journalctl@ceph.mds.i.smithi181.stdout:Dec 11 17:44:08 smithi181 podman[141515]: 2023-12-11T17:44:08.303 INFO:journalctl@ceph.mds.i.smithi181.stdout:Dec 11 17:44:08 smithi181 bash[141515]: 3de1d6fedb9046bb1eb54839e7285c2875324a460a9b13c89466270009d791bd 2023-12-11T17:44:08.303 INFO:journalctl@ceph.mds.i.smithi181.stdout:Dec 11 17:44:08 smithi181 ceph-mds[141555]: -- 172.21.15.181:0/1311550363 <== mon.1 v2:172.21.15.181:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55ae28f33520 con 0x55ae28f08800 2023-12-11T17:44:08.303 INFO:journalctl@ceph.mds.i.smithi181.stdout:Dec 11 17:44:08 smithi181 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mds-i[141531]: starting mds.i at 2023-12-11T17:44:08.645 INFO:journalctl@ceph.mds.i.smithi181.stdout:Dec 11 17:44:08 smithi181 systemd[1]: Started Ceph mds.i for d0a07eaa-984b-11ee-95a3-87774f69a715. 2023-12-11T17:44:09.015 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-11T17:44:09.155 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:09.152+0000 7ff3f0418700 1 -- 172.21.15.59:0/813784826 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff3e8101570 msgr2=0x7ff3e81019d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:09.155 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:09.152+0000 7ff3f0418700 1 --2- 172.21.15.59:0/813784826 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff3e8101570 0x7ff3e81019d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff3d80099c0 tx=0x7ff3d8009cd0 comp rx=0 tx=0).stop 2023-12-11T17:44:09.155 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:09.152+0000 7ff3f0418700 1 -- 172.21.15.59:0/813784826 shutdown_connections 2023-12-11T17:44:09.155 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:09.152+0000 7ff3f0418700 1 --2- 172.21.15.59:0/813784826 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff3e8101f10 0x7ff3e81087c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:09.155 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:09.152+0000 7ff3f0418700 1 --2- 172.21.15.59:0/813784826 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3e8100380 0x7ff3e81007a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:09.155 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:09.152+0000 7ff3f0418700 1 --2- 172.21.15.59:0/813784826 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff3e8101570 0x7ff3e81019d0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:09.155 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:09.152+0000 7ff3f0418700 1 -- 172.21.15.59:0/813784826 >> 172.21.15.59:0/813784826 conn(0x7ff3e80fb8f0 msgr2=0x7ff3e80fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:44:09.155 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:09.152+0000 7ff3f0418700 1 -- 172.21.15.59:0/813784826 shutdown_connections 2023-12-11T17:44:09.155 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:09.152+0000 7ff3f0418700 1 -- 172.21.15.59:0/813784826 wait complete. 2023-12-11T17:44:09.156 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:09.153+0000 7ff3f0418700 1 Processor -- start 2023-12-11T17:44:09.156 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:09.153+0000 7ff3f0418700 1 -- start start 2023-12-11T17:44:09.158 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:09.153+0000 7ff3f0418700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff3e8100380 0x7ff3e8196320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:09.158 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:09.153+0000 7ff3f0418700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3e8101570 0x7ff3e8196860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:09.158 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:09.153+0000 7ff3f0418700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff3e8101f10 0x7ff3e819d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:09.158 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:09.153+0000 7ff3f0418700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3e819de50 con 0x7ff3e8100380 2023-12-11T17:44:09.158 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:09.153+0000 7ff3f0418700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3e819df90 con 0x7ff3e8101f10 2023-12-11T17:44:09.158 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:09.153+0000 7ff3f0418700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3e819e120 con 0x7ff3e8101570 2023-12-11T17:44:09.158 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:09.153+0000 7ff3ee1b4700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff3e8100380 0x7ff3e8196320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:09.158 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:09.153+0000 7ff3ee1b4700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff3e8100380 0x7ff3e8196320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60928/0 (socket says 172.21.15.59:60928) 2023-12-11T17:44:09.158 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:09.153+0000 7ff3ee1b4700 1 -- 172.21.15.59:0/720481869 learned_addr learned my addr 172.21.15.59:0/720481869 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:44:09.159 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:09.153+0000 7ff3ee9b5700 1 --2- 172.21.15.59:0/720481869 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff3e8101f10 0x7ff3e819d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:09.159 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:09.153+0000 7ff3ed9b3700 1 --2- 172.21.15.59:0/720481869 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3e8101570 0x7ff3e8196860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:09.159 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:09.153+0000 7ff3ee1b4700 1 -- 172.21.15.59:0/720481869 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3e8101570 msgr2=0x7ff3e8196860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:09.159 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:09.153+0000 7ff3ee1b4700 1 --2- 172.21.15.59:0/720481869 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3e8101570 0x7ff3e8196860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:09.159 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:09.153+0000 7ff3ee1b4700 1 -- 172.21.15.59:0/720481869 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff3e8101f10 msgr2=0x7ff3e819d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:09.159 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:09.153+0000 7ff3ee1b4700 1 --2- 172.21.15.59:0/720481869 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff3e8101f10 0x7ff3e819d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:09.159 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:09.153+0000 7ff3ee1b4700 1 -- 172.21.15.59:0/720481869 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3d80095f0 con 0x7ff3e8100380 2023-12-11T17:44:09.159 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:09.154+0000 7ff3ee1b4700 1 --2- 172.21.15.59:0/720481869 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff3e8100380 0x7ff3e8196320 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff3e400d980 tx=0x7ff3e400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:44:09.159 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:09.154+0000 7ff3df7fe700 1 -- 172.21.15.59:0/720481869 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3e4004020 con 0x7ff3e8100380 2023-12-11T17:44:09.159 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:09.154+0000 7ff3f0418700 1 -- 172.21.15.59:0/720481869 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3e819e400 con 0x7ff3e8100380 2023-12-11T17:44:09.159 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:09.154+0000 7ff3df7fe700 1 -- 172.21.15.59:0/720481869 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff3e400f690 con 0x7ff3e8100380 2023-12-11T17:44:09.159 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:09.154+0000 7ff3df7fe700 1 -- 172.21.15.59:0/720481869 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3e4003a60 con 0x7ff3e8100380 2023-12-11T17:44:09.159 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:09.154+0000 7ff3f0418700 1 -- 172.21.15.59:0/720481869 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3e819e960 con 0x7ff3e8100380 2023-12-11T17:44:09.159 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:09.155+0000 7ff3f0418700 1 -- 172.21.15.59:0/720481869 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3e8107750 con 0x7ff3e8100380 2023-12-11T17:44:09.160 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:09.156+0000 7ff3df7fe700 1 -- 172.21.15.59:0/720481869 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7ff3e400fb00 con 0x7ff3e8100380 2023-12-11T17:44:09.160 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:09.157+0000 7ff3df7fe700 1 --2- 172.21.15.59:0/720481869 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7ff3d40a1480 0x7ff3d40a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:09.160 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:09.157+0000 7ff3ed9b3700 1 --2- 172.21.15.59:0/720481869 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7ff3d40a1480 0x7ff3d40a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:09.160 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:09.157+0000 7ff3df7fe700 1 -- 172.21.15.59:0/720481869 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ff3e40cc790 con 0x7ff3e8100380 2023-12-11T17:44:09.160 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:09.157+0000 7ff3ed9b3700 1 --2- 172.21.15.59:0/720481869 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7ff3d40a1480 0x7ff3d40a3940 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff3d8006010 tx=0x7ff3d80037c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:44:09.162 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:09.160+0000 7ff3df7fe700 1 -- 172.21.15.59:0/720481869 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7ff3e4096290 con 0x7ff3e8100380 2023-12-11T17:44:09.416 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:09.413+0000 7ff3f0418700 1 -- 172.21.15.59:0/720481869 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff3e8068230 con 0x7ff3e8100380 2023-12-11T17:44:09.416 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:09.414+0000 7ff3df7fe700 1 -- 172.21.15.59:0/720481869 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+7673 (secure 0 0 0) 0x7ff3e40998b0 con 0x7ff3e8100380 2023-12-11T17:44:09.417 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-12-11T17:44:09.417 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/2184151593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":2184151593},{"type":"v1","addr":"172.21.15.59:6835","nonce":2184151593}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/2817614886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2817614886},{"type":"v1","addr":"172.21.15.59:6837","nonce":2817614886}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/395684042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":395684042},{"type":"v1","addr":"172.21.15.181:6833","nonce":395684042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6833/3214606191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3214606191},{"type":"v1","addr":"172.21.15.115:6833","nonce":3214606191}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24449,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/1874159657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1874159657},{"type":"v1","addr":"172.21.15.115:6835","nonce":1874159657}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/4221183620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":4221183620},{"type":"v1","addr":"172.21.15.181:6835","nonce":4221183620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24466,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/3580309407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":3580309407},{"type":"v1","addr":"172.21.15.181:6837","nonce":3580309407}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24482,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/1209365460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":1209365460},{"type":"v1","addr":"172.21.15.115:6837","nonce":1209365460}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24484,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6839/2214384806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":2214384806},{"type":"v1","addr":"172.21.15.181:6839","nonce":2214384806}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2023-12-11T17:44:09.419 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:09.416+0000 7ff3f0418700 1 -- 172.21.15.59:0/720481869 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7ff3d40a1480 msgr2=0x7ff3d40a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:09.419 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:09.416+0000 7ff3f0418700 1 --2- 172.21.15.59:0/720481869 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7ff3d40a1480 0x7ff3d40a3940 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff3d8006010 tx=0x7ff3d80037c0 comp rx=0 tx=0).stop 2023-12-11T17:44:09.419 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:09.416+0000 7ff3f0418700 1 -- 172.21.15.59:0/720481869 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff3e8100380 msgr2=0x7ff3e8196320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:09.419 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:09.416+0000 7ff3f0418700 1 --2- 172.21.15.59:0/720481869 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff3e8100380 0x7ff3e8196320 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff3e400d980 tx=0x7ff3e400dc90 comp rx=0 tx=0).stop 2023-12-11T17:44:09.419 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:09.416+0000 7ff3f0418700 1 -- 172.21.15.59:0/720481869 shutdown_connections 2023-12-11T17:44:09.419 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:09.416+0000 7ff3f0418700 1 --2- 172.21.15.59:0/720481869 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7ff3d40a1480 0x7ff3d40a3940 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:09.419 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:09.416+0000 7ff3f0418700 1 --2- 172.21.15.59:0/720481869 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff3e8101f10 0x7ff3e819d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:09.419 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:09.416+0000 7ff3f0418700 1 --2- 172.21.15.59:0/720481869 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff3e8100380 0x7ff3e8196320 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:09.419 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:09.416+0000 7ff3f0418700 1 --2- 172.21.15.59:0/720481869 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3e8101570 0x7ff3e8196860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:09.419 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:09.416+0000 7ff3f0418700 1 -- 172.21.15.59:0/720481869 >> 172.21.15.59:0/720481869 conn(0x7ff3e80fb8f0 msgr2=0x7ff3e8104890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:44:09.419 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:09.416+0000 7ff3f0418700 1 -- 172.21.15.59:0/720481869 shutdown_connections 2023-12-11T17:44:09.419 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:09.416+0000 7ff3f0418700 1 -- 172.21.15.59:0/720481869 wait complete. 2023-12-11T17:44:09.420 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 10 2023-12-11T17:44:09.582 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:09 smithi059 ceph-mon[98847]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T17:44:09.583 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:09 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:44:09.583 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:09 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:44:09.583 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:09 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:44:09.583 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:09 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-11T17:44:09.583 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:09 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-11T17:44:09.583 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:09 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:44:09.583 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:09 smithi059 ceph-mon[98847]: Deploying daemon mds.g on smithi059 2023-12-11T17:44:09.583 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:09 smithi059 ceph-mon[98847]: mds.? [v2:172.21.15.181:6838/2214384806,v1:172.21.15.181:6839/2214384806] up:boot 2023-12-11T17:44:09.583 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:09 smithi059 ceph-mon[98847]: fsmap 9 up:standby 2023-12-11T17:44:09.583 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:09 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-12-11T17:44:09.645 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:09 smithi181 ceph-mon[103627]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T17:44:09.645 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:09 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:44:09.645 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:09 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:44:09.645 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:09 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:44:09.645 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:09 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-11T17:44:09.646 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:09 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-11T17:44:09.646 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:09 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:44:09.646 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:09 smithi181 ceph-mon[103627]: Deploying daemon mds.g on smithi059 2023-12-11T17:44:09.646 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:09 smithi181 ceph-mon[103627]: mds.? [v2:172.21.15.181:6838/2214384806,v1:172.21.15.181:6839/2214384806] up:boot 2023-12-11T17:44:09.646 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:09 smithi181 ceph-mon[103627]: fsmap 9 up:standby 2023-12-11T17:44:09.646 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:09 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-12-11T17:44:09.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:09 smithi115 ceph-mon[102748]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T17:44:09.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:09 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:44:09.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:09 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:44:09.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:09 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:44:09.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:09 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-11T17:44:09.784 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:09 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-11T17:44:09.784 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:09 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:44:09.784 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:09 smithi115 ceph-mon[102748]: Deploying daemon mds.g on smithi059 2023-12-11T17:44:09.784 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:09 smithi115 ceph-mon[102748]: mds.? [v2:172.21.15.181:6838/2214384806,v1:172.21.15.181:6839/2214384806] up:boot 2023-12-11T17:44:09.784 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:09 smithi115 ceph-mon[102748]: fsmap 9 up:standby 2023-12-11T17:44:09.784 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:09 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-12-11T17:44:10.255 INFO:journalctl@ceph.mds.g.smithi059.stdout:Dec 11 17:44:09 smithi059 systemd[1]: Starting Ceph mds.g for d0a07eaa-984b-11ee-95a3-87774f69a715... 2023-12-11T17:44:10.506 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:10 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/720481869' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T17:44:10.506 INFO:journalctl@ceph.mds.g.smithi059.stdout:Dec 11 17:44:10 smithi059 podman[151072]: 2023-12-11T17:44:10.506 INFO:journalctl@ceph.mds.g.smithi059.stdout:Dec 11 17:44:10 smithi059 bash[151072]: 23e954a0e0d8153f292fcb6ba7e58ad17ed6a95a4f5ab73fdb78c5d8392c0702 2023-12-11T17:44:10.506 INFO:journalctl@ceph.mds.g.smithi059.stdout:Dec 11 17:44:10 smithi059 ceph-mds[151113]: -- 172.21.15.59:0/2679069939 <== mon.0 v2:172.21.15.59:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5633de01b520 con 0x5633ddff0c00 2023-12-11T17:44:10.645 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:10 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/720481869' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T17:44:10.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:10 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/720481869' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T17:44:10.904 INFO:journalctl@ceph.mds.g.smithi059.stdout:Dec 11 17:44:10 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mds-g[151089]: starting mds.g at 2023-12-11T17:44:10.904 INFO:journalctl@ceph.mds.g.smithi059.stdout:Dec 11 17:44:10 smithi059 systemd[1]: Started Ceph mds.g for d0a07eaa-984b-11ee-95a3-87774f69a715. 2023-12-11T17:44:11.432 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-11T17:44:11.568 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:11.565+0000 7f66c0523700 1 -- 172.21.15.59:0/3712824734 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f66b81003b0 msgr2=0x7f66b81007d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:11.568 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:11.566+0000 7f66c0523700 1 --2- 172.21.15.59:0/3712824734 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f66b81003b0 0x7f66b81007d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f66b40099c0 tx=0x7f66b4009cd0 comp rx=0 tx=0).stop 2023-12-11T17:44:11.569 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:11.566+0000 7f66c0523700 1 -- 172.21.15.59:0/3712824734 shutdown_connections 2023-12-11T17:44:11.569 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:11.566+0000 7f66c0523700 1 --2- 172.21.15.59:0/3712824734 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f66b8101f40 0x7f66b81087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:11.569 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:11.566+0000 7f66c0523700 1 --2- 172.21.15.59:0/3712824734 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f66b81003b0 0x7f66b81007d0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:11.569 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:11.566+0000 7f66c0523700 1 --2- 172.21.15.59:0/3712824734 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f66b81015a0 0x7f66b8101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:11.569 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:11.566+0000 7f66c0523700 1 -- 172.21.15.59:0/3712824734 >> 172.21.15.59:0/3712824734 conn(0x7f66b80fb960 msgr2=0x7f66b80fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:44:11.569 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:11.566+0000 7f66c0523700 1 -- 172.21.15.59:0/3712824734 shutdown_connections 2023-12-11T17:44:11.569 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:11.566+0000 7f66c0523700 1 -- 172.21.15.59:0/3712824734 wait complete. 2023-12-11T17:44:11.571 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:11.566+0000 7f66c0523700 1 Processor -- start 2023-12-11T17:44:11.571 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:11.566+0000 7f66c0523700 1 -- start start 2023-12-11T17:44:11.571 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:11.567+0000 7f66c0523700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f66b81003b0 0x7f66b8197400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:11.571 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:11.567+0000 7f66c0523700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f66b81015a0 0x7f66b8197940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:11.571 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:11.567+0000 7f66c0523700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f66b8101f40 0x7f66b819e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:11.571 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:11.567+0000 7f66c0523700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66b819ef30 con 0x7f66b81015a0 2023-12-11T17:44:11.571 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:11.567+0000 7f66c0523700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66b819f070 con 0x7f66b81003b0 2023-12-11T17:44:11.571 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:11.567+0000 7f66c0523700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66b819f200 con 0x7f66b8101f40 2023-12-11T17:44:11.571 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:11.567+0000 7f66bdabe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f66b81015a0 0x7f66b8197940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:11.571 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:11.567+0000 7f66bdabe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f66b81015a0 0x7f66b8197940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60958/0 (socket says 172.21.15.59:60958) 2023-12-11T17:44:11.571 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:11.567+0000 7f66bdabe700 1 -- 172.21.15.59:0/2310907146 learned_addr learned my addr 172.21.15.59:0/2310907146 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:44:11.571 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:11.567+0000 7f66beac0700 1 --2- 172.21.15.59:0/2310907146 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f66b8101f40 0x7f66b819e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:11.572 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:11.567+0000 7f66be2bf700 1 --2- 172.21.15.59:0/2310907146 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f66b81003b0 0x7f66b8197400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:11.572 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:11.567+0000 7f66bdabe700 1 -- 172.21.15.59:0/2310907146 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f66b8101f40 msgr2=0x7f66b819e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:11.572 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:11.567+0000 7f66bdabe700 1 --2- 172.21.15.59:0/2310907146 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f66b8101f40 0x7f66b819e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:11.572 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:11.567+0000 7f66bdabe700 1 -- 172.21.15.59:0/2310907146 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f66b81003b0 msgr2=0x7f66b8197400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:11.572 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:11.567+0000 7f66bdabe700 1 --2- 172.21.15.59:0/2310907146 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f66b81003b0 0x7f66b8197400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:11.572 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:11.567+0000 7f66bdabe700 1 -- 172.21.15.59:0/2310907146 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66b40095f0 con 0x7f66b81015a0 2023-12-11T17:44:11.572 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:11.567+0000 7f66be2bf700 1 --2- 172.21.15.59:0/2310907146 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f66b81003b0 0x7f66b8197400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T17:44:11.572 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:11.567+0000 7f66bdabe700 1 --2- 172.21.15.59:0/2310907146 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f66b81015a0 0x7f66b8197940 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f66a800d980 tx=0x7f66a800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:44:11.572 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:11.568+0000 7f66af7fe700 1 -- 172.21.15.59:0/2310907146 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66a8004220 con 0x7f66b81015a0 2023-12-11T17:44:11.572 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:11.568+0000 7f66af7fe700 1 -- 172.21.15.59:0/2310907146 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f66a8007640 con 0x7f66b81015a0 2023-12-11T17:44:11.572 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:11.568+0000 7f66af7fe700 1 -- 172.21.15.59:0/2310907146 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66a8005610 con 0x7f66b81015a0 2023-12-11T17:44:11.572 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:11.568+0000 7f66c0523700 1 -- 172.21.15.59:0/2310907146 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66b81028f0 con 0x7f66b81015a0 2023-12-11T17:44:11.572 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:11.568+0000 7f66c0523700 1 -- 172.21.15.59:0/2310907146 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66b8102e00 con 0x7f66b81015a0 2023-12-11T17:44:11.573 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:11.570+0000 7f66c0523700 1 -- 172.21.15.59:0/2310907146 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66b8068680 con 0x7f66b81015a0 2023-12-11T17:44:11.573 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:11.571+0000 7f66af7fe700 1 -- 172.21.15.59:0/2310907146 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f66a8004750 con 0x7f66b81015a0 2023-12-11T17:44:11.574 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:11.571+0000 7f66af7fe700 1 --2- 172.21.15.59:0/2310907146 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f66a40a5910 0x7f66a40a7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:11.574 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:11.572+0000 7f66af7fe700 1 -- 172.21.15.59:0/2310907146 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f66a80cc970 con 0x7f66b81015a0 2023-12-11T17:44:11.574 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:11.572+0000 7f66be2bf700 1 --2- 172.21.15.59:0/2310907146 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f66a40a5910 0x7f66a40a7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:11.574 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:11.572+0000 7f66be2bf700 1 --2- 172.21.15.59:0/2310907146 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f66a40a5910 0x7f66a40a7dd0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f66b400b580 tx=0x7f66b4005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:44:11.577 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:11.575+0000 7f66af7fe700 1 -- 172.21.15.59:0/2310907146 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f66a80963d0 con 0x7f66b81015a0 2023-12-11T17:44:11.832 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:11.829+0000 7f66c0523700 1 -- 172.21.15.59:0/2310907146 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f66b810bc10 con 0x7f66b81015a0 2023-12-11T17:44:11.832 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:11.830+0000 7f66af7fe700 1 -- 172.21.15.59:0/2310907146 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+8462 (secure 0 0 0) 0x7f66a80999f0 con 0x7f66b81015a0 2023-12-11T17:44:11.833 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-12-11T17:44:11.833 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/2184151593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":2184151593},{"type":"v1","addr":"172.21.15.59:6835","nonce":2184151593}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/2817614886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2817614886},{"type":"v1","addr":"172.21.15.59:6837","nonce":2817614886}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6839/229873490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":229873490},{"type":"v1","addr":"172.21.15.59:6839","nonce":229873490}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/395684042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":395684042},{"type":"v1","addr":"172.21.15.181:6833","nonce":395684042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6833/3214606191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3214606191},{"type":"v1","addr":"172.21.15.115:6833","nonce":3214606191}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24449,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/1874159657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1874159657},{"type":"v1","addr":"172.21.15.115:6835","nonce":1874159657}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/4221183620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":4221183620},{"type":"v1","addr":"172.21.15.181:6835","nonce":4221183620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24466,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/3580309407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":3580309407},{"type":"v1","addr":"172.21.15.181:6837","nonce":3580309407}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24482,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/1209365460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":1209365460},{"type":"v1","addr":"172.21.15.115:6837","nonce":1209365460}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24484,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6839/2214384806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":2214384806},{"type":"v1","addr":"172.21.15.181:6839","nonce":2214384806}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2023-12-11T17:44:11.835 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:11.832+0000 7f66c0523700 1 -- 172.21.15.59:0/2310907146 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f66a40a5910 msgr2=0x7f66a40a7dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:11.835 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:11.832+0000 7f66c0523700 1 --2- 172.21.15.59:0/2310907146 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f66a40a5910 0x7f66a40a7dd0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f66b400b580 tx=0x7f66b4005fb0 comp rx=0 tx=0).stop 2023-12-11T17:44:11.835 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:11.832+0000 7f66c0523700 1 -- 172.21.15.59:0/2310907146 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f66b81015a0 msgr2=0x7f66b8197940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:11.835 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:11.832+0000 7f66c0523700 1 --2- 172.21.15.59:0/2310907146 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f66b81015a0 0x7f66b8197940 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f66a800d980 tx=0x7f66a800dc90 comp rx=0 tx=0).stop 2023-12-11T17:44:11.835 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:11.832+0000 7f66c0523700 1 -- 172.21.15.59:0/2310907146 shutdown_connections 2023-12-11T17:44:11.835 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:11.832+0000 7f66c0523700 1 --2- 172.21.15.59:0/2310907146 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f66a40a5910 0x7f66a40a7dd0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:11.835 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:11.832+0000 7f66c0523700 1 --2- 172.21.15.59:0/2310907146 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f66b8101f40 0x7f66b819e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:11.835 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:11.832+0000 7f66c0523700 1 --2- 172.21.15.59:0/2310907146 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f66b81003b0 0x7f66b8197400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:11.835 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:11.832+0000 7f66c0523700 1 --2- 172.21.15.59:0/2310907146 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f66b81015a0 0x7f66b8197940 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:11.835 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:11.832+0000 7f66c0523700 1 -- 172.21.15.59:0/2310907146 >> 172.21.15.59:0/2310907146 conn(0x7f66b80fb960 msgr2=0x7f66b8069a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:44:11.836 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:11.832+0000 7f66c0523700 1 -- 172.21.15.59:0/2310907146 shutdown_connections 2023-12-11T17:44:11.836 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:11.832+0000 7f66c0523700 1 -- 172.21.15.59:0/2310907146 wait complete. 2023-12-11T17:44:11.836 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 11 2023-12-11T17:44:11.846 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:11 smithi059 ceph-mon[98847]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T17:44:11.846 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:11 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:44:11.846 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:11 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:44:11.846 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:11 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:44:11.846 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:11 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-11T17:44:11.846 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:11 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-11T17:44:11.846 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:11 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:44:11.846 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:11 smithi059 ceph-mon[98847]: Deploying daemon mds.a on smithi059 2023-12-11T17:44:11.846 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:11 smithi059 ceph-mon[98847]: mds.? [v2:172.21.15.59:6838/229873490,v1:172.21.15.59:6839/229873490] up:boot 2023-12-11T17:44:11.847 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:11 smithi059 ceph-mon[98847]: fsmap 10 up:standby 2023-12-11T17:44:11.847 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:11 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-12-11T17:44:11.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:11 smithi181 ceph-mon[103627]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T17:44:11.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:11 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:44:11.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:11 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:44:11.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:11 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:44:11.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:11 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-11T17:44:11.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:11 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-11T17:44:11.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:11 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:44:11.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:11 smithi181 ceph-mon[103627]: Deploying daemon mds.a on smithi059 2023-12-11T17:44:11.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:11 smithi181 ceph-mon[103627]: mds.? [v2:172.21.15.59:6838/229873490,v1:172.21.15.59:6839/229873490] up:boot 2023-12-11T17:44:11.896 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:11 smithi181 ceph-mon[103627]: fsmap 10 up:standby 2023-12-11T17:44:11.896 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:11 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-12-11T17:44:12.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:11 smithi115 ceph-mon[102748]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T17:44:12.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:11 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:44:12.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:11 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:44:12.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:11 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:44:12.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:11 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-11T17:44:12.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:11 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-11T17:44:12.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:11 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:44:12.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:11 smithi115 ceph-mon[102748]: Deploying daemon mds.a on smithi059 2023-12-11T17:44:12.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:11 smithi115 ceph-mon[102748]: mds.? [v2:172.21.15.59:6838/229873490,v1:172.21.15.59:6839/229873490] up:boot 2023-12-11T17:44:12.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:11 smithi115 ceph-mon[102748]: fsmap 10 up:standby 2023-12-11T17:44:12.034 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:11 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-12-11T17:44:12.488 INFO:journalctl@ceph.mds.a.smithi059.stdout:Dec 11 17:44:12 smithi059 systemd[1]: Starting Ceph mds.a for d0a07eaa-984b-11ee-95a3-87774f69a715... 2023-12-11T17:44:12.788 INFO:journalctl@ceph.mds.a.smithi059.stdout:Dec 11 17:44:12 smithi059 podman[151460]: 2023-12-11T17:44:12.788 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:12 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/2310907146' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T17:44:12.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:12 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/2310907146' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T17:44:13.032 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:12 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/2310907146' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T17:44:13.154 INFO:journalctl@ceph.mds.a.smithi059.stdout:Dec 11 17:44:12 smithi059 ceph-mds[151501]: -- 172.21.15.59:0/43342875 <== mon.0 v2:172.21.15.59:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x560801337520 con 0x56080130cc00 2023-12-11T17:44:13.154 INFO:journalctl@ceph.mds.a.smithi059.stdout:Dec 11 17:44:12 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mds-a[151476]: starting mds.a at 2023-12-11T17:44:13.154 INFO:journalctl@ceph.mds.a.smithi059.stdout:Dec 11 17:44:12 smithi059 bash[151460]: d6d01ee907b4f59a3e87aa5d48730589a9c7949cc90f4d722084d431c59bdfa1 2023-12-11T17:44:13.155 INFO:journalctl@ceph.mds.a.smithi059.stdout:Dec 11 17:44:12 smithi059 systemd[1]: Started Ceph mds.a for d0a07eaa-984b-11ee-95a3-87774f69a715. 2023-12-11T17:44:13.848 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-11T17:44:13.996 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:13.993+0000 7f2f80e2c700 1 -- 172.21.15.59:0/2703918217 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f7c1048a0 msgr2=0x7f2f7c106c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:13.996 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:13.993+0000 7f2f80e2c700 1 --2- 172.21.15.59:0/2703918217 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f7c1048a0 0x7f2f7c106c90 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f2f70009990 tx=0x7f2f70009ca0 comp rx=0 tx=0).stop 2023-12-11T17:44:13.996 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:13.993+0000 7f2f80e2c700 1 -- 172.21.15.59:0/2703918217 shutdown_connections 2023-12-11T17:44:13.997 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:13.993+0000 7f2f80e2c700 1 --2- 172.21.15.59:0/2703918217 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2f7c1071d0 0x7f2f7c1095e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:13.997 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:13.993+0000 7f2f80e2c700 1 --2- 172.21.15.59:0/2703918217 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2f7c101f70 0x7f2f7c104360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:13.997 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:13.993+0000 7f2f80e2c700 1 --2- 172.21.15.59:0/2703918217 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f7c1048a0 0x7f2f7c106c90 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:13.997 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:13.993+0000 7f2f80e2c700 1 -- 172.21.15.59:0/2703918217 >> 172.21.15.59:0/2703918217 conn(0x7f2f7c0fb920 msgr2=0x7f2f7c0fdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:44:13.997 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:13.993+0000 7f2f80e2c700 1 -- 172.21.15.59:0/2703918217 shutdown_connections 2023-12-11T17:44:13.997 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:13.993+0000 7f2f80e2c700 1 -- 172.21.15.59:0/2703918217 wait complete. 2023-12-11T17:44:13.999 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:13.994+0000 7f2f80e2c700 1 Processor -- start 2023-12-11T17:44:13.999 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:13.995+0000 7f2f80e2c700 1 -- start start 2023-12-11T17:44:13.999 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:13.995+0000 7f2f80e2c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2f7c101f70 0x7f2f7c197430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:13.999 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:13.995+0000 7f2f80e2c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2f7c1048a0 0x7f2f7c197970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:13.999 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:13.995+0000 7f2f80e2c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f7c1071d0 0x7f2f7c19e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:14.000 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:13.995+0000 7f2f80e2c700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f7c19ef60 con 0x7f2f7c1071d0 2023-12-11T17:44:14.000 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:13.995+0000 7f2f80e2c700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f7c19f0a0 con 0x7f2f7c1048a0 2023-12-11T17:44:14.000 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:13.995+0000 7f2f80e2c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f7c19f230 con 0x7f2f7c101f70 2023-12-11T17:44:14.000 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:13.995+0000 7f2f7ad9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f7c1071d0 0x7f2f7c19e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:14.000 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:13.995+0000 7f2f7ad9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f7c1071d0 0x7f2f7c19e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60984/0 (socket says 172.21.15.59:60984) 2023-12-11T17:44:14.000 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:13.995+0000 7f2f7ad9d700 1 -- 172.21.15.59:0/2745533808 learned_addr learned my addr 172.21.15.59:0/2745533808 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:44:14.000 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:13.995+0000 7f2f7a59c700 1 --2- 172.21.15.59:0/2745533808 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2f7c101f70 0x7f2f7c197430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:14.000 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:13.995+0000 7f2f79d9b700 1 --2- 172.21.15.59:0/2745533808 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2f7c1048a0 0x7f2f7c197970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:14.000 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:13.995+0000 7f2f7ad9d700 1 -- 172.21.15.59:0/2745533808 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2f7c101f70 msgr2=0x7f2f7c197430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:14.000 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:13.995+0000 7f2f7ad9d700 1 --2- 172.21.15.59:0/2745533808 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2f7c101f70 0x7f2f7c197430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:14.000 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:13.995+0000 7f2f7ad9d700 1 -- 172.21.15.59:0/2745533808 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2f7c1048a0 msgr2=0x7f2f7c197970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:14.000 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:13.995+0000 7f2f7ad9d700 1 --2- 172.21.15.59:0/2745533808 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2f7c1048a0 0x7f2f7c197970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:14.000 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:13.995+0000 7f2f7ad9d700 1 -- 172.21.15.59:0/2745533808 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f700095f0 con 0x7f2f7c1071d0 2023-12-11T17:44:14.000 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:13.995+0000 7f2f7ad9d700 1 --2- 172.21.15.59:0/2745533808 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f7c1071d0 0x7f2f7c19e990 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2f74007ee0 tx=0x7f2f7400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:44:14.001 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:13.995+0000 7f2f6b7fe700 1 -- 172.21.15.59:0/2745533808 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f7400ddd0 con 0x7f2f7c1071d0 2023-12-11T17:44:14.001 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:13.996+0000 7f2f80e2c700 1 -- 172.21.15.59:0/2745533808 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f7c19f510 con 0x7f2f7c1071d0 2023-12-11T17:44:14.001 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:13.996+0000 7f2f6b7fe700 1 -- 172.21.15.59:0/2745533808 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2f7400f040 con 0x7f2f7c1071d0 2023-12-11T17:44:14.001 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:13.996+0000 7f2f6b7fe700 1 -- 172.21.15.59:0/2745533808 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f740048e0 con 0x7f2f7c1071d0 2023-12-11T17:44:14.001 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:13.996+0000 7f2f80e2c700 1 -- 172.21.15.59:0/2745533808 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f7c19fa20 con 0x7f2f7c1071d0 2023-12-11T17:44:14.001 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:13.997+0000 7f2f80e2c700 1 -- 172.21.15.59:0/2745533808 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f7c0fd4a0 con 0x7f2f7c1071d0 2023-12-11T17:44:14.001 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:13.998+0000 7f2f6b7fe700 1 -- 172.21.15.59:0/2745533808 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f2f74003a00 con 0x7f2f7c1071d0 2023-12-11T17:44:14.001 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:13.999+0000 7f2f6b7fe700 1 --2- 172.21.15.59:0/2745533808 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f2f640a1480 0x7f2f640a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:14.002 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:13.999+0000 7f2f7a59c700 1 --2- 172.21.15.59:0/2745533808 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f2f640a1480 0x7f2f640a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:14.002 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:13.999+0000 7f2f6b7fe700 1 -- 172.21.15.59:0/2745533808 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f2f740cca80 con 0x7f2f7c1071d0 2023-12-11T17:44:14.002 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:13.999+0000 7f2f7a59c700 1 --2- 172.21.15.59:0/2745533808 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f2f640a1480 0x7f2f640a3940 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c005950 tx=0x7f2f6c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:44:14.005 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:14.002+0000 7f2f6b7fe700 1 -- 172.21.15.59:0/2745533808 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f2f740964e0 con 0x7f2f7c1071d0 2023-12-11T17:44:14.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:13 smithi059 ceph-mon[98847]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T17:44:14.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:13 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:44:14.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:13 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:44:14.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:13 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:44:14.155 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:13 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-11T17:44:14.155 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:13 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-11T17:44:14.155 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:13 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:44:14.155 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:13 smithi059 ceph-mon[98847]: Deploying daemon mds.b on smithi115 2023-12-11T17:44:14.155 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:13 smithi059 ceph-mon[98847]: mds.? [v2:172.21.15.59:6840/967998755,v1:172.21.15.59:6841/967998755] up:boot 2023-12-11T17:44:14.155 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:13 smithi059 ceph-mon[98847]: fsmap 11 up:standby 2023-12-11T17:44:14.155 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:13 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-12-11T17:44:14.222 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:13 smithi115 ceph-mon[102748]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T17:44:14.222 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:13 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:44:14.222 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:13 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:44:14.222 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:13 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:44:14.222 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:13 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-11T17:44:14.222 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:13 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-11T17:44:14.222 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:13 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:44:14.222 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:13 smithi115 ceph-mon[102748]: Deploying daemon mds.b on smithi115 2023-12-11T17:44:14.222 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:13 smithi115 ceph-mon[102748]: mds.? [v2:172.21.15.59:6840/967998755,v1:172.21.15.59:6841/967998755] up:boot 2023-12-11T17:44:14.222 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:13 smithi115 ceph-mon[102748]: fsmap 11 up:standby 2023-12-11T17:44:14.223 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:13 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-12-11T17:44:14.254 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:14.252+0000 7f2f80e2c700 1 -- 172.21.15.59:0/2745533808 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2f7c10ccf0 con 0x7f2f7c1071d0 2023-12-11T17:44:14.255 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:14.253+0000 7f2f6b7fe700 1 -- 172.21.15.59:0/2745533808 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+9251 (secure 0 0 0) 0x7f2f74099b00 con 0x7f2f7c1071d0 2023-12-11T17:44:14.255 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-12-11T17:44:14.255 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/2184151593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":2184151593},{"type":"v1","addr":"172.21.15.59:6835","nonce":2184151593}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/2817614886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2817614886},{"type":"v1","addr":"172.21.15.59:6837","nonce":2817614886}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6839/229873490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":229873490},{"type":"v1","addr":"172.21.15.59:6839","nonce":229873490}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6841/967998755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":967998755},{"type":"v1","addr":"172.21.15.59:6841","nonce":967998755}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/395684042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":395684042},{"type":"v1","addr":"172.21.15.181:6833","nonce":395684042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6833/3214606191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3214606191},{"type":"v1","addr":"172.21.15.115:6833","nonce":3214606191}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24449,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/1874159657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1874159657},{"type":"v1","addr":"172.21.15.115:6835","nonce":1874159657}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/4221183620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":4221183620},{"type":"v1","addr":"172.21.15.181:6835","nonce":4221183620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24466,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/3580309407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":3580309407},{"type":"v1","addr":"172.21.15.181:6837","nonce":3580309407}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24482,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/1209365460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":1209365460},{"type":"v1","addr":"172.21.15.115:6837","nonce":1209365460}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24484,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6839/2214384806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":2214384806},{"type":"v1","addr":"172.21.15.181:6839","nonce":2214384806}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2023-12-11T17:44:14.257 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:14.255+0000 7f2f80e2c700 1 -- 172.21.15.59:0/2745533808 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f2f640a1480 msgr2=0x7f2f640a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:14.258 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:14.255+0000 7f2f80e2c700 1 --2- 172.21.15.59:0/2745533808 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f2f640a1480 0x7f2f640a3940 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c005950 tx=0x7f2f6c00b410 comp rx=0 tx=0).stop 2023-12-11T17:44:14.258 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:14.255+0000 7f2f80e2c700 1 -- 172.21.15.59:0/2745533808 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f7c1071d0 msgr2=0x7f2f7c19e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:14.258 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:14.255+0000 7f2f80e2c700 1 --2- 172.21.15.59:0/2745533808 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f7c1071d0 0x7f2f7c19e990 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2f74007ee0 tx=0x7f2f7400d3b0 comp rx=0 tx=0).stop 2023-12-11T17:44:14.258 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:14.255+0000 7f2f80e2c700 1 -- 172.21.15.59:0/2745533808 shutdown_connections 2023-12-11T17:44:14.258 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:14.255+0000 7f2f80e2c700 1 --2- 172.21.15.59:0/2745533808 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f2f640a1480 0x7f2f640a3940 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:14.258 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:14.255+0000 7f2f80e2c700 1 --2- 172.21.15.59:0/2745533808 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f7c1071d0 0x7f2f7c19e990 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:14.258 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:14.255+0000 7f2f80e2c700 1 --2- 172.21.15.59:0/2745533808 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2f7c101f70 0x7f2f7c197430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:14.258 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:14.255+0000 7f2f80e2c700 1 --2- 172.21.15.59:0/2745533808 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2f7c1048a0 0x7f2f7c197970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:14.258 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:14.255+0000 7f2f80e2c700 1 -- 172.21.15.59:0/2745533808 >> 172.21.15.59:0/2745533808 conn(0x7f2f7c0fb920 msgr2=0x7f2f7c1029b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:44:14.258 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:14.255+0000 7f2f80e2c700 1 -- 172.21.15.59:0/2745533808 shutdown_connections 2023-12-11T17:44:14.259 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:14.255+0000 7f2f80e2c700 1 -- 172.21.15.59:0/2745533808 wait complete. 2023-12-11T17:44:14.259 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 12 2023-12-11T17:44:14.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:13 smithi181 ceph-mon[103627]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T17:44:14.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:13 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:44:14.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:13 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:44:14.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:13 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:44:14.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:13 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-11T17:44:14.396 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:13 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-11T17:44:14.396 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:13 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:44:14.396 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:13 smithi181 ceph-mon[103627]: Deploying daemon mds.b on smithi115 2023-12-11T17:44:14.396 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:13 smithi181 ceph-mon[103627]: mds.? [v2:172.21.15.59:6840/967998755,v1:172.21.15.59:6841/967998755] up:boot 2023-12-11T17:44:14.396 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:13 smithi181 ceph-mon[103627]: fsmap 11 up:standby 2023-12-11T17:44:14.396 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:13 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-12-11T17:44:14.783 INFO:journalctl@ceph.mds.b.smithi115.stdout:Dec 11 17:44:14 smithi115 systemd[1]: Starting Ceph mds.b for d0a07eaa-984b-11ee-95a3-87774f69a715... 2023-12-11T17:44:15.167 INFO:journalctl@ceph.mds.b.smithi115.stdout:Dec 11 17:44:14 smithi115 podman[139675]: 2023-12-11T17:44:15.168 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:15 smithi115 ceph-mon[102748]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T17:44:15.168 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:15 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/2745533808' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T17:44:15.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:15 smithi181 ceph-mon[103627]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T17:44:15.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:15 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/2745533808' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T17:44:15.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:15 smithi059 ceph-mon[98847]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T17:44:15.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:15 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/2745533808' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T17:44:15.533 INFO:journalctl@ceph.mds.b.smithi115.stdout:Dec 11 17:44:15 smithi115 ceph-mds[139715]: -- 172.21.15.115:0/685818820 <== mon.2 v2:172.21.15.115:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55dd693db520 con 0x55dd693b0800 2023-12-11T17:44:15.533 INFO:journalctl@ceph.mds.b.smithi115.stdout:Dec 11 17:44:15 smithi115 bash[139675]: c68d62e00c79f556713f7f9aa7249a3b633bb066552c35e1723e26938d00831f 2023-12-11T17:44:15.533 INFO:journalctl@ceph.mds.b.smithi115.stdout:Dec 11 17:44:15 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mds-b[139691]: starting mds.b at 2023-12-11T17:44:15.533 INFO:journalctl@ceph.mds.b.smithi115.stdout:Dec 11 17:44:15 smithi115 systemd[1]: Started Ceph mds.b for d0a07eaa-984b-11ee-95a3-87774f69a715. 2023-12-11T17:44:16.271 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-11T17:44:16.405 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:16.402+0000 7f8ddf04d700 1 -- 172.21.15.59:0/1750820726 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8dd8101f40 msgr2=0x7f8dd81087f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:16.405 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:16.402+0000 7f8ddf04d700 1 --2- 172.21.15.59:0/1750820726 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8dd8101f40 0x7f8dd81087f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f8dd000b0d0 tx=0x7f8dd000b3e0 comp rx=0 tx=0).stop 2023-12-11T17:44:16.405 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:16.402+0000 7f8ddf04d700 1 -- 172.21.15.59:0/1750820726 shutdown_connections 2023-12-11T17:44:16.405 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:16.402+0000 7f8ddf04d700 1 --2- 172.21.15.59:0/1750820726 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8dd8101f40 0x7f8dd81087f0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:16.405 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:16.402+0000 7f8ddf04d700 1 --2- 172.21.15.59:0/1750820726 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8dd81003b0 0x7f8dd81007d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:16.406 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:16.402+0000 7f8ddf04d700 1 --2- 172.21.15.59:0/1750820726 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8dd81015a0 0x7f8dd8101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:16.406 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:16.402+0000 7f8ddf04d700 1 -- 172.21.15.59:0/1750820726 >> 172.21.15.59:0/1750820726 conn(0x7f8dd80fb920 msgr2=0x7f8dd80fdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:44:16.406 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:16.402+0000 7f8ddf04d700 1 -- 172.21.15.59:0/1750820726 shutdown_connections 2023-12-11T17:44:16.406 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:16.402+0000 7f8ddf04d700 1 -- 172.21.15.59:0/1750820726 wait complete. 2023-12-11T17:44:16.406 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:16.403+0000 7f8ddf04d700 1 Processor -- start 2023-12-11T17:44:16.406 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:16.403+0000 7f8ddf04d700 1 -- start start 2023-12-11T17:44:16.408 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:16.403+0000 7f8ddf04d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8dd81003b0 0x7f8dd81963e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:16.408 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:16.403+0000 7f8ddf04d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8dd81015a0 0x7f8dd8196920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:16.408 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:16.403+0000 7f8ddf04d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8dd8101f40 0x7f8dd819d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:16.408 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:16.403+0000 7f8ddf04d700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dd819df10 con 0x7f8dd81003b0 2023-12-11T17:44:16.408 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:16.403+0000 7f8ddf04d700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dd819e050 con 0x7f8dd81015a0 2023-12-11T17:44:16.408 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:16.403+0000 7f8ddf04d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dd819e1e0 con 0x7f8dd8101f40 2023-12-11T17:44:16.408 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:16.403+0000 7f8ddcde9700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8dd81003b0 0x7f8dd81963e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:16.408 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:16.403+0000 7f8ddcde9700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8dd81003b0 0x7f8dd81963e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60994/0 (socket says 172.21.15.59:60994) 2023-12-11T17:44:16.409 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:16.403+0000 7f8ddcde9700 1 -- 172.21.15.59:0/1091256749 learned_addr learned my addr 172.21.15.59:0/1091256749 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:44:16.409 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:16.403+0000 7f8dd7fff700 1 --2- 172.21.15.59:0/1091256749 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8dd81015a0 0x7f8dd8196920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:16.409 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:16.403+0000 7f8ddd5ea700 1 --2- 172.21.15.59:0/1091256749 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8dd8101f40 0x7f8dd819d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:16.409 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:16.403+0000 7f8ddcde9700 1 -- 172.21.15.59:0/1091256749 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8dd8101f40 msgr2=0x7f8dd819d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:16.409 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:16.403+0000 7f8ddcde9700 1 --2- 172.21.15.59:0/1091256749 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8dd8101f40 0x7f8dd819d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:16.409 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:16.403+0000 7f8ddcde9700 1 -- 172.21.15.59:0/1091256749 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8dd81015a0 msgr2=0x7f8dd8196920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:16.409 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:16.403+0000 7f8ddcde9700 1 --2- 172.21.15.59:0/1091256749 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8dd81015a0 0x7f8dd8196920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:16.409 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:16.404+0000 7f8ddcde9700 1 -- 172.21.15.59:0/1091256749 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dd0009d00 con 0x7f8dd81003b0 2023-12-11T17:44:16.409 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:16.404+0000 7f8ddcde9700 1 --2- 172.21.15.59:0/1091256749 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8dd81003b0 0x7f8dd81963e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f8dcc00d980 tx=0x7f8dcc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:44:16.409 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:16.404+0000 7f8dd5ffb700 1 -- 172.21.15.59:0/1091256749 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dcc004020 con 0x7f8dd81003b0 2023-12-11T17:44:16.409 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:16.404+0000 7f8ddf04d700 1 -- 172.21.15.59:0/1091256749 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8dd819e4c0 con 0x7f8dd81003b0 2023-12-11T17:44:16.409 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:16.404+0000 7f8dd5ffb700 1 -- 172.21.15.59:0/1091256749 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8dcc00f690 con 0x7f8dd81003b0 2023-12-11T17:44:16.409 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:16.404+0000 7f8dd5ffb700 1 -- 172.21.15.59:0/1091256749 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dcc003a60 con 0x7f8dd81003b0 2023-12-11T17:44:16.409 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:16.404+0000 7f8ddf04d700 1 -- 172.21.15.59:0/1091256749 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8dd819ea20 con 0x7f8dd81003b0 2023-12-11T17:44:16.410 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:16.405+0000 7f8ddf04d700 1 -- 172.21.15.59:0/1091256749 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8dd8107780 con 0x7f8dd81003b0 2023-12-11T17:44:16.410 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:16.406+0000 7f8dd5ffb700 1 -- 172.21.15.59:0/1091256749 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f8dcc00fb00 con 0x7f8dd81003b0 2023-12-11T17:44:16.410 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:16.407+0000 7f8dd5ffb700 1 --2- 172.21.15.59:0/1091256749 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f8dc80a1480 0x7f8dc80a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:16.410 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:16.407+0000 7f8dd7fff700 1 --2- 172.21.15.59:0/1091256749 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f8dc80a1480 0x7f8dc80a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:16.411 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:16.407+0000 7f8dd5ffb700 1 -- 172.21.15.59:0/1091256749 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f8dcc0cc830 con 0x7f8dd81003b0 2023-12-11T17:44:16.411 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:16.407+0000 7f8dd7fff700 1 --2- 172.21.15.59:0/1091256749 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f8dc80a1480 0x7f8dc80a3940 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8dc4005950 tx=0x7f8dc400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:44:16.412 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:16.410+0000 7f8dd5ffb700 1 -- 172.21.15.59:0/1091256749 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f8dcc096290 con 0x7f8dd81003b0 2023-12-11T17:44:16.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:16 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:44:16.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:16 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:44:16.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:16 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:44:16.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:16 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:44:16.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:16 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:44:16.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:16 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:44:16.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:16 smithi115 ceph-mon[102748]: mds.? [v2:172.21.15.115:6838/4122922866,v1:172.21.15.115:6839/4122922866] up:boot 2023-12-11T17:44:16.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:16 smithi115 ceph-mon[102748]: fsmap 12 up:standby 2023-12-11T17:44:16.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:16 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-12-11T17:44:16.645 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:16 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:44:16.645 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:16 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:44:16.645 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:16 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:44:16.645 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:16 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:44:16.645 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:16 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:44:16.646 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:16 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:44:16.646 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:16 smithi181 ceph-mon[103627]: mds.? [v2:172.21.15.115:6838/4122922866,v1:172.21.15.115:6839/4122922866] up:boot 2023-12-11T17:44:16.646 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:16 smithi181 ceph-mon[103627]: fsmap 12 up:standby 2023-12-11T17:44:16.646 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:16 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-12-11T17:44:16.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:16 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:44:16.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:16 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:44:16.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:16 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:44:16.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:16 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:44:16.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:16 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:44:16.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:16 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:44:16.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:16 smithi059 ceph-mon[98847]: mds.? [v2:172.21.15.115:6838/4122922866,v1:172.21.15.115:6839/4122922866] up:boot 2023-12-11T17:44:16.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:16 smithi059 ceph-mon[98847]: fsmap 12 up:standby 2023-12-11T17:44:16.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:16 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-12-11T17:44:16.668 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:16.666+0000 7f8ddf04d700 1 -- 172.21.15.59:0/1091256749 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8dd810be00 con 0x7f8dd81003b0 2023-12-11T17:44:16.669 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:16.667+0000 7f8dd5ffb700 1 -- 172.21.15.59:0/1091256749 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+10046 (secure 0 0 0) 0x7f8dcc0998b0 con 0x7f8dd81003b0 2023-12-11T17:44:16.669 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-12-11T17:44:16.669 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/2184151593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":2184151593},{"type":"v1","addr":"172.21.15.59:6835","nonce":2184151593}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/2817614886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2817614886},{"type":"v1","addr":"172.21.15.59:6837","nonce":2817614886}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6839/229873490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":229873490},{"type":"v1","addr":"172.21.15.59:6839","nonce":229873490}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6841/967998755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":967998755},{"type":"v1","addr":"172.21.15.59:6841","nonce":967998755}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/395684042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":395684042},{"type":"v1","addr":"172.21.15.181:6833","nonce":395684042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6833/3214606191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3214606191},{"type":"v1","addr":"172.21.15.115:6833","nonce":3214606191}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24449,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/1874159657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1874159657},{"type":"v1","addr":"172.21.15.115:6835","nonce":1874159657}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/4221183620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":4221183620},{"type":"v1","addr":"172.21.15.181:6835","nonce":4221183620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24466,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/3580309407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":3580309407},{"type":"v1","addr":"172.21.15.181:6837","nonce":3580309407}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24482,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/1209365460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":1209365460},{"type":"v1","addr":"172.21.15.115:6837","nonce":1209365460}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24484,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6839/2214384806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":2214384806},{"type":"v1","addr":"172.21.15.181:6839","nonce":2214384806}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24521,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6839/4122922866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":4122922866},{"type":"v1","addr":"172.21.15.115:6839","nonce":4122922866}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[]} 2023-12-11T17:44:16.671 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:16.669+0000 7f8ddf04d700 1 -- 172.21.15.59:0/1091256749 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f8dc80a1480 msgr2=0x7f8dc80a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:16.671 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:16.669+0000 7f8ddf04d700 1 --2- 172.21.15.59:0/1091256749 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f8dc80a1480 0x7f8dc80a3940 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8dc4005950 tx=0x7f8dc400b410 comp rx=0 tx=0).stop 2023-12-11T17:44:16.671 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:16.669+0000 7f8ddf04d700 1 -- 172.21.15.59:0/1091256749 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8dd81003b0 msgr2=0x7f8dd81963e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:16.671 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:16.669+0000 7f8ddf04d700 1 --2- 172.21.15.59:0/1091256749 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8dd81003b0 0x7f8dd81963e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f8dcc00d980 tx=0x7f8dcc00dc90 comp rx=0 tx=0).stop 2023-12-11T17:44:16.672 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:16.669+0000 7f8ddf04d700 1 -- 172.21.15.59:0/1091256749 shutdown_connections 2023-12-11T17:44:16.672 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:16.669+0000 7f8ddf04d700 1 --2- 172.21.15.59:0/1091256749 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f8dc80a1480 0x7f8dc80a3940 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:16.672 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:16.669+0000 7f8ddf04d700 1 --2- 172.21.15.59:0/1091256749 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8dd8101f40 0x7f8dd819d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:16.672 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:16.669+0000 7f8ddf04d700 1 --2- 172.21.15.59:0/1091256749 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8dd81003b0 0x7f8dd81963e0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:16.672 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:16.669+0000 7f8ddf04d700 1 --2- 172.21.15.59:0/1091256749 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8dd81015a0 0x7f8dd8196920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:16.672 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:16.669+0000 7f8ddf04d700 1 -- 172.21.15.59:0/1091256749 >> 172.21.15.59:0/1091256749 conn(0x7f8dd80fb920 msgr2=0x7f8dd81048c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:44:16.672 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:16.669+0000 7f8ddf04d700 1 -- 172.21.15.59:0/1091256749 shutdown_connections 2023-12-11T17:44:16.672 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:16.669+0000 7f8ddf04d700 1 -- 172.21.15.59:0/1091256749 wait complete. 2023-12-11T17:44:16.672 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 13 2023-12-11T17:44:16.684 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-11T17:44:16.819 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:16.814+0000 7f9027e6c700 1 -- 172.21.15.59:0/3534461894 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f90201015a0 msgr2=0x7f9020101a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:16.819 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:16.814+0000 7f9027e6c700 1 --2- 172.21.15.59:0/3534461894 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f90201015a0 0x7f9020101a00 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f90140099c0 tx=0x7f9014009cd0 comp rx=0 tx=0).stop 2023-12-11T17:44:16.819 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:16.814+0000 7f9027e6c700 1 -- 172.21.15.59:0/3534461894 shutdown_connections 2023-12-11T17:44:16.819 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:16.814+0000 7f9027e6c700 1 --2- 172.21.15.59:0/3534461894 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9020101f40 0x7f90201087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:16.819 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:16.814+0000 7f9027e6c700 1 --2- 172.21.15.59:0/3534461894 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f90201003b0 0x7f90201007d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:16.819 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:16.814+0000 7f9027e6c700 1 --2- 172.21.15.59:0/3534461894 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f90201015a0 0x7f9020101a00 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:16.819 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:16.814+0000 7f9027e6c700 1 -- 172.21.15.59:0/3534461894 >> 172.21.15.59:0/3534461894 conn(0x7f90200fb920 msgr2=0x7f90200fdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:44:16.819 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:16.814+0000 7f9027e6c700 1 -- 172.21.15.59:0/3534461894 shutdown_connections 2023-12-11T17:44:16.819 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:16.814+0000 7f9027e6c700 1 -- 172.21.15.59:0/3534461894 wait complete. 2023-12-11T17:44:16.820 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:16.814+0000 7f9027e6c700 1 Processor -- start 2023-12-11T17:44:16.820 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:16.814+0000 7f9027e6c700 1 -- start start 2023-12-11T17:44:16.820 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:16.815+0000 7f9027e6c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f90201003b0 0x7f9020196270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:16.820 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:16.815+0000 7f9027e6c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f90201015a0 0x7f90201967b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:16.820 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:16.815+0000 7f9027e6c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9020101f40 0x7f902019d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:16.820 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:16.815+0000 7f9027e6c700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f902019dda0 con 0x7f90201015a0 2023-12-11T17:44:16.820 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:16.815+0000 7f9027e6c700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f902019dee0 con 0x7f9020101f40 2023-12-11T17:44:16.820 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:16.815+0000 7f9027e6c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f902019e070 con 0x7f90201003b0 2023-12-11T17:44:16.820 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:16.815+0000 7f9025407700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f90201015a0 0x7f90201967b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:16.820 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:16.815+0000 7f9025407700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f90201015a0 0x7f90201967b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:32778/0 (socket says 172.21.15.59:32778) 2023-12-11T17:44:16.820 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:16.815+0000 7f9025407700 1 -- 172.21.15.59:0/2388543461 learned_addr learned my addr 172.21.15.59:0/2388543461 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:44:16.820 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:16.815+0000 7f9026409700 1 --2- 172.21.15.59:0/2388543461 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9020101f40 0x7f902019d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:16.820 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:16.815+0000 7f9025c08700 1 --2- 172.21.15.59:0/2388543461 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f90201003b0 0x7f9020196270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:16.820 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:16.815+0000 7f9025407700 1 -- 172.21.15.59:0/2388543461 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f90201003b0 msgr2=0x7f9020196270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:16.821 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:16.815+0000 7f9025407700 1 --2- 172.21.15.59:0/2388543461 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f90201003b0 0x7f9020196270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:16.821 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:16.815+0000 7f9025407700 1 -- 172.21.15.59:0/2388543461 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9020101f40 msgr2=0x7f902019d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:16.821 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:16.815+0000 7f9025407700 1 --2- 172.21.15.59:0/2388543461 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9020101f40 0x7f902019d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:16.821 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:16.815+0000 7f9025407700 1 -- 172.21.15.59:0/2388543461 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90140095f0 con 0x7f90201015a0 2023-12-11T17:44:16.821 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:16.815+0000 7f9025c08700 1 --2- 172.21.15.59:0/2388543461 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f90201003b0 0x7f9020196270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T17:44:16.821 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:16.815+0000 7f9025407700 1 --2- 172.21.15.59:0/2388543461 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f90201015a0 0x7f90201967b0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f90140036f0 tx=0x7f90140047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:44:16.821 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:16.815+0000 7f9012ffd700 1 -- 172.21.15.59:0/2388543461 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f901401f070 con 0x7f90201015a0 2023-12-11T17:44:16.821 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:16.815+0000 7f9012ffd700 1 -- 172.21.15.59:0/2388543461 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f901401abd0 con 0x7f90201015a0 2023-12-11T17:44:16.821 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:16.815+0000 7f9027e6c700 1 -- 172.21.15.59:0/2388543461 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9020069a50 con 0x7f90201015a0 2023-12-11T17:44:16.821 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:16.815+0000 7f9012ffd700 1 -- 172.21.15.59:0/2388543461 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90140040b0 con 0x7f90201015a0 2023-12-11T17:44:16.821 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:16.815+0000 7f9027e6c700 1 -- 172.21.15.59:0/2388543461 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9020069f00 con 0x7f90201015a0 2023-12-11T17:44:16.821 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:16.816+0000 7f9027e6c700 1 -- 172.21.15.59:0/2388543461 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9020068680 con 0x7f90201015a0 2023-12-11T17:44:16.822 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:16.818+0000 7f9012ffd700 1 -- 172.21.15.59:0/2388543461 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f9014023070 con 0x7f90201015a0 2023-12-11T17:44:16.822 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:16.818+0000 7f9012ffd700 1 --2- 172.21.15.59:0/2388543461 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f900c0a1480 0x7f900c0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:16.822 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:16.818+0000 7f9012ffd700 1 -- 172.21.15.59:0/2388543461 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f90140cfcc0 con 0x7f90201015a0 2023-12-11T17:44:16.822 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:16.818+0000 7f9025c08700 1 --2- 172.21.15.59:0/2388543461 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f900c0a1480 0x7f900c0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:16.823 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:16.819+0000 7f9025c08700 1 --2- 172.21.15.59:0/2388543461 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f900c0a1480 0x7f900c0a3940 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9020101400 tx=0x7f901c00a680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:44:16.824 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:16.822+0000 7f9012ffd700 1 -- 172.21.15.59:0/2388543461 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f9014099850 con 0x7f90201015a0 2023-12-11T17:44:17.037 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.034+0000 7f9027e6c700 1 -- 172.21.15.59:0/2388543461 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f902010bbc0 con 0x7f90201015a0 2023-12-11T17:44:17.038 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.036+0000 7f9012ffd700 1 -- 172.21.15.59:0/2388543461 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) v1 ==== 74+0+18422 (secure 0 0 0) 0x7f901409ce70 con 0x7f90201015a0 2023-12-11T17:44:17.039 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-12-11T17:44:17.039 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":78,"fsid":"d0a07eaa-984b-11ee-95a3-87774f69a715","created":"2023-12-11T17:37:14.526539+0000","modified":"2023-12-11T17:43:39.541636+0000","last_up_change":"2023-12-11T17:43:38.535503+0000","last_in_change":"2023-12-11T17:43:21.014888+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-11T17:40:26.596029+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3c0b3218-4b2b-40f8-8bef-ed8cd5633607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":1040759398},{"type":"v1","addr":"172.21.15.59:6803","nonce":1040759398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":1040759398},{"type":"v1","addr":"172.21.15.59:6805","nonce":1040759398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":1040759398},{"type":"v1","addr":"172.21.15.59:6809","nonce":1040759398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":1040759398},{"type":"v1","addr":"172.21.15.59:6807","nonce":1040759398}]},"public_addr":"172.21.15.59:6803/1040759398","cluster_addr":"172.21.15.59:6805/1040759398","heartbeat_back_addr":"172.21.15.59:6809/1040759398","heartbeat_front_addr":"172.21.15.59:6807/1040759398","state":["exists","up"]},{"osd":1,"uuid":"2ead7578-765b-4f5a-a72c-863731ed69e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":2776354245},{"type":"v1","addr":"172.21.15.115:6801","nonce":2776354245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":2776354245},{"type":"v1","addr":"172.21.15.115:6803","nonce":2776354245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":2776354245},{"type":"v1","addr":"172.21.15.115:6807","nonce":2776354245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":2776354245},{"type":"v1","addr":"172.21.15.115:6805","nonce":2776354245}]},"public_addr":"172.21.15.115:6801/2776354245","cluster_addr":"172.21.15.115:6803/2776354245","heartbeat_back_addr":"172.21.15.115:6807/2776354245","heartbeat_front_addr":"172.21.15.115:6805/2776354245","state":["exists","up"]},{"osd":2,"uuid":"8cb1d7ec-53cc-4672-b0b6-70d73797c44f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4287675481},{"type":"v1","addr":"172.21.15.181:6801","nonce":4287675481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4287675481},{"type":"v1","addr":"172.21.15.181:6803","nonce":4287675481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4287675481},{"type":"v1","addr":"172.21.15.181:6807","nonce":4287675481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4287675481},{"type":"v1","addr":"172.21.15.181:6805","nonce":4287675481}]},"public_addr":"172.21.15.181:6801/4287675481","cluster_addr":"172.21.15.181:6803/4287675481","heartbeat_back_addr":"172.21.15.181:6807/4287675481","heartbeat_front_addr":"172.21.15.181:6805/4287675481","state":["exists","up"]},{"osd":3,"uuid":"5d64d9af-831b-4d0a-8ace-a452169351d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":219909995},{"type":"v1","addr":"172.21.15.59:6811","nonce":219909995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":219909995},{"type":"v1","addr":"172.21.15.59:6813","nonce":219909995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":219909995},{"type":"v1","addr":"172.21.15.59:6817","nonce":219909995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":219909995},{"type":"v1","addr":"172.21.15.59:6815","nonce":219909995}]},"public_addr":"172.21.15.59:6811/219909995","cluster_addr":"172.21.15.59:6813/219909995","heartbeat_back_addr":"172.21.15.59:6817/219909995","heartbeat_front_addr":"172.21.15.59:6815/219909995","state":["exists","up"]},{"osd":4,"uuid":"ef8fad28-1db5-4cba-b5f2-10e54e8e1549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":1021473826},{"type":"v1","addr":"172.21.15.115:6809","nonce":1021473826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":1021473826},{"type":"v1","addr":"172.21.15.115:6811","nonce":1021473826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":1021473826},{"type":"v1","addr":"172.21.15.115:6815","nonce":1021473826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":1021473826},{"type":"v1","addr":"172.21.15.115:6813","nonce":1021473826}]},"public_addr":"172.21.15.115:6809/1021473826","cluster_addr":"172.21.15.115:6811/1021473826","heartbeat_back_addr":"172.21.15.115:6815/1021473826","heartbeat_front_addr":"172.21.15.115:6813/1021473826","state":["exists","up"]},{"osd":5,"uuid":"4233a512-9e8f-4db8-aed2-6c3af706e589","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":2405172521},{"type":"v1","addr":"172.21.15.181:6809","nonce":2405172521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":2405172521},{"type":"v1","addr":"172.21.15.181:6811","nonce":2405172521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2405172521},{"type":"v1","addr":"172.21.15.181:6815","nonce":2405172521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":2405172521},{"type":"v1","addr":"172.21.15.181:6813","nonce":2405172521}]},"public_addr":"172.21.15.181:6809/2405172521","cluster_addr":"172.21.15.181:6811/2405172521","heartbeat_back_addr":"172.21.15.181:6815/2405172521","heartbeat_front_addr":"172.21.15.181:6813/2405172521","state":["exists","up"]},{"osd":6,"uuid":"8faab332-016e-4c77-8370-8bef95fc6f34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":815374853},{"type":"v1","addr":"172.21.15.59:6819","nonce":815374853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":815374853},{"type":"v1","addr":"172.21.15.59:6821","nonce":815374853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":815374853},{"type":"v1","addr":"172.21.15.59:6825","nonce":815374853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":815374853},{"type":"v1","addr":"172.21.15.59:6823","nonce":815374853}]},"public_addr":"172.21.15.59:6819/815374853","cluster_addr":"172.21.15.59:6821/815374853","heartbeat_back_addr":"172.21.15.59:6825/815374853","heartbeat_front_addr":"172.21.15.59:6823/815374853","state":["exists","up"]},{"osd":7,"uuid":"f20f470a-0d40-4ed3-8e0e-fd58675cf143","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":1314971870},{"type":"v1","addr":"172.21.15.115:6817","nonce":1314971870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":1314971870},{"type":"v1","addr":"172.21.15.115:6819","nonce":1314971870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":1314971870},{"type":"v1","addr":"172.21.15.115:6823","nonce":1314971870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":1314971870},{"type":"v1","addr":"172.21.15.115:6821","nonce":1314971870}]},"public_addr":"172.21.15.115:6817/1314971870","cluster_addr":"172.21.15.115:6819/1314971870","heartbeat_back_addr":"172.21.15.115:6823/1314971870","heartbeat_front_addr":"172.21.15.115:6821/1314971870","state":["exists","up"]},{"osd":8,"uuid":"eb657d32-1a95-4fd7-a293-d0a970837fc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":3954509873},{"type":"v1","addr":"172.21.15.181:6817","nonce":3954509873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":3954509873},{"type":"v1","addr":"172.21.15.181:6819","nonce":3954509873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":3954509873},{"type":"v1","addr":"172.21.15.181:6823","nonce":3954509873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":3954509873},{"type":"v1","addr":"172.21.15.181:6821","nonce":3954509873}]},"public_addr":"172.21.15.181:6817/3954509873","cluster_addr":"172.21.15.181:6819/3954509873","heartbeat_back_addr":"172.21.15.181:6823/3954509873","heartbeat_front_addr":"172.21.15.181:6821/3954509873","state":["exists","up"]},{"osd":9,"uuid":"8f7717d4-b6f2-4ec1-9b58-62feb0c9a146","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":1163095677},{"type":"v1","addr":"172.21.15.59:6827","nonce":1163095677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":1163095677},{"type":"v1","addr":"172.21.15.59:6829","nonce":1163095677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":1163095677},{"type":"v1","addr":"172.21.15.59:6833","nonce":1163095677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":1163095677},{"type":"v1","addr":"172.21.15.59:6831","nonce":1163095677}]},"public_addr":"172.21.15.59:6827/1163095677","cluster_addr":"172.21.15.59:6829/1163095677","heartbeat_back_addr":"172.21.15.59:6833/1163095677","heartbeat_front_addr":"172.21.15.59:6831/1163095677","state":["exists","up"]},{"osd":10,"uuid":"b75c179a-875d-43bc-b4c1-e8eb80fb64fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":2199610269},{"type":"v1","addr":"172.21.15.115:6825","nonce":2199610269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":2199610269},{"type":"v1","addr":"172.21.15.115:6827","nonce":2199610269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":2199610269},{"type":"v1","addr":"172.21.15.115:6831","nonce":2199610269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":2199610269},{"type":"v1","addr":"172.21.15.115:6829","nonce":2199610269}]},"public_addr":"172.21.15.115:6825/2199610269","cluster_addr":"172.21.15.115:6827/2199610269","heartbeat_back_addr":"172.21.15.115:6831/2199610269","heartbeat_front_addr":"172.21.15.115:6829/2199610269","state":["exists","up"]},{"osd":11,"uuid":"5e141dfe-8a00-4bbc-8204-2dc9be24e18d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":226506989},{"type":"v1","addr":"172.21.15.181:6825","nonce":226506989}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":226506989},{"type":"v1","addr":"172.21.15.181:6827","nonce":226506989}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":226506989},{"type":"v1","addr":"172.21.15.181:6831","nonce":226506989}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":226506989},{"type":"v1","addr":"172.21.15.181:6829","nonce":226506989}]},"public_addr":"172.21.15.181:6825/226506989","cluster_addr":"172.21.15.181:6827/226506989","heartbeat_back_addr":"172.21.15.181:6831/226506989","heartbeat_front_addr":"172.21.15.181:6829/226506989","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:39:42.687084+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:40:03.903448+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:40:24.899908+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:40:46.600251+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:41:07.690037+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:41:28.488855+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:41:49.459274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:42:10.863881+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:42:32.248391+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:42:54.112265+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:43:15.634872+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:43:36.922534+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.59:6800/1259916482":"2023-12-12T17:38:14.037922+0000","172.21.15.59:0/17457774":"2023-12-12T17:38:14.037922+0000","172.21.15.59:6801/1672530462":"2023-12-12T17:37:44.773925+0000","172.21.15.59:0/1053944547":"2023-12-12T17:38:14.037922+0000","172.21.15.59:6801/1259916482":"2023-12-12T17:38:14.037922+0000","172.21.15.59:6800/1672530462":"2023-12-12T17:37:44.773925+0000","172.21.15.59:0/3570767650":"2023-12-12T17:37:44.773925+0000","172.21.15.59:0/3853462899":"2023-12-12T17:37:44.773925+0000","172.21.15.59:0/2859415380":"2023-12-12T17:38:14.037922+0000","172.21.15.59:0/2192742819":"2023-12-12T17:37:44.773925+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-11T17:44:17.041 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.038+0000 7f9027e6c700 1 -- 172.21.15.59:0/2388543461 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f900c0a1480 msgr2=0x7f900c0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:17.041 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.038+0000 7f9027e6c700 1 --2- 172.21.15.59:0/2388543461 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f900c0a1480 0x7f900c0a3940 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9020101400 tx=0x7f901c00a680 comp rx=0 tx=0).stop 2023-12-11T17:44:17.041 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.038+0000 7f9027e6c700 1 -- 172.21.15.59:0/2388543461 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f90201015a0 msgr2=0x7f90201967b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:17.041 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.038+0000 7f9027e6c700 1 --2- 172.21.15.59:0/2388543461 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f90201015a0 0x7f90201967b0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f90140036f0 tx=0x7f90140047b0 comp rx=0 tx=0).stop 2023-12-11T17:44:17.041 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.038+0000 7f9027e6c700 1 -- 172.21.15.59:0/2388543461 shutdown_connections 2023-12-11T17:44:17.041 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.038+0000 7f9027e6c700 1 --2- 172.21.15.59:0/2388543461 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f900c0a1480 0x7f900c0a3940 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:17.041 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.038+0000 7f9027e6c700 1 --2- 172.21.15.59:0/2388543461 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9020101f40 0x7f902019d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:17.041 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.038+0000 7f9027e6c700 1 --2- 172.21.15.59:0/2388543461 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f90201003b0 0x7f9020196270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:17.041 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.038+0000 7f9027e6c700 1 --2- 172.21.15.59:0/2388543461 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f90201015a0 0x7f90201967b0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:17.041 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.038+0000 7f9027e6c700 1 -- 172.21.15.59:0/2388543461 >> 172.21.15.59:0/2388543461 conn(0x7f90200fb920 msgr2=0x7f90201048a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:44:17.041 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.038+0000 7f9027e6c700 1 -- 172.21.15.59:0/2388543461 shutdown_connections 2023-12-11T17:44:17.041 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.038+0000 7f9027e6c700 1 -- 172.21.15.59:0/2388543461 wait complete. 2023-12-11T17:44:17.055 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-11T17:40:26.596029+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-11T17:44:17.056 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-11T17:44:17.192 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.187+0000 7fd41f062700 1 -- 172.21.15.59:0/2726766810 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd4181015a0 msgr2=0x7fd4181019c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:17.192 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.187+0000 7fd41f062700 1 --2- 172.21.15.59:0/2726766810 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd4181015a0 0x7fd4181019c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd40c0099c0 tx=0x7fd40c009cd0 comp rx=0 tx=0).stop 2023-12-11T17:44:17.192 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.187+0000 7fd41f062700 1 -- 172.21.15.59:0/2726766810 shutdown_connections 2023-12-11T17:44:17.192 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.187+0000 7fd41f062700 1 --2- 172.21.15.59:0/2726766810 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd418103130 0x7fd4181099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:17.193 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.187+0000 7fd41f062700 1 --2- 172.21.15.59:0/2726766810 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd4181015a0 0x7fd4181019c0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:17.193 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.187+0000 7fd41f062700 1 --2- 172.21.15.59:0/2726766810 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd418102790 0x7fd418102bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:17.193 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.188+0000 7fd41f062700 1 -- 172.21.15.59:0/2726766810 >> 172.21.15.59:0/2726766810 conn(0x7fd4180fcb10 msgr2=0x7fd4180fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:44:17.193 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.188+0000 7fd41f062700 1 -- 172.21.15.59:0/2726766810 shutdown_connections 2023-12-11T17:44:17.193 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.188+0000 7fd41f062700 1 -- 172.21.15.59:0/2726766810 wait complete. 2023-12-11T17:44:17.193 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.188+0000 7fd41f062700 1 Processor -- start 2023-12-11T17:44:17.193 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.188+0000 7fd41f062700 1 -- start start 2023-12-11T17:44:17.193 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.188+0000 7fd41f062700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd4181015a0 0x7fd4181974e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:17.193 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.188+0000 7fd41f062700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd418102790 0x7fd418197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:17.193 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.188+0000 7fd41f062700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd418103130 0x7fd41819ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:17.193 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.188+0000 7fd41f062700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd41819f010 con 0x7fd4181015a0 2023-12-11T17:44:17.193 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.188+0000 7fd41f062700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd41819f150 con 0x7fd418102790 2023-12-11T17:44:17.193 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.188+0000 7fd41f062700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd41819f2e0 con 0x7fd418103130 2023-12-11T17:44:17.193 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.189+0000 7fd41cdfe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd4181015a0 0x7fd4181974e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:17.193 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.189+0000 7fd41cdfe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd4181015a0 0x7fd4181974e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:32790/0 (socket says 172.21.15.59:32790) 2023-12-11T17:44:17.194 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.189+0000 7fd41cdfe700 1 -- 172.21.15.59:0/3866718071 learned_addr learned my addr 172.21.15.59:0/3866718071 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:44:17.194 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.189+0000 7fd41d5ff700 1 --2- 172.21.15.59:0/3866718071 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd418103130 0x7fd41819ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:17.194 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.189+0000 7fd417fff700 1 --2- 172.21.15.59:0/3866718071 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd418102790 0x7fd418197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:17.194 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.189+0000 7fd41cdfe700 1 -- 172.21.15.59:0/3866718071 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd418103130 msgr2=0x7fd41819ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:17.194 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.189+0000 7fd41cdfe700 1 --2- 172.21.15.59:0/3866718071 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd418103130 0x7fd41819ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:17.194 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.189+0000 7fd41cdfe700 1 -- 172.21.15.59:0/3866718071 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd418102790 msgr2=0x7fd418197a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:17.194 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.189+0000 7fd41cdfe700 1 --2- 172.21.15.59:0/3866718071 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd418102790 0x7fd418197a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:17.194 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.189+0000 7fd41cdfe700 1 -- 172.21.15.59:0/3866718071 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd40c0095f0 con 0x7fd4181015a0 2023-12-11T17:44:17.194 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.189+0000 7fd41cdfe700 1 --2- 172.21.15.59:0/3866718071 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd4181015a0 0x7fd4181974e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd40c009fd0 tx=0x7fd40c005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:44:17.194 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.189+0000 7fd415ffb700 1 -- 172.21.15.59:0/3866718071 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd40c01d070 con 0x7fd4181015a0 2023-12-11T17:44:17.194 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.189+0000 7fd415ffb700 1 -- 172.21.15.59:0/3866718071 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd40c0059a0 con 0x7fd4181015a0 2023-12-11T17:44:17.194 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.189+0000 7fd41f062700 1 -- 172.21.15.59:0/3866718071 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd41819f560 con 0x7fd4181015a0 2023-12-11T17:44:17.194 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.189+0000 7fd415ffb700 1 -- 172.21.15.59:0/3866718071 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd40c0173f0 con 0x7fd4181015a0 2023-12-11T17:44:17.195 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.190+0000 7fd41f062700 1 -- 172.21.15.59:0/3866718071 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd41819fa60 con 0x7fd4181015a0 2023-12-11T17:44:17.195 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.191+0000 7fd41f062700 1 -- 172.21.15.59:0/3866718071 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd418108a20 con 0x7fd4181015a0 2023-12-11T17:44:17.195 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.192+0000 7fd415ffb700 1 -- 172.21.15.59:0/3866718071 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7fd40c022600 con 0x7fd4181015a0 2023-12-11T17:44:17.195 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.193+0000 7fd415ffb700 1 --2- 172.21.15.59:0/3866718071 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fd4080a1500 0x7fd4080a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:17.196 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.193+0000 7fd415ffb700 1 -- 172.21.15.59:0/3866718071 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fd40c0cd940 con 0x7fd4181015a0 2023-12-11T17:44:17.196 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.193+0000 7fd417fff700 1 --2- 172.21.15.59:0/3866718071 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fd4080a1500 0x7fd4080a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:17.196 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.193+0000 7fd417fff700 1 --2- 172.21.15.59:0/3866718071 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fd4080a1500 0x7fd4080a39c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd404009c60 tx=0x7fd404009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:44:17.200 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.198+0000 7fd415ffb700 1 -- 172.21.15.59:0/3866718071 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fd40c0974d0 con 0x7fd4181015a0 2023-12-11T17:44:17.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:17 smithi059 ceph-mon[98847]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T17:44:17.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:17 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/1091256749' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T17:44:17.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:17 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/2388543461' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-11T17:44:17.406 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.404+0000 7fd41f062700 1 -- 172.21.15.59:0/3866718071 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd418108400 con 0x7fd4181015a0 2023-12-11T17:44:17.407 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.405+0000 7fd415ffb700 1 -- 172.21.15.59:0/3866718071 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v78) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd40c09aaf0 con 0x7fd4181015a0 2023-12-11T17:44:17.407 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2023-12-11T17:44:17.410 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.407+0000 7fd41f062700 1 -- 172.21.15.59:0/3866718071 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fd4080a1500 msgr2=0x7fd4080a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:17.410 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.407+0000 7fd41f062700 1 --2- 172.21.15.59:0/3866718071 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fd4080a1500 0x7fd4080a39c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd404009c60 tx=0x7fd404009260 comp rx=0 tx=0).stop 2023-12-11T17:44:17.410 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.407+0000 7fd41f062700 1 -- 172.21.15.59:0/3866718071 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd4181015a0 msgr2=0x7fd4181974e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:17.410 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.407+0000 7fd41f062700 1 --2- 172.21.15.59:0/3866718071 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd4181015a0 0x7fd4181974e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd40c009fd0 tx=0x7fd40c005f10 comp rx=0 tx=0).stop 2023-12-11T17:44:17.410 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.407+0000 7fd41f062700 1 -- 172.21.15.59:0/3866718071 shutdown_connections 2023-12-11T17:44:17.410 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.407+0000 7fd41f062700 1 --2- 172.21.15.59:0/3866718071 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fd4080a1500 0x7fd4080a39c0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:17.410 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.407+0000 7fd41f062700 1 --2- 172.21.15.59:0/3866718071 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd418103130 0x7fd41819ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:17.410 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.407+0000 7fd41f062700 1 --2- 172.21.15.59:0/3866718071 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd4181015a0 0x7fd4181974e0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:17.410 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.407+0000 7fd41f062700 1 --2- 172.21.15.59:0/3866718071 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd418102790 0x7fd418197a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:17.410 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.407+0000 7fd41f062700 1 -- 172.21.15.59:0/3866718071 >> 172.21.15.59:0/3866718071 conn(0x7fd4180fcb10 msgr2=0x7fd418105b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:44:17.410 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.407+0000 7fd41f062700 1 -- 172.21.15.59:0/3866718071 shutdown_connections 2023-12-11T17:44:17.410 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.407+0000 7fd41f062700 1 -- 172.21.15.59:0/3866718071 wait complete. 2023-12-11T17:44:17.423 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-12-11T17:44:17.560 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.555+0000 7f824689f700 1 -- 172.21.15.59:0/1968161644 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8240102720 msgr2=0x7f8240102b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:17.560 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.555+0000 7f824689f700 1 --2- 172.21.15.59:0/1968161644 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8240102720 0x7f8240102b80 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f8234009990 tx=0x7f8234009ca0 comp rx=0 tx=0).stop 2023-12-11T17:44:17.560 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.555+0000 7f824689f700 1 -- 172.21.15.59:0/1968161644 shutdown_connections 2023-12-11T17:44:17.560 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.555+0000 7f824689f700 1 --2- 172.21.15.59:0/1968161644 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82401030c0 0x7f8240109970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:17.560 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.555+0000 7f824689f700 1 --2- 172.21.15.59:0/1968161644 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f82400ff750 0x7f82400ffb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:17.560 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.555+0000 7f824689f700 1 --2- 172.21.15.59:0/1968161644 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8240102720 0x7f8240102b80 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:17.561 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.555+0000 7f824689f700 1 -- 172.21.15.59:0/1968161644 >> 172.21.15.59:0/1968161644 conn(0x7f82400fb0a0 msgr2=0x7f82400fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:44:17.561 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.555+0000 7f824689f700 1 -- 172.21.15.59:0/1968161644 shutdown_connections 2023-12-11T17:44:17.561 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.556+0000 7f824689f700 1 -- 172.21.15.59:0/1968161644 wait complete. 2023-12-11T17:44:17.561 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.556+0000 7f824689f700 1 Processor -- start 2023-12-11T17:44:17.561 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.556+0000 7f824689f700 1 -- start start 2023-12-11T17:44:17.561 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.556+0000 7f824689f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82400ff750 0x7f8240102040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:17.561 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.556+0000 7f824689f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8240102720 0x7f82401006d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:17.561 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.556+0000 7f824689f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f82401030c0 0x7f8240100c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:17.561 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.556+0000 7f824689f700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82401013b0 con 0x7f8240102720 2023-12-11T17:44:17.561 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.556+0000 7f824689f700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8240101520 con 0x7f82400ff750 2023-12-11T17:44:17.561 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.556+0000 7f824689f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82401a1450 con 0x7f82401030c0 2023-12-11T17:44:17.561 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.557+0000 7f823f7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8240102720 0x7f82401006d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:17.561 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.557+0000 7f823f7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8240102720 0x7f82401006d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:32806/0 (socket says 172.21.15.59:32806) 2023-12-11T17:44:17.562 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.557+0000 7f823f7fe700 1 -- 172.21.15.59:0/3892199125 learned_addr learned my addr 172.21.15.59:0/3892199125 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:44:17.562 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.557+0000 7f8244e3c700 1 --2- 172.21.15.59:0/3892199125 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f82401030c0 0x7f8240100c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:17.562 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.557+0000 7f823ffff700 1 --2- 172.21.15.59:0/3892199125 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82400ff750 0x7f8240102040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:17.562 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.557+0000 7f823f7fe700 1 -- 172.21.15.59:0/3892199125 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f82401030c0 msgr2=0x7f8240100c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:17.562 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.557+0000 7f823f7fe700 1 --2- 172.21.15.59:0/3892199125 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f82401030c0 0x7f8240100c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:17.562 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.557+0000 7f823f7fe700 1 -- 172.21.15.59:0/3892199125 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82400ff750 msgr2=0x7f8240102040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:17.562 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.557+0000 7f823f7fe700 1 --2- 172.21.15.59:0/3892199125 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82400ff750 0x7f8240102040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:17.562 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.557+0000 7f823f7fe700 1 -- 172.21.15.59:0/3892199125 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82340095f0 con 0x7f8240102720 2023-12-11T17:44:17.562 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.557+0000 7f823f7fe700 1 --2- 172.21.15.59:0/3892199125 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8240102720 0x7f82401006d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f82340095c0 tx=0x7f823400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:44:17.562 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.557+0000 7f823d7fa700 1 -- 172.21.15.59:0/3892199125 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f823401d070 con 0x7f8240102720 2023-12-11T17:44:17.562 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.557+0000 7f823d7fa700 1 -- 172.21.15.59:0/3892199125 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8234005510 con 0x7f8240102720 2023-12-11T17:44:17.562 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.557+0000 7f824689f700 1 -- 172.21.15.59:0/3892199125 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82401a1680 con 0x7f8240102720 2023-12-11T17:44:17.562 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.557+0000 7f823d7fa700 1 -- 172.21.15.59:0/3892199125 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8234004b70 con 0x7f8240102720 2023-12-11T17:44:17.563 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.557+0000 7f824689f700 1 -- 172.21.15.59:0/3892199125 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82401a1a40 con 0x7f8240102720 2023-12-11T17:44:17.564 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.558+0000 7f824689f700 1 -- 172.21.15.59:0/3892199125 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8240108a00 con 0x7f8240102720 2023-12-11T17:44:17.564 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.559+0000 7f823d7fa700 1 -- 172.21.15.59:0/3892199125 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f823400fb90 con 0x7f8240102720 2023-12-11T17:44:17.564 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.560+0000 7f823d7fa700 1 --2- 172.21.15.59:0/3892199125 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f82280a1480 0x7f82280a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:17.564 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.560+0000 7f823d7fa700 1 -- 172.21.15.59:0/3892199125 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f82340cd7d0 con 0x7f8240102720 2023-12-11T17:44:17.564 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.560+0000 7f823ffff700 1 --2- 172.21.15.59:0/3892199125 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f82280a1480 0x7f82280a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:17.564 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.560+0000 7f823ffff700 1 --2- 172.21.15.59:0/3892199125 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f82280a1480 0x7f82280a3940 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8240068410 tx=0x7f8230007400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:44:17.566 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.564+0000 7f823d7fa700 1 -- 172.21.15.59:0/3892199125 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f8234097360 con 0x7f8240102720 2023-12-11T17:44:17.645 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:17 smithi181 ceph-mon[103627]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T17:44:17.645 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:17 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/1091256749' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T17:44:17.645 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:17 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/2388543461' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-11T17:44:17.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:17 smithi115 ceph-mon[102748]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T17:44:17.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:17 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/1091256749' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T17:44:17.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:17 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/2388543461' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-11T17:44:17.784 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.782+0000 7f824689f700 1 -- 172.21.15.59:0/3892199125 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f824010cca0 con 0x7f8240102720 2023-12-11T17:44:17.784 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.782+0000 7f823d7fa700 1 -- 172.21.15.59:0/3892199125 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v78) v1 ==== 84+0+65 (secure 0 0 0) 0x7f823409a980 con 0x7f8240102720 2023-12-11T17:44:17.784 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-12-11T17:44:17.785 INFO:teuthology.orchestra.run.smithi059.stdout:[ 2023-12-11T17:44:17.785 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-12-11T17:44:17.785 INFO:teuthology.orchestra.run.smithi059.stdout: "poolnum": 1, 2023-12-11T17:44:17.785 INFO:teuthology.orchestra.run.smithi059.stdout: "poolname": ".mgr" 2023-12-11T17:44:17.785 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T17:44:17.785 INFO:teuthology.orchestra.run.smithi059.stdout:] 2023-12-11T17:44:17.787 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.784+0000 7f824689f700 1 -- 172.21.15.59:0/3892199125 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f82280a1480 msgr2=0x7f82280a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:17.787 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.784+0000 7f824689f700 1 --2- 172.21.15.59:0/3892199125 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f82280a1480 0x7f82280a3940 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8240068410 tx=0x7f8230007400 comp rx=0 tx=0).stop 2023-12-11T17:44:17.787 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.784+0000 7f824689f700 1 -- 172.21.15.59:0/3892199125 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8240102720 msgr2=0x7f82401006d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:17.787 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.784+0000 7f824689f700 1 --2- 172.21.15.59:0/3892199125 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8240102720 0x7f82401006d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f82340095c0 tx=0x7f823400f690 comp rx=0 tx=0).stop 2023-12-11T17:44:17.787 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.784+0000 7f824689f700 1 -- 172.21.15.59:0/3892199125 shutdown_connections 2023-12-11T17:44:17.787 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.784+0000 7f824689f700 1 --2- 172.21.15.59:0/3892199125 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f82280a1480 0x7f82280a3940 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:17.787 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.784+0000 7f824689f700 1 --2- 172.21.15.59:0/3892199125 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f82401030c0 0x7f8240100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:17.787 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.784+0000 7f824689f700 1 --2- 172.21.15.59:0/3892199125 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82400ff750 0x7f8240102040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:17.787 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.784+0000 7f824689f700 1 --2- 172.21.15.59:0/3892199125 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8240102720 0x7f82401006d0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:17.787 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.784+0000 7f824689f700 1 -- 172.21.15.59:0/3892199125 >> 172.21.15.59:0/3892199125 conn(0x7f82400fb0a0 msgr2=0x7f8240105b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:44:17.787 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.784+0000 7f824689f700 1 -- 172.21.15.59:0/3892199125 shutdown_connections 2023-12-11T17:44:17.788 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.784+0000 7f824689f700 1 -- 172.21.15.59:0/3892199125 wait complete. 2023-12-11T17:44:17.800 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-12-11T17:44:17.801 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-12-11T17:44:17.935 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.930+0000 7f73fffa5700 1 -- 172.21.15.59:0/2859457200 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f73f8100480 msgr2=0x7f73f81008a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:17.935 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.930+0000 7f73fffa5700 1 --2- 172.21.15.59:0/2859457200 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f73f8100480 0x7f73f81008a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f73e80099c0 tx=0x7f73e8009cd0 comp rx=0 tx=0).stop 2023-12-11T17:44:17.935 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.930+0000 7f73fffa5700 1 -- 172.21.15.59:0/2859457200 shutdown_connections 2023-12-11T17:44:17.936 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.930+0000 7f73fffa5700 1 --2- 172.21.15.59:0/2859457200 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f73f8102010 0x7f73f81088c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:17.936 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.930+0000 7f73fffa5700 1 --2- 172.21.15.59:0/2859457200 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f73f8100480 0x7f73f81008a0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:17.936 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.930+0000 7f73fffa5700 1 --2- 172.21.15.59:0/2859457200 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f73f8101670 0x7f73f8101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:17.936 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.930+0000 7f73fffa5700 1 -- 172.21.15.59:0/2859457200 >> 172.21.15.59:0/2859457200 conn(0x7f73f80fb9f0 msgr2=0x7f73f80fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:44:17.936 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.931+0000 7f73fffa5700 1 -- 172.21.15.59:0/2859457200 shutdown_connections 2023-12-11T17:44:17.937 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.931+0000 7f73fffa5700 1 -- 172.21.15.59:0/2859457200 wait complete. 2023-12-11T17:44:17.937 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.931+0000 7f73fffa5700 1 Processor -- start 2023-12-11T17:44:17.937 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.931+0000 7f73fffa5700 1 -- start start 2023-12-11T17:44:17.937 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.931+0000 7f73fffa5700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f73f8100480 0x7f73f8196450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:17.937 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.931+0000 7f73fffa5700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f73f8101670 0x7f73f8196990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:17.937 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.931+0000 7f73fffa5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f73f8102010 0x7f73f819d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:17.937 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.931+0000 7f73fffa5700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73f819df80 con 0x7f73f8100480 2023-12-11T17:44:17.937 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.931+0000 7f73fffa5700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73f819e0c0 con 0x7f73f8101670 2023-12-11T17:44:17.937 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.931+0000 7f73fffa5700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73f819e250 con 0x7f73f8102010 2023-12-11T17:44:17.937 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.931+0000 7f73fdd41700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f73f8100480 0x7f73f8196450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:17.937 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.931+0000 7f73fdd41700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f73f8100480 0x7f73f8196450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:32818/0 (socket says 172.21.15.59:32818) 2023-12-11T17:44:17.937 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.931+0000 7f73fdd41700 1 -- 172.21.15.59:0/854047649 learned_addr learned my addr 172.21.15.59:0/854047649 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:44:17.938 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.931+0000 7f73fd540700 1 --2- 172.21.15.59:0/854047649 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f73f8101670 0x7f73f8196990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:17.938 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.932+0000 7f73fe542700 1 --2- 172.21.15.59:0/854047649 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f73f8102010 0x7f73f819d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:17.938 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.932+0000 7f73fdd41700 1 -- 172.21.15.59:0/854047649 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f73f8102010 msgr2=0x7f73f819d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:17.938 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.932+0000 7f73fdd41700 1 --2- 172.21.15.59:0/854047649 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f73f8102010 0x7f73f819d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:17.938 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.932+0000 7f73fdd41700 1 -- 172.21.15.59:0/854047649 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f73f8101670 msgr2=0x7f73f8196990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:17.938 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.932+0000 7f73fdd41700 1 --2- 172.21.15.59:0/854047649 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f73f8101670 0x7f73f8196990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:17.938 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.932+0000 7f73fdd41700 1 -- 172.21.15.59:0/854047649 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73e80095f0 con 0x7f73f8100480 2023-12-11T17:44:17.938 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.932+0000 7f73fdd41700 1 --2- 172.21.15.59:0/854047649 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f73f8100480 0x7f73f8196450 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f73e8006010 tx=0x7f73e8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:44:17.938 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.932+0000 7f73eeffd700 1 -- 172.21.15.59:0/854047649 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73e801d070 con 0x7f73f8100480 2023-12-11T17:44:17.938 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.932+0000 7f73eeffd700 1 -- 172.21.15.59:0/854047649 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f73e8004050 con 0x7f73f8100480 2023-12-11T17:44:17.938 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.932+0000 7f73fffa5700 1 -- 172.21.15.59:0/854047649 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73f819e4d0 con 0x7f73f8100480 2023-12-11T17:44:17.938 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.932+0000 7f73eeffd700 1 -- 172.21.15.59:0/854047649 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73e8005280 con 0x7f73f8100480 2023-12-11T17:44:17.938 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.932+0000 7f73fffa5700 1 -- 172.21.15.59:0/854047649 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73f819e8f0 con 0x7f73f8100480 2023-12-11T17:44:17.939 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.933+0000 7f73fffa5700 1 -- 172.21.15.59:0/854047649 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73f8107900 con 0x7f73f8100480 2023-12-11T17:44:17.939 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.934+0000 7f73eeffd700 1 -- 172.21.15.59:0/854047649 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f73e80048e0 con 0x7f73f8100480 2023-12-11T17:44:17.939 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.935+0000 7f73eeffd700 1 --2- 172.21.15.59:0/854047649 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f73e40a1500 0x7f73e40a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:17.939 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.935+0000 7f73eeffd700 1 -- 172.21.15.59:0/854047649 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f73e80cd590 con 0x7f73f8100480 2023-12-11T17:44:17.939 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.935+0000 7f73fd540700 1 --2- 172.21.15.59:0/854047649 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f73e40a1500 0x7f73e40a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:17.940 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.935+0000 7f73fd540700 1 --2- 172.21.15.59:0/854047649 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f73e40a1500 0x7f73e40a39c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f73f4005950 tx=0x7f73f400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:44:17.941 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:17.939+0000 7f73eeffd700 1 -- 172.21.15.59:0/854047649 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f73e8097080 con 0x7f73f8100480 2023-12-11T17:44:18.155 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:18.153+0000 7f73fffa5700 1 -- 172.21.15.59:0/854047649 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f73f810bce0 con 0x7f73f8100480 2023-12-11T17:44:18.639 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:18.637+0000 7f73eeffd700 1 -- 172.21.15.59:0/854047649 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v79) v1 ==== 138+0+0 (secure 0 0 0) 0x7f73e809a6a0 con 0x7f73f8100480 2023-12-11T17:44:18.642 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:18.639+0000 7f73fffa5700 1 -- 172.21.15.59:0/854047649 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f73e40a1500 msgr2=0x7f73e40a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:18.642 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:18.639+0000 7f73fffa5700 1 --2- 172.21.15.59:0/854047649 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f73e40a1500 0x7f73e40a39c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f73f4005950 tx=0x7f73f400b410 comp rx=0 tx=0).stop 2023-12-11T17:44:18.642 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:18.639+0000 7f73fffa5700 1 -- 172.21.15.59:0/854047649 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f73f8100480 msgr2=0x7f73f8196450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:18.642 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:18.639+0000 7f73fffa5700 1 --2- 172.21.15.59:0/854047649 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f73f8100480 0x7f73f8196450 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f73e8006010 tx=0x7f73e8004730 comp rx=0 tx=0).stop 2023-12-11T17:44:18.642 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:18.639+0000 7f73fffa5700 1 -- 172.21.15.59:0/854047649 shutdown_connections 2023-12-11T17:44:18.642 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:18.639+0000 7f73fffa5700 1 --2- 172.21.15.59:0/854047649 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f73e40a1500 0x7f73e40a39c0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:18.642 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:18.639+0000 7f73fffa5700 1 --2- 172.21.15.59:0/854047649 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f73f8102010 0x7f73f819d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:18.642 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:18.639+0000 7f73fffa5700 1 --2- 172.21.15.59:0/854047649 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f73f8100480 0x7f73f8196450 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:18.642 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:18.639+0000 7f73fffa5700 1 --2- 172.21.15.59:0/854047649 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f73f8101670 0x7f73f8196990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:18.642 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:18.639+0000 7f73fffa5700 1 -- 172.21.15.59:0/854047649 >> 172.21.15.59:0/854047649 conn(0x7f73f80fb9f0 msgr2=0x7f73f8104a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:44:18.642 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:18.639+0000 7f73fffa5700 1 -- 172.21.15.59:0/854047649 shutdown_connections 2023-12-11T17:44:18.642 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:18.639+0000 7f73fffa5700 1 -- 172.21.15.59:0/854047649 wait complete. 2023-12-11T17:44:18.643 INFO:teuthology.orchestra.run.smithi059.stderr:pool 'cephfs_metadata' created 2023-12-11T17:44:18.658 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2023-12-11T17:44:18.795 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:18.790+0000 7fc781d88700 1 -- 172.21.15.59:0/3559816534 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc77c1004b0 msgr2=0x7fc77c1008d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:18.795 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:18.790+0000 7fc781d88700 1 --2- 172.21.15.59:0/3559816534 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc77c1004b0 0x7fc77c1008d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc76c0099c0 tx=0x7fc76c009cd0 comp rx=0 tx=0).stop 2023-12-11T17:44:18.795 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:18.790+0000 7fc781d88700 1 -- 172.21.15.59:0/3559816534 shutdown_connections 2023-12-11T17:44:18.795 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:18.790+0000 7fc781d88700 1 --2- 172.21.15.59:0/3559816534 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc77c102040 0x7fc77c1088f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:18.795 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:18.790+0000 7fc781d88700 1 --2- 172.21.15.59:0/3559816534 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc77c1004b0 0x7fc77c1008d0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:18.795 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:18.790+0000 7fc781d88700 1 --2- 172.21.15.59:0/3559816534 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc77c1016a0 0x7fc77c101b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:18.795 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:18.790+0000 7fc781d88700 1 -- 172.21.15.59:0/3559816534 >> 172.21.15.59:0/3559816534 conn(0x7fc77c0fba20 msgr2=0x7fc77c0fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:44:18.795 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:18.790+0000 7fc781d88700 1 -- 172.21.15.59:0/3559816534 shutdown_connections 2023-12-11T17:44:18.796 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:18.790+0000 7fc781d88700 1 -- 172.21.15.59:0/3559816534 wait complete. 2023-12-11T17:44:18.796 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:18.791+0000 7fc781d88700 1 Processor -- start 2023-12-11T17:44:18.796 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:18.791+0000 7fc781d88700 1 -- start start 2023-12-11T17:44:18.796 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:18.791+0000 7fc781d88700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc77c1004b0 0x7fc77c196450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:18.796 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:18.791+0000 7fc781d88700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc77c1016a0 0x7fc77c196990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:18.796 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:18.791+0000 7fc781d88700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc77c102040 0x7fc77c19d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:18.796 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:18.791+0000 7fc781d88700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc77c19df80 con 0x7fc77c1016a0 2023-12-11T17:44:18.796 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:18.791+0000 7fc781d88700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc77c19e0c0 con 0x7fc77c102040 2023-12-11T17:44:18.796 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:18.791+0000 7fc781d88700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc77c19e250 con 0x7fc77c1004b0 2023-12-11T17:44:18.796 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:18.791+0000 7fc77affd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc77c1016a0 0x7fc77c196990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:18.796 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:18.791+0000 7fc77affd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc77c1016a0 0x7fc77c196990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:32834/0 (socket says 172.21.15.59:32834) 2023-12-11T17:44:18.796 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:18.791+0000 7fc77affd700 1 -- 172.21.15.59:0/3268115365 learned_addr learned my addr 172.21.15.59:0/3268115365 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:44:18.796 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:18.791+0000 7fc77b7fe700 1 --2- 172.21.15.59:0/3268115365 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc77c1004b0 0x7fc77c196450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:18.796 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:18.791+0000 7fc77bfff700 1 --2- 172.21.15.59:0/3268115365 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc77c102040 0x7fc77c19d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:18.796 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:18.791+0000 7fc77affd700 1 -- 172.21.15.59:0/3268115365 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc77c1004b0 msgr2=0x7fc77c196450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:18.797 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:18.791+0000 7fc77affd700 1 --2- 172.21.15.59:0/3268115365 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc77c1004b0 0x7fc77c196450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:18.797 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:18.791+0000 7fc77affd700 1 -- 172.21.15.59:0/3268115365 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc77c102040 msgr2=0x7fc77c19d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:18.797 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:18.791+0000 7fc77affd700 1 --2- 172.21.15.59:0/3268115365 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc77c102040 0x7fc77c19d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:18.797 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:18.791+0000 7fc77affd700 1 -- 172.21.15.59:0/3268115365 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc76c0095f0 con 0x7fc77c1016a0 2023-12-11T17:44:18.797 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:18.792+0000 7fc77affd700 1 --2- 172.21.15.59:0/3268115365 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc77c1016a0 0x7fc77c196990 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc76400e9c0 tx=0x7fc76400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:44:18.797 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:18.792+0000 7fc778ff9700 1 -- 172.21.15.59:0/3268115365 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7640097b0 con 0x7fc77c1016a0 2023-12-11T17:44:18.797 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:18.792+0000 7fc781d88700 1 -- 172.21.15.59:0/3268115365 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc77c069810 con 0x7fc77c1016a0 2023-12-11T17:44:18.797 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:18.792+0000 7fc778ff9700 1 -- 172.21.15.59:0/3268115365 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc76400ee60 con 0x7fc77c1016a0 2023-12-11T17:44:18.797 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:18.792+0000 7fc778ff9700 1 -- 172.21.15.59:0/3268115365 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7640054f0 con 0x7fc77c1016a0 2023-12-11T17:44:18.797 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:18.792+0000 7fc781d88700 1 -- 172.21.15.59:0/3268115365 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc77c069cd0 con 0x7fc77c1016a0 2023-12-11T17:44:18.797 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:18.793+0000 7fc781d88700 1 -- 172.21.15.59:0/3268115365 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc77c067a80 con 0x7fc77c1016a0 2023-12-11T17:44:18.798 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:18.794+0000 7fc778ff9700 1 -- 172.21.15.59:0/3268115365 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7fc764004750 con 0x7fc77c1016a0 2023-12-11T17:44:18.798 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:18.795+0000 7fc778ff9700 1 --2- 172.21.15.59:0/3268115365 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fc7680a1480 0x7fc7680a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:18.798 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:18.795+0000 7fc77b7fe700 1 --2- 172.21.15.59:0/3268115365 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fc7680a1480 0x7fc7680a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:18.798 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:18.795+0000 7fc778ff9700 1 -- 172.21.15.59:0/3268115365 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7497+0+0 (secure 0 0 0) 0x7fc764014070 con 0x7fc77c1016a0 2023-12-11T17:44:18.798 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:18.795+0000 7fc77b7fe700 1 --2- 172.21.15.59:0/3268115365 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fc7680a1480 0x7fc7680a3940 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc76c003730 tx=0x7fc76c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:44:18.800 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:18.798+0000 7fc778ff9700 1 -- 172.21.15.59:0/3268115365 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fc764096750 con 0x7fc77c1016a0 2023-12-11T17:44:18.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:18 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/3866718071' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-11T17:44:18.896 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:18 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/3892199125' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-12-11T17:44:18.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:18 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/3866718071' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-11T17:44:18.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:18 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/3892199125' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-12-11T17:44:19.002 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:18.999+0000 7fc781d88700 1 -- 172.21.15.59:0/3268115365 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7fc77c10bce0 con 0x7fc77c1016a0 2023-12-11T17:44:19.032 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:18 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/3866718071' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-11T17:44:19.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:18 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/3892199125' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-12-11T17:44:19.643 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:19.640+0000 7fc778ff9700 1 -- 172.21.15.59:0/3268115365 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v80) v1 ==== 171+0+0 (secure 0 0 0) 0x7fc764099d70 con 0x7fc77c1016a0 2023-12-11T17:44:19.646 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:19.643+0000 7fc781d88700 1 -- 172.21.15.59:0/3268115365 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fc7680a1480 msgr2=0x7fc7680a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:19.646 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:19.643+0000 7fc781d88700 1 --2- 172.21.15.59:0/3268115365 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fc7680a1480 0x7fc7680a3940 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc76c003730 tx=0x7fc76c000bc0 comp rx=0 tx=0).stop 2023-12-11T17:44:19.646 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:19.643+0000 7fc781d88700 1 -- 172.21.15.59:0/3268115365 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc77c1016a0 msgr2=0x7fc77c196990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:19.646 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:19.643+0000 7fc781d88700 1 --2- 172.21.15.59:0/3268115365 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc77c1016a0 0x7fc77c196990 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc76400e9c0 tx=0x7fc76400ecd0 comp rx=0 tx=0).stop 2023-12-11T17:44:19.646 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:19.643+0000 7fc781d88700 1 -- 172.21.15.59:0/3268115365 shutdown_connections 2023-12-11T17:44:19.646 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:19.643+0000 7fc781d88700 1 --2- 172.21.15.59:0/3268115365 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fc7680a1480 0x7fc7680a3940 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:19.647 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:19.643+0000 7fc781d88700 1 --2- 172.21.15.59:0/3268115365 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc77c102040 0x7fc77c19d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:19.647 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:19.643+0000 7fc781d88700 1 --2- 172.21.15.59:0/3268115365 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc77c1004b0 0x7fc77c196450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:19.647 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:19.643+0000 7fc781d88700 1 --2- 172.21.15.59:0/3268115365 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc77c1016a0 0x7fc77c196990 secure :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc76400e9c0 tx=0x7fc76400ecd0 comp rx=0 tx=0).stop 2023-12-11T17:44:19.647 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:19.643+0000 7fc781d88700 1 -- 172.21.15.59:0/3268115365 >> 172.21.15.59:0/3268115365 conn(0x7fc77c0fba20 msgr2=0x7fc77c104ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:44:19.647 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:19.644+0000 7fc781d88700 1 -- 172.21.15.59:0/3268115365 shutdown_connections 2023-12-11T17:44:19.647 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:19.644+0000 7fc781d88700 1 -- 172.21.15.59:0/3268115365 wait complete. 2023-12-11T17:44:19.667 INFO:teuthology.orchestra.run.smithi059.stderr:pool 'cephfs_data' created 2023-12-11T17:44:19.679 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-12-11T17:44:19.811 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:19.808+0000 7f3aa96e1700 1 -- 172.21.15.59:0/26073392 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3aa4102000 msgr2=0x7f3aa41043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:19.811 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:19.808+0000 7f3aa96e1700 1 --2- 172.21.15.59:0/26073392 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3aa4102000 0x7f3aa41043f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f3a94009990 tx=0x7f3a94009ca0 comp rx=0 tx=0).stop 2023-12-11T17:44:19.811 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:19.808+0000 7f3aa96e1700 1 -- 172.21.15.59:0/26073392 shutdown_connections 2023-12-11T17:44:19.812 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:19.808+0000 7f3aa96e1700 1 --2- 172.21.15.59:0/26073392 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3aa4107260 0x7f3aa41096d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:19.812 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:19.808+0000 7f3aa96e1700 1 --2- 172.21.15.59:0/26073392 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3aa4102000 0x7f3aa41043f0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:19.812 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:19.808+0000 7f3aa96e1700 1 --2- 172.21.15.59:0/26073392 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3aa4104930 0x7f3aa4106d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:19.812 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:19.808+0000 7f3aa96e1700 1 -- 172.21.15.59:0/26073392 >> 172.21.15.59:0/26073392 conn(0x7f3aa40fb9f0 msgr2=0x7f3aa40fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:44:19.812 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:19.808+0000 7f3aa96e1700 1 -- 172.21.15.59:0/26073392 shutdown_connections 2023-12-11T17:44:19.812 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:19.808+0000 7f3aa96e1700 1 -- 172.21.15.59:0/26073392 wait complete. 2023-12-11T17:44:19.812 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:19.809+0000 7f3aa96e1700 1 Processor -- start 2023-12-11T17:44:19.812 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:19.809+0000 7f3aa96e1700 1 -- start start 2023-12-11T17:44:19.814 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:19.809+0000 7f3aa96e1700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3aa4102000 0x7f3aa41974f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:19.815 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:19.809+0000 7f3aa96e1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3aa4104930 0x7f3aa4197a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:19.815 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:19.809+0000 7f3aa96e1700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3aa4107260 0x7f3aa419ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:19.815 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:19.809+0000 7f3aa96e1700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa419f020 con 0x7f3aa4107260 2023-12-11T17:44:19.815 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:19.809+0000 7f3aa96e1700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa419f160 con 0x7f3aa4102000 2023-12-11T17:44:19.815 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:19.809+0000 7f3aa96e1700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa419f2f0 con 0x7f3aa4104930 2023-12-11T17:44:19.815 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:19.810+0000 7f3aa37fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3aa4107260 0x7f3aa419ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:19.815 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:19.810+0000 7f3aa2ffd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3aa4102000 0x7f3aa41974f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:19.815 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:19.810+0000 7f3aa27fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3aa4104930 0x7f3aa4197a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:19.815 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:19.810+0000 7f3aa37fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3aa4107260 0x7f3aa419ea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:32850/0 (socket says 172.21.15.59:32850) 2023-12-11T17:44:19.815 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:19.810+0000 7f3aa37fe700 1 -- 172.21.15.59:0/492300905 learned_addr learned my addr 172.21.15.59:0/492300905 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:44:19.815 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:19.810+0000 7f3aa37fe700 1 -- 172.21.15.59:0/492300905 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3aa4104930 msgr2=0x7f3aa4197a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:19.815 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:19.810+0000 7f3aa37fe700 1 --2- 172.21.15.59:0/492300905 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3aa4104930 0x7f3aa4197a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:19.815 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:19.810+0000 7f3aa37fe700 1 -- 172.21.15.59:0/492300905 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3aa4102000 msgr2=0x7f3aa41974f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:19.815 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:19.810+0000 7f3aa37fe700 1 --2- 172.21.15.59:0/492300905 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3aa4102000 0x7f3aa41974f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:19.815 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:19.810+0000 7f3aa37fe700 1 -- 172.21.15.59:0/492300905 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a940095f0 con 0x7f3aa4107260 2023-12-11T17:44:19.816 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:19.810+0000 7f3aa2ffd700 1 --2- 172.21.15.59:0/492300905 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3aa4102000 0x7f3aa41974f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T17:44:19.816 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:19.810+0000 7f3aa27fc700 1 --2- 172.21.15.59:0/492300905 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3aa4104930 0x7f3aa4197a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T17:44:19.816 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:19.810+0000 7f3aa37fe700 1 --2- 172.21.15.59:0/492300905 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3aa4107260 0x7f3aa419ea50 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c00ef30 tx=0x7f3a9c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:44:19.816 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:19.810+0000 7f3a8bfff700 1 -- 172.21.15.59:0/492300905 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a9c018070 con 0x7f3aa4107260 2023-12-11T17:44:19.816 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:19.810+0000 7f3aa96e1700 1 -- 172.21.15.59:0/492300905 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3aa419f570 con 0x7f3aa4107260 2023-12-11T17:44:19.816 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:19.810+0000 7f3a8bfff700 1 -- 172.21.15.59:0/492300905 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3a9c00f040 con 0x7f3aa4107260 2023-12-11T17:44:19.816 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:19.810+0000 7f3a8bfff700 1 -- 172.21.15.59:0/492300905 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a9c00cd00 con 0x7f3aa4107260 2023-12-11T17:44:19.816 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:19.810+0000 7f3aa96e1700 1 -- 172.21.15.59:0/492300905 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3aa419fa30 con 0x7f3aa4107260 2023-12-11T17:44:19.816 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:19.812+0000 7f3aa96e1700 1 -- 172.21.15.59:0/492300905 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3aa40fd570 con 0x7f3aa4107260 2023-12-11T17:44:19.817 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:19.813+0000 7f3a8bfff700 1 -- 172.21.15.59:0/492300905 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f3a9c003a00 con 0x7f3aa4107260 2023-12-11T17:44:19.817 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:19.813+0000 7f3a8bfff700 1 --2- 172.21.15.59:0/492300905 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f3a8c0a1480 0x7f3a8c0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:19.817 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:19.813+0000 7f3a8bfff700 1 -- 172.21.15.59:0/492300905 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7879+0+0 (secure 0 0 0) 0x7f3a9c0cc6c0 con 0x7f3aa4107260 2023-12-11T17:44:19.817 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:19.813+0000 7f3aa2ffd700 1 --2- 172.21.15.59:0/492300905 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f3a8c0a1480 0x7f3a8c0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:19.817 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:19.814+0000 7f3aa2ffd700 1 --2- 172.21.15.59:0/492300905 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f3a8c0a1480 0x7f3a8c0a3940 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f3a9400b2e0 tx=0x7f3a94005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:44:19.820 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:19.818+0000 7f3a8bfff700 1 -- 172.21.15.59:0/492300905 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f3a9c095f60 con 0x7f3aa4107260 2023-12-11T17:44:19.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:19 smithi181 ceph-mon[103627]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T17:44:19.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:19 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/854047649' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-12-11T17:44:19.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:19 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:44:19.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:19 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:44:19.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:19 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:44:19.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:19 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:44:19.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:19 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:44:19.896 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:19 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:44:19.896 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:19 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/854047649' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-12-11T17:44:19.896 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:19 smithi181 ceph-mon[103627]: osdmap e79: 12 total, 12 up, 12 in 2023-12-11T17:44:19.896 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:19 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/3268115365' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-12-11T17:44:19.896 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:19 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:44:19.896 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:19 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:44:19.896 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:19 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:44:19.896 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:19 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:44:19.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:19 smithi059 ceph-mon[98847]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T17:44:19.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:19 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/854047649' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-12-11T17:44:19.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:19 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:44:19.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:19 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:44:19.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:19 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:44:19.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:19 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:44:19.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:19 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:44:19.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:19 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:44:19.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:19 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/854047649' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-12-11T17:44:19.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:19 smithi059 ceph-mon[98847]: osdmap e79: 12 total, 12 up, 12 in 2023-12-11T17:44:19.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:19 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/3268115365' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-12-11T17:44:19.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:19 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:44:19.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:19 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:44:19.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:19 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:44:19.906 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:19 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:44:20.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:19 smithi115 ceph-mon[102748]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T17:44:20.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:19 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/854047649' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-12-11T17:44:20.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:19 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:44:20.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:19 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:44:20.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:19 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:44:20.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:19 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:44:20.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:19 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:44:20.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:19 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:44:20.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:19 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/854047649' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-12-11T17:44:20.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:19 smithi115 ceph-mon[102748]: osdmap e79: 12 total, 12 up, 12 in 2023-12-11T17:44:20.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:19 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/3268115365' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-12-11T17:44:20.034 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:19 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:44:20.034 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:19 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:44:20.034 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:19 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:44:20.034 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:19 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:44:20.077 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:20.075+0000 7f3aa96e1700 1 -- 172.21.15.59:0/492300905 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f3aa40682e0 con 0x7f3aa4107260 2023-12-11T17:44:20.648 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:20.645+0000 7f3a8bfff700 1 -- 172.21.15.59:0/492300905 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7f3a9c099580 con 0x7f3aa4107260 2023-12-11T17:44:20.651 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:20.648+0000 7f3aa96e1700 1 -- 172.21.15.59:0/492300905 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f3a8c0a1480 msgr2=0x7f3a8c0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:20.651 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:20.648+0000 7f3aa96e1700 1 --2- 172.21.15.59:0/492300905 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f3a8c0a1480 0x7f3a8c0a3940 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f3a9400b2e0 tx=0x7f3a94005fd0 comp rx=0 tx=0).stop 2023-12-11T17:44:20.651 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:20.648+0000 7f3aa96e1700 1 -- 172.21.15.59:0/492300905 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3aa4107260 msgr2=0x7f3aa419ea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:20.651 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:20.648+0000 7f3aa96e1700 1 --2- 172.21.15.59:0/492300905 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3aa4107260 0x7f3aa419ea50 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c00ef30 tx=0x7f3a9c00c3a0 comp rx=0 tx=0).stop 2023-12-11T17:44:20.651 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:20.648+0000 7f3aa96e1700 1 -- 172.21.15.59:0/492300905 shutdown_connections 2023-12-11T17:44:20.651 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:20.648+0000 7f3aa96e1700 1 --2- 172.21.15.59:0/492300905 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f3a8c0a1480 0x7f3a8c0a3940 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:20.651 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:20.648+0000 7f3aa96e1700 1 --2- 172.21.15.59:0/492300905 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3aa4107260 0x7f3aa419ea50 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:20.651 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:20.648+0000 7f3aa96e1700 1 --2- 172.21.15.59:0/492300905 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3aa4102000 0x7f3aa41974f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:20.652 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:20.648+0000 7f3aa96e1700 1 --2- 172.21.15.59:0/492300905 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3aa4104930 0x7f3aa4197a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:20.652 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:20.648+0000 7f3aa96e1700 1 -- 172.21.15.59:0/492300905 >> 172.21.15.59:0/492300905 conn(0x7f3aa40fb9f0 msgr2=0x7f3aa4103ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:44:20.652 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:20.648+0000 7f3aa96e1700 1 -- 172.21.15.59:0/492300905 shutdown_connections 2023-12-11T17:44:20.652 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:20.648+0000 7f3aa96e1700 1 -- 172.21.15.59:0/492300905 wait complete. 2023-12-11T17:44:20.652 INFO:teuthology.orchestra.run.smithi059.stderr:new fs with metadata pool 2 and data pool 3 2023-12-11T17:44:20.665 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2023-12-11T17:44:20.665 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2023-12-11T17:44:20.801 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:20.798+0000 7f4a66c48700 1 -- 172.21.15.59:0/1787277722 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4a60104990 msgr2=0x7f4a60106d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:20.801 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:20.798+0000 7f4a66c48700 1 --2- 172.21.15.59:0/1787277722 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4a60104990 0x7f4a60106d80 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c009990 tx=0x7f4a4c009ca0 comp rx=0 tx=0).stop 2023-12-11T17:44:20.802 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:20.798+0000 7f4a66c48700 1 -- 172.21.15.59:0/1787277722 shutdown_connections 2023-12-11T17:44:20.802 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:20.798+0000 7f4a66c48700 1 --2- 172.21.15.59:0/1787277722 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4a601072c0 0x7f4a601096d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:20.802 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:20.798+0000 7f4a66c48700 1 --2- 172.21.15.59:0/1787277722 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4a60102060 0x7f4a60104450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:20.802 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:20.798+0000 7f4a66c48700 1 --2- 172.21.15.59:0/1787277722 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4a60104990 0x7f4a60106d80 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:20.802 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:20.798+0000 7f4a66c48700 1 -- 172.21.15.59:0/1787277722 >> 172.21.15.59:0/1787277722 conn(0x7f4a600fb9f0 msgr2=0x7f4a600fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:44:20.802 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:20.799+0000 7f4a66c48700 1 -- 172.21.15.59:0/1787277722 shutdown_connections 2023-12-11T17:44:20.802 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:20.799+0000 7f4a66c48700 1 -- 172.21.15.59:0/1787277722 wait complete. 2023-12-11T17:44:20.804 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:20.799+0000 7f4a66c48700 1 Processor -- start 2023-12-11T17:44:20.804 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:20.799+0000 7f4a66c48700 1 -- start start 2023-12-11T17:44:20.805 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:20.800+0000 7f4a66c48700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4a60102060 0x7f4a60101980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:20.805 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:20.800+0000 7f4a66c48700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4a60104990 0x7f4a60100010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:20.805 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:20.800+0000 7f4a66c48700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4a601072c0 0x7f4a60100550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:20.805 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:20.800+0000 7f4a66c48700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a60100cf0 con 0x7f4a601072c0 2023-12-11T17:44:20.805 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:20.800+0000 7f4a66c48700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a60100e60 con 0x7f4a60104990 2023-12-11T17:44:20.805 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:20.800+0000 7f4a66c48700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a601a1550 con 0x7f4a60102060 2023-12-11T17:44:20.805 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:20.800+0000 7f4a651e5700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4a601072c0 0x7f4a60100550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:20.805 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:20.800+0000 7f4a649e4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4a60102060 0x7f4a60101980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:20.805 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:20.800+0000 7f4a651e5700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4a601072c0 0x7f4a60100550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:32872/0 (socket says 172.21.15.59:32872) 2023-12-11T17:44:20.805 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:20.800+0000 7f4a649e4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4a60102060 0x7f4a60101980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.59:37686/0 (socket says 172.21.15.59:37686) 2023-12-11T17:44:20.805 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:20.800+0000 7f4a651e5700 1 -- 172.21.15.59:0/918611493 learned_addr learned my addr 172.21.15.59:0/918611493 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:44:20.805 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:20.800+0000 7f4a5ffff700 1 --2- 172.21.15.59:0/918611493 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4a60104990 0x7f4a60100010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:20.805 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:20.800+0000 7f4a651e5700 1 -- 172.21.15.59:0/918611493 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4a60102060 msgr2=0x7f4a60101980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:20.805 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:20.800+0000 7f4a651e5700 1 --2- 172.21.15.59:0/918611493 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4a60102060 0x7f4a60101980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:20.806 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:20.800+0000 7f4a651e5700 1 -- 172.21.15.59:0/918611493 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4a60104990 msgr2=0x7f4a60100010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:20.806 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:20.800+0000 7f4a651e5700 1 --2- 172.21.15.59:0/918611493 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4a60104990 0x7f4a60100010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:20.806 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:20.800+0000 7f4a651e5700 1 -- 172.21.15.59:0/918611493 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a4c0095f0 con 0x7f4a601072c0 2023-12-11T17:44:20.806 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:20.800+0000 7f4a649e4700 1 --2- 172.21.15.59:0/918611493 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4a60102060 0x7f4a60101980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T17:44:20.806 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:20.800+0000 7f4a651e5700 1 --2- 172.21.15.59:0/918611493 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4a601072c0 0x7f4a60100550 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f4a5800ef30 tx=0x7f4a5800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:44:20.806 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:20.801+0000 7f4a5dffb700 1 -- 172.21.15.59:0/918611493 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a58018070 con 0x7f4a601072c0 2023-12-11T17:44:20.806 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:20.801+0000 7f4a5dffb700 1 -- 172.21.15.59:0/918611493 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a5800f040 con 0x7f4a601072c0 2023-12-11T17:44:20.806 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:20.801+0000 7f4a66c48700 1 -- 172.21.15.59:0/918611493 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a601a1780 con 0x7f4a601072c0 2023-12-11T17:44:20.806 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:20.801+0000 7f4a5dffb700 1 -- 172.21.15.59:0/918611493 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a5800cca0 con 0x7f4a601072c0 2023-12-11T17:44:20.806 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:20.801+0000 7f4a66c48700 1 -- 172.21.15.59:0/918611493 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a601a1ce0 con 0x7f4a601072c0 2023-12-11T17:44:20.807 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:20.802+0000 7f4a66c48700 1 -- 172.21.15.59:0/918611493 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a600fd570 con 0x7f4a601072c0 2023-12-11T17:44:20.808 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:20.804+0000 7f4a5dffb700 1 -- 172.21.15.59:0/918611493 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f4a58003a00 con 0x7f4a601072c0 2023-12-11T17:44:20.808 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:20.804+0000 7f4a5dffb700 1 --2- 172.21.15.59:0/918611493 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f4a500a1500 0x7f4a500a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:20.808 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:20.804+0000 7f4a5dffb700 1 -- 172.21.15.59:0/918611493 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f4a580cc5b0 con 0x7f4a601072c0 2023-12-11T17:44:20.808 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:20.804+0000 7f4a649e4700 1 --2- 172.21.15.59:0/918611493 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f4a500a1500 0x7f4a500a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:20.808 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:20.805+0000 7f4a649e4700 1 --2- 172.21.15.59:0/918611493 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f4a500a1500 0x7f4a500a39c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4a54011490 tx=0x7f4a54009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:44:20.811 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:20.809+0000 7f4a5dffb700 1 -- 172.21.15.59:0/918611493 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f4a58095df0 con 0x7f4a601072c0 2023-12-11T17:44:20.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:20 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:44:20.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:20 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/3268115365' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2023-12-11T17:44:20.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:20 smithi181 ceph-mon[103627]: osdmap e80: 12 total, 12 up, 12 in 2023-12-11T17:44:20.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:20 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/492300905' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-12-11T17:44:20.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:20 smithi181 ceph-mon[103627]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-12-11T17:44:20.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:20 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:44:20.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:20 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/3268115365' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2023-12-11T17:44:20.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:20 smithi059 ceph-mon[98847]: osdmap e80: 12 total, 12 up, 12 in 2023-12-11T17:44:20.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:20 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/492300905' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-12-11T17:44:20.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:20 smithi059 ceph-mon[98847]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-12-11T17:44:21.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:20 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:44:21.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:20 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/3268115365' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2023-12-11T17:44:21.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:20 smithi115 ceph-mon[102748]: osdmap e80: 12 total, 12 up, 12 in 2023-12-11T17:44:21.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:20 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/492300905' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-12-11T17:44:21.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:20 smithi115 ceph-mon[102748]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-12-11T17:44:21.036 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:21.034+0000 7f4a66c48700 1 -- 172.21.15.59:0/918611493 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7f4a600682e0 con 0x7f4a601072c0 2023-12-11T17:44:21.598 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:21.596+0000 7f4a5dffb700 1 -- 172.21.15.59:0/918611493 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v82) v1 ==== 157+0+0 (secure 0 0 0) 0x7f4a58099410 con 0x7f4a601072c0 2023-12-11T17:44:21.601 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:21.598+0000 7f4a66c48700 1 -- 172.21.15.59:0/918611493 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f4a500a1500 msgr2=0x7f4a500a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:21.601 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:21.598+0000 7f4a66c48700 1 --2- 172.21.15.59:0/918611493 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f4a500a1500 0x7f4a500a39c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4a54011490 tx=0x7f4a54009260 comp rx=0 tx=0).stop 2023-12-11T17:44:21.601 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:21.598+0000 7f4a66c48700 1 -- 172.21.15.59:0/918611493 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4a601072c0 msgr2=0x7f4a60100550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:21.601 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:21.598+0000 7f4a66c48700 1 --2- 172.21.15.59:0/918611493 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4a601072c0 0x7f4a60100550 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f4a5800ef30 tx=0x7f4a5800c3a0 comp rx=0 tx=0).stop 2023-12-11T17:44:21.601 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:21.598+0000 7f4a66c48700 1 -- 172.21.15.59:0/918611493 shutdown_connections 2023-12-11T17:44:21.601 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:21.598+0000 7f4a66c48700 1 --2- 172.21.15.59:0/918611493 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f4a500a1500 0x7f4a500a39c0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:21.601 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:21.598+0000 7f4a66c48700 1 --2- 172.21.15.59:0/918611493 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4a601072c0 0x7f4a60100550 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:21.602 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:21.598+0000 7f4a66c48700 1 --2- 172.21.15.59:0/918611493 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4a60102060 0x7f4a60101980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:21.602 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:21.598+0000 7f4a66c48700 1 --2- 172.21.15.59:0/918611493 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4a60104990 0x7f4a60100010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:21.602 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:21.598+0000 7f4a66c48700 1 -- 172.21.15.59:0/918611493 >> 172.21.15.59:0/918611493 conn(0x7f4a600fb9f0 msgr2=0x7f4a60103f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:44:21.602 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:21.598+0000 7f4a66c48700 1 -- 172.21.15.59:0/918611493 shutdown_connections 2023-12-11T17:44:21.602 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:21.598+0000 7f4a66c48700 1 -- 172.21.15.59:0/918611493 wait complete. 2023-12-11T17:44:21.615 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2023-12-11T17:44:21.757 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:21.752+0000 7fc5f1d7c700 1 -- 172.21.15.59:0/779604080 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5ec102690 msgr2=0x7fc5ec102af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:21.757 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:21.752+0000 7fc5f1d7c700 1 --2- 172.21.15.59:0/779604080 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5ec102690 0x7fc5ec102af0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc5d40099c0 tx=0x7fc5d4009cd0 comp rx=0 tx=0).stop 2023-12-11T17:44:21.757 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:21.752+0000 7fc5f1d7c700 1 -- 172.21.15.59:0/779604080 shutdown_connections 2023-12-11T17:44:21.757 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:21.752+0000 7fc5f1d7c700 1 --2- 172.21.15.59:0/779604080 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc5ec103030 0x7fc5ec1098e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:21.757 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:21.752+0000 7fc5f1d7c700 1 --2- 172.21.15.59:0/779604080 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5ec1014a0 0x7fc5ec1018c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:21.757 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:21.752+0000 7fc5f1d7c700 1 --2- 172.21.15.59:0/779604080 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5ec102690 0x7fc5ec102af0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:21.758 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:21.752+0000 7fc5f1d7c700 1 -- 172.21.15.59:0/779604080 >> 172.21.15.59:0/779604080 conn(0x7fc5ec0fca10 msgr2=0x7fc5ec0fee50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:44:21.758 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:21.752+0000 7fc5f1d7c700 1 -- 172.21.15.59:0/779604080 shutdown_connections 2023-12-11T17:44:21.758 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:21.752+0000 7fc5f1d7c700 1 -- 172.21.15.59:0/779604080 wait complete. 2023-12-11T17:44:21.758 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:21.753+0000 7fc5f1d7c700 1 Processor -- start 2023-12-11T17:44:21.758 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:21.753+0000 7fc5f1d7c700 1 -- start start 2023-12-11T17:44:21.758 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:21.753+0000 7fc5f1d7c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5ec1014a0 0x7fc5ec1972c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:21.758 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:21.753+0000 7fc5f1d7c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc5ec102690 0x7fc5ec197800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:21.758 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:21.753+0000 7fc5f1d7c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5ec103030 0x7fc5ec19e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:21.758 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:21.753+0000 7fc5f1d7c700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5ec19edf0 con 0x7fc5ec1014a0 2023-12-11T17:44:21.758 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:21.753+0000 7fc5f1d7c700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5ec19ef30 con 0x7fc5ec102690 2023-12-11T17:44:21.758 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:21.753+0000 7fc5f1d7c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5ec19f0c0 con 0x7fc5ec103030 2023-12-11T17:44:21.758 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:21.753+0000 7fc5eb7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5ec1014a0 0x7fc5ec1972c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:21.758 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:21.753+0000 7fc5eb7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5ec1014a0 0x7fc5ec1972c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:32880/0 (socket says 172.21.15.59:32880) 2023-12-11T17:44:21.758 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:21.753+0000 7fc5eb7fe700 1 -- 172.21.15.59:0/2538219663 learned_addr learned my addr 172.21.15.59:0/2538219663 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:44:21.759 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:21.754+0000 7fc5ebfff700 1 --2- 172.21.15.59:0/2538219663 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5ec103030 0x7fc5ec19e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:21.759 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:21.754+0000 7fc5eaffd700 1 --2- 172.21.15.59:0/2538219663 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc5ec102690 0x7fc5ec197800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:21.759 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:21.754+0000 7fc5eb7fe700 1 -- 172.21.15.59:0/2538219663 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5ec103030 msgr2=0x7fc5ec19e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:21.759 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:21.754+0000 7fc5eb7fe700 1 --2- 172.21.15.59:0/2538219663 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5ec103030 0x7fc5ec19e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:21.759 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:21.754+0000 7fc5eb7fe700 1 -- 172.21.15.59:0/2538219663 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc5ec102690 msgr2=0x7fc5ec197800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:21.759 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:21.754+0000 7fc5eb7fe700 1 --2- 172.21.15.59:0/2538219663 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc5ec102690 0x7fc5ec197800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:21.759 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:21.754+0000 7fc5eb7fe700 1 -- 172.21.15.59:0/2538219663 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5d40095f0 con 0x7fc5ec1014a0 2023-12-11T17:44:21.759 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:21.754+0000 7fc5ebfff700 1 --2- 172.21.15.59:0/2538219663 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5ec103030 0x7fc5ec19e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T17:44:21.759 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:21.754+0000 7fc5eb7fe700 1 --2- 172.21.15.59:0/2538219663 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5ec1014a0 0x7fc5ec1972c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc00c990 tx=0x7fc5dc00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:44:21.759 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:21.754+0000 7fc5e8ff9700 1 -- 172.21.15.59:0/2538219663 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5dc007b00 con 0x7fc5ec1014a0 2023-12-11T17:44:21.759 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:21.754+0000 7fc5e8ff9700 1 -- 172.21.15.59:0/2538219663 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc5dc007c60 con 0x7fc5ec1014a0 2023-12-11T17:44:21.759 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:21.754+0000 7fc5f1d7c700 1 -- 172.21.15.59:0/2538219663 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5ec19f3a0 con 0x7fc5ec1014a0 2023-12-11T17:44:21.759 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:21.754+0000 7fc5e8ff9700 1 -- 172.21.15.59:0/2538219663 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5dc018660 con 0x7fc5ec1014a0 2023-12-11T17:44:21.759 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:21.754+0000 7fc5f1d7c700 1 -- 172.21.15.59:0/2538219663 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5ec19f8b0 con 0x7fc5ec1014a0 2023-12-11T17:44:21.760 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:21.755+0000 7fc5f1d7c700 1 -- 172.21.15.59:0/2538219663 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5ec108850 con 0x7fc5ec1014a0 2023-12-11T17:44:21.760 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:21.756+0000 7fc5e8ff9700 1 -- 172.21.15.59:0/2538219663 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7fc5dc00f450 con 0x7fc5ec1014a0 2023-12-11T17:44:21.761 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:21.757+0000 7fc5e8ff9700 1 --2- 172.21.15.59:0/2538219663 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fc5d80a1470 0x7fc5d80a3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:21.761 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:21.757+0000 7fc5eaffd700 1 --2- 172.21.15.59:0/2538219663 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fc5d80a1470 0x7fc5d80a3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:21.761 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:21.757+0000 7fc5e8ff9700 1 -- 172.21.15.59:0/2538219663 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8205+0+0 (secure 0 0 0) 0x7fc5dc0ccd30 con 0x7fc5ec1014a0 2023-12-11T17:44:21.761 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:21.757+0000 7fc5eaffd700 1 --2- 172.21.15.59:0/2538219663 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fc5d80a1470 0x7fc5d80a3930 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fc5d4000c00 tx=0x7fc5d4005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:44:21.763 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:21.761+0000 7fc5e8ff9700 1 -- 172.21.15.59:0/2538219663 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fc5dc096400 con 0x7fc5ec1014a0 2023-12-11T17:44:21.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:21 smithi181 ceph-mon[103627]: pgmap v253: 97 pgs: 11 creating+peering, 12 active+clean, 74 unknown; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T17:44:21.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:21 smithi181 ceph-mon[103627]: osdmap e81: 12 total, 12 up, 12 in 2023-12-11T17:44:21.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:21 smithi181 ceph-mon[103627]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-12-11T17:44:21.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:21 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/492300905' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-12-11T17:44:21.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:21 smithi181 ceph-mon[103627]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-12-11T17:44:21.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:21 smithi181 ceph-mon[103627]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-12-11T17:44:21.896 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:21 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/918611493' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-12-11T17:44:21.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:21 smithi059 ceph-mon[98847]: pgmap v253: 97 pgs: 11 creating+peering, 12 active+clean, 74 unknown; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T17:44:21.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:21 smithi059 ceph-mon[98847]: osdmap e81: 12 total, 12 up, 12 in 2023-12-11T17:44:21.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:21 smithi059 ceph-mon[98847]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-12-11T17:44:21.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:21 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/492300905' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-12-11T17:44:21.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:21 smithi059 ceph-mon[98847]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-12-11T17:44:21.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:21 smithi059 ceph-mon[98847]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-12-11T17:44:21.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:21 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/918611493' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-12-11T17:44:21.978 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:21.976+0000 7fc5f1d7c700 1 -- 172.21.15.59:0/2538219663 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fc5ec068680 con 0x7fc5ec1014a0 2023-12-11T17:44:22.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:21 smithi115 ceph-mon[102748]: pgmap v253: 97 pgs: 11 creating+peering, 12 active+clean, 74 unknown; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T17:44:22.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:21 smithi115 ceph-mon[102748]: osdmap e81: 12 total, 12 up, 12 in 2023-12-11T17:44:22.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:21 smithi115 ceph-mon[102748]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-12-11T17:44:22.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:21 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/492300905' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-12-11T17:44:22.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:21 smithi115 ceph-mon[102748]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-12-11T17:44:22.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:21 smithi115 ceph-mon[102748]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-12-11T17:44:22.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:21 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/918611493' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-12-11T17:44:22.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:22 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/918611493' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2023-12-11T17:44:22.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:22 smithi181 ceph-mon[103627]: osdmap e82: 12 total, 12 up, 12 in 2023-12-11T17:44:22.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:22 smithi181 ceph-mon[103627]: mds.? [v2:172.21.15.115:6838/4122922866,v1:172.21.15.115:6839/4122922866] up:active 2023-12-11T17:44:22.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:22 smithi181 ceph-mon[103627]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-12-11T17:44:22.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:22 smithi181 ceph-mon[103627]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-12-11T17:44:22.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:22 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/2538219663' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-12-11T17:44:22.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:22 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:44:22.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:22 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:44:22.896 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:22 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:44:22.896 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:22 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:44:22.896 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:22 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:44:22.896 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:22 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:44:22.896 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:22 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:44:22.896 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:22 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:44:22.896 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:22 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:44:22.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:22 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/918611493' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2023-12-11T17:44:22.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:22 smithi059 ceph-mon[98847]: osdmap e82: 12 total, 12 up, 12 in 2023-12-11T17:44:22.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:22 smithi059 ceph-mon[98847]: mds.? [v2:172.21.15.115:6838/4122922866,v1:172.21.15.115:6839/4122922866] up:active 2023-12-11T17:44:22.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:22 smithi059 ceph-mon[98847]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-12-11T17:44:22.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:22 smithi059 ceph-mon[98847]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-12-11T17:44:22.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:22 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/2538219663' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-12-11T17:44:22.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:22 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:44:22.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:22 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:44:22.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:22 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:44:22.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:22 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:44:22.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:22 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:44:22.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:22 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:44:22.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:22 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:44:22.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:22 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:44:22.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:22 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:44:23.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:22 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/918611493' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2023-12-11T17:44:23.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:22 smithi115 ceph-mon[102748]: osdmap e82: 12 total, 12 up, 12 in 2023-12-11T17:44:23.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:22 smithi115 ceph-mon[102748]: mds.? [v2:172.21.15.115:6838/4122922866,v1:172.21.15.115:6839/4122922866] up:active 2023-12-11T17:44:23.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:22 smithi115 ceph-mon[102748]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-12-11T17:44:23.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:22 smithi115 ceph-mon[102748]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-12-11T17:44:23.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:22 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/2538219663' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-12-11T17:44:23.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:22 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:44:23.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:22 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:44:23.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:22 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:44:23.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:22 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:44:23.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:22 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:44:23.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:22 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:44:23.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:22 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:44:23.034 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:22 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:44:23.034 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:22 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:44:23.667 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:23.664+0000 7fc5e8ff9700 1 -- 172.21.15.59:0/2538219663 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v84) v1 ==== 228+0+0 (secure 0 0 0) 0x7fc5dc099a20 con 0x7fc5ec1014a0 2023-12-11T17:44:23.671 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:23.668+0000 7fc5f1d7c700 1 -- 172.21.15.59:0/2538219663 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fc5d80a1470 msgr2=0x7fc5d80a3930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:23.671 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:23.668+0000 7fc5f1d7c700 1 --2- 172.21.15.59:0/2538219663 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fc5d80a1470 0x7fc5d80a3930 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fc5d4000c00 tx=0x7fc5d4005fd0 comp rx=0 tx=0).stop 2023-12-11T17:44:23.671 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:23.668+0000 7fc5f1d7c700 1 -- 172.21.15.59:0/2538219663 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5ec1014a0 msgr2=0x7fc5ec1972c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:23.671 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:23.668+0000 7fc5f1d7c700 1 --2- 172.21.15.59:0/2538219663 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5ec1014a0 0x7fc5ec1972c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc00c990 tx=0x7fc5dc00cd50 comp rx=0 tx=0).stop 2023-12-11T17:44:23.676 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:23.672+0000 7fc5f1d7c700 1 -- 172.21.15.59:0/2538219663 shutdown_connections 2023-12-11T17:44:23.677 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:23.672+0000 7fc5f1d7c700 1 --2- 172.21.15.59:0/2538219663 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fc5d80a1470 0x7fc5d80a3930 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:23.677 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:23.672+0000 7fc5f1d7c700 1 --2- 172.21.15.59:0/2538219663 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5ec103030 0x7fc5ec19e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:23.677 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:23.672+0000 7fc5f1d7c700 1 --2- 172.21.15.59:0/2538219663 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5ec1014a0 0x7fc5ec1972c0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:23.677 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:23.672+0000 7fc5f1d7c700 1 --2- 172.21.15.59:0/2538219663 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc5ec102690 0x7fc5ec197800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:23.677 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:23.672+0000 7fc5f1d7c700 1 -- 172.21.15.59:0/2538219663 >> 172.21.15.59:0/2538219663 conn(0x7fc5ec0fca10 msgr2=0x7fc5ec105990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:44:23.679 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:23.677+0000 7fc5f1d7c700 1 -- 172.21.15.59:0/2538219663 shutdown_connections 2023-12-11T17:44:23.680 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:23.678+0000 7fc5f1d7c700 1 -- 172.21.15.59:0/2538219663 wait complete. 2023-12-11T17:44:23.681 INFO:teuthology.orchestra.run.smithi059.stderr:pool 'cephfs_data_ec' created 2023-12-11T17:44:23.695 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2023-12-11T17:44:23.834 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:23.829+0000 7f8438776700 1 -- 172.21.15.59:0/3396226602 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8430107260 msgr2=0x7f84301096d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:23.835 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:23.829+0000 7f8438776700 1 --2- 172.21.15.59:0/3396226602 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8430107260 0x7f84301096d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f842800b0a0 tx=0x7f842800b3b0 comp rx=0 tx=0).stop 2023-12-11T17:44:23.835 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:23.830+0000 7f8438776700 1 -- 172.21.15.59:0/3396226602 shutdown_connections 2023-12-11T17:44:23.835 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:23.830+0000 7f8438776700 1 --2- 172.21.15.59:0/3396226602 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8430107260 0x7f84301096d0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:23.835 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:23.830+0000 7f8438776700 1 --2- 172.21.15.59:0/3396226602 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8430102000 0x7f84301043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:23.835 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:23.830+0000 7f8438776700 1 --2- 172.21.15.59:0/3396226602 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8430104930 0x7f8430106d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:23.835 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:23.830+0000 7f8438776700 1 -- 172.21.15.59:0/3396226602 >> 172.21.15.59:0/3396226602 conn(0x7f84300fb9f0 msgr2=0x7f84300fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:44:23.835 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:23.830+0000 7f8438776700 1 -- 172.21.15.59:0/3396226602 shutdown_connections 2023-12-11T17:44:23.835 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:23.830+0000 7f8438776700 1 -- 172.21.15.59:0/3396226602 wait complete. 2023-12-11T17:44:23.835 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:23.830+0000 7f8438776700 1 Processor -- start 2023-12-11T17:44:23.835 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:23.830+0000 7f8438776700 1 -- start start 2023-12-11T17:44:23.835 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:23.830+0000 7f8438776700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8430102000 0x7f8430197590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:23.835 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:23.830+0000 7f8438776700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8430104930 0x7f8430197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:23.835 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:23.830+0000 7f8438776700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8430107260 0x7f843019eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:23.835 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:23.830+0000 7f8438776700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f843019f0c0 con 0x7f8430102000 2023-12-11T17:44:23.836 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:23.830+0000 7f8438776700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f843019f200 con 0x7f8430107260 2023-12-11T17:44:23.836 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:23.831+0000 7f8438776700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f843019f390 con 0x7f8430104930 2023-12-11T17:44:23.836 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:23.831+0000 7f8436512700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8430102000 0x7f8430197590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:23.836 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:23.831+0000 7f8436512700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8430102000 0x7f8430197590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:32906/0 (socket says 172.21.15.59:32906) 2023-12-11T17:44:23.836 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:23.831+0000 7f8436512700 1 -- 172.21.15.59:0/3835944011 learned_addr learned my addr 172.21.15.59:0/3835944011 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:44:23.836 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:23.831+0000 7f8435d11700 1 --2- 172.21.15.59:0/3835944011 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8430104930 0x7f8430197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:23.836 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:23.831+0000 7f8436d13700 1 --2- 172.21.15.59:0/3835944011 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8430107260 0x7f843019eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:23.836 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:23.831+0000 7f8436512700 1 -- 172.21.15.59:0/3835944011 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8430104930 msgr2=0x7f8430197ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:23.836 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:23.831+0000 7f8436512700 1 --2- 172.21.15.59:0/3835944011 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8430104930 0x7f8430197ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:23.836 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:23.831+0000 7f8436512700 1 -- 172.21.15.59:0/3835944011 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8430107260 msgr2=0x7f843019eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:23.836 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:23.831+0000 7f8436512700 1 --2- 172.21.15.59:0/3835944011 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8430107260 0x7f843019eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:23.836 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:23.831+0000 7f8436512700 1 -- 172.21.15.59:0/3835944011 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8428009d00 con 0x7f8430102000 2023-12-11T17:44:23.836 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:23.831+0000 7f8436d13700 1 --2- 172.21.15.59:0/3835944011 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8430107260 0x7f843019eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T17:44:23.836 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:23.831+0000 7f8436512700 1 --2- 172.21.15.59:0/3835944011 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8430102000 0x7f8430197590 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f842c00e9a0 tx=0x7f842c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:44:23.837 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:23.831+0000 7f84277fe700 1 -- 172.21.15.59:0/3835944011 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f842c009790 con 0x7f8430102000 2023-12-11T17:44:23.837 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:23.831+0000 7f84277fe700 1 -- 172.21.15.59:0/3835944011 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f842c00cbd0 con 0x7f8430102000 2023-12-11T17:44:23.837 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:23.831+0000 7f8438776700 1 -- 172.21.15.59:0/3835944011 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f843019f670 con 0x7f8430102000 2023-12-11T17:44:23.837 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:23.831+0000 7f84277fe700 1 -- 172.21.15.59:0/3835944011 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f842c018680 con 0x7f8430102000 2023-12-11T17:44:23.837 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:23.831+0000 7f8438776700 1 -- 172.21.15.59:0/3835944011 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f843019fbd0 con 0x7f8430102000 2023-12-11T17:44:23.838 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:23.832+0000 7f8438776700 1 -- 172.21.15.59:0/3835944011 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84300fd570 con 0x7f8430102000 2023-12-11T17:44:23.838 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:23.833+0000 7f84277fe700 1 -- 172.21.15.59:0/3835944011 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f842c00cd40 con 0x7f8430102000 2023-12-11T17:44:23.838 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:23.834+0000 7f84277fe700 1 --2- 172.21.15.59:0/3835944011 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f841c0a1500 0x7f841c0a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:23.838 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:23.834+0000 7f84277fe700 1 -- 172.21.15.59:0/3835944011 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8698+0+0 (secure 0 0 0) 0x7f842c014070 con 0x7f8430102000 2023-12-11T17:44:23.838 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:23.834+0000 7f8435d11700 1 --2- 172.21.15.59:0/3835944011 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f841c0a1500 0x7f841c0a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:23.838 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:23.834+0000 7f8435d11700 1 --2- 172.21.15.59:0/3835944011 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f841c0a1500 0x7f841c0a39c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8420009c60 tx=0x7f8420009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:44:23.840 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:23.838+0000 7f84277fe700 1 -- 172.21.15.59:0/3835944011 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f842c096890 con 0x7f8430102000 2023-12-11T17:44:23.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:23 smithi181 ceph-mon[103627]: pgmap v256: 97 pgs: 2 creating+activating, 19 creating+peering, 43 active+clean, 33 unknown; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 2 op/s 2023-12-11T17:44:23.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:23 smithi181 ceph-mon[103627]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2023-12-11T17:44:23.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:23 smithi181 ceph-mon[103627]: Cluster is now healthy 2023-12-11T17:44:23.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:23 smithi181 ceph-mon[103627]: osdmap e83: 12 total, 12 up, 12 in 2023-12-11T17:44:23.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:23 smithi059 ceph-mon[98847]: pgmap v256: 97 pgs: 2 creating+activating, 19 creating+peering, 43 active+clean, 33 unknown; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 2 op/s 2023-12-11T17:44:23.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:23 smithi059 ceph-mon[98847]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2023-12-11T17:44:23.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:23 smithi059 ceph-mon[98847]: Cluster is now healthy 2023-12-11T17:44:23.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:23 smithi059 ceph-mon[98847]: osdmap e83: 12 total, 12 up, 12 in 2023-12-11T17:44:24.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:23 smithi115 ceph-mon[102748]: pgmap v256: 97 pgs: 2 creating+activating, 19 creating+peering, 43 active+clean, 33 unknown; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 2 op/s 2023-12-11T17:44:24.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:23 smithi115 ceph-mon[102748]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2023-12-11T17:44:24.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:23 smithi115 ceph-mon[102748]: Cluster is now healthy 2023-12-11T17:44:24.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:23 smithi115 ceph-mon[102748]: osdmap e83: 12 total, 12 up, 12 in 2023-12-11T17:44:24.044 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:24.042+0000 7f8438776700 1 -- 172.21.15.59:0/3835944011 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7f843010d090 con 0x7f8430102000 2023-12-11T17:44:24.674 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:24.671+0000 7f84277fe700 1 -- 172.21.15.59:0/3835944011 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v85) v1 ==== 169+0+0 (secure 0 0 0) 0x7f842c0260c0 con 0x7f8430102000 2023-12-11T17:44:24.678 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:24.675+0000 7f8438776700 1 -- 172.21.15.59:0/3835944011 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f841c0a1500 msgr2=0x7f841c0a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:24.678 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:24.676+0000 7f8438776700 1 --2- 172.21.15.59:0/3835944011 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f841c0a1500 0x7f841c0a39c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8420009c60 tx=0x7f8420009260 comp rx=0 tx=0).stop 2023-12-11T17:44:24.678 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:24.676+0000 7f8438776700 1 -- 172.21.15.59:0/3835944011 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8430102000 msgr2=0x7f8430197590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:24.679 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:24.676+0000 7f8438776700 1 --2- 172.21.15.59:0/3835944011 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8430102000 0x7f8430197590 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f842c00e9a0 tx=0x7f842c00ecb0 comp rx=0 tx=0).stop 2023-12-11T17:44:24.679 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:24.676+0000 7f8438776700 1 -- 172.21.15.59:0/3835944011 shutdown_connections 2023-12-11T17:44:24.679 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:24.676+0000 7f8438776700 1 --2- 172.21.15.59:0/3835944011 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f841c0a1500 0x7f841c0a39c0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:24.679 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:24.676+0000 7f8438776700 1 --2- 172.21.15.59:0/3835944011 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8430107260 0x7f843019eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:24.679 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:24.676+0000 7f8438776700 1 --2- 172.21.15.59:0/3835944011 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8430102000 0x7f8430197590 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:24.679 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:24.676+0000 7f8438776700 1 --2- 172.21.15.59:0/3835944011 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8430104930 0x7f8430197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:24.679 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:24.676+0000 7f8438776700 1 -- 172.21.15.59:0/3835944011 >> 172.21.15.59:0/3835944011 conn(0x7f84300fb9f0 msgr2=0x7f84301029e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:44:24.679 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:24.676+0000 7f8438776700 1 -- 172.21.15.59:0/3835944011 shutdown_connections 2023-12-11T17:44:24.679 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:24.676+0000 7f8438776700 1 -- 172.21.15.59:0/3835944011 wait complete. 2023-12-11T17:44:24.692 INFO:teuthology.orchestra.run.smithi059.stderr:set pool 4 allow_ec_overwrites to true 2023-12-11T17:44:24.707 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2023-12-11T17:44:24.843 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:24.838+0000 7fb201d90700 1 -- 172.21.15.59:0/2138552321 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1fc102010 msgr2=0x7fb1fc1088c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:24.843 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:24.838+0000 7fb201d90700 1 --2- 172.21.15.59:0/2138552321 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1fc102010 0x7fb1fc1088c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fb1f400b0d0 tx=0x7fb1f400b3e0 comp rx=0 tx=0).stop 2023-12-11T17:44:24.843 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:24.838+0000 7fb201d90700 1 -- 172.21.15.59:0/2138552321 shutdown_connections 2023-12-11T17:44:24.843 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:24.838+0000 7fb201d90700 1 --2- 172.21.15.59:0/2138552321 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1fc102010 0x7fb1fc1088c0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:24.843 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:24.838+0000 7fb201d90700 1 --2- 172.21.15.59:0/2138552321 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb1fc100480 0x7fb1fc1008a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:24.843 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:24.838+0000 7fb201d90700 1 --2- 172.21.15.59:0/2138552321 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb1fc101670 0x7fb1fc101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:24.843 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:24.838+0000 7fb201d90700 1 -- 172.21.15.59:0/2138552321 >> 172.21.15.59:0/2138552321 conn(0x7fb1fc0fb9f0 msgr2=0x7fb1fc0fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:44:24.843 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:24.838+0000 7fb201d90700 1 -- 172.21.15.59:0/2138552321 shutdown_connections 2023-12-11T17:44:24.843 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:24.838+0000 7fb201d90700 1 -- 172.21.15.59:0/2138552321 wait complete. 2023-12-11T17:44:24.843 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:24.839+0000 7fb201d90700 1 Processor -- start 2023-12-11T17:44:24.843 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:24.839+0000 7fb201d90700 1 -- start start 2023-12-11T17:44:24.843 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:24.839+0000 7fb201d90700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb1fc100480 0x7fb1fc1975f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:24.844 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:24.839+0000 7fb201d90700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1fc101670 0x7fb1fc197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:24.844 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:24.839+0000 7fb201d90700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb1fc102010 0x7fb1fc19eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:24.844 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:24.839+0000 7fb201d90700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1fc19f120 con 0x7fb1fc101670 2023-12-11T17:44:24.844 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:24.839+0000 7fb201d90700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1fc19f260 con 0x7fb1fc100480 2023-12-11T17:44:24.844 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:24.839+0000 7fb201d90700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1fc19f3f0 con 0x7fb1fc102010 2023-12-11T17:44:24.844 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:24.839+0000 7fb1faffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1fc101670 0x7fb1fc197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:24.844 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:24.839+0000 7fb1faffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1fc101670 0x7fb1fc197b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:32922/0 (socket says 172.21.15.59:32922) 2023-12-11T17:44:24.844 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:24.839+0000 7fb1faffd700 1 -- 172.21.15.59:0/3269008731 learned_addr learned my addr 172.21.15.59:0/3269008731 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:44:24.844 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:24.839+0000 7fb1fb7fe700 1 --2- 172.21.15.59:0/3269008731 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb1fc100480 0x7fb1fc1975f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:24.844 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:24.839+0000 7fb1fbfff700 1 --2- 172.21.15.59:0/3269008731 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb1fc102010 0x7fb1fc19eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:24.844 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:24.839+0000 7fb1faffd700 1 -- 172.21.15.59:0/3269008731 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb1fc102010 msgr2=0x7fb1fc19eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:24.844 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:24.839+0000 7fb1faffd700 1 --2- 172.21.15.59:0/3269008731 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb1fc102010 0x7fb1fc19eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:24.844 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:24.839+0000 7fb1faffd700 1 -- 172.21.15.59:0/3269008731 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb1fc100480 msgr2=0x7fb1fc1975f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:24.844 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:24.839+0000 7fb1faffd700 1 --2- 172.21.15.59:0/3269008731 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb1fc100480 0x7fb1fc1975f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:24.845 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:24.839+0000 7fb1faffd700 1 -- 172.21.15.59:0/3269008731 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1f4009d00 con 0x7fb1fc101670 2023-12-11T17:44:24.845 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:24.839+0000 7fb1faffd700 1 --2- 172.21.15.59:0/3269008731 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1fc101670 0x7fb1fc197b30 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb1ec00e9a0 tx=0x7fb1ec00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:44:24.845 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:24.840+0000 7fb1f8ff9700 1 -- 172.21.15.59:0/3269008731 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1ec009790 con 0x7fb1fc101670 2023-12-11T17:44:24.845 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:24.840+0000 7fb1f8ff9700 1 -- 172.21.15.59:0/3269008731 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb1ec010430 con 0x7fb1fc101670 2023-12-11T17:44:24.845 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:24.840+0000 7fb201d90700 1 -- 172.21.15.59:0/3269008731 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1fc19f6d0 con 0x7fb1fc101670 2023-12-11T17:44:24.845 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:24.840+0000 7fb1f8ff9700 1 -- 172.21.15.59:0/3269008731 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1ec018680 con 0x7fb1fc101670 2023-12-11T17:44:24.845 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:24.840+0000 7fb201d90700 1 -- 172.21.15.59:0/3269008731 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1fc19fbe0 con 0x7fb1fc101670 2023-12-11T17:44:24.846 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:24.841+0000 7fb201d90700 1 -- 172.21.15.59:0/3269008731 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1fc0682f0 con 0x7fb1fc101670 2023-12-11T17:44:24.846 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:24.842+0000 7fb1f8ff9700 1 -- 172.21.15.59:0/3269008731 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7fb1ec0105a0 con 0x7fb1fc101670 2023-12-11T17:44:24.846 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:24.842+0000 7fb1f8ff9700 1 --2- 172.21.15.59:0/3269008731 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fb1e40a1480 0x7fb1e40a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:24.846 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:24.842+0000 7fb1fb7fe700 1 --2- 172.21.15.59:0/3269008731 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fb1e40a1480 0x7fb1e40a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:24.846 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:24.842+0000 7fb1f8ff9700 1 -- 172.21.15.59:0/3269008731 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8698+0+0 (secure 0 0 0) 0x7fb1ec014070 con 0x7fb1fc101670 2023-12-11T17:44:24.846 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:24.843+0000 7fb1fb7fe700 1 --2- 172.21.15.59:0/3269008731 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fb1e40a1480 0x7fb1e40a3940 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fb1fc1014d0 tx=0x7fb1f0007400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:44:24.849 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:24.847+0000 7fb1f8ff9700 1 -- 172.21.15.59:0/3269008731 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fb1ec0522f0 con 0x7fb1fc101670 2023-12-11T17:44:25.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:24 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/2538219663' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-12-11T17:44:25.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:24 smithi115 ceph-mon[102748]: osdmap e84: 12 total, 12 up, 12 in 2023-12-11T17:44:25.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:24 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/3835944011' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2023-12-11T17:44:25.097 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:25.094+0000 7fb201d90700 1 -- 172.21.15.59:0/3269008731 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7fb1fc10bca0 con 0x7fb1fc101670 2023-12-11T17:44:25.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:24 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/2538219663' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-12-11T17:44:25.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:24 smithi181 ceph-mon[103627]: osdmap e84: 12 total, 12 up, 12 in 2023-12-11T17:44:25.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:24 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/3835944011' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2023-12-11T17:44:25.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:24 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/2538219663' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-12-11T17:44:25.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:24 smithi059 ceph-mon[98847]: osdmap e84: 12 total, 12 up, 12 in 2023-12-11T17:44:25.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:24 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/3835944011' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2023-12-11T17:44:25.675 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:25.673+0000 7fb1f8ff9700 1 -- 172.21.15.59:0/3269008731 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) v1 ==== 137+0+0 (secure 0 0 0) 0x7fb1ec099f40 con 0x7fb1fc101670 2023-12-11T17:44:25.679 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:25.677+0000 7fb201d90700 1 -- 172.21.15.59:0/3269008731 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fb1e40a1480 msgr2=0x7fb1e40a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:25.679 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:25.677+0000 7fb201d90700 1 --2- 172.21.15.59:0/3269008731 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fb1e40a1480 0x7fb1e40a3940 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fb1fc1014d0 tx=0x7fb1f0007400 comp rx=0 tx=0).stop 2023-12-11T17:44:25.680 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:25.677+0000 7fb201d90700 1 -- 172.21.15.59:0/3269008731 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1fc101670 msgr2=0x7fb1fc197b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:25.680 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:25.677+0000 7fb201d90700 1 --2- 172.21.15.59:0/3269008731 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1fc101670 0x7fb1fc197b30 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb1ec00e9a0 tx=0x7fb1ec00ecb0 comp rx=0 tx=0).stop 2023-12-11T17:44:25.680 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:25.677+0000 7fb201d90700 1 -- 172.21.15.59:0/3269008731 shutdown_connections 2023-12-11T17:44:25.680 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:25.677+0000 7fb201d90700 1 --2- 172.21.15.59:0/3269008731 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fb1e40a1480 0x7fb1e40a3940 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:25.680 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:25.677+0000 7fb201d90700 1 --2- 172.21.15.59:0/3269008731 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb1fc102010 0x7fb1fc19eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:25.680 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:25.677+0000 7fb201d90700 1 --2- 172.21.15.59:0/3269008731 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb1fc100480 0x7fb1fc1975f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:25.680 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:25.677+0000 7fb201d90700 1 --2- 172.21.15.59:0/3269008731 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1fc101670 0x7fb1fc197b30 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:25.680 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:25.677+0000 7fb201d90700 1 -- 172.21.15.59:0/3269008731 >> 172.21.15.59:0/3269008731 conn(0x7fb1fc0fb9f0 msgr2=0x7fb1fc0fd4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:44:25.680 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:25.677+0000 7fb201d90700 1 -- 172.21.15.59:0/3269008731 shutdown_connections 2023-12-11T17:44:25.680 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:25.677+0000 7fb201d90700 1 -- 172.21.15.59:0/3269008731 wait complete. 2023-12-11T17:44:25.680 INFO:teuthology.orchestra.run.smithi059.stderr:added data pool 4 to fsmap 2023-12-11T17:44:25.695 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-11T17:44:25.829 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:25.826+0000 7f29055d3700 1 -- 172.21.15.59:0/1381334466 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2900101570 msgr2=0x7f29001019d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:25.829 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:25.826+0000 7f29055d3700 1 --2- 172.21.15.59:0/1381334466 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2900101570 0x7f29001019d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f28f40099c0 tx=0x7f28f4009cd0 comp rx=0 tx=0).stop 2023-12-11T17:44:25.829 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:25.826+0000 7f29055d3700 1 -- 172.21.15.59:0/1381334466 shutdown_connections 2023-12-11T17:44:25.829 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:25.826+0000 7f29055d3700 1 --2- 172.21.15.59:0/1381334466 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2900101f10 0x7f29001087c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:25.829 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:25.826+0000 7f29055d3700 1 --2- 172.21.15.59:0/1381334466 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2900100380 0x7f29001007a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:25.829 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:25.826+0000 7f29055d3700 1 --2- 172.21.15.59:0/1381334466 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2900101570 0x7f29001019d0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:25.829 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:25.826+0000 7f29055d3700 1 -- 172.21.15.59:0/1381334466 >> 172.21.15.59:0/1381334466 conn(0x7f29000fb8f0 msgr2=0x7f29000fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:44:25.829 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:25.826+0000 7f29055d3700 1 -- 172.21.15.59:0/1381334466 shutdown_connections 2023-12-11T17:44:25.829 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:25.826+0000 7f29055d3700 1 -- 172.21.15.59:0/1381334466 wait complete. 2023-12-11T17:44:25.830 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:25.827+0000 7f29055d3700 1 Processor -- start 2023-12-11T17:44:25.831 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:25.827+0000 7f29055d3700 1 -- start start 2023-12-11T17:44:25.832 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:25.827+0000 7f29055d3700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2900100380 0x7f29001973f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:25.832 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:25.827+0000 7f29055d3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2900101570 0x7f2900197930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:25.832 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:25.827+0000 7f29055d3700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2900101f10 0x7f290019e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:25.832 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:25.827+0000 7f29055d3700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f290019ef20 con 0x7f2900100380 2023-12-11T17:44:25.832 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:25.827+0000 7f29055d3700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f290019f060 con 0x7f2900101f10 2023-12-11T17:44:25.832 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:25.827+0000 7f29055d3700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f290019f1f0 con 0x7f2900101570 2023-12-11T17:44:25.832 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:25.827+0000 7f28feffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2900100380 0x7f29001973f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:25.832 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:25.827+0000 7f28feffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2900100380 0x7f29001973f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:32934/0 (socket says 172.21.15.59:32934) 2023-12-11T17:44:25.832 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:25.827+0000 7f28feffd700 1 -- 172.21.15.59:0/3714767746 learned_addr learned my addr 172.21.15.59:0/3714767746 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:44:25.832 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:25.827+0000 7f28fe7fc700 1 --2- 172.21.15.59:0/3714767746 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2900101570 0x7f2900197930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:25.832 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:25.827+0000 7f28ff7fe700 1 --2- 172.21.15.59:0/3714767746 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2900101f10 0x7f290019e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:25.832 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:25.828+0000 7f28feffd700 1 -- 172.21.15.59:0/3714767746 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2900101570 msgr2=0x7f2900197930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:25.832 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:25.828+0000 7f28feffd700 1 --2- 172.21.15.59:0/3714767746 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2900101570 0x7f2900197930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:25.832 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:25.828+0000 7f28feffd700 1 -- 172.21.15.59:0/3714767746 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2900101f10 msgr2=0x7f290019e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:25.832 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:25.828+0000 7f28feffd700 1 --2- 172.21.15.59:0/3714767746 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2900101f10 0x7f290019e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:25.833 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:25.828+0000 7f28feffd700 1 -- 172.21.15.59:0/3714767746 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28f40095f0 con 0x7f2900100380 2023-12-11T17:44:25.833 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:25.828+0000 7f28feffd700 1 --2- 172.21.15.59:0/3714767746 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2900100380 0x7f29001973f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f28f000d980 tx=0x7f28f000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:44:25.833 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:25.828+0000 7f28e7fff700 1 -- 172.21.15.59:0/3714767746 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f0004020 con 0x7f2900100380 2023-12-11T17:44:25.833 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:25.828+0000 7f29055d3700 1 -- 172.21.15.59:0/3714767746 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f290019f4d0 con 0x7f2900100380 2023-12-11T17:44:25.833 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:25.828+0000 7f28e7fff700 1 -- 172.21.15.59:0/3714767746 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f28f000f690 con 0x7f2900100380 2023-12-11T17:44:25.833 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:25.828+0000 7f28e7fff700 1 -- 172.21.15.59:0/3714767746 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f0003a60 con 0x7f2900100380 2023-12-11T17:44:25.833 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:25.828+0000 7f29055d3700 1 -- 172.21.15.59:0/3714767746 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f290019fa30 con 0x7f2900100380 2023-12-11T17:44:25.833 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:25.829+0000 7f29055d3700 1 -- 172.21.15.59:0/3714767746 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2900068230 con 0x7f2900100380 2023-12-11T17:44:25.834 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:25.830+0000 7f28e7fff700 1 -- 172.21.15.59:0/3714767746 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f28f000fb00 con 0x7f2900100380 2023-12-11T17:44:25.834 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:25.831+0000 7f28e7fff700 1 --2- 172.21.15.59:0/3714767746 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f28e80a1500 0x7f28e80a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:25.834 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:25.831+0000 7f28fe7fc700 1 --2- 172.21.15.59:0/3714767746 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f28e80a1500 0x7f28e80a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:25.834 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:25.831+0000 7f28e7fff700 1 -- 172.21.15.59:0/3714767746 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f28f00ccda0 con 0x7f2900100380 2023-12-11T17:44:25.834 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:25.831+0000 7f28fe7fc700 1 --2- 172.21.15.59:0/3714767746 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f28e80a1500 0x7f28e80a39c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f28f4009990 tx=0x7f28f4005ad0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:44:25.837 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:25.834+0000 7f28e7fff700 1 -- 172.21.15.59:0/3714767746 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f28f0096290 con 0x7f2900100380 2023-12-11T17:44:26.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:25 smithi115 ceph-mon[102748]: pgmap v259: 129 pgs: 2 creating+activating, 19 creating+peering, 71 active+clean, 37 unknown; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s wr, 3 op/s 2023-12-11T17:44:26.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:25 smithi115 ceph-mon[102748]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-12-11T17:44:26.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:25 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/3835944011' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2023-12-11T17:44:26.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:25 smithi115 ceph-mon[102748]: osdmap e85: 12 total, 12 up, 12 in 2023-12-11T17:44:26.034 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:25 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/3269008731' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2023-12-11T17:44:26.087 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.085+0000 7f29055d3700 1 -- 172.21.15.59:0/3714767746 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f290010be50 con 0x7f2900100380 2023-12-11T17:44:26.088 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.086+0000 7f28e7fff700 1 -- 172.21.15.59:0/3714767746 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+11264 (secure 0 0 0) 0x7f28f00998b0 con 0x7f2900100380 2023-12-11T17:44:26.088 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-12-11T17:44:26.088 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/2184151593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":2184151593},{"type":"v1","addr":"172.21.15.59:6835","nonce":2184151593}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/2817614886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2817614886},{"type":"v1","addr":"172.21.15.59:6837","nonce":2817614886}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6839/229873490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":229873490},{"type":"v1","addr":"172.21.15.59:6839","nonce":229873490}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6841/967998755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":967998755},{"type":"v1","addr":"172.21.15.59:6841","nonce":967998755}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/395684042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":395684042},{"type":"v1","addr":"172.21.15.181:6833","nonce":395684042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6833/3214606191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3214606191},{"type":"v1","addr":"172.21.15.115:6833","nonce":3214606191}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24449,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/1874159657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1874159657},{"type":"v1","addr":"172.21.15.115:6835","nonce":1874159657}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/4221183620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":4221183620},{"type":"v1","addr":"172.21.15.181:6835","nonce":4221183620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24466,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/3580309407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":3580309407},{"type":"v1","addr":"172.21.15.181:6837","nonce":3580309407}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24482,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/1209365460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":1209365460},{"type":"v1","addr":"172.21.15.115:6837","nonce":1209365460}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24484,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6839/2214384806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":2214384806},{"type":"v1","addr":"172.21.15.181:6839","nonce":2214384806}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-11T17:44:20.076660+0000","modified":"2023-12-11T17:44:25.096263+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24521},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.115:6839/4122922866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":4122922866},{"type":"v1","addr":"172.21.15.115:6839","nonce":4122922866}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2023-12-11T17:44:26.090 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.088+0000 7f29055d3700 1 -- 172.21.15.59:0/3714767746 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f28e80a1500 msgr2=0x7f28e80a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:26.090 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.088+0000 7f29055d3700 1 --2- 172.21.15.59:0/3714767746 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f28e80a1500 0x7f28e80a39c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f28f4009990 tx=0x7f28f4005ad0 comp rx=0 tx=0).stop 2023-12-11T17:44:26.090 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.088+0000 7f29055d3700 1 -- 172.21.15.59:0/3714767746 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2900100380 msgr2=0x7f29001973f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:26.090 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.088+0000 7f29055d3700 1 --2- 172.21.15.59:0/3714767746 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2900100380 0x7f29001973f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f28f000d980 tx=0x7f28f000dc90 comp rx=0 tx=0).stop 2023-12-11T17:44:26.091 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.088+0000 7f29055d3700 1 -- 172.21.15.59:0/3714767746 shutdown_connections 2023-12-11T17:44:26.091 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.088+0000 7f29055d3700 1 --2- 172.21.15.59:0/3714767746 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f28e80a1500 0x7f28e80a39c0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:26.091 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.088+0000 7f29055d3700 1 --2- 172.21.15.59:0/3714767746 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2900101f10 0x7f290019e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:26.091 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.088+0000 7f29055d3700 1 --2- 172.21.15.59:0/3714767746 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2900100380 0x7f29001973f0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:26.091 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.088+0000 7f29055d3700 1 --2- 172.21.15.59:0/3714767746 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2900101570 0x7f2900197930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:26.091 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.088+0000 7f29055d3700 1 -- 172.21.15.59:0/3714767746 >> 172.21.15.59:0/3714767746 conn(0x7f29000fb8f0 msgr2=0x7f29000fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:44:26.091 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.088+0000 7f29055d3700 1 -- 172.21.15.59:0/3714767746 shutdown_connections 2023-12-11T17:44:26.091 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.088+0000 7f29055d3700 1 -- 172.21.15.59:0/3714767746 wait complete. 2023-12-11T17:44:26.091 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 17 2023-12-11T17:44:26.101 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:25 smithi059 ceph-mon[98847]: pgmap v259: 129 pgs: 2 creating+activating, 19 creating+peering, 71 active+clean, 37 unknown; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s wr, 3 op/s 2023-12-11T17:44:26.102 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:25 smithi059 ceph-mon[98847]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-12-11T17:44:26.102 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:25 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/3835944011' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2023-12-11T17:44:26.102 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:25 smithi059 ceph-mon[98847]: osdmap e85: 12 total, 12 up, 12 in 2023-12-11T17:44:26.102 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:25 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/3269008731' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2023-12-11T17:44:26.103 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-11T17:44:26.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:25 smithi181 ceph-mon[103627]: pgmap v259: 129 pgs: 2 creating+activating, 19 creating+peering, 71 active+clean, 37 unknown; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s wr, 3 op/s 2023-12-11T17:44:26.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:25 smithi181 ceph-mon[103627]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-12-11T17:44:26.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:25 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/3835944011' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2023-12-11T17:44:26.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:25 smithi181 ceph-mon[103627]: osdmap e85: 12 total, 12 up, 12 in 2023-12-11T17:44:26.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:25 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/3269008731' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2023-12-11T17:44:26.229 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.226+0000 7ff389e9c700 1 -- 172.21.15.59:0/3455357448 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff384101f40 msgr2=0x7ff3841087f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:26.229 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.226+0000 7ff389e9c700 1 --2- 172.21.15.59:0/3455357448 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff384101f40 0x7ff3841087f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff37c00b0d0 tx=0x7ff37c00b3e0 comp rx=0 tx=0).stop 2023-12-11T17:44:26.229 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.226+0000 7ff389e9c700 1 -- 172.21.15.59:0/3455357448 shutdown_connections 2023-12-11T17:44:26.229 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.226+0000 7ff389e9c700 1 --2- 172.21.15.59:0/3455357448 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff384101f40 0x7ff3841087f0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:26.229 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.226+0000 7ff389e9c700 1 --2- 172.21.15.59:0/3455357448 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3841003b0 0x7ff3841007d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:26.229 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.226+0000 7ff389e9c700 1 --2- 172.21.15.59:0/3455357448 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff3841015a0 0x7ff384101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:26.229 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.226+0000 7ff389e9c700 1 -- 172.21.15.59:0/3455357448 >> 172.21.15.59:0/3455357448 conn(0x7ff3840fb920 msgr2=0x7ff3840fdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:44:26.229 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.226+0000 7ff389e9c700 1 -- 172.21.15.59:0/3455357448 shutdown_connections 2023-12-11T17:44:26.229 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.226+0000 7ff389e9c700 1 -- 172.21.15.59:0/3455357448 wait complete. 2023-12-11T17:44:26.229 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.227+0000 7ff389e9c700 1 Processor -- start 2023-12-11T17:44:26.232 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.227+0000 7ff389e9c700 1 -- start start 2023-12-11T17:44:26.232 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.227+0000 7ff389e9c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3841003b0 0x7ff384196380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:26.232 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.227+0000 7ff389e9c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff3841015a0 0x7ff3841968c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:26.232 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.227+0000 7ff389e9c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff384101f40 0x7ff38419d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:26.232 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.227+0000 7ff389e9c700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff38419deb0 con 0x7ff384101f40 2023-12-11T17:44:26.232 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.227+0000 7ff389e9c700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff38419dff0 con 0x7ff3841015a0 2023-12-11T17:44:26.232 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.227+0000 7ff389e9c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff38419e180 con 0x7ff3841003b0 2023-12-11T17:44:26.232 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.227+0000 7ff383fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff384101f40 0x7ff38419d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:26.232 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.227+0000 7ff383fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff384101f40 0x7ff38419d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:32952/0 (socket says 172.21.15.59:32952) 2023-12-11T17:44:26.232 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.227+0000 7ff383fff700 1 -- 172.21.15.59:0/3758537040 learned_addr learned my addr 172.21.15.59:0/3758537040 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:44:26.232 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.227+0000 7ff3837fe700 1 --2- 172.21.15.59:0/3758537040 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3841003b0 0x7ff384196380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:26.232 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.227+0000 7ff382ffd700 1 --2- 172.21.15.59:0/3758537040 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff3841015a0 0x7ff3841968c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:26.232 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.227+0000 7ff383fff700 1 -- 172.21.15.59:0/3758537040 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3841003b0 msgr2=0x7ff384196380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:26.232 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.227+0000 7ff383fff700 1 --2- 172.21.15.59:0/3758537040 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3841003b0 0x7ff384196380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:26.233 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.227+0000 7ff383fff700 1 -- 172.21.15.59:0/3758537040 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff3841015a0 msgr2=0x7ff3841968c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:26.233 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.227+0000 7ff383fff700 1 --2- 172.21.15.59:0/3758537040 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff3841015a0 0x7ff3841968c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:26.233 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.227+0000 7ff383fff700 1 -- 172.21.15.59:0/3758537040 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff37c009d00 con 0x7ff384101f40 2023-12-11T17:44:26.233 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.227+0000 7ff382ffd700 1 --2- 172.21.15.59:0/3758537040 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff3841015a0 0x7ff3841968c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-11T17:44:26.233 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.227+0000 7ff3837fe700 1 --2- 172.21.15.59:0/3758537040 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3841003b0 0x7ff384196380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T17:44:26.233 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.228+0000 7ff383fff700 1 --2- 172.21.15.59:0/3758537040 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff384101f40 0x7ff38419d8e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff37c007bd0 tx=0x7ff37c0094f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:44:26.233 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.228+0000 7ff380ff9700 1 -- 172.21.15.59:0/3758537040 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff37c00e040 con 0x7ff384101f40 2023-12-11T17:44:26.233 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.228+0000 7ff380ff9700 1 -- 172.21.15.59:0/3758537040 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff37c012dc0 con 0x7ff384101f40 2023-12-11T17:44:26.233 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.228+0000 7ff389e9c700 1 -- 172.21.15.59:0/3758537040 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3840699f0 con 0x7ff384101f40 2023-12-11T17:44:26.233 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.228+0000 7ff380ff9700 1 -- 172.21.15.59:0/3758537040 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff37c01b450 con 0x7ff384101f40 2023-12-11T17:44:26.233 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.228+0000 7ff389e9c700 1 -- 172.21.15.59:0/3758537040 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff384069f00 con 0x7ff384101f40 2023-12-11T17:44:26.233 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.229+0000 7ff389e9c700 1 -- 172.21.15.59:0/3758537040 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff384068680 con 0x7ff384101f40 2023-12-11T17:44:26.234 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.230+0000 7ff380ff9700 1 -- 172.21.15.59:0/3758537040 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7ff37c01d610 con 0x7ff384101f40 2023-12-11T17:44:26.234 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.231+0000 7ff380ff9700 1 --2- 172.21.15.59:0/3758537040 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7ff3700a3690 0x7ff3700a5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:26.234 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.231+0000 7ff3837fe700 1 --2- 172.21.15.59:0/3758537040 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7ff3700a3690 0x7ff3700a5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:26.234 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.231+0000 7ff380ff9700 1 -- 172.21.15.59:0/3758537040 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7ff37c0cede0 con 0x7ff384101f40 2023-12-11T17:44:26.234 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.231+0000 7ff3837fe700 1 --2- 172.21.15.59:0/3758537040 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7ff3700a3690 0x7ff3700a5b50 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff384101400 tx=0x7ff36c00a680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:44:26.237 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.235+0000 7ff380ff9700 1 -- 172.21.15.59:0/3758537040 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7ff37c098280 con 0x7ff384101f40 2023-12-11T17:44:26.441 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.439+0000 7ff389e9c700 1 -- 172.21.15.59:0/3758537040 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff38406a8f0 con 0x7ff384101f40 2023-12-11T17:44:26.443 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.441+0000 7ff380ff9700 1 -- 172.21.15.59:0/3758537040 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+23960 (secure 0 0 0) 0x7ff37c09b8a0 con 0x7ff384101f40 2023-12-11T17:44:26.444 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-12-11T17:44:26.444 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":86,"fsid":"d0a07eaa-984b-11ee-95a3-87774f69a715","created":"2023-12-11T17:37:14.526539+0000","modified":"2023-12-11T17:44:25.096094+0000","last_up_change":"2023-12-11T17:43:38.535503+0000","last_in_change":"2023-12-11T17:43:21.014888+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-11T17:40:26.596029+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-11T17:44:18.586290+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-11T17:44:19.432952+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-11T17:44:23.097308+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3c0b3218-4b2b-40f8-8bef-ed8cd5633607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":1040759398},{"type":"v1","addr":"172.21.15.59:6803","nonce":1040759398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":1040759398},{"type":"v1","addr":"172.21.15.59:6805","nonce":1040759398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":1040759398},{"type":"v1","addr":"172.21.15.59:6809","nonce":1040759398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":1040759398},{"type":"v1","addr":"172.21.15.59:6807","nonce":1040759398}]},"public_addr":"172.21.15.59:6803/1040759398","cluster_addr":"172.21.15.59:6805/1040759398","heartbeat_back_addr":"172.21.15.59:6809/1040759398","heartbeat_front_addr":"172.21.15.59:6807/1040759398","state":["exists","up"]},{"osd":1,"uuid":"2ead7578-765b-4f5a-a72c-863731ed69e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":2776354245},{"type":"v1","addr":"172.21.15.115:6801","nonce":2776354245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":2776354245},{"type":"v1","addr":"172.21.15.115:6803","nonce":2776354245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":2776354245},{"type":"v1","addr":"172.21.15.115:6807","nonce":2776354245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":2776354245},{"type":"v1","addr":"172.21.15.115:6805","nonce":2776354245}]},"public_addr":"172.21.15.115:6801/2776354245","cluster_addr":"172.21.15.115:6803/2776354245","heartbeat_back_addr":"172.21.15.115:6807/2776354245","heartbeat_front_addr":"172.21.15.115:6805/2776354245","state":["exists","up"]},{"osd":2,"uuid":"8cb1d7ec-53cc-4672-b0b6-70d73797c44f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4287675481},{"type":"v1","addr":"172.21.15.181:6801","nonce":4287675481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4287675481},{"type":"v1","addr":"172.21.15.181:6803","nonce":4287675481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4287675481},{"type":"v1","addr":"172.21.15.181:6807","nonce":4287675481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4287675481},{"type":"v1","addr":"172.21.15.181:6805","nonce":4287675481}]},"public_addr":"172.21.15.181:6801/4287675481","cluster_addr":"172.21.15.181:6803/4287675481","heartbeat_back_addr":"172.21.15.181:6807/4287675481","heartbeat_front_addr":"172.21.15.181:6805/4287675481","state":["exists","up"]},{"osd":3,"uuid":"5d64d9af-831b-4d0a-8ace-a452169351d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":219909995},{"type":"v1","addr":"172.21.15.59:6811","nonce":219909995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":219909995},{"type":"v1","addr":"172.21.15.59:6813","nonce":219909995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":219909995},{"type":"v1","addr":"172.21.15.59:6817","nonce":219909995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":219909995},{"type":"v1","addr":"172.21.15.59:6815","nonce":219909995}]},"public_addr":"172.21.15.59:6811/219909995","cluster_addr":"172.21.15.59:6813/219909995","heartbeat_back_addr":"172.21.15.59:6817/219909995","heartbeat_front_addr":"172.21.15.59:6815/219909995","state":["exists","up"]},{"osd":4,"uuid":"ef8fad28-1db5-4cba-b5f2-10e54e8e1549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":1021473826},{"type":"v1","addr":"172.21.15.115:6809","nonce":1021473826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":1021473826},{"type":"v1","addr":"172.21.15.115:6811","nonce":1021473826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":1021473826},{"type":"v1","addr":"172.21.15.115:6815","nonce":1021473826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":1021473826},{"type":"v1","addr":"172.21.15.115:6813","nonce":1021473826}]},"public_addr":"172.21.15.115:6809/1021473826","cluster_addr":"172.21.15.115:6811/1021473826","heartbeat_back_addr":"172.21.15.115:6815/1021473826","heartbeat_front_addr":"172.21.15.115:6813/1021473826","state":["exists","up"]},{"osd":5,"uuid":"4233a512-9e8f-4db8-aed2-6c3af706e589","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":2405172521},{"type":"v1","addr":"172.21.15.181:6809","nonce":2405172521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":2405172521},{"type":"v1","addr":"172.21.15.181:6811","nonce":2405172521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2405172521},{"type":"v1","addr":"172.21.15.181:6815","nonce":2405172521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":2405172521},{"type":"v1","addr":"172.21.15.181:6813","nonce":2405172521}]},"public_addr":"172.21.15.181:6809/2405172521","cluster_addr":"172.21.15.181:6811/2405172521","heartbeat_back_addr":"172.21.15.181:6815/2405172521","heartbeat_front_addr":"172.21.15.181:6813/2405172521","state":["exists","up"]},{"osd":6,"uuid":"8faab332-016e-4c77-8370-8bef95fc6f34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":815374853},{"type":"v1","addr":"172.21.15.59:6819","nonce":815374853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":815374853},{"type":"v1","addr":"172.21.15.59:6821","nonce":815374853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":815374853},{"type":"v1","addr":"172.21.15.59:6825","nonce":815374853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":815374853},{"type":"v1","addr":"172.21.15.59:6823","nonce":815374853}]},"public_addr":"172.21.15.59:6819/815374853","cluster_addr":"172.21.15.59:6821/815374853","heartbeat_back_addr":"172.21.15.59:6825/815374853","heartbeat_front_addr":"172.21.15.59:6823/815374853","state":["exists","up"]},{"osd":7,"uuid":"f20f470a-0d40-4ed3-8e0e-fd58675cf143","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":1314971870},{"type":"v1","addr":"172.21.15.115:6817","nonce":1314971870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":1314971870},{"type":"v1","addr":"172.21.15.115:6819","nonce":1314971870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":1314971870},{"type":"v1","addr":"172.21.15.115:6823","nonce":1314971870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":1314971870},{"type":"v1","addr":"172.21.15.115:6821","nonce":1314971870}]},"public_addr":"172.21.15.115:6817/1314971870","cluster_addr":"172.21.15.115:6819/1314971870","heartbeat_back_addr":"172.21.15.115:6823/1314971870","heartbeat_front_addr":"172.21.15.115:6821/1314971870","state":["exists","up"]},{"osd":8,"uuid":"eb657d32-1a95-4fd7-a293-d0a970837fc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":3954509873},{"type":"v1","addr":"172.21.15.181:6817","nonce":3954509873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":3954509873},{"type":"v1","addr":"172.21.15.181:6819","nonce":3954509873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":3954509873},{"type":"v1","addr":"172.21.15.181:6823","nonce":3954509873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":3954509873},{"type":"v1","addr":"172.21.15.181:6821","nonce":3954509873}]},"public_addr":"172.21.15.181:6817/3954509873","cluster_addr":"172.21.15.181:6819/3954509873","heartbeat_back_addr":"172.21.15.181:6823/3954509873","heartbeat_front_addr":"172.21.15.181:6821/3954509873","state":["exists","up"]},{"osd":9,"uuid":"8f7717d4-b6f2-4ec1-9b58-62feb0c9a146","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":1163095677},{"type":"v1","addr":"172.21.15.59:6827","nonce":1163095677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":1163095677},{"type":"v1","addr":"172.21.15.59:6829","nonce":1163095677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":1163095677},{"type":"v1","addr":"172.21.15.59:6833","nonce":1163095677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":1163095677},{"type":"v1","addr":"172.21.15.59:6831","nonce":1163095677}]},"public_addr":"172.21.15.59:6827/1163095677","cluster_addr":"172.21.15.59:6829/1163095677","heartbeat_back_addr":"172.21.15.59:6833/1163095677","heartbeat_front_addr":"172.21.15.59:6831/1163095677","state":["exists","up"]},{"osd":10,"uuid":"b75c179a-875d-43bc-b4c1-e8eb80fb64fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":2199610269},{"type":"v1","addr":"172.21.15.115:6825","nonce":2199610269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":2199610269},{"type":"v1","addr":"172.21.15.115:6827","nonce":2199610269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":2199610269},{"type":"v1","addr":"172.21.15.115:6831","nonce":2199610269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":2199610269},{"type":"v1","addr":"172.21.15.115:6829","nonce":2199610269}]},"public_addr":"172.21.15.115:6825/2199610269","cluster_addr":"172.21.15.115:6827/2199610269","heartbeat_back_addr":"172.21.15.115:6831/2199610269","heartbeat_front_addr":"172.21.15.115:6829/2199610269","state":["exists","up"]},{"osd":11,"uuid":"5e141dfe-8a00-4bbc-8204-2dc9be24e18d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":226506989},{"type":"v1","addr":"172.21.15.181:6825","nonce":226506989}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":226506989},{"type":"v1","addr":"172.21.15.181:6827","nonce":226506989}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":226506989},{"type":"v1","addr":"172.21.15.181:6831","nonce":226506989}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":226506989},{"type":"v1","addr":"172.21.15.181:6829","nonce":226506989}]},"public_addr":"172.21.15.181:6825/226506989","cluster_addr":"172.21.15.181:6827/226506989","heartbeat_back_addr":"172.21.15.181:6831/226506989","heartbeat_front_addr":"172.21.15.181:6829/226506989","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:39:42.687084+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:40:03.903448+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:40:24.899908+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:40:46.600251+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:41:07.690037+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:41:28.488855+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:41:49.459274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:42:10.863881+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:42:32.248391+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:42:54.112265+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:43:15.634872+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:43:36.922534+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.59:6800/1259916482":"2023-12-12T17:38:14.037922+0000","172.21.15.59:0/17457774":"2023-12-12T17:38:14.037922+0000","172.21.15.59:6801/1672530462":"2023-12-12T17:37:44.773925+0000","172.21.15.59:0/1053944547":"2023-12-12T17:38:14.037922+0000","172.21.15.59:6801/1259916482":"2023-12-12T17:38:14.037922+0000","172.21.15.59:6800/1672530462":"2023-12-12T17:37:44.773925+0000","172.21.15.59:0/3570767650":"2023-12-12T17:37:44.773925+0000","172.21.15.59:0/3853462899":"2023-12-12T17:37:44.773925+0000","172.21.15.59:0/2859415380":"2023-12-12T17:38:14.037922+0000","172.21.15.59:0/2192742819":"2023-12-12T17:37:44.773925+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-11T17:44:26.445 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.443+0000 7ff389e9c700 1 -- 172.21.15.59:0/3758537040 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7ff3700a3690 msgr2=0x7ff3700a5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:26.445 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.443+0000 7ff389e9c700 1 --2- 172.21.15.59:0/3758537040 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7ff3700a3690 0x7ff3700a5b50 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff384101400 tx=0x7ff36c00a680 comp rx=0 tx=0).stop 2023-12-11T17:44:26.446 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.443+0000 7ff389e9c700 1 -- 172.21.15.59:0/3758537040 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff384101f40 msgr2=0x7ff38419d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:26.446 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.443+0000 7ff389e9c700 1 --2- 172.21.15.59:0/3758537040 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff384101f40 0x7ff38419d8e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff37c007bd0 tx=0x7ff37c0094f0 comp rx=0 tx=0).stop 2023-12-11T17:44:26.446 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.443+0000 7ff389e9c700 1 -- 172.21.15.59:0/3758537040 shutdown_connections 2023-12-11T17:44:26.446 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.443+0000 7ff389e9c700 1 --2- 172.21.15.59:0/3758537040 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7ff3700a3690 0x7ff3700a5b50 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:26.446 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.443+0000 7ff389e9c700 1 --2- 172.21.15.59:0/3758537040 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff384101f40 0x7ff38419d8e0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:26.446 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.443+0000 7ff389e9c700 1 --2- 172.21.15.59:0/3758537040 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3841003b0 0x7ff384196380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:26.446 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.443+0000 7ff389e9c700 1 --2- 172.21.15.59:0/3758537040 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff3841015a0 0x7ff3841968c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:26.446 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.443+0000 7ff389e9c700 1 -- 172.21.15.59:0/3758537040 >> 172.21.15.59:0/3758537040 conn(0x7ff3840fb920 msgr2=0x7ff3841047d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:44:26.446 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.443+0000 7ff389e9c700 1 -- 172.21.15.59:0/3758537040 shutdown_connections 2023-12-11T17:44:26.446 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.443+0000 7ff389e9c700 1 -- 172.21.15.59:0/3758537040 wait complete. 2023-12-11T17:44:26.459 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-11T17:44:26.533 INFO:journalctl@ceph.mds.b.smithi115.stdout:Dec 11 17:44:26 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mds-b[139691]: 2023-12-11T17:44:26.127+0000 7fd52fbae700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-11T17:44:26.585 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.582+0000 7efe2d7c2700 1 -- 172.21.15.59:0/873405575 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efe281016d0 msgr2=0x7efe28107f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:26.585 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.582+0000 7efe2d7c2700 1 --2- 172.21.15.59:0/873405575 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efe281016d0 0x7efe28107f60 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7efe2000b0a0 tx=0x7efe2000b3b0 comp rx=0 tx=0).stop 2023-12-11T17:44:26.585 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.582+0000 7efe2d7c2700 1 -- 172.21.15.59:0/873405575 shutdown_connections 2023-12-11T17:44:26.586 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.582+0000 7efe2d7c2700 1 --2- 172.21.15.59:0/873405575 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efe281016d0 0x7efe28107f60 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:26.586 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.582+0000 7efe2d7c2700 1 --2- 172.21.15.59:0/873405575 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efe280ffc60 0x7efe28100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:26.586 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.582+0000 7efe2d7c2700 1 --2- 172.21.15.59:0/873405575 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe28100d10 0x7efe28101190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:26.586 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.582+0000 7efe2d7c2700 1 -- 172.21.15.59:0/873405575 >> 172.21.15.59:0/873405575 conn(0x7efe280fac30 msgr2=0x7efe280fd070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:44:26.586 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.582+0000 7efe2d7c2700 1 -- 172.21.15.59:0/873405575 shutdown_connections 2023-12-11T17:44:26.586 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.582+0000 7efe2d7c2700 1 -- 172.21.15.59:0/873405575 wait complete. 2023-12-11T17:44:26.586 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.583+0000 7efe2d7c2700 1 Processor -- start 2023-12-11T17:44:26.588 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.583+0000 7efe2d7c2700 1 -- start start 2023-12-11T17:44:26.588 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.584+0000 7efe2d7c2700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efe280ffc60 0x7efe28195b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:26.588 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.584+0000 7efe2d7c2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efe28100d10 0x7efe28196090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:26.588 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.584+0000 7efe2d7c2700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe281016d0 0x7efe2819d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:26.588 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.584+0000 7efe2d7c2700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe2819d680 con 0x7efe280ffc60 2023-12-11T17:44:26.588 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.584+0000 7efe2d7c2700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe2819d7c0 con 0x7efe281016d0 2023-12-11T17:44:26.589 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.584+0000 7efe2d7c2700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe2819d950 con 0x7efe28100d10 2023-12-11T17:44:26.589 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.584+0000 7efe26ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efe280ffc60 0x7efe28195b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:26.589 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.584+0000 7efe26ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efe280ffc60 0x7efe28195b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:32962/0 (socket says 172.21.15.59:32962) 2023-12-11T17:44:26.589 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.584+0000 7efe26ffd700 1 -- 172.21.15.59:0/797105838 learned_addr learned my addr 172.21.15.59:0/797105838 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:44:26.589 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.584+0000 7efe267fc700 1 --2- 172.21.15.59:0/797105838 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efe28100d10 0x7efe28196090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:26.589 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.584+0000 7efe26ffd700 1 -- 172.21.15.59:0/797105838 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efe28100d10 msgr2=0x7efe28196090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:26.589 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.584+0000 7efe26ffd700 1 --2- 172.21.15.59:0/797105838 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efe28100d10 0x7efe28196090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:26.589 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.584+0000 7efe26ffd700 1 -- 172.21.15.59:0/797105838 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe281016d0 msgr2=0x7efe2819d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:26.589 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.584+0000 7efe26ffd700 1 --2- 172.21.15.59:0/797105838 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe281016d0 0x7efe2819d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:26.589 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.584+0000 7efe26ffd700 1 -- 172.21.15.59:0/797105838 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe20009d00 con 0x7efe280ffc60 2023-12-11T17:44:26.589 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.584+0000 7efe267fc700 1 --2- 172.21.15.59:0/797105838 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efe28100d10 0x7efe28196090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-11T17:44:26.589 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.584+0000 7efe26ffd700 1 --2- 172.21.15.59:0/797105838 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efe280ffc60 0x7efe28195b50 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7efe1800c920 tx=0x7efe1800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:44:26.589 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.584+0000 7efe0ffff700 1 -- 172.21.15.59:0/797105838 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe18007b00 con 0x7efe280ffc60 2023-12-11T17:44:26.589 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.584+0000 7efe2d7c2700 1 -- 172.21.15.59:0/797105838 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe280ff090 con 0x7efe280ffc60 2023-12-11T17:44:26.589 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.584+0000 7efe0ffff700 1 -- 172.21.15.59:0/797105838 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efe18004cf0 con 0x7efe280ffc60 2023-12-11T17:44:26.590 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.584+0000 7efe0ffff700 1 -- 172.21.15.59:0/797105838 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe18005520 con 0x7efe280ffc60 2023-12-11T17:44:26.590 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.584+0000 7efe2d7c2700 1 -- 172.21.15.59:0/797105838 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe280ff5f0 con 0x7efe280ffc60 2023-12-11T17:44:26.590 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.585+0000 7efe2d7c2700 1 -- 172.21.15.59:0/797105838 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe28106f50 con 0x7efe280ffc60 2023-12-11T17:44:26.590 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.586+0000 7efe0ffff700 1 -- 172.21.15.59:0/797105838 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7efe18004750 con 0x7efe280ffc60 2023-12-11T17:44:26.591 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.587+0000 7efe0ffff700 1 --2- 172.21.15.59:0/797105838 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7efe100a1480 0x7efe100a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:26.591 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.587+0000 7efe267fc700 1 --2- 172.21.15.59:0/797105838 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7efe100a1480 0x7efe100a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:26.591 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.588+0000 7efe0ffff700 1 -- 172.21.15.59:0/797105838 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7efe180cd920 con 0x7efe280ffc60 2023-12-11T17:44:26.591 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.588+0000 7efe267fc700 1 --2- 172.21.15.59:0/797105838 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7efe100a1480 0x7efe100a3940 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7efe1c005d90 tx=0x7efe1c005d00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:44:26.594 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.592+0000 7efe0ffff700 1 -- 172.21.15.59:0/797105838 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7efe180502f0 con 0x7efe280ffc60 2023-12-11T17:44:26.798 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.795+0000 7efe2d7c2700 1 -- 172.21.15.59:0/797105838 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efe28068680 con 0x7efe280ffc60 2023-12-11T17:44:26.799 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.797+0000 7efe0ffff700 1 -- 172.21.15.59:0/797105838 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+23960 (secure 0 0 0) 0x7efe18096d30 con 0x7efe280ffc60 2023-12-11T17:44:26.800 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-12-11T17:44:26.800 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":86,"fsid":"d0a07eaa-984b-11ee-95a3-87774f69a715","created":"2023-12-11T17:37:14.526539+0000","modified":"2023-12-11T17:44:25.096094+0000","last_up_change":"2023-12-11T17:43:38.535503+0000","last_in_change":"2023-12-11T17:43:21.014888+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-11T17:40:26.596029+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-11T17:44:18.586290+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-11T17:44:19.432952+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-11T17:44:23.097308+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3c0b3218-4b2b-40f8-8bef-ed8cd5633607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":1040759398},{"type":"v1","addr":"172.21.15.59:6803","nonce":1040759398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":1040759398},{"type":"v1","addr":"172.21.15.59:6805","nonce":1040759398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":1040759398},{"type":"v1","addr":"172.21.15.59:6809","nonce":1040759398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":1040759398},{"type":"v1","addr":"172.21.15.59:6807","nonce":1040759398}]},"public_addr":"172.21.15.59:6803/1040759398","cluster_addr":"172.21.15.59:6805/1040759398","heartbeat_back_addr":"172.21.15.59:6809/1040759398","heartbeat_front_addr":"172.21.15.59:6807/1040759398","state":["exists","up"]},{"osd":1,"uuid":"2ead7578-765b-4f5a-a72c-863731ed69e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":2776354245},{"type":"v1","addr":"172.21.15.115:6801","nonce":2776354245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":2776354245},{"type":"v1","addr":"172.21.15.115:6803","nonce":2776354245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":2776354245},{"type":"v1","addr":"172.21.15.115:6807","nonce":2776354245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":2776354245},{"type":"v1","addr":"172.21.15.115:6805","nonce":2776354245}]},"public_addr":"172.21.15.115:6801/2776354245","cluster_addr":"172.21.15.115:6803/2776354245","heartbeat_back_addr":"172.21.15.115:6807/2776354245","heartbeat_front_addr":"172.21.15.115:6805/2776354245","state":["exists","up"]},{"osd":2,"uuid":"8cb1d7ec-53cc-4672-b0b6-70d73797c44f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4287675481},{"type":"v1","addr":"172.21.15.181:6801","nonce":4287675481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4287675481},{"type":"v1","addr":"172.21.15.181:6803","nonce":4287675481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4287675481},{"type":"v1","addr":"172.21.15.181:6807","nonce":4287675481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4287675481},{"type":"v1","addr":"172.21.15.181:6805","nonce":4287675481}]},"public_addr":"172.21.15.181:6801/4287675481","cluster_addr":"172.21.15.181:6803/4287675481","heartbeat_back_addr":"172.21.15.181:6807/4287675481","heartbeat_front_addr":"172.21.15.181:6805/4287675481","state":["exists","up"]},{"osd":3,"uuid":"5d64d9af-831b-4d0a-8ace-a452169351d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":219909995},{"type":"v1","addr":"172.21.15.59:6811","nonce":219909995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":219909995},{"type":"v1","addr":"172.21.15.59:6813","nonce":219909995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":219909995},{"type":"v1","addr":"172.21.15.59:6817","nonce":219909995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":219909995},{"type":"v1","addr":"172.21.15.59:6815","nonce":219909995}]},"public_addr":"172.21.15.59:6811/219909995","cluster_addr":"172.21.15.59:6813/219909995","heartbeat_back_addr":"172.21.15.59:6817/219909995","heartbeat_front_addr":"172.21.15.59:6815/219909995","state":["exists","up"]},{"osd":4,"uuid":"ef8fad28-1db5-4cba-b5f2-10e54e8e1549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":1021473826},{"type":"v1","addr":"172.21.15.115:6809","nonce":1021473826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":1021473826},{"type":"v1","addr":"172.21.15.115:6811","nonce":1021473826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":1021473826},{"type":"v1","addr":"172.21.15.115:6815","nonce":1021473826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":1021473826},{"type":"v1","addr":"172.21.15.115:6813","nonce":1021473826}]},"public_addr":"172.21.15.115:6809/1021473826","cluster_addr":"172.21.15.115:6811/1021473826","heartbeat_back_addr":"172.21.15.115:6815/1021473826","heartbeat_front_addr":"172.21.15.115:6813/1021473826","state":["exists","up"]},{"osd":5,"uuid":"4233a512-9e8f-4db8-aed2-6c3af706e589","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":2405172521},{"type":"v1","addr":"172.21.15.181:6809","nonce":2405172521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":2405172521},{"type":"v1","addr":"172.21.15.181:6811","nonce":2405172521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2405172521},{"type":"v1","addr":"172.21.15.181:6815","nonce":2405172521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":2405172521},{"type":"v1","addr":"172.21.15.181:6813","nonce":2405172521}]},"public_addr":"172.21.15.181:6809/2405172521","cluster_addr":"172.21.15.181:6811/2405172521","heartbeat_back_addr":"172.21.15.181:6815/2405172521","heartbeat_front_addr":"172.21.15.181:6813/2405172521","state":["exists","up"]},{"osd":6,"uuid":"8faab332-016e-4c77-8370-8bef95fc6f34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":815374853},{"type":"v1","addr":"172.21.15.59:6819","nonce":815374853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":815374853},{"type":"v1","addr":"172.21.15.59:6821","nonce":815374853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":815374853},{"type":"v1","addr":"172.21.15.59:6825","nonce":815374853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":815374853},{"type":"v1","addr":"172.21.15.59:6823","nonce":815374853}]},"public_addr":"172.21.15.59:6819/815374853","cluster_addr":"172.21.15.59:6821/815374853","heartbeat_back_addr":"172.21.15.59:6825/815374853","heartbeat_front_addr":"172.21.15.59:6823/815374853","state":["exists","up"]},{"osd":7,"uuid":"f20f470a-0d40-4ed3-8e0e-fd58675cf143","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":1314971870},{"type":"v1","addr":"172.21.15.115:6817","nonce":1314971870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":1314971870},{"type":"v1","addr":"172.21.15.115:6819","nonce":1314971870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":1314971870},{"type":"v1","addr":"172.21.15.115:6823","nonce":1314971870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":1314971870},{"type":"v1","addr":"172.21.15.115:6821","nonce":1314971870}]},"public_addr":"172.21.15.115:6817/1314971870","cluster_addr":"172.21.15.115:6819/1314971870","heartbeat_back_addr":"172.21.15.115:6823/1314971870","heartbeat_front_addr":"172.21.15.115:6821/1314971870","state":["exists","up"]},{"osd":8,"uuid":"eb657d32-1a95-4fd7-a293-d0a970837fc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":3954509873},{"type":"v1","addr":"172.21.15.181:6817","nonce":3954509873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":3954509873},{"type":"v1","addr":"172.21.15.181:6819","nonce":3954509873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":3954509873},{"type":"v1","addr":"172.21.15.181:6823","nonce":3954509873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":3954509873},{"type":"v1","addr":"172.21.15.181:6821","nonce":3954509873}]},"public_addr":"172.21.15.181:6817/3954509873","cluster_addr":"172.21.15.181:6819/3954509873","heartbeat_back_addr":"172.21.15.181:6823/3954509873","heartbeat_front_addr":"172.21.15.181:6821/3954509873","state":["exists","up"]},{"osd":9,"uuid":"8f7717d4-b6f2-4ec1-9b58-62feb0c9a146","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":1163095677},{"type":"v1","addr":"172.21.15.59:6827","nonce":1163095677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":1163095677},{"type":"v1","addr":"172.21.15.59:6829","nonce":1163095677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":1163095677},{"type":"v1","addr":"172.21.15.59:6833","nonce":1163095677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":1163095677},{"type":"v1","addr":"172.21.15.59:6831","nonce":1163095677}]},"public_addr":"172.21.15.59:6827/1163095677","cluster_addr":"172.21.15.59:6829/1163095677","heartbeat_back_addr":"172.21.15.59:6833/1163095677","heartbeat_front_addr":"172.21.15.59:6831/1163095677","state":["exists","up"]},{"osd":10,"uuid":"b75c179a-875d-43bc-b4c1-e8eb80fb64fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":2199610269},{"type":"v1","addr":"172.21.15.115:6825","nonce":2199610269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":2199610269},{"type":"v1","addr":"172.21.15.115:6827","nonce":2199610269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":2199610269},{"type":"v1","addr":"172.21.15.115:6831","nonce":2199610269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":2199610269},{"type":"v1","addr":"172.21.15.115:6829","nonce":2199610269}]},"public_addr":"172.21.15.115:6825/2199610269","cluster_addr":"172.21.15.115:6827/2199610269","heartbeat_back_addr":"172.21.15.115:6831/2199610269","heartbeat_front_addr":"172.21.15.115:6829/2199610269","state":["exists","up"]},{"osd":11,"uuid":"5e141dfe-8a00-4bbc-8204-2dc9be24e18d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":226506989},{"type":"v1","addr":"172.21.15.181:6825","nonce":226506989}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":226506989},{"type":"v1","addr":"172.21.15.181:6827","nonce":226506989}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":226506989},{"type":"v1","addr":"172.21.15.181:6831","nonce":226506989}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":226506989},{"type":"v1","addr":"172.21.15.181:6829","nonce":226506989}]},"public_addr":"172.21.15.181:6825/226506989","cluster_addr":"172.21.15.181:6827/226506989","heartbeat_back_addr":"172.21.15.181:6831/226506989","heartbeat_front_addr":"172.21.15.181:6829/226506989","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:39:42.687084+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:40:03.903448+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:40:24.899908+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:40:46.600251+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:41:07.690037+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:41:28.488855+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:41:49.459274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:42:10.863881+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:42:32.248391+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:42:54.112265+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:43:15.634872+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:43:36.922534+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.59:6800/1259916482":"2023-12-12T17:38:14.037922+0000","172.21.15.59:0/17457774":"2023-12-12T17:38:14.037922+0000","172.21.15.59:6801/1672530462":"2023-12-12T17:37:44.773925+0000","172.21.15.59:0/1053944547":"2023-12-12T17:38:14.037922+0000","172.21.15.59:6801/1259916482":"2023-12-12T17:38:14.037922+0000","172.21.15.59:6800/1672530462":"2023-12-12T17:37:44.773925+0000","172.21.15.59:0/3570767650":"2023-12-12T17:37:44.773925+0000","172.21.15.59:0/3853462899":"2023-12-12T17:37:44.773925+0000","172.21.15.59:0/2859415380":"2023-12-12T17:38:14.037922+0000","172.21.15.59:0/2192742819":"2023-12-12T17:37:44.773925+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-11T17:44:26.802 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.799+0000 7efe2d7c2700 1 -- 172.21.15.59:0/797105838 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7efe100a1480 msgr2=0x7efe100a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:26.802 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.799+0000 7efe2d7c2700 1 --2- 172.21.15.59:0/797105838 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7efe100a1480 0x7efe100a3940 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7efe1c005d90 tx=0x7efe1c005d00 comp rx=0 tx=0).stop 2023-12-11T17:44:26.802 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.799+0000 7efe2d7c2700 1 -- 172.21.15.59:0/797105838 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efe280ffc60 msgr2=0x7efe28195b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:26.802 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.799+0000 7efe2d7c2700 1 --2- 172.21.15.59:0/797105838 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efe280ffc60 0x7efe28195b50 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7efe1800c920 tx=0x7efe1800cce0 comp rx=0 tx=0).stop 2023-12-11T17:44:26.802 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.799+0000 7efe2d7c2700 1 -- 172.21.15.59:0/797105838 shutdown_connections 2023-12-11T17:44:26.802 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.799+0000 7efe2d7c2700 1 --2- 172.21.15.59:0/797105838 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7efe100a1480 0x7efe100a3940 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:26.802 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.799+0000 7efe2d7c2700 1 --2- 172.21.15.59:0/797105838 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe281016d0 0x7efe2819d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:26.803 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.799+0000 7efe2d7c2700 1 --2- 172.21.15.59:0/797105838 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efe280ffc60 0x7efe28195b50 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:26.803 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.799+0000 7efe2d7c2700 1 --2- 172.21.15.59:0/797105838 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efe28100d10 0x7efe28196090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:26.803 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.799+0000 7efe2d7c2700 1 -- 172.21.15.59:0/797105838 >> 172.21.15.59:0/797105838 conn(0x7efe280fac30 msgr2=0x7efe28104090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:44:26.803 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.799+0000 7efe2d7c2700 1 -- 172.21.15.59:0/797105838 shutdown_connections 2023-12-11T17:44:26.803 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.799+0000 7efe2d7c2700 1 -- 172.21.15.59:0/797105838 wait complete. 2023-12-11T17:44:26.813 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:26 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/3269008731' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2023-12-11T17:44:26.813 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:26 smithi059 ceph-mon[98847]: osdmap e86: 12 total, 12 up, 12 in 2023-12-11T17:44:26.813 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:26 smithi059 ceph-mon[98847]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-12-11T17:44:26.813 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:26 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/3714767746' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T17:44:26.813 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:26 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/3758537040' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-11T17:44:26.829 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-11T17:44:26.956 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.952+0000 7eff77409700 1 -- 172.21.15.59:0/21220100 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7eff70100380 msgr2=0x7eff701007a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:26.957 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.952+0000 7eff77409700 1 --2- 172.21.15.59:0/21220100 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7eff70100380 0x7eff701007a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7eff600099c0 tx=0x7eff60009cd0 comp rx=0 tx=0).stop 2023-12-11T17:44:26.957 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.952+0000 7eff77409700 1 -- 172.21.15.59:0/21220100 shutdown_connections 2023-12-11T17:44:26.957 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.952+0000 7eff77409700 1 --2- 172.21.15.59:0/21220100 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff70101f10 0x7eff701087c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:26.957 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.952+0000 7eff77409700 1 --2- 172.21.15.59:0/21220100 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7eff70100380 0x7eff701007a0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:26.957 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.952+0000 7eff77409700 1 --2- 172.21.15.59:0/21220100 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7eff70101570 0x7eff701019d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:26.957 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.952+0000 7eff77409700 1 -- 172.21.15.59:0/21220100 >> 172.21.15.59:0/21220100 conn(0x7eff700fb8f0 msgr2=0x7eff700fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:44:26.957 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.952+0000 7eff77409700 1 -- 172.21.15.59:0/21220100 shutdown_connections 2023-12-11T17:44:26.957 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.952+0000 7eff77409700 1 -- 172.21.15.59:0/21220100 wait complete. 2023-12-11T17:44:26.957 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.952+0000 7eff77409700 1 Processor -- start 2023-12-11T17:44:26.957 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.952+0000 7eff77409700 1 -- start start 2023-12-11T17:44:26.957 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.953+0000 7eff77409700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff70100380 0x7eff70069150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:26.957 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.953+0000 7eff77409700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7eff70101570 0x7eff7019c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:26.958 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.953+0000 7eff77409700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7eff70101f10 0x7eff7019e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:26.958 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.953+0000 7eff77409700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff7019ef10 con 0x7eff70101570 2023-12-11T17:44:26.958 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.953+0000 7eff77409700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff7019f080 con 0x7eff70101f10 2023-12-11T17:44:26.958 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.953+0000 7eff77409700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff7019f240 con 0x7eff70100380 2023-12-11T17:44:26.958 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.953+0000 7eff749a4700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7eff70101570 0x7eff7019c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:26.958 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.953+0000 7eff751a5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff70100380 0x7eff70069150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:26.958 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.953+0000 7eff749a4700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7eff70101570 0x7eff7019c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:32978/0 (socket says 172.21.15.59:32978) 2023-12-11T17:44:26.958 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.953+0000 7eff749a4700 1 -- 172.21.15.59:0/1496931243 learned_addr learned my addr 172.21.15.59:0/1496931243 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:44:26.958 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.953+0000 7eff759a6700 1 --2- 172.21.15.59:0/1496931243 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7eff70101f10 0x7eff7019e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:26.958 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.953+0000 7eff749a4700 1 -- 172.21.15.59:0/1496931243 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff70100380 msgr2=0x7eff70069150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:26.958 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.953+0000 7eff749a4700 1 --2- 172.21.15.59:0/1496931243 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff70100380 0x7eff70069150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:26.958 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.953+0000 7eff749a4700 1 -- 172.21.15.59:0/1496931243 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7eff70101f10 msgr2=0x7eff7019e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:26.958 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.953+0000 7eff749a4700 1 --2- 172.21.15.59:0/1496931243 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7eff70101f10 0x7eff7019e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:26.959 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.953+0000 7eff749a4700 1 -- 172.21.15.59:0/1496931243 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff600095f0 con 0x7eff70101570 2023-12-11T17:44:26.959 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.953+0000 7eff759a6700 1 --2- 172.21.15.59:0/1496931243 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7eff70101f10 0x7eff7019e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T17:44:26.959 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.953+0000 7eff749a4700 1 --2- 172.21.15.59:0/1496931243 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7eff70101570 0x7eff7019c100 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7eff6c00d980 tx=0x7eff6c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:44:26.959 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.953+0000 7eff667fc700 1 -- 172.21.15.59:0/1496931243 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff6c004220 con 0x7eff70101570 2023-12-11T17:44:26.959 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.953+0000 7eff667fc700 1 -- 172.21.15.59:0/1496931243 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7eff6c007640 con 0x7eff70101570 2023-12-11T17:44:26.959 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.953+0000 7eff77409700 1 -- 172.21.15.59:0/1496931243 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff7019f4c0 con 0x7eff70101570 2023-12-11T17:44:26.959 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.953+0000 7eff667fc700 1 -- 172.21.15.59:0/1496931243 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff6c005610 con 0x7eff70101570 2023-12-11T17:44:26.959 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.953+0000 7eff77409700 1 -- 172.21.15.59:0/1496931243 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff7019fa20 con 0x7eff70101570 2023-12-11T17:44:26.959 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.954+0000 7eff77409700 1 -- 172.21.15.59:0/1496931243 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff701077a0 con 0x7eff70101570 2023-12-11T17:44:26.960 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.956+0000 7eff667fc700 1 -- 172.21.15.59:0/1496931243 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7eff6c004750 con 0x7eff70101570 2023-12-11T17:44:26.960 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.956+0000 7eff667fc700 1 --2- 172.21.15.59:0/1496931243 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7eff5c0a1500 0x7eff5c0a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:26.960 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.956+0000 7eff751a5700 1 --2- 172.21.15.59:0/1496931243 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7eff5c0a1500 0x7eff5c0a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:26.960 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.956+0000 7eff667fc700 1 -- 172.21.15.59:0/1496931243 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7eff6c0cd1c0 con 0x7eff70101570 2023-12-11T17:44:26.960 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.957+0000 7eff751a5700 1 --2- 172.21.15.59:0/1496931243 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7eff5c0a1500 0x7eff5c0a39c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7eff60003730 tx=0x7eff60000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:44:26.962 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:26.960+0000 7eff667fc700 1 -- 172.21.15.59:0/1496931243 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7eff6c0966d0 con 0x7eff70101570 2023-12-11T17:44:27.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:26 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/3269008731' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2023-12-11T17:44:27.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:26 smithi115 ceph-mon[102748]: osdmap e86: 12 total, 12 up, 12 in 2023-12-11T17:44:27.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:26 smithi115 ceph-mon[102748]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-12-11T17:44:27.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:26 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/3714767746' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T17:44:27.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:26 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/3758537040' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-11T17:44:27.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:26 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/3269008731' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2023-12-11T17:44:27.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:26 smithi181 ceph-mon[103627]: osdmap e86: 12 total, 12 up, 12 in 2023-12-11T17:44:27.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:26 smithi181 ceph-mon[103627]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-12-11T17:44:27.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:26 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/3714767746' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T17:44:27.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:26 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/3758537040' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-11T17:44:27.212 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.210+0000 7eff77409700 1 -- 172.21.15.59:0/1496931243 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7eff70068230 con 0x7eff70101570 2023-12-11T17:44:27.213 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.210+0000 7eff667fc700 1 -- 172.21.15.59:0/1496931243 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+11264 (secure 0 0 0) 0x7eff6c0b1c10 con 0x7eff70101570 2023-12-11T17:44:27.213 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-12-11T17:44:27.213 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/2184151593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":2184151593},{"type":"v1","addr":"172.21.15.59:6835","nonce":2184151593}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/2817614886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2817614886},{"type":"v1","addr":"172.21.15.59:6837","nonce":2817614886}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6839/229873490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":229873490},{"type":"v1","addr":"172.21.15.59:6839","nonce":229873490}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6841/967998755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":967998755},{"type":"v1","addr":"172.21.15.59:6841","nonce":967998755}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/395684042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":395684042},{"type":"v1","addr":"172.21.15.181:6833","nonce":395684042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6833/3214606191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3214606191},{"type":"v1","addr":"172.21.15.115:6833","nonce":3214606191}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24449,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/1874159657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1874159657},{"type":"v1","addr":"172.21.15.115:6835","nonce":1874159657}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/4221183620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":4221183620},{"type":"v1","addr":"172.21.15.181:6835","nonce":4221183620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24466,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/3580309407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":3580309407},{"type":"v1","addr":"172.21.15.181:6837","nonce":3580309407}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24482,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/1209365460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":1209365460},{"type":"v1","addr":"172.21.15.115:6837","nonce":1209365460}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24484,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6839/2214384806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":2214384806},{"type":"v1","addr":"172.21.15.181:6839","nonce":2214384806}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-11T17:44:20.076660+0000","modified":"2023-12-11T17:44:25.096263+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24521},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.115:6839/4122922866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":4122922866},{"type":"v1","addr":"172.21.15.115:6839","nonce":4122922866}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2023-12-11T17:44:27.215 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.212+0000 7eff77409700 1 -- 172.21.15.59:0/1496931243 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7eff5c0a1500 msgr2=0x7eff5c0a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:27.215 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.212+0000 7eff77409700 1 --2- 172.21.15.59:0/1496931243 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7eff5c0a1500 0x7eff5c0a39c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7eff60003730 tx=0x7eff60000bc0 comp rx=0 tx=0).stop 2023-12-11T17:44:27.215 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.212+0000 7eff77409700 1 -- 172.21.15.59:0/1496931243 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7eff70101570 msgr2=0x7eff7019c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:27.215 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.212+0000 7eff77409700 1 --2- 172.21.15.59:0/1496931243 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7eff70101570 0x7eff7019c100 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7eff6c00d980 tx=0x7eff6c00dc90 comp rx=0 tx=0).stop 2023-12-11T17:44:27.215 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.213+0000 7eff77409700 1 -- 172.21.15.59:0/1496931243 shutdown_connections 2023-12-11T17:44:27.215 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.213+0000 7eff77409700 1 --2- 172.21.15.59:0/1496931243 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7eff5c0a1500 0x7eff5c0a39c0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:27.215 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.213+0000 7eff77409700 1 --2- 172.21.15.59:0/1496931243 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7eff70101f10 0x7eff7019e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:27.216 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.213+0000 7eff77409700 1 --2- 172.21.15.59:0/1496931243 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff70100380 0x7eff70069150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:27.216 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.213+0000 7eff77409700 1 --2- 172.21.15.59:0/1496931243 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7eff70101570 0x7eff7019c100 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:27.216 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.213+0000 7eff77409700 1 -- 172.21.15.59:0/1496931243 >> 172.21.15.59:0/1496931243 conn(0x7eff700fb8f0 msgr2=0x7eff701048e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:44:27.216 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.213+0000 7eff77409700 1 -- 172.21.15.59:0/1496931243 shutdown_connections 2023-12-11T17:44:27.216 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.213+0000 7eff77409700 1 -- 172.21.15.59:0/1496931243 wait complete. 2023-12-11T17:44:27.216 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 17 2023-12-11T17:44:27.227 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-11T17:44:27.356 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.351+0000 7f3c9d8a3700 1 -- 172.21.15.59:0/745600943 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3c98101f10 msgr2=0x7f3c981087c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:27.356 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.351+0000 7f3c9d8a3700 1 --2- 172.21.15.59:0/745600943 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3c98101f10 0x7f3c981087c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f3c9000b0d0 tx=0x7f3c9000b3e0 comp rx=0 tx=0).stop 2023-12-11T17:44:27.356 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.351+0000 7f3c9d8a3700 1 -- 172.21.15.59:0/745600943 shutdown_connections 2023-12-11T17:44:27.356 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.351+0000 7f3c9d8a3700 1 --2- 172.21.15.59:0/745600943 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3c98101f10 0x7f3c981087c0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:27.356 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.351+0000 7f3c9d8a3700 1 --2- 172.21.15.59:0/745600943 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c98100380 0x7f3c981007a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:27.356 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.351+0000 7f3c9d8a3700 1 --2- 172.21.15.59:0/745600943 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3c98101570 0x7f3c981019d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:27.356 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.351+0000 7f3c9d8a3700 1 -- 172.21.15.59:0/745600943 >> 172.21.15.59:0/745600943 conn(0x7f3c980fb8f0 msgr2=0x7f3c980fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:44:27.356 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.351+0000 7f3c9d8a3700 1 -- 172.21.15.59:0/745600943 shutdown_connections 2023-12-11T17:44:27.356 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.351+0000 7f3c9d8a3700 1 -- 172.21.15.59:0/745600943 wait complete. 2023-12-11T17:44:27.356 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.352+0000 7f3c9d8a3700 1 Processor -- start 2023-12-11T17:44:27.356 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.352+0000 7f3c9d8a3700 1 -- start start 2023-12-11T17:44:27.357 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.352+0000 7f3c9d8a3700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3c98100380 0x7f3c981974d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:27.357 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.352+0000 7f3c9d8a3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c98101570 0x7f3c98197a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:27.357 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.352+0000 7f3c9d8a3700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3c98101f10 0x7f3c9819ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:27.357 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.352+0000 7f3c9d8a3700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c9819f000 con 0x7f3c98100380 2023-12-11T17:44:27.357 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.352+0000 7f3c9d8a3700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c9819f140 con 0x7f3c98101f10 2023-12-11T17:44:27.357 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.352+0000 7f3c9d8a3700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c9819f2d0 con 0x7f3c98101570 2023-12-11T17:44:27.357 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.352+0000 7f3c96ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3c98100380 0x7f3c981974d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:27.357 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.352+0000 7f3c96ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3c98100380 0x7f3c981974d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:32990/0 (socket says 172.21.15.59:32990) 2023-12-11T17:44:27.357 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.352+0000 7f3c96ffd700 1 -- 172.21.15.59:0/1477563605 learned_addr learned my addr 172.21.15.59:0/1477563605 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:44:27.357 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.352+0000 7f3c967fc700 1 --2- 172.21.15.59:0/1477563605 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c98101570 0x7f3c98197a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:27.357 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.352+0000 7f3c977fe700 1 --2- 172.21.15.59:0/1477563605 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3c98101f10 0x7f3c9819ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:27.357 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.352+0000 7f3c96ffd700 1 -- 172.21.15.59:0/1477563605 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c98101570 msgr2=0x7f3c98197a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:27.357 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.352+0000 7f3c96ffd700 1 --2- 172.21.15.59:0/1477563605 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c98101570 0x7f3c98197a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:27.357 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.352+0000 7f3c96ffd700 1 -- 172.21.15.59:0/1477563605 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3c98101f10 msgr2=0x7f3c9819ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:27.357 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.352+0000 7f3c96ffd700 1 --2- 172.21.15.59:0/1477563605 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3c98101f10 0x7f3c9819ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:27.357 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.352+0000 7f3c96ffd700 1 -- 172.21.15.59:0/1477563605 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c90009d00 con 0x7f3c98100380 2023-12-11T17:44:27.358 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.352+0000 7f3c967fc700 1 --2- 172.21.15.59:0/1477563605 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c98101570 0x7f3c98197a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T17:44:27.358 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.353+0000 7f3c96ffd700 1 --2- 172.21.15.59:0/1477563605 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3c98100380 0x7f3c981974d0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f3c8800d980 tx=0x7f3c8800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:44:27.358 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.353+0000 7f3c9c8a1700 1 -- 172.21.15.59:0/1477563605 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c8800bb60 con 0x7f3c98100380 2023-12-11T17:44:27.358 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.353+0000 7f3c9c8a1700 1 -- 172.21.15.59:0/1477563605 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3c88007620 con 0x7f3c98100380 2023-12-11T17:44:27.358 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.353+0000 7f3c9d8a3700 1 -- 172.21.15.59:0/1477563605 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c981028c0 con 0x7f3c98100380 2023-12-11T17:44:27.358 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.353+0000 7f3c9c8a1700 1 -- 172.21.15.59:0/1477563605 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c8800f620 con 0x7f3c98100380 2023-12-11T17:44:27.358 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.353+0000 7f3c9d8a3700 1 -- 172.21.15.59:0/1477563605 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c98102dd0 con 0x7f3c98100380 2023-12-11T17:44:27.359 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.354+0000 7f3c9d8a3700 1 -- 172.21.15.59:0/1477563605 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c98068230 con 0x7f3c98100380 2023-12-11T17:44:27.359 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.355+0000 7f3c9c8a1700 1 -- 172.21.15.59:0/1477563605 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f3c88010460 con 0x7f3c98100380 2023-12-11T17:44:27.359 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.355+0000 7f3c9c8a1700 1 --2- 172.21.15.59:0/1477563605 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f3c840a1480 0x7f3c840a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:27.359 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.355+0000 7f3c967fc700 1 --2- 172.21.15.59:0/1477563605 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f3c840a1480 0x7f3c840a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:27.359 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.355+0000 7f3c9c8a1700 1 -- 172.21.15.59:0/1477563605 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f3c880ccf10 con 0x7f3c98100380 2023-12-11T17:44:27.359 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.356+0000 7f3c967fc700 1 --2- 172.21.15.59:0/1477563605 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f3c840a1480 0x7f3c840a3940 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3c80005e10 tx=0x7f3c80005d60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:44:27.362 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.360+0000 7f3c9c8a1700 1 -- 172.21.15.59:0/1477563605 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f3c88096380 con 0x7f3c98100380 2023-12-11T17:44:27.564 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.562+0000 7f3c9d8a3700 1 -- 172.21.15.59:0/1477563605 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3c9810bb90 con 0x7f3c98100380 2023-12-11T17:44:27.566 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.563+0000 7f3c9c8a1700 1 -- 172.21.15.59:0/1477563605 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+23960 (secure 0 0 0) 0x7f3c88016090 con 0x7f3c98100380 2023-12-11T17:44:27.566 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-12-11T17:44:27.566 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":86,"fsid":"d0a07eaa-984b-11ee-95a3-87774f69a715","created":"2023-12-11T17:37:14.526539+0000","modified":"2023-12-11T17:44:25.096094+0000","last_up_change":"2023-12-11T17:43:38.535503+0000","last_in_change":"2023-12-11T17:43:21.014888+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-11T17:40:26.596029+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-11T17:44:18.586290+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-11T17:44:19.432952+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-11T17:44:23.097308+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3c0b3218-4b2b-40f8-8bef-ed8cd5633607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":1040759398},{"type":"v1","addr":"172.21.15.59:6803","nonce":1040759398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":1040759398},{"type":"v1","addr":"172.21.15.59:6805","nonce":1040759398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":1040759398},{"type":"v1","addr":"172.21.15.59:6809","nonce":1040759398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":1040759398},{"type":"v1","addr":"172.21.15.59:6807","nonce":1040759398}]},"public_addr":"172.21.15.59:6803/1040759398","cluster_addr":"172.21.15.59:6805/1040759398","heartbeat_back_addr":"172.21.15.59:6809/1040759398","heartbeat_front_addr":"172.21.15.59:6807/1040759398","state":["exists","up"]},{"osd":1,"uuid":"2ead7578-765b-4f5a-a72c-863731ed69e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":2776354245},{"type":"v1","addr":"172.21.15.115:6801","nonce":2776354245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":2776354245},{"type":"v1","addr":"172.21.15.115:6803","nonce":2776354245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":2776354245},{"type":"v1","addr":"172.21.15.115:6807","nonce":2776354245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":2776354245},{"type":"v1","addr":"172.21.15.115:6805","nonce":2776354245}]},"public_addr":"172.21.15.115:6801/2776354245","cluster_addr":"172.21.15.115:6803/2776354245","heartbeat_back_addr":"172.21.15.115:6807/2776354245","heartbeat_front_addr":"172.21.15.115:6805/2776354245","state":["exists","up"]},{"osd":2,"uuid":"8cb1d7ec-53cc-4672-b0b6-70d73797c44f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4287675481},{"type":"v1","addr":"172.21.15.181:6801","nonce":4287675481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4287675481},{"type":"v1","addr":"172.21.15.181:6803","nonce":4287675481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4287675481},{"type":"v1","addr":"172.21.15.181:6807","nonce":4287675481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4287675481},{"type":"v1","addr":"172.21.15.181:6805","nonce":4287675481}]},"public_addr":"172.21.15.181:6801/4287675481","cluster_addr":"172.21.15.181:6803/4287675481","heartbeat_back_addr":"172.21.15.181:6807/4287675481","heartbeat_front_addr":"172.21.15.181:6805/4287675481","state":["exists","up"]},{"osd":3,"uuid":"5d64d9af-831b-4d0a-8ace-a452169351d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":219909995},{"type":"v1","addr":"172.21.15.59:6811","nonce":219909995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":219909995},{"type":"v1","addr":"172.21.15.59:6813","nonce":219909995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":219909995},{"type":"v1","addr":"172.21.15.59:6817","nonce":219909995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":219909995},{"type":"v1","addr":"172.21.15.59:6815","nonce":219909995}]},"public_addr":"172.21.15.59:6811/219909995","cluster_addr":"172.21.15.59:6813/219909995","heartbeat_back_addr":"172.21.15.59:6817/219909995","heartbeat_front_addr":"172.21.15.59:6815/219909995","state":["exists","up"]},{"osd":4,"uuid":"ef8fad28-1db5-4cba-b5f2-10e54e8e1549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":1021473826},{"type":"v1","addr":"172.21.15.115:6809","nonce":1021473826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":1021473826},{"type":"v1","addr":"172.21.15.115:6811","nonce":1021473826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":1021473826},{"type":"v1","addr":"172.21.15.115:6815","nonce":1021473826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":1021473826},{"type":"v1","addr":"172.21.15.115:6813","nonce":1021473826}]},"public_addr":"172.21.15.115:6809/1021473826","cluster_addr":"172.21.15.115:6811/1021473826","heartbeat_back_addr":"172.21.15.115:6815/1021473826","heartbeat_front_addr":"172.21.15.115:6813/1021473826","state":["exists","up"]},{"osd":5,"uuid":"4233a512-9e8f-4db8-aed2-6c3af706e589","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":2405172521},{"type":"v1","addr":"172.21.15.181:6809","nonce":2405172521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":2405172521},{"type":"v1","addr":"172.21.15.181:6811","nonce":2405172521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2405172521},{"type":"v1","addr":"172.21.15.181:6815","nonce":2405172521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":2405172521},{"type":"v1","addr":"172.21.15.181:6813","nonce":2405172521}]},"public_addr":"172.21.15.181:6809/2405172521","cluster_addr":"172.21.15.181:6811/2405172521","heartbeat_back_addr":"172.21.15.181:6815/2405172521","heartbeat_front_addr":"172.21.15.181:6813/2405172521","state":["exists","up"]},{"osd":6,"uuid":"8faab332-016e-4c77-8370-8bef95fc6f34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":815374853},{"type":"v1","addr":"172.21.15.59:6819","nonce":815374853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":815374853},{"type":"v1","addr":"172.21.15.59:6821","nonce":815374853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":815374853},{"type":"v1","addr":"172.21.15.59:6825","nonce":815374853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":815374853},{"type":"v1","addr":"172.21.15.59:6823","nonce":815374853}]},"public_addr":"172.21.15.59:6819/815374853","cluster_addr":"172.21.15.59:6821/815374853","heartbeat_back_addr":"172.21.15.59:6825/815374853","heartbeat_front_addr":"172.21.15.59:6823/815374853","state":["exists","up"]},{"osd":7,"uuid":"f20f470a-0d40-4ed3-8e0e-fd58675cf143","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":1314971870},{"type":"v1","addr":"172.21.15.115:6817","nonce":1314971870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":1314971870},{"type":"v1","addr":"172.21.15.115:6819","nonce":1314971870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":1314971870},{"type":"v1","addr":"172.21.15.115:6823","nonce":1314971870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":1314971870},{"type":"v1","addr":"172.21.15.115:6821","nonce":1314971870}]},"public_addr":"172.21.15.115:6817/1314971870","cluster_addr":"172.21.15.115:6819/1314971870","heartbeat_back_addr":"172.21.15.115:6823/1314971870","heartbeat_front_addr":"172.21.15.115:6821/1314971870","state":["exists","up"]},{"osd":8,"uuid":"eb657d32-1a95-4fd7-a293-d0a970837fc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":3954509873},{"type":"v1","addr":"172.21.15.181:6817","nonce":3954509873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":3954509873},{"type":"v1","addr":"172.21.15.181:6819","nonce":3954509873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":3954509873},{"type":"v1","addr":"172.21.15.181:6823","nonce":3954509873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":3954509873},{"type":"v1","addr":"172.21.15.181:6821","nonce":3954509873}]},"public_addr":"172.21.15.181:6817/3954509873","cluster_addr":"172.21.15.181:6819/3954509873","heartbeat_back_addr":"172.21.15.181:6823/3954509873","heartbeat_front_addr":"172.21.15.181:6821/3954509873","state":["exists","up"]},{"osd":9,"uuid":"8f7717d4-b6f2-4ec1-9b58-62feb0c9a146","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":1163095677},{"type":"v1","addr":"172.21.15.59:6827","nonce":1163095677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":1163095677},{"type":"v1","addr":"172.21.15.59:6829","nonce":1163095677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":1163095677},{"type":"v1","addr":"172.21.15.59:6833","nonce":1163095677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":1163095677},{"type":"v1","addr":"172.21.15.59:6831","nonce":1163095677}]},"public_addr":"172.21.15.59:6827/1163095677","cluster_addr":"172.21.15.59:6829/1163095677","heartbeat_back_addr":"172.21.15.59:6833/1163095677","heartbeat_front_addr":"172.21.15.59:6831/1163095677","state":["exists","up"]},{"osd":10,"uuid":"b75c179a-875d-43bc-b4c1-e8eb80fb64fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":2199610269},{"type":"v1","addr":"172.21.15.115:6825","nonce":2199610269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":2199610269},{"type":"v1","addr":"172.21.15.115:6827","nonce":2199610269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":2199610269},{"type":"v1","addr":"172.21.15.115:6831","nonce":2199610269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":2199610269},{"type":"v1","addr":"172.21.15.115:6829","nonce":2199610269}]},"public_addr":"172.21.15.115:6825/2199610269","cluster_addr":"172.21.15.115:6827/2199610269","heartbeat_back_addr":"172.21.15.115:6831/2199610269","heartbeat_front_addr":"172.21.15.115:6829/2199610269","state":["exists","up"]},{"osd":11,"uuid":"5e141dfe-8a00-4bbc-8204-2dc9be24e18d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":226506989},{"type":"v1","addr":"172.21.15.181:6825","nonce":226506989}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":226506989},{"type":"v1","addr":"172.21.15.181:6827","nonce":226506989}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":226506989},{"type":"v1","addr":"172.21.15.181:6831","nonce":226506989}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":226506989},{"type":"v1","addr":"172.21.15.181:6829","nonce":226506989}]},"public_addr":"172.21.15.181:6825/226506989","cluster_addr":"172.21.15.181:6827/226506989","heartbeat_back_addr":"172.21.15.181:6831/226506989","heartbeat_front_addr":"172.21.15.181:6829/226506989","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:39:42.687084+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:40:03.903448+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:40:24.899908+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:40:46.600251+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:41:07.690037+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:41:28.488855+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:41:49.459274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:42:10.863881+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:42:32.248391+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:42:54.112265+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:43:15.634872+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:43:36.922534+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.59:6800/1259916482":"2023-12-12T17:38:14.037922+0000","172.21.15.59:0/17457774":"2023-12-12T17:38:14.037922+0000","172.21.15.59:6801/1672530462":"2023-12-12T17:37:44.773925+0000","172.21.15.59:0/1053944547":"2023-12-12T17:38:14.037922+0000","172.21.15.59:6801/1259916482":"2023-12-12T17:38:14.037922+0000","172.21.15.59:6800/1672530462":"2023-12-12T17:37:44.773925+0000","172.21.15.59:0/3570767650":"2023-12-12T17:37:44.773925+0000","172.21.15.59:0/3853462899":"2023-12-12T17:37:44.773925+0000","172.21.15.59:0/2859415380":"2023-12-12T17:38:14.037922+0000","172.21.15.59:0/2192742819":"2023-12-12T17:37:44.773925+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-11T17:44:27.568 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.565+0000 7f3c9d8a3700 1 -- 172.21.15.59:0/1477563605 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f3c840a1480 msgr2=0x7f3c840a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:27.568 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.565+0000 7f3c9d8a3700 1 --2- 172.21.15.59:0/1477563605 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f3c840a1480 0x7f3c840a3940 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3c80005e10 tx=0x7f3c80005d60 comp rx=0 tx=0).stop 2023-12-11T17:44:27.568 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.565+0000 7f3c9d8a3700 1 -- 172.21.15.59:0/1477563605 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3c98100380 msgr2=0x7f3c981974d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:27.569 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.565+0000 7f3c9d8a3700 1 --2- 172.21.15.59:0/1477563605 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3c98100380 0x7f3c981974d0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f3c8800d980 tx=0x7f3c8800dd40 comp rx=0 tx=0).stop 2023-12-11T17:44:27.569 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.566+0000 7f3c9d8a3700 1 -- 172.21.15.59:0/1477563605 shutdown_connections 2023-12-11T17:44:27.569 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.566+0000 7f3c9d8a3700 1 --2- 172.21.15.59:0/1477563605 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f3c840a1480 0x7f3c840a3940 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:27.569 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.566+0000 7f3c9d8a3700 1 --2- 172.21.15.59:0/1477563605 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3c98101f10 0x7f3c9819ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:27.569 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.566+0000 7f3c9d8a3700 1 --2- 172.21.15.59:0/1477563605 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3c98100380 0x7f3c981974d0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:27.569 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.566+0000 7f3c9d8a3700 1 --2- 172.21.15.59:0/1477563605 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c98101570 0x7f3c98197a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:27.569 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.566+0000 7f3c9d8a3700 1 -- 172.21.15.59:0/1477563605 >> 172.21.15.59:0/1477563605 conn(0x7f3c980fb8f0 msgr2=0x7f3c98068610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:44:27.569 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.566+0000 7f3c9d8a3700 1 -- 172.21.15.59:0/1477563605 shutdown_connections 2023-12-11T17:44:27.569 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.566+0000 7f3c9d8a3700 1 -- 172.21.15.59:0/1477563605 wait complete. 2023-12-11T17:44:27.581 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-12-11T17:44:20.076660+0000', 'modified': '2023-12-11T17:44:25.096263+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24521}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24521': {'gid': 24521, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.115:6839/4122922866', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6838', 'nonce': 4122922866}, {'type': 'v1', 'addr': '172.21.15.115:6839', 'nonce': 4122922866}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1} 2023-12-11T17:44:27.581 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-12-11T17:44:27.582 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-12-11T17:44:27.710 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.707+0000 7f52241dd700 1 -- 172.21.15.59:0/1836485097 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f521c102010 msgr2=0x7f521c1088c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:27.710 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.707+0000 7f52241dd700 1 --2- 172.21.15.59:0/1836485097 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f521c102010 0x7f521c1088c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f521400b0d0 tx=0x7f521400b3e0 comp rx=0 tx=0).stop 2023-12-11T17:44:27.711 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.707+0000 7f52241dd700 1 -- 172.21.15.59:0/1836485097 shutdown_connections 2023-12-11T17:44:27.711 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.707+0000 7f52241dd700 1 --2- 172.21.15.59:0/1836485097 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f521c102010 0x7f521c1088c0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:27.711 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.707+0000 7f52241dd700 1 --2- 172.21.15.59:0/1836485097 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f521c100480 0x7f521c1008a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:27.711 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.707+0000 7f52241dd700 1 --2- 172.21.15.59:0/1836485097 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f521c101670 0x7f521c101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:27.711 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.707+0000 7f52241dd700 1 -- 172.21.15.59:0/1836485097 >> 172.21.15.59:0/1836485097 conn(0x7f521c0fb9f0 msgr2=0x7f521c0fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:44:27.711 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.707+0000 7f52241dd700 1 -- 172.21.15.59:0/1836485097 shutdown_connections 2023-12-11T17:44:27.711 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.707+0000 7f52241dd700 1 -- 172.21.15.59:0/1836485097 wait complete. 2023-12-11T17:44:27.711 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.708+0000 7f52241dd700 1 Processor -- start 2023-12-11T17:44:27.711 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.708+0000 7f52241dd700 1 -- start start 2023-12-11T17:44:27.713 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.708+0000 7f52241dd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f521c100480 0x7f521c1964d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:27.713 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.708+0000 7f52241dd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f521c101670 0x7f521c196a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:27.713 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.708+0000 7f52241dd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f521c102010 0x7f521c19da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:27.714 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.708+0000 7f52241dd700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f521c19e000 con 0x7f521c102010 2023-12-11T17:44:27.714 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.709+0000 7f52241dd700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f521c19e140 con 0x7f521c100480 2023-12-11T17:44:27.714 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.709+0000 7f52241dd700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f521c19e2d0 con 0x7f521c101670 2023-12-11T17:44:27.714 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.709+0000 7f522277a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f521c102010 0x7f521c19da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:27.714 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.709+0000 7f522277a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f521c102010 0x7f521c19da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33008/0 (socket says 172.21.15.59:33008) 2023-12-11T17:44:27.714 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.709+0000 7f522277a700 1 -- 172.21.15.59:0/4013665469 learned_addr learned my addr 172.21.15.59:0/4013665469 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:44:27.714 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.709+0000 7f5221778700 1 --2- 172.21.15.59:0/4013665469 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f521c101670 0x7f521c196a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:27.714 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.709+0000 7f5221f79700 1 --2- 172.21.15.59:0/4013665469 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f521c100480 0x7f521c1964d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:27.714 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.709+0000 7f522277a700 1 -- 172.21.15.59:0/4013665469 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f521c101670 msgr2=0x7f521c196a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:27.714 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.709+0000 7f522277a700 1 --2- 172.21.15.59:0/4013665469 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f521c101670 0x7f521c196a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:27.714 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.709+0000 7f522277a700 1 -- 172.21.15.59:0/4013665469 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f521c100480 msgr2=0x7f521c1964d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:27.715 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.709+0000 7f522277a700 1 --2- 172.21.15.59:0/4013665469 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f521c100480 0x7f521c1964d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:27.715 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.709+0000 7f522277a700 1 -- 172.21.15.59:0/4013665469 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5214009d00 con 0x7f521c102010 2023-12-11T17:44:27.715 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.709+0000 7f5221f79700 1 --2- 172.21.15.59:0/4013665469 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f521c100480 0x7f521c1964d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T17:44:27.715 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.709+0000 7f522277a700 1 --2- 172.21.15.59:0/4013665469 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f521c102010 0x7f521c19da30 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f5214007b60 tx=0x7f5214008f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:44:27.715 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.709+0000 7f5212ffd700 1 -- 172.21.15.59:0/4013665469 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f521400e040 con 0x7f521c102010 2023-12-11T17:44:27.715 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.709+0000 7f52241dd700 1 -- 172.21.15.59:0/4013665469 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f521c19e550 con 0x7f521c102010 2023-12-11T17:44:27.715 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.709+0000 7f5212ffd700 1 -- 172.21.15.59:0/4013665469 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f521400be30 con 0x7f521c102010 2023-12-11T17:44:27.715 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.709+0000 7f5212ffd700 1 -- 172.21.15.59:0/4013665469 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f521401bae0 con 0x7f521c102010 2023-12-11T17:44:27.715 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.709+0000 7f52241dd700 1 -- 172.21.15.59:0/4013665469 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f521c19e980 con 0x7f521c102010 2023-12-11T17:44:27.723 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.711+0000 7f5212ffd700 1 -- 172.21.15.59:0/4013665469 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f521401d670 con 0x7f521c102010 2023-12-11T17:44:27.723 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.712+0000 7f5212ffd700 1 --2- 172.21.15.59:0/4013665469 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f52080a1480 0x7f52080a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:27.723 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.712+0000 7f5221f79700 1 --2- 172.21.15.59:0/4013665469 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f52080a1480 0x7f52080a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:27.723 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.712+0000 7f5212ffd700 1 -- 172.21.15.59:0/4013665469 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f52140ce7c0 con 0x7f521c102010 2023-12-11T17:44:27.723 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.713+0000 7f5221f79700 1 --2- 172.21.15.59:0/4013665469 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f52080a1480 0x7f52080a3940 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f521c1014d0 tx=0x7f5218009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:44:27.723 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.713+0000 7f52303d6b80 1 Processor -- start 2023-12-11T17:44:27.723 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.713+0000 7f52303d6b80 1 -- start start 2023-12-11T17:44:27.724 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.713+0000 7f52303d6b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e02f1f4590 0x55e02f1f49b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:27.724 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.713+0000 7f52303d6b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e02f1f4ef0 0x55e02f1f5800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:27.724 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.713+0000 7f52303d6b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e02f1f5de0 0x55e02f1f6260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:27.724 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.713+0000 7f52303d6b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e02f14db00 con 0x55e02f1f4590 2023-12-11T17:44:27.724 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.713+0000 7f52303d6b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e02f14b420 con 0x55e02f1f4ef0 2023-12-11T17:44:27.724 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.713+0000 7f52303d6b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e02f146d10 con 0x55e02f1f5de0 2023-12-11T17:44:27.724 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.713+0000 7f5221f79700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e02f1f4590 0x55e02f1f49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:27.724 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.713+0000 7f5221f79700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e02f1f4590 0x55e02f1f49b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33012/0 (socket says 172.21.15.59:33012) 2023-12-11T17:44:27.724 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.713+0000 7f5221f79700 1 -- 172.21.15.59:0/3046677347 learned_addr learned my addr 172.21.15.59:0/3046677347 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:44:27.724 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.713+0000 7f522277a700 1 --2- 172.21.15.59:0/3046677347 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e02f1f5de0 0x55e02f1f6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:27.724 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.713+0000 7f5221778700 1 --2- 172.21.15.59:0/3046677347 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e02f1f4ef0 0x55e02f1f5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:27.725 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.714+0000 7f5221f79700 1 -- 172.21.15.59:0/3046677347 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e02f1f5de0 msgr2=0x55e02f1f6260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T17:44:27.725 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.714+0000 7f5221f79700 1 --2- 172.21.15.59:0/3046677347 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e02f1f5de0 0x55e02f1f6260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:27.725 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.714+0000 7f5221f79700 1 -- 172.21.15.59:0/3046677347 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e02f1f4ef0 msgr2=0x55e02f1f5800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T17:44:27.725 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.714+0000 7f5221f79700 1 --2- 172.21.15.59:0/3046677347 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e02f1f4ef0 0x55e02f1f5800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:27.725 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.714+0000 7f5221f79700 1 -- 172.21.15.59:0/3046677347 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e02efc35e0 con 0x55e02f1f4590 2023-12-11T17:44:27.725 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.714+0000 7f5221f79700 1 --2- 172.21.15.59:0/3046677347 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e02f1f4590 0x55e02f1f49b0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f5218011ed0 tx=0x7f5218014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=392f3e14346d2d62 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:44:27.725 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.714+0000 7f5207fff700 1 -- 172.21.15.59:0/3046677347 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f521801f620 con 0x55e02f1f4590 2023-12-11T17:44:27.725 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.714+0000 7f5207fff700 1 -- 172.21.15.59:0/3046677347 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5218016070 con 0x55e02f1f4590 2023-12-11T17:44:27.725 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.714+0000 7f5207fff700 1 -- 172.21.15.59:0/3046677347 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f521801daf0 con 0x55e02f1f4590 2023-12-11T17:44:27.725 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.714+0000 7f52303d6b80 1 -- 172.21.15.59:0/3046677347 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e02f1f4590 msgr2=0x55e02f1f49b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:27.725 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.715+0000 7f52303d6b80 1 --2- 172.21.15.59:0/3046677347 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e02f1f4590 0x55e02f1f49b0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f5218011ed0 tx=0x7f5218014000 comp rx=0 tx=0).stop 2023-12-11T17:44:27.725 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.715+0000 7f52303d6b80 1 -- 172.21.15.59:0/3046677347 shutdown_connections 2023-12-11T17:44:27.726 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.715+0000 7f52303d6b80 1 --2- 172.21.15.59:0/3046677347 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e02f1f5de0 0x55e02f1f6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:27.726 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.715+0000 7f52303d6b80 1 --2- 172.21.15.59:0/3046677347 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e02f1f4590 0x55e02f1f49b0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:27.726 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.715+0000 7f52303d6b80 1 --2- 172.21.15.59:0/3046677347 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e02f1f4ef0 0x55e02f1f5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:27.726 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.715+0000 7f52303d6b80 1 -- 172.21.15.59:0/3046677347 >> 172.21.15.59:0/3046677347 conn(0x55e02f121790 msgr2=0x55e02f1eed40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:44:27.726 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.715+0000 7f52303d6b80 1 -- 172.21.15.59:0/3046677347 shutdown_connections 2023-12-11T17:44:27.726 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.715+0000 7f52303d6b80 1 -- 172.21.15.59:0/3046677347 wait complete. 2023-12-11T17:44:27.726 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.715+0000 7f52303d6b80 1 -- start start 2023-12-11T17:44:27.726 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.715+0000 7f52303d6b80 1 Processor -- start 2023-12-11T17:44:27.726 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.715+0000 7f52303d6b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e02f1f4590 0x55e02f1fe330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:27.726 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.715+0000 7f52303d6b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e02f1f4ef0 0x55e02f1fe870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:27.726 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.715+0000 7f52303d6b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e02f1f5de0 0x55e02f2058b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:27.727 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.715+0000 7f52303d6b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e02f100510 con 0x55e02f1f4590 2023-12-11T17:44:27.727 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.715+0000 7f52303d6b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e02f1001b0 con 0x55e02f1f4ef0 2023-12-11T17:44:27.727 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.715+0000 7f52303d6b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e02efc58d0 con 0x55e02f1f5de0 2023-12-11T17:44:27.727 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.716+0000 7f52303d6b80 10 client.0 _ll_register_callbacks cb 0x55e02f02ba30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T17:44:27.727 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.716+0000 7f5221f79700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e02f1f4590 0x55e02f1fe330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:27.727 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.716+0000 7f5221f79700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e02f1f4590 0x55e02f1fe330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33018/0 (socket says 172.21.15.59:33018) 2023-12-11T17:44:27.727 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.716+0000 7f5221f79700 1 -- 172.21.15.59:0/246975211 learned_addr learned my addr 172.21.15.59:0/246975211 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:44:27.727 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.716+0000 7f522277a700 1 --2- 172.21.15.59:0/246975211 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e02f1f5de0 0x55e02f2058b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:27.727 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.716+0000 7f5221778700 1 --2- 172.21.15.59:0/246975211 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e02f1f4ef0 0x55e02f1fe870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:27.727 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.716+0000 7f5221f79700 1 -- 172.21.15.59:0/246975211 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e02f1f5de0 msgr2=0x55e02f2058b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T17:44:27.727 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.716+0000 7f5221f79700 1 --2- 172.21.15.59:0/246975211 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e02f1f5de0 0x55e02f2058b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:27.727 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.716+0000 7f5221f79700 1 -- 172.21.15.59:0/246975211 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e02f1f4ef0 msgr2=0x55e02f1fe870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T17:44:27.728 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.716+0000 7f5221f79700 1 --2- 172.21.15.59:0/246975211 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e02f1f4ef0 0x55e02f1fe870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:27.728 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.716+0000 7f5221f79700 1 -- 172.21.15.59:0/246975211 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5218011c60 con 0x55e02f1f4590 2023-12-11T17:44:27.728 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.716+0000 7f5221f79700 1 --2- 172.21.15.59:0/246975211 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e02f1f4590 0x55e02f1fe330 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f5218005d80 tx=0x7f5218011f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cd3555519c2e18 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:44:27.728 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.716+0000 7f52057fa700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2023-12-11T17:44:27.728 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.716+0000 7f52057fa700 1 -- 172.21.15.59:0/246975211 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f521802d070 con 0x55e02f1f4590 2023-12-11T17:44:27.728 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.716+0000 7f52057fa700 1 -- 172.21.15.59:0/246975211 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f5218018070 con 0x55e02f1f4590 2023-12-11T17:44:27.728 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.716+0000 7f52303d6b80 1 -- 172.21.15.59:0/246975211 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e02f1e5af0 con 0x55e02f1f4590 2023-12-11T17:44:27.728 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.716+0000 7f52057fa700 1 -- 172.21.15.59:0/246975211 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5218005260 con 0x55e02f1f4590 2023-12-11T17:44:27.728 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.716+0000 7f52303d6b80 10 client.14715 fetch_fsmap learned FSMap version 17 2023-12-11T17:44:27.728 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.716+0000 7f52303d6b80 1 -- 172.21.15.59:0/246975211 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) v3 -- 0x55e02ef6fd90 con 0x55e02f1f4590 2023-12-11T17:44:27.728 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.717+0000 7f52057fa700 1 -- 172.21.15.59:0/246975211 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e02f1e5af0 con 0x55e02f1f4590 2023-12-11T17:44:27.729 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.717+0000 7f52057fa700 1 -- 172.21.15.59:0/246975211 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 17) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f521801f420 con 0x55e02f1f4590 2023-12-11T17:44:27.729 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.717+0000 7f52303d6b80 10 client.14715 fetch_fsmap finished waiting for FSMap version 17 2023-12-11T17:44:27.729 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.717+0000 7f52303d6b80 10 client.14715 resolve_mds: resolved name 'b' to daemon mds.b 2023-12-11T17:44:27.729 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.717+0000 7f52303d6b80 20 client.14715 populate_metadata read hostname 'smithi059' 2023-12-11T17:44:27.729 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.717+0000 7f52303d6b80 1 --2- 172.21.15.59:0/246975211 >> [v2:172.21.15.115:6838/4122922866,v1:172.21.15.115:6839/4122922866] conn(0x55e02f209f60 0x55e02f2089f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:27.729 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.717+0000 7f52303d6b80 4 client.14715 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-11T17:44:27.729 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.717+0000 7f52303d6b80 1 -- 172.21.15.59:0/246975211 --> [v2:172.21.15.115:6838/4122922866,v1:172.21.15.115:6839/4122922866] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e02f1c1ef0 con 0x55e02f209f60 2023-12-11T17:44:27.729 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.717+0000 7f5221778700 1 --2- 172.21.15.59:0/246975211 >> [v2:172.21.15.115:6838/4122922866,v1:172.21.15.115:6839/4122922866] conn(0x55e02f209f60 0x55e02f2089f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:27.729 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.718+0000 7f52057fa700 10 client.14715 ms_handle_connect on v2:172.21.15.115:6838/4122922866 2023-12-11T17:44:27.729 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.718+0000 7f5221778700 1 --2- 172.21.15.59:0/246975211 >> [v2:172.21.15.115:6838/4122922866,v1:172.21.15.115:6839/4122922866] conn(0x55e02f209f60 0x55e02f2089f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13155c3a725eeda3 server_cookie=22f1e02adfa132ed in_seq=0 out_seq=0 2023-12-11T17:44:27.729 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.719+0000 7f52057fa700 1 -- 172.21.15.59:0/246975211 <== mds.0 v2:172.21.15.115:6838/4122922866 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e02f1c1ef0 con 0x55e02f209f60 2023-12-11T17:44:27.729 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.719+0000 7f52057fa700 10 client.14715 handle_command_reply: tid=0 2023-12-11T17:44:27.729 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.719+0000 7f52303d6b80 1 client.14715 shutdown 2023-12-11T17:44:27.730 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.719+0000 7f52303d6b80 1 -- 172.21.15.59:0/246975211 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e02f1f4590 msgr2=0x55e02f1fe330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:27.730 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.719+0000 7f52303d6b80 1 --2- 172.21.15.59:0/246975211 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e02f1f4590 0x55e02f1fe330 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f5218005d80 tx=0x7f5218011f00 comp rx=0 tx=0).stop 2023-12-11T17:44:27.730 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.719+0000 7f52303d6b80 1 -- 172.21.15.59:0/246975211 shutdown_connections 2023-12-11T17:44:27.730 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.719+0000 7f52303d6b80 1 --2- 172.21.15.59:0/246975211 >> [v2:172.21.15.115:6838/4122922866,v1:172.21.15.115:6839/4122922866] conn(0x55e02f209f60 0x55e02f2089f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:27.730 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.719+0000 7f52303d6b80 1 --2- 172.21.15.59:0/246975211 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e02f1f5de0 0x55e02f2058b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:27.730 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.719+0000 7f52057fa700 0 client.14715 ms_handle_reset on v2:172.21.15.115:6838/4122922866 2023-12-11T17:44:27.730 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.719+0000 7f52303d6b80 1 --2- 172.21.15.59:0/246975211 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e02f1f4590 0x55e02f1fe330 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:27.730 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.719+0000 7f52303d6b80 1 --2- 172.21.15.59:0/246975211 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e02f1f4ef0 0x55e02f1fe870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:27.731 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.719+0000 7f52303d6b80 1 -- 172.21.15.59:0/246975211 >> 172.21.15.59:0/246975211 conn(0x55e02f121790 msgr2=0x55e02f1ed9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:44:27.731 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.719+0000 7f52303d6b80 1 -- 172.21.15.59:0/246975211 shutdown_connections 2023-12-11T17:44:27.731 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.719+0000 7f52303d6b80 1 -- 172.21.15.59:0/246975211 wait complete. 2023-12-11T17:44:27.731 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.719+0000 7f52303d6b80 20 client.14715 trim_cache size 0 max 16384 2023-12-11T17:44:27.744 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.737+0000 7f52303d6b80 1 Processor -- start 2023-12-11T17:44:27.744 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.737+0000 7f52303d6b80 1 -- start start 2023-12-11T17:44:27.745 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.737+0000 7f52303d6b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e02f1f4590 0x55e02f1fd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:27.745 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.737+0000 7f52303d6b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e02f1f4ef0 0x55e02f205870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:27.745 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.737+0000 7f52303d6b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e02f1f5de0 0x55e02f1fda30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:27.745 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.737+0000 7f52303d6b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e02f0e9830 con 0x55e02f1f4590 2023-12-11T17:44:27.745 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.737+0000 7f52303d6b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e02f1e9740 con 0x55e02f1f4ef0 2023-12-11T17:44:27.745 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.737+0000 7f52303d6b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e02f206a30 con 0x55e02f1f5de0 2023-12-11T17:44:27.745 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.737+0000 7f5221f79700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e02f1f4590 0x55e02f1fd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:27.745 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.737+0000 7f5221f79700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e02f1f4590 0x55e02f1fd4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33026/0 (socket says 172.21.15.59:33026) 2023-12-11T17:44:27.745 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.737+0000 7f5221f79700 1 -- 172.21.15.59:0/795923313 learned_addr learned my addr 172.21.15.59:0/795923313 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:44:27.745 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.737+0000 7f5221778700 1 --2- 172.21.15.59:0/795923313 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e02f1f4ef0 0x55e02f205870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:27.745 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.737+0000 7f522277a700 1 --2- 172.21.15.59:0/795923313 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e02f1f5de0 0x55e02f1fda30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:27.745 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.737+0000 7f5221f79700 1 -- 172.21.15.59:0/795923313 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e02f1f5de0 msgr2=0x55e02f1fda30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T17:44:27.746 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.737+0000 7f5221f79700 1 --2- 172.21.15.59:0/795923313 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e02f1f5de0 0x55e02f1fda30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:27.746 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.737+0000 7f5221f79700 1 -- 172.21.15.59:0/795923313 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e02f1f4ef0 msgr2=0x55e02f205870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T17:44:27.746 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.738+0000 7f5221f79700 1 --2- 172.21.15.59:0/795923313 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e02f1f4ef0 0x55e02f205870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:27.746 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.738+0000 7f5221f79700 1 -- 172.21.15.59:0/795923313 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e02ef58c10 con 0x55e02f1f4590 2023-12-11T17:44:27.746 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.738+0000 7f522277a700 1 --2- 172.21.15.59:0/795923313 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e02f1f5de0 0x55e02f1fda30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-11T17:44:27.746 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.738+0000 7f5221f79700 1 --2- 172.21.15.59:0/795923313 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e02f1f4590 0x55e02f1fd4f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f5218009840 tx=0x7f5218011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5a83decfa54616 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:44:27.746 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.738+0000 7f5210ff9700 1 -- 172.21.15.59:0/795923313 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f521802d070 con 0x55e02f1f4590 2023-12-11T17:44:27.746 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.738+0000 7f5210ff9700 1 -- 172.21.15.59:0/795923313 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5218009350 con 0x55e02f1f4590 2023-12-11T17:44:27.746 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.738+0000 7f5210ff9700 1 -- 172.21.15.59:0/795923313 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5218005680 con 0x55e02f1f4590 2023-12-11T17:44:27.746 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.738+0000 7f52303d6b80 1 -- 172.21.15.59:0/795923313 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e02f1f4590 msgr2=0x55e02f1fd4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:27.747 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.738+0000 7f52303d6b80 1 --2- 172.21.15.59:0/795923313 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e02f1f4590 0x55e02f1fd4f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f5218009840 tx=0x7f5218011a50 comp rx=0 tx=0).stop 2023-12-11T17:44:27.747 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.738+0000 7f52303d6b80 1 -- 172.21.15.59:0/795923313 shutdown_connections 2023-12-11T17:44:27.747 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.738+0000 7f52303d6b80 1 --2- 172.21.15.59:0/795923313 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e02f1f5de0 0x55e02f1fda30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:27.747 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.738+0000 7f52303d6b80 1 --2- 172.21.15.59:0/795923313 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e02f1f4590 0x55e02f1fd4f0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:27.747 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.738+0000 7f52303d6b80 1 --2- 172.21.15.59:0/795923313 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e02f1f4ef0 0x55e02f205870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:27.747 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.738+0000 7f52303d6b80 1 -- 172.21.15.59:0/795923313 >> 172.21.15.59:0/795923313 conn(0x55e02f121790 msgr2=0x55e02f1f8d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:44:27.747 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.739+0000 7f52303d6b80 1 -- 172.21.15.59:0/795923313 shutdown_connections 2023-12-11T17:44:27.747 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.739+0000 7f52303d6b80 1 -- 172.21.15.59:0/795923313 wait complete. 2023-12-11T17:44:27.747 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.739+0000 7f52303d6b80 1 -- start start 2023-12-11T17:44:27.747 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.739+0000 7f52303d6b80 1 Processor -- start 2023-12-11T17:44:27.747 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.739+0000 7f52303d6b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e02f1f4590 0x55e02f1ff9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:27.748 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.739+0000 7f52303d6b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e02f1f4ef0 0x55e02f1fffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:27.748 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.739+0000 7f52303d6b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e02f1f5de0 0x55e02f204c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:27.748 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.739+0000 7f52303d6b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e02f2051d0 con 0x55e02f1f5de0 2023-12-11T17:44:27.748 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.739+0000 7f52303d6b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e02f205340 con 0x55e02f1f4590 2023-12-11T17:44:27.748 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.739+0000 7f52303d6b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e02f205500 con 0x55e02f1f4ef0 2023-12-11T17:44:27.748 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.739+0000 7f52303d6b80 10 client.0 _ll_register_callbacks cb 0x55e02f02ba30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T17:44:27.748 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.739+0000 7f522277a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e02f1f5de0 0x55e02f204c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:27.748 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.739+0000 7f522277a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e02f1f5de0 0x55e02f204c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33036/0 (socket says 172.21.15.59:33036) 2023-12-11T17:44:27.748 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.739+0000 7f522277a700 1 -- 172.21.15.59:0/3878033467 learned_addr learned my addr 172.21.15.59:0/3878033467 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:44:27.748 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.739+0000 7f5221f79700 1 --2- 172.21.15.59:0/3878033467 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e02f1f4590 0x55e02f1ff9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:27.748 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.740+0000 7f5221778700 1 --2- 172.21.15.59:0/3878033467 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e02f1f4ef0 0x55e02f1fffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:27.749 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.740+0000 7f522277a700 1 -- 172.21.15.59:0/3878033467 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e02f1f4ef0 msgr2=0x55e02f1fffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T17:44:27.749 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.740+0000 7f522277a700 1 --2- 172.21.15.59:0/3878033467 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e02f1f4ef0 0x55e02f1fffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:27.749 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.740+0000 7f522277a700 1 -- 172.21.15.59:0/3878033467 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e02f1f4590 msgr2=0x55e02f1ff9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T17:44:27.749 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.740+0000 7f522277a700 1 --2- 172.21.15.59:0/3878033467 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e02f1f4590 0x55e02f1ff9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:27.749 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.740+0000 7f522277a700 1 -- 172.21.15.59:0/3878033467 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e02ef6fd90 con 0x55e02f1f5de0 2023-12-11T17:44:27.749 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.740+0000 7f5221778700 1 --2- 172.21.15.59:0/3878033467 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e02f1f4ef0 0x55e02f1fffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-11T17:44:27.749 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.740+0000 7f522277a700 1 --2- 172.21.15.59:0/3878033467 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e02f1f5de0 0x55e02f204c90 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f5214098390 tx=0x7f521409b570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1891bed5af012fc server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:44:27.749 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.740+0000 7f52057fa700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2023-12-11T17:44:27.749 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.740+0000 7f52057fa700 1 -- 172.21.15.59:0/3878033467 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f521409fa70 con 0x55e02f1f5de0 2023-12-11T17:44:27.749 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.740+0000 7f52057fa700 1 -- 172.21.15.59:0/3878033467 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f52140a3c20 con 0x55e02f1f5de0 2023-12-11T17:44:27.749 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.740+0000 7f52303d6b80 1 -- 172.21.15.59:0/3878033467 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5218016070 con 0x55e02f1f5de0 2023-12-11T17:44:27.749 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.740+0000 7f52057fa700 1 -- 172.21.15.59:0/3878033467 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f521401d920 con 0x55e02f1f5de0 2023-12-11T17:44:27.750 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.740+0000 7f52303d6b80 10 client.14721 fetch_fsmap learned FSMap version 17 2023-12-11T17:44:27.750 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.740+0000 7f52303d6b80 1 -- 172.21.15.59:0/3878033467 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) v3 -- 0x55e02ef96200 con 0x55e02f1f5de0 2023-12-11T17:44:27.750 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.740+0000 7f52057fa700 1 -- 172.21.15.59:0/3878033467 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5218016070 con 0x55e02f1f5de0 2023-12-11T17:44:27.750 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.740+0000 7f52057fa700 1 -- 172.21.15.59:0/3878033467 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 17) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f521409ae10 con 0x55e02f1f5de0 2023-12-11T17:44:27.750 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.741+0000 7f52303d6b80 10 client.14721 fetch_fsmap finished waiting for FSMap version 17 2023-12-11T17:44:27.750 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.741+0000 7f52303d6b80 10 client.14721 resolve_mds: resolved name 'b' to daemon mds.b 2023-12-11T17:44:27.750 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.741+0000 7f52303d6b80 20 client.14721 populate_metadata read hostname 'smithi059' 2023-12-11T17:44:27.750 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.741+0000 7f52303d6b80 1 --2- 172.21.15.59:0/3878033467 >> [v2:172.21.15.115:6838/4122922866,v1:172.21.15.115:6839/4122922866] conn(0x55e02f264770 0x55e02f260bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:27.750 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.741+0000 7f52303d6b80 4 client.14721 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "status"}] 2023-12-11T17:44:27.750 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.741+0000 7f52303d6b80 1 -- 172.21.15.59:0/3878033467 --> [v2:172.21.15.115:6838/4122922866,v1:172.21.15.115:6839/4122922866] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55e02f10cf80 con 0x55e02f264770 2023-12-11T17:44:27.750 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.741+0000 7f5221f79700 1 --2- 172.21.15.59:0/3878033467 >> [v2:172.21.15.115:6838/4122922866,v1:172.21.15.115:6839/4122922866] conn(0x55e02f264770 0x55e02f260bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:27.750 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.741+0000 7f52057fa700 10 client.14721 ms_handle_connect on v2:172.21.15.115:6838/4122922866 2023-12-11T17:44:27.751 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.741+0000 7f5221f79700 1 --2- 172.21.15.59:0/3878033467 >> [v2:172.21.15.115:6838/4122922866,v1:172.21.15.115:6839/4122922866] conn(0x55e02f264770 0x55e02f260bd0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4d290f7051019a9 server_cookie=4b244bfccb618029 in_seq=0 out_seq=0 2023-12-11T17:44:27.751 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.742+0000 7f52057fa700 1 -- 172.21.15.59:0/3878033467 <== mds.0 v2:172.21.15.115:6838/4122922866 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x55e02f10cf80 con 0x55e02f264770 2023-12-11T17:44:27.751 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.742+0000 7f52057fa700 10 client.14721 handle_command_reply: tid=0 2023-12-11T17:44:27.751 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.742+0000 7f52303d6b80 1 client.14721 shutdown 2023-12-11T17:44:27.751 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.742+0000 7f52303d6b80 1 -- 172.21.15.59:0/3878033467 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e02f1f5de0 msgr2=0x55e02f204c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:27.751 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.742+0000 7f52303d6b80 1 --2- 172.21.15.59:0/3878033467 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e02f1f5de0 0x55e02f204c90 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f5214098390 tx=0x7f521409b570 comp rx=0 tx=0).stop 2023-12-11T17:44:27.751 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.742+0000 7f52303d6b80 1 -- 172.21.15.59:0/3878033467 shutdown_connections 2023-12-11T17:44:27.751 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.742+0000 7f52303d6b80 1 --2- 172.21.15.59:0/3878033467 >> [v2:172.21.15.115:6838/4122922866,v1:172.21.15.115:6839/4122922866] conn(0x55e02f264770 0x55e02f260bd0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:27.751 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.742+0000 7f52303d6b80 1 --2- 172.21.15.59:0/3878033467 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e02f1f5de0 0x55e02f204c90 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:27.751 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.742+0000 7f52303d6b80 1 --2- 172.21.15.59:0/3878033467 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e02f1f4590 0x55e02f1ff9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:27.751 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.742+0000 7f52303d6b80 1 --2- 172.21.15.59:0/3878033467 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e02f1f4ef0 0x55e02f1fffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:27.751 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.742+0000 7f52057fa700 0 client.14721 ms_handle_reset on v2:172.21.15.115:6838/4122922866 2023-12-11T17:44:27.751 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.742+0000 7f52303d6b80 1 -- 172.21.15.59:0/3878033467 >> 172.21.15.59:0/3878033467 conn(0x55e02f121790 msgr2=0x55e02f1f8d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:44:27.751 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.742+0000 7f52303d6b80 1 -- 172.21.15.59:0/3878033467 shutdown_connections 2023-12-11T17:44:27.751 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.742+0000 7f52303d6b80 1 -- 172.21.15.59:0/3878033467 wait complete. 2023-12-11T17:44:27.752 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.742+0000 7f52303d6b80 20 client.14721 trim_cache size 0 max 16384 2023-12-11T17:44:27.752 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2023-12-11T17:44:27.752 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_fsid": "d0a07eaa-984b-11ee-95a3-87774f69a715", 2023-12-11T17:44:27.752 INFO:teuthology.orchestra.run.smithi059.stdout: "whoami": 0, 2023-12-11T17:44:27.752 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 24521, 2023-12-11T17:44:27.752 INFO:teuthology.orchestra.run.smithi059.stdout: "want_state": "up:active", 2023-12-11T17:44:27.752 INFO:teuthology.orchestra.run.smithi059.stdout: "state": "up:active", 2023-12-11T17:44:27.752 INFO:teuthology.orchestra.run.smithi059.stdout: "fs_name": "cephfs", 2023-12-11T17:44:27.752 INFO:teuthology.orchestra.run.smithi059.stdout: "rank_uptime": 7.0952546249999999, 2023-12-11T17:44:27.753 INFO:teuthology.orchestra.run.smithi059.stdout: "mdsmap_epoch": 17, 2023-12-11T17:44:27.753 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch": 86, 2023-12-11T17:44:27.753 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch_barrier": 85, 2023-12-11T17:44:27.753 INFO:teuthology.orchestra.run.smithi059.stdout: "uptime": 12.574712597 2023-12-11T17:44:27.753 INFO:teuthology.orchestra.run.smithi059.stdout:} 2023-12-11T17:44:27.754 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.742+0000 7f52241dd700 1 -- 172.21.15.59:0/4013665469 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f52080a1480 msgr2=0x7f52080a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:27.754 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.742+0000 7f52241dd700 1 --2- 172.21.15.59:0/4013665469 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f52080a1480 0x7f52080a3940 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f521c1014d0 tx=0x7f5218009260 comp rx=0 tx=0).stop 2023-12-11T17:44:27.754 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.742+0000 7f52241dd700 1 -- 172.21.15.59:0/4013665469 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f521c102010 msgr2=0x7f521c19da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:27.754 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.742+0000 7f52241dd700 1 --2- 172.21.15.59:0/4013665469 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f521c102010 0x7f521c19da30 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f5214007b60 tx=0x7f5214008f10 comp rx=0 tx=0).stop 2023-12-11T17:44:27.754 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.742+0000 7f52241dd700 1 -- 172.21.15.59:0/4013665469 shutdown_connections 2023-12-11T17:44:27.754 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.743+0000 7f52241dd700 1 --2- 172.21.15.59:0/4013665469 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f52080a1480 0x7f52080a3940 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:27.754 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.743+0000 7f52241dd700 1 --2- 172.21.15.59:0/4013665469 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f521c102010 0x7f521c19da30 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:27.755 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.743+0000 7f52241dd700 1 --2- 172.21.15.59:0/4013665469 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f521c100480 0x7f521c1964d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:27.755 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.743+0000 7f52241dd700 1 --2- 172.21.15.59:0/4013665469 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f521c101670 0x7f521c196a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:27.755 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.743+0000 7f52241dd700 1 -- 172.21.15.59:0/4013665469 >> 172.21.15.59:0/4013665469 conn(0x7f521c0fb9f0 msgr2=0x7f521c104970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:44:27.755 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.743+0000 7f52241dd700 1 -- 172.21.15.59:0/4013665469 shutdown_connections 2023-12-11T17:44:27.755 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:27.743+0000 7f52241dd700 1 -- 172.21.15.59:0/4013665469 wait complete. 2023-12-11T17:44:27.757 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-12-11T17:44:27.757 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2023-12-11T17:44:27.758 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell --fsid d0a07eaa-984b-11ee-95a3-87774f69a715 -- ceph fs ls 2023-12-11T17:44:27.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:27 smithi059 ceph-mon[98847]: pgmap v262: 129 pgs: 4 creating+activating, 5 creating+peering, 107 active+clean, 13 unknown; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 8 op/s 2023-12-11T17:44:27.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:27 smithi059 ceph-mon[98847]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2023-12-11T17:44:27.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:27 smithi059 ceph-mon[98847]: Cluster is now healthy 2023-12-11T17:44:27.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:27 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/797105838' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-11T17:44:27.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:27 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/1496931243' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T17:44:27.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:27 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/1477563605' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-11T17:44:28.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:27 smithi115 ceph-mon[102748]: pgmap v262: 129 pgs: 4 creating+activating, 5 creating+peering, 107 active+clean, 13 unknown; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 8 op/s 2023-12-11T17:44:28.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:27 smithi115 ceph-mon[102748]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2023-12-11T17:44:28.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:27 smithi115 ceph-mon[102748]: Cluster is now healthy 2023-12-11T17:44:28.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:27 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/797105838' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-11T17:44:28.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:27 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/1496931243' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T17:44:28.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:27 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/1477563605' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-11T17:44:28.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:27 smithi181 ceph-mon[103627]: pgmap v262: 129 pgs: 4 creating+activating, 5 creating+peering, 107 active+clean, 13 unknown; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 8 op/s 2023-12-11T17:44:28.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:27 smithi181 ceph-mon[103627]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2023-12-11T17:44:28.146 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:27 smithi181 ceph-mon[103627]: Cluster is now healthy 2023-12-11T17:44:28.146 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:27 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/797105838' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-11T17:44:28.146 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:27 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/1496931243' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T17:44:28.146 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:27 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/1477563605' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-11T17:44:28.354 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/mon.a/config 2023-12-11T17:44:29.692 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:29.689+0000 7fcc7116b700 1 -- 172.21.15.59:0/3856713343 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc6c101680 msgr2=0x7fcc6c101b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:29.692 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:29.689+0000 7fcc7116b700 1 --2- 172.21.15.59:0/3856713343 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc6c101680 0x7fcc6c101b00 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fcc60009990 tx=0x7fcc60009ca0 comp rx=0 tx=0).stop 2023-12-11T17:44:29.692 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:29.689+0000 7fcc7116b700 1 -- 172.21.15.59:0/3856713343 shutdown_connections 2023-12-11T17:44:29.692 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:29.689+0000 7fcc7116b700 1 --2- 172.21.15.59:0/3856713343 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcc6c102040 0x7fcc6c10e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:29.692 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:29.689+0000 7fcc7116b700 1 --2- 172.21.15.59:0/3856713343 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcc6c1076f0 0x7fcc6c107ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:29.692 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:29.689+0000 7fcc7116b700 1 --2- 172.21.15.59:0/3856713343 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc6c101680 0x7fcc6c101b00 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:29.692 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:29.689+0000 7fcc7116b700 1 -- 172.21.15.59:0/3856713343 >> 172.21.15.59:0/3856713343 conn(0x7fcc6c0787c0 msgr2=0x7fcc6c078bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:44:29.692 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:29.689+0000 7fcc7116b700 1 -- 172.21.15.59:0/3856713343 shutdown_connections 2023-12-11T17:44:29.692 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:29.689+0000 7fcc7116b700 1 -- 172.21.15.59:0/3856713343 wait complete. 2023-12-11T17:44:29.694 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:29.690+0000 7fcc7116b700 1 Processor -- start 2023-12-11T17:44:29.694 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:29.690+0000 7fcc7116b700 1 -- start start 2023-12-11T17:44:29.694 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:29.690+0000 7fcc7116b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcc6c101680 0x7fcc6c0738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:29.694 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:29.690+0000 7fcc7116b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcc6c102040 0x7fcc6c073e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:29.695 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:29.690+0000 7fcc7116b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc6c1076f0 0x7fcc6c074350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:29.695 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:29.690+0000 7fcc7116b700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc6c1a5d20 con 0x7fcc6c1076f0 2023-12-11T17:44:29.695 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:29.690+0000 7fcc7116b700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc6c1a5e90 con 0x7fcc6c101680 2023-12-11T17:44:29.695 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:29.690+0000 7fcc7116b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc6c1a6050 con 0x7fcc6c102040 2023-12-11T17:44:29.695 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:29.690+0000 7fcc6b59e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc6c1076f0 0x7fcc6c074350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:29.695 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:29.690+0000 7fcc6a59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcc6c102040 0x7fcc6c073e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:29.695 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:29.690+0000 7fcc6b59e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc6c1076f0 0x7fcc6c074350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33050/0 (socket says 172.21.15.59:33050) 2023-12-11T17:44:29.695 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:29.690+0000 7fcc6ad9d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcc6c101680 0x7fcc6c0738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:29.695 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:29.690+0000 7fcc6b59e700 1 -- 172.21.15.59:0/3522277034 learned_addr learned my addr 172.21.15.59:0/3522277034 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:44:29.695 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:29.691+0000 7fcc6b59e700 1 -- 172.21.15.59:0/3522277034 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcc6c102040 msgr2=0x7fcc6c073e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:29.695 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:29.691+0000 7fcc6b59e700 1 --2- 172.21.15.59:0/3522277034 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcc6c102040 0x7fcc6c073e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:29.695 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:29.691+0000 7fcc6b59e700 1 -- 172.21.15.59:0/3522277034 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcc6c101680 msgr2=0x7fcc6c0738d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:29.696 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:29.691+0000 7fcc6b59e700 1 --2- 172.21.15.59:0/3522277034 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcc6c101680 0x7fcc6c0738d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:29.696 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:29.691+0000 7fcc6b59e700 1 -- 172.21.15.59:0/3522277034 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc600095f0 con 0x7fcc6c1076f0 2023-12-11T17:44:29.696 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:29.691+0000 7fcc6b59e700 1 --2- 172.21.15.59:0/3522277034 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc6c1076f0 0x7fcc6c074350 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fcc64007f10 tx=0x7fcc6400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:44:29.696 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:29.691+0000 7fcc53fff700 1 -- 172.21.15.59:0/3522277034 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc64017070 con 0x7fcc6c1076f0 2023-12-11T17:44:29.696 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:29.691+0000 7fcc7116b700 1 -- 172.21.15.59:0/3522277034 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc6c1a6330 con 0x7fcc6c1076f0 2023-12-11T17:44:29.696 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:29.691+0000 7fcc53fff700 1 -- 172.21.15.59:0/3522277034 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcc6400f040 con 0x7fcc6c1076f0 2023-12-11T17:44:29.696 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:29.691+0000 7fcc53fff700 1 -- 172.21.15.59:0/3522277034 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc640054b0 con 0x7fcc6c1076f0 2023-12-11T17:44:29.696 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:29.691+0000 7fcc7116b700 1 -- 172.21.15.59:0/3522277034 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc6c1a6800 con 0x7fcc6c1076f0 2023-12-11T17:44:29.697 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:29.692+0000 7fcc7116b700 1 -- 172.21.15.59:0/3522277034 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc6c109b80 con 0x7fcc6c1076f0 2023-12-11T17:44:29.697 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:29.693+0000 7fcc53fff700 1 -- 172.21.15.59:0/3522277034 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7fcc64004ad0 con 0x7fcc6c1076f0 2023-12-11T17:44:29.697 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:29.694+0000 7fcc53fff700 1 --2- 172.21.15.59:0/3522277034 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fcc540a1480 0x7fcc540a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:29.697 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:29.694+0000 7fcc6ad9d700 1 --2- 172.21.15.59:0/3522277034 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fcc540a1480 0x7fcc540a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:29.697 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:29.694+0000 7fcc53fff700 1 -- 172.21.15.59:0/3522277034 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fcc640cc5b0 con 0x7fcc6c1076f0 2023-12-11T17:44:29.697 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:29.694+0000 7fcc6ad9d700 1 --2- 172.21.15.59:0/3522277034 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fcc540a1480 0x7fcc540a3940 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c005950 tx=0x7fcc5c00a300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:44:29.702 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:29.700+0000 7fcc53fff700 1 -- 172.21.15.59:0/3522277034 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fcc64095a20 con 0x7fcc6c1076f0 2023-12-11T17:44:29.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:29 smithi059 ceph-mon[98847]: pgmap v263: 129 pgs: 4 creating+activating, 5 creating+peering, 120 active+clean; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 563 B/s wr, 5 op/s 2023-12-11T17:44:29.947 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:29.944+0000 7fcc7116b700 1 -- 172.21.15.59:0/3522277034 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fcc6c04f290 con 0x7fcc6c1076f0 2023-12-11T17:44:29.947 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:29.944+0000 7fcc53fff700 1 -- 172.21.15.59:0/3522277034 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+88 (secure 0 0 0) 0x7fcc64099040 con 0x7fcc6c1076f0 2023-12-11T17:44:29.948 INFO:teuthology.orchestra.run.smithi059.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-12-11T17:44:29.949 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:29.946+0000 7fcc7116b700 1 -- 172.21.15.59:0/3522277034 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fcc540a1480 msgr2=0x7fcc540a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:29.949 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:29.946+0000 7fcc7116b700 1 --2- 172.21.15.59:0/3522277034 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fcc540a1480 0x7fcc540a3940 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c005950 tx=0x7fcc5c00a300 comp rx=0 tx=0).stop 2023-12-11T17:44:29.949 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:29.946+0000 7fcc7116b700 1 -- 172.21.15.59:0/3522277034 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc6c1076f0 msgr2=0x7fcc6c074350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:29.949 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:29.946+0000 7fcc7116b700 1 --2- 172.21.15.59:0/3522277034 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc6c1076f0 0x7fcc6c074350 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fcc64007f10 tx=0x7fcc6400d3b0 comp rx=0 tx=0).stop 2023-12-11T17:44:29.949 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:29.946+0000 7fcc7116b700 1 -- 172.21.15.59:0/3522277034 shutdown_connections 2023-12-11T17:44:29.949 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:29.946+0000 7fcc7116b700 1 --2- 172.21.15.59:0/3522277034 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fcc540a1480 0x7fcc540a3940 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:29.949 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:29.946+0000 7fcc7116b700 1 --2- 172.21.15.59:0/3522277034 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc6c1076f0 0x7fcc6c074350 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:29.949 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:29.946+0000 7fcc7116b700 1 --2- 172.21.15.59:0/3522277034 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcc6c101680 0x7fcc6c0738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:29.950 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:29.946+0000 7fcc7116b700 1 --2- 172.21.15.59:0/3522277034 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcc6c102040 0x7fcc6c073e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:29.950 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:29.946+0000 7fcc7116b700 1 -- 172.21.15.59:0/3522277034 >> 172.21.15.59:0/3522277034 conn(0x7fcc6c0787c0 msgr2=0x7fcc6c100330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:44:29.950 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:29.946+0000 7fcc7116b700 1 -- 172.21.15.59:0/3522277034 shutdown_connections 2023-12-11T17:44:29.950 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:29.947+0000 7fcc7116b700 1 -- 172.21.15.59:0/3522277034 wait complete. 2023-12-11T17:44:30.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:29 smithi115 ceph-mon[102748]: pgmap v263: 129 pgs: 4 creating+activating, 5 creating+peering, 120 active+clean; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 563 B/s wr, 5 op/s 2023-12-11T17:44:30.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:29 smithi181 ceph-mon[103627]: pgmap v263: 129 pgs: 4 creating+activating, 5 creating+peering, 120 active+clean; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 563 B/s wr, 5 op/s 2023-12-11T17:44:30.419 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-12-11T17:44:30.419 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-11T17:44:30.419 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi059.front.sepia.ceph.com 2023-12-11T17:44:30.419 INFO:tasks.cephfs.mount:self.client.name = client.admin 2023-12-11T17:44:30.419 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2023-12-11T17:44:30.420 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-12-11T17:44:30.420 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-12-11T17:44:30.420 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-11T17:44:30.420 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2023-12-11T17:44:30.420 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-11T17:44:30.420 DEBUG:teuthology.orchestra.run.smithi059:> ip addr 2023-12-11T17:44:30.439 INFO:teuthology.orchestra.run.smithi059.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-11T17:44:30.440 INFO:teuthology.orchestra.run.smithi059.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-11T17:44:30.440 INFO:teuthology.orchestra.run.smithi059.stdout: inet 127.0.0.1/8 scope host lo 2023-12-11T17:44:30.440 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2023-12-11T17:44:30.440 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 ::1/128 scope host 2023-12-11T17:44:30.440 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2023-12-11T17:44:30.440 INFO:teuthology.orchestra.run.smithi059.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-11T17:44:30.440 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:6b:fe:ca brd ff:ff:ff:ff:ff:ff 2023-12-11T17:44:30.440 INFO:teuthology.orchestra.run.smithi059.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-11T17:44:30.440 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:6b:fe:cb brd ff:ff:ff:ff:ff:ff 2023-12-11T17:44:30.440 INFO:teuthology.orchestra.run.smithi059.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-11T17:44:30.440 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:bd:19:2a brd ff:ff:ff:ff:ff:ff 2023-12-11T17:44:30.440 INFO:teuthology.orchestra.run.smithi059.stdout: inet 172.21.15.59/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-12-11T17:44:30.441 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft 41892sec preferred_lft 41892sec 2023-12-11T17:44:30.441 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::ec4:7aff:febd:192a/64 scope link 2023-12-11T17:44:30.441 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2023-12-11T17:44:30.441 INFO:teuthology.orchestra.run.smithi059.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-11T17:44:30.441 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:bd:19:2b brd ff:ff:ff:ff:ff:ff 2023-12-11T17:44:30.442 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-12-11T17:44:30.442 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-11T17:44:30.442 DEBUG:teuthology.orchestra.run.smithi059:> (cd / && exec bash -c ' 2023-12-11T17:44:30.442 DEBUG:teuthology.orchestra.run.smithi059:> set -e 2023-12-11T17:44:30.442 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link add name ceph-brx type bridge 2023-12-11T17:44:30.442 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip addr flush dev ceph-brx 2023-12-11T17:44:30.442 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link set ceph-brx up 2023-12-11T17:44:30.442 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-12-11T17:44:30.442 DEBUG:teuthology.orchestra.run.smithi059:> ') 2023-12-11T17:44:30.618 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-11T17:44:30.618 DEBUG:teuthology.orchestra.run.smithi059:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-11T17:44:30.647 INFO:teuthology.orchestra.run.smithi059.stdout:1 2023-12-11T17:44:30.648 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-11T17:44:30.648 DEBUG:teuthology.orchestra.run.smithi059:> route 2023-12-11T17:44:30.716 INFO:teuthology.orchestra.run.smithi059.stdout:Kernel IP routing table 2023-12-11T17:44:30.716 INFO:teuthology.orchestra.run.smithi059.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-11T17:44:30.716 INFO:teuthology.orchestra.run.smithi059.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-12-11T17:44:30.717 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-12-11T17:44:30.717 INFO:teuthology.orchestra.run.smithi059.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-11T17:44:30.717 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-11T17:44:30.717 DEBUG:teuthology.orchestra.run.smithi059:> (cd / && exec bash -c ' 2023-12-11T17:44:30.717 DEBUG:teuthology.orchestra.run.smithi059:> set -e 2023-12-11T17:44:30.717 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-12-11T17:44:30.717 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-12-11T17:44:30.717 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-12-11T17:44:30.718 DEBUG:teuthology.orchestra.run.smithi059:> ') 2023-12-11T17:44:30.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:30 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/3522277034' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-12-11T17:44:30.996 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-11T17:44:30.997 DEBUG:teuthology.orchestra.run.smithi059:> ip netns list 2023-12-11T17:44:31.014 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-11T17:44:31.014 DEBUG:teuthology.orchestra.run.smithi059:> ip netns list-id 2023-12-11T17:44:31.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:30 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/3522277034' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-12-11T17:44:31.071 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-11T17:44:31.071 DEBUG:teuthology.orchestra.run.smithi059:> (cd / && exec bash -c ' 2023-12-11T17:44:31.071 DEBUG:teuthology.orchestra.run.smithi059:> set -e 2023-12-11T17:44:31.071 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-12-11T17:44:31.071 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2023-12-11T17:44:31.071 DEBUG:teuthology.orchestra.run.smithi059:> ') 2023-12-11T17:44:31.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:30 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/3522277034' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-12-11T17:44:31.169 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2023-12-11T17:44:31.169 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-11T17:44:31.169 DEBUG:teuthology.orchestra.run.smithi059:> (cd / && exec bash -c ' 2023-12-11T17:44:31.169 DEBUG:teuthology.orchestra.run.smithi059:> set -e 2023-12-11T17:44:31.169 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2023-12-11T17:44:31.170 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-12-11T17:44:31.170 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2023-12-11T17:44:31.170 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2023-12-11T17:44:31.170 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2023-12-11T17:44:31.170 DEBUG:teuthology.orchestra.run.smithi059:> ') 2023-12-11T17:44:31.399 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-11T17:44:31.399 DEBUG:teuthology.orchestra.run.smithi059:> (cd / && exec bash -c ' 2023-12-11T17:44:31.399 DEBUG:teuthology.orchestra.run.smithi059:> set -e 2023-12-11T17:44:31.399 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link set brx.0 up 2023-12-11T17:44:31.399 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link set dev brx.0 master ceph-brx 2023-12-11T17:44:31.399 DEBUG:teuthology.orchestra.run.smithi059:> ') 2023-12-11T17:44:31.500 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2023-12-11T17:44:31.500 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-11T17:44:31.500 DEBUG:teuthology.orchestra.run.smithi059:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2023-12-11T17:44:31.517 INFO:teuthology.orchestra.run.smithi059.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2023-12-11T17:44:31.517 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-11T17:44:31.517 DEBUG:teuthology.orchestra.run.smithi059:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2023-12-11T17:44:31.574 DEBUG:teuthology.orchestra.run.smithi059:> sudo modprobe fuse 2023-12-11T17:44:31.642 DEBUG:teuthology.orchestra.run.smithi059:> cat /proc/self/mounts | awk '{print $2}' 2023-12-11T17:44:31.701 INFO:teuthology.orchestra.run.smithi059.stdout:/sys 2023-12-11T17:44:31.701 INFO:teuthology.orchestra.run.smithi059.stdout:/proc 2023-12-11T17:44:31.701 INFO:teuthology.orchestra.run.smithi059.stdout:/dev 2023-12-11T17:44:31.701 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/security 2023-12-11T17:44:31.701 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/shm 2023-12-11T17:44:31.701 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/pts 2023-12-11T17:44:31.701 INFO:teuthology.orchestra.run.smithi059.stdout:/run 2023-12-11T17:44:31.701 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/cgroup 2023-12-11T17:44:31.701 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/cgroup/systemd 2023-12-11T17:44:31.702 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/pstore 2023-12-11T17:44:31.702 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/bpf 2023-12-11T17:44:31.702 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/cgroup/devices 2023-12-11T17:44:31.702 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/cgroup/blkio 2023-12-11T17:44:31.702 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/cgroup/cpu,cpuacct 2023-12-11T17:44:31.702 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/cgroup/freezer 2023-12-11T17:44:31.702 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/cgroup/cpuset 2023-12-11T17:44:31.702 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/cgroup/memory 2023-12-11T17:44:31.702 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/cgroup/net_cls,net_prio 2023-12-11T17:44:31.702 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/cgroup/rdma 2023-12-11T17:44:31.702 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/cgroup/hugetlb 2023-12-11T17:44:31.702 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/cgroup/pids 2023-12-11T17:44:31.703 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/cgroup/perf_event 2023-12-11T17:44:31.703 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/tracing 2023-12-11T17:44:31.703 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/config 2023-12-11T17:44:31.703 INFO:teuthology.orchestra.run.smithi059.stdout:/ 2023-12-11T17:44:31.703 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/selinux 2023-12-11T17:44:31.703 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/mqueue 2023-12-11T17:44:31.703 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/hugepages 2023-12-11T17:44:31.703 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/debug 2023-12-11T17:44:31.703 INFO:teuthology.orchestra.run.smithi059.stdout:/proc/sys/fs/binfmt_misc 2023-12-11T17:44:31.703 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/fuse/connections 2023-12-11T17:44:31.703 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/nfs/rpc_pipefs 2023-12-11T17:44:31.703 INFO:teuthology.orchestra.run.smithi059.stdout:/run/user/1000 2023-12-11T17:44:31.704 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph 2023-12-11T17:44:31.704 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/containers/storage/overlay 2023-12-11T17:44:31.704 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/containers/storage/overlay/2455e591f6739cbbbcaacd0827bf7d505a85a6fd5399d3dbeb16e7bf9c8caf31/merged 2023-12-11T17:44:31.704 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/containers/storage/overlay/7d363dfe8ee758db94cc885ec1a5345a5cc3214bc9a83b07bba8f871c61a954a/merged 2023-12-11T17:44:31.704 INFO:teuthology.orchestra.run.smithi059.stdout:/run/user/0 2023-12-11T17:44:31.704 INFO:teuthology.orchestra.run.smithi059.stdout:/proc/sys/fs/binfmt_misc 2023-12-11T17:44:31.704 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/containers/storage/overlay/158893c42a76fbe340d855bb5af1f85abd6c72ea2e136f335db666b3d4fa8336/merged 2023-12-11T17:44:31.704 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/containers/storage/overlay/9d30922a22bf35fa43b8469febb855d6837bd073531aea9e11c65e92843550b5/merged 2023-12-11T17:44:31.704 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/containers/storage/overlay/c1a3277f2167ca817995c42be034555f45b8a82144e3887ac238f4ead92bdc4c/merged 2023-12-11T17:44:31.704 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/containers/storage/overlay/5b9f2a294febc5f83a642cc187ff9fcb103bee1c91192e1a6b901bb9b92116b2/merged 2023-12-11T17:44:31.704 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/containers/storage/overlay/f3799e0bd998885d8fa7f7e1f3ced2e24765baa1a4e11187d961193cf4b90bcd/merged 2023-12-11T17:44:31.705 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/containers/storage/overlay/0bb37326e5761e4012444235fe6d29fae399919297b684b8f704ed0aaa98cbdc/merged 2023-12-11T17:44:31.705 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/containers/storage/overlay/83fbb5a430291679c6b087a9c3715297825f25f4777ca051d6ba9fb0f4cd0f39/merged 2023-12-11T17:44:31.705 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/containers/storage/overlay/1e65e5d92631d33b15a5ef793701103b4ef4092fce46241b6d452048aef17302/merged 2023-12-11T17:44:31.705 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns 2023-12-11T17:44:31.705 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-12-11T17:44:31.705 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-12-11T17:44:31.705 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-11T17:44:31.705 DEBUG:teuthology.orchestra.run.smithi059:> ls /sys/fs/fuse/connections 2023-12-11T17:44:31.758 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-12-11T17:44:31.759 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2023-12-11T17:44:31.803 DEBUG:teuthology.orchestra.run.smithi059:> sudo modprobe fuse 2023-12-11T17:44:31.826 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:31 smithi059 ceph-mon[98847]: pgmap v264: 129 pgs: 129 active+clean; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 476 B/s wr, 4 op/s 2023-12-11T17:44:31.831 DEBUG:teuthology.orchestra.run.smithi059:> cat /proc/self/mounts | awk '{print $2}' 2023-12-11T17:44:31.890 INFO:teuthology.orchestra.run.smithi059.stdout:/sys 2023-12-11T17:44:31.890 INFO:teuthology.orchestra.run.smithi059.stdout:/proc 2023-12-11T17:44:31.890 INFO:teuthology.orchestra.run.smithi059.stdout:/dev 2023-12-11T17:44:31.890 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/security 2023-12-11T17:44:31.891 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/shm 2023-12-11T17:44:31.891 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/pts 2023-12-11T17:44:31.891 INFO:teuthology.orchestra.run.smithi059.stdout:/run 2023-12-11T17:44:31.891 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/cgroup 2023-12-11T17:44:31.891 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/cgroup/systemd 2023-12-11T17:44:31.891 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/pstore 2023-12-11T17:44:31.891 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/bpf 2023-12-11T17:44:31.891 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/cgroup/devices 2023-12-11T17:44:31.891 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/cgroup/blkio 2023-12-11T17:44:31.891 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/cgroup/cpu,cpuacct 2023-12-11T17:44:31.891 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/cgroup/freezer 2023-12-11T17:44:31.892 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/cgroup/cpuset 2023-12-11T17:44:31.892 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/cgroup/memory 2023-12-11T17:44:31.892 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/cgroup/net_cls,net_prio 2023-12-11T17:44:31.892 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/cgroup/rdma 2023-12-11T17:44:31.892 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/cgroup/hugetlb 2023-12-11T17:44:31.892 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/cgroup/pids 2023-12-11T17:44:31.892 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/cgroup/perf_event 2023-12-11T17:44:31.892 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/tracing 2023-12-11T17:44:31.892 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/config 2023-12-11T17:44:31.893 INFO:teuthology.orchestra.run.smithi059.stdout:/ 2023-12-11T17:44:31.893 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/selinux 2023-12-11T17:44:31.893 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/mqueue 2023-12-11T17:44:31.893 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/hugepages 2023-12-11T17:44:31.893 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/debug 2023-12-11T17:44:31.893 INFO:teuthology.orchestra.run.smithi059.stdout:/proc/sys/fs/binfmt_misc 2023-12-11T17:44:31.893 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/fuse/connections 2023-12-11T17:44:31.893 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/nfs/rpc_pipefs 2023-12-11T17:44:31.893 INFO:teuthology.orchestra.run.smithi059.stdout:/run/user/1000 2023-12-11T17:44:31.894 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph 2023-12-11T17:44:31.894 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/containers/storage/overlay 2023-12-11T17:44:31.894 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/containers/storage/overlay/2455e591f6739cbbbcaacd0827bf7d505a85a6fd5399d3dbeb16e7bf9c8caf31/merged 2023-12-11T17:44:31.894 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/containers/storage/overlay/7d363dfe8ee758db94cc885ec1a5345a5cc3214bc9a83b07bba8f871c61a954a/merged 2023-12-11T17:44:31.894 INFO:teuthology.orchestra.run.smithi059.stdout:/run/user/0 2023-12-11T17:44:31.894 INFO:teuthology.orchestra.run.smithi059.stdout:/proc/sys/fs/binfmt_misc 2023-12-11T17:44:31.895 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/containers/storage/overlay/158893c42a76fbe340d855bb5af1f85abd6c72ea2e136f335db666b3d4fa8336/merged 2023-12-11T17:44:31.895 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/containers/storage/overlay/9d30922a22bf35fa43b8469febb855d6837bd073531aea9e11c65e92843550b5/merged 2023-12-11T17:44:31.895 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/containers/storage/overlay/c1a3277f2167ca817995c42be034555f45b8a82144e3887ac238f4ead92bdc4c/merged 2023-12-11T17:44:31.895 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/containers/storage/overlay/5b9f2a294febc5f83a642cc187ff9fcb103bee1c91192e1a6b901bb9b92116b2/merged 2023-12-11T17:44:31.895 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/containers/storage/overlay/f3799e0bd998885d8fa7f7e1f3ced2e24765baa1a4e11187d961193cf4b90bcd/merged 2023-12-11T17:44:31.895 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/containers/storage/overlay/0bb37326e5761e4012444235fe6d29fae399919297b684b8f704ed0aaa98cbdc/merged 2023-12-11T17:44:31.895 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/containers/storage/overlay/83fbb5a430291679c6b087a9c3715297825f25f4777ca051d6ba9fb0f4cd0f39/merged 2023-12-11T17:44:31.895 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/containers/storage/overlay/1e65e5d92631d33b15a5ef793701103b4ef4092fce46241b6d452048aef17302/merged 2023-12-11T17:44:31.896 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns 2023-12-11T17:44:31.896 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-12-11T17:44:31.896 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-12-11T17:44:31.896 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-11T17:44:31.896 DEBUG:teuthology.orchestra.run.smithi059:> ls /sys/fs/fuse/connections 2023-12-11T17:44:31.920 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi059.stderr:2023-12-11T17:44:31.918+0000 7f432d7075c0 -1 init, newargv = 0x55880548fa30 newargc=15 2023-12-11T17:44:31.920 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi059.stderr:ceph-fuse[153949]: starting ceph client 2023-12-11T17:44:31.927 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi059.stderr:ceph-fuse[153949]: starting fuse 2023-12-11T17:44:31.940 INFO:teuthology.orchestra.run.smithi059.stdout:124 2023-12-11T17:44:31.940 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [124] 2023-12-11T17:44:31.940 DEBUG:teuthology.orchestra.run.smithi059:> sudo stdin-killer -- python3 -c ' 2023-12-11T17:44:31.940 DEBUG:teuthology.orchestra.run.smithi059:> import glob 2023-12-11T17:44:31.941 DEBUG:teuthology.orchestra.run.smithi059:> import re 2023-12-11T17:44:31.941 DEBUG:teuthology.orchestra.run.smithi059:> import os 2023-12-11T17:44:31.941 DEBUG:teuthology.orchestra.run.smithi059:> import subprocess 2023-12-11T17:44:31.941 DEBUG:teuthology.orchestra.run.smithi059:> 2023-12-11T17:44:31.941 DEBUG:teuthology.orchestra.run.smithi059:> def _find_admin_socket(client_name): 2023-12-11T17:44:31.941 DEBUG:teuthology.orchestra.run.smithi059:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2023-12-11T17:44:31.941 DEBUG:teuthology.orchestra.run.smithi059:> files = glob.glob(asok_path) 2023-12-11T17:44:31.941 DEBUG:teuthology.orchestra.run.smithi059:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2023-12-11T17:44:31.941 DEBUG:teuthology.orchestra.run.smithi059:> 2023-12-11T17:44:31.941 DEBUG:teuthology.orchestra.run.smithi059:> # Given a non-glob path, it better be there 2023-12-11T17:44:31.941 DEBUG:teuthology.orchestra.run.smithi059:> if "*" not in asok_path: 2023-12-11T17:44:31.941 DEBUG:teuthology.orchestra.run.smithi059:> assert(len(files) == 1) 2023-12-11T17:44:31.941 DEBUG:teuthology.orchestra.run.smithi059:> return files[0] 2023-12-11T17:44:31.941 DEBUG:teuthology.orchestra.run.smithi059:> 2023-12-11T17:44:31.941 DEBUG:teuthology.orchestra.run.smithi059:> for f in files: 2023-12-11T17:44:31.941 DEBUG:teuthology.orchestra.run.smithi059:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-12-11T17:44:31.942 DEBUG:teuthology.orchestra.run.smithi059:> if os.path.exists("/proc/{0}".format(pid)): 2023-12-11T17:44:31.942 DEBUG:teuthology.orchestra.run.smithi059:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-12-11T17:44:31.942 DEBUG:teuthology.orchestra.run.smithi059:> contents = proc_f.read() 2023-12-11T17:44:31.942 DEBUG:teuthology.orchestra.run.smithi059:> if mountpoint in contents: 2023-12-11T17:44:31.942 DEBUG:teuthology.orchestra.run.smithi059:> return f 2023-12-11T17:44:31.942 DEBUG:teuthology.orchestra.run.smithi059:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-12-11T17:44:31.942 DEBUG:teuthology.orchestra.run.smithi059:> 2023-12-11T17:44:31.942 DEBUG:teuthology.orchestra.run.smithi059:> print(_find_admin_socket("client.admin")) 2023-12-11T17:44:31.942 DEBUG:teuthology.orchestra.run.smithi059:> ' 2023-12-11T17:44:32.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:31 smithi115 ceph-mon[102748]: pgmap v264: 129 pgs: 129 active+clean; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 476 B/s wr, 4 op/s 2023-12-11T17:44:32.066 INFO:teuthology.orchestra.run.smithi059.stdout:/var/run/ceph/ceph-client.admin.153949.asok 2023-12-11T17:44:32.069 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-12-11T17:44:32.075 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.153949.asok 2023-12-11T17:44:32.075 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-11T17:44:32.075 DEBUG:teuthology.orchestra.run.smithi059:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.153949.asok status 2023-12-11T17:44:32.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:31 smithi181 ceph-mon[103627]: pgmap v264: 129 pgs: 129 active+clean; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 476 B/s wr, 4 op/s 2023-12-11T17:44:32.236 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2023-12-11T17:44:32.236 INFO:teuthology.orchestra.run.smithi059.stdout: "metadata": { 2023-12-11T17:44:32.236 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_sha1": "a62fda73efba5520d409bb5a21bd8373289dda5d", 2023-12-11T17:44:32.236 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_version": "ceph version 19.0.0-119-ga62fda73 (a62fda73efba5520d409bb5a21bd8373289dda5d) reef (dev)", 2023-12-11T17:44:32.236 INFO:teuthology.orchestra.run.smithi059.stdout: "entity_id": "admin", 2023-12-11T17:44:32.237 INFO:teuthology.orchestra.run.smithi059.stdout: "hostname": "smithi059", 2023-12-11T17:44:32.237 INFO:teuthology.orchestra.run.smithi059.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2023-12-11T17:44:32.237 INFO:teuthology.orchestra.run.smithi059.stdout: "pid": "153949", 2023-12-11T17:44:32.237 INFO:teuthology.orchestra.run.smithi059.stdout: "root": "/" 2023-12-11T17:44:32.237 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T17:44:32.237 INFO:teuthology.orchestra.run.smithi059.stdout: "dentry_count": 0, 2023-12-11T17:44:32.237 INFO:teuthology.orchestra.run.smithi059.stdout: "dentry_pinned_count": 0, 2023-12-11T17:44:32.237 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 14733, 2023-12-11T17:44:32.237 INFO:teuthology.orchestra.run.smithi059.stdout: "inst": { 2023-12-11T17:44:32.237 INFO:teuthology.orchestra.run.smithi059.stdout: "name": { 2023-12-11T17:44:32.237 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "client", 2023-12-11T17:44:32.237 INFO:teuthology.orchestra.run.smithi059.stdout: "num": 14733 2023-12-11T17:44:32.237 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T17:44:32.237 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": { 2023-12-11T17:44:32.238 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2023-12-11T17:44:32.238 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "192.168.0.1:0", 2023-12-11T17:44:32.238 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 2940576799 2023-12-11T17:44:32.238 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T17:44:32.238 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T17:44:32.238 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": { 2023-12-11T17:44:32.238 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2023-12-11T17:44:32.238 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "192.168.0.1:0", 2023-12-11T17:44:32.238 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 2940576799 2023-12-11T17:44:32.238 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T17:44:32.238 INFO:teuthology.orchestra.run.smithi059.stdout: "inst_str": "client.14733 192.168.0.1:0/2940576799", 2023-12-11T17:44:32.238 INFO:teuthology.orchestra.run.smithi059.stdout: "addr_str": "192.168.0.1:0/2940576799", 2023-12-11T17:44:32.238 INFO:teuthology.orchestra.run.smithi059.stdout: "inode_count": 1, 2023-12-11T17:44:32.238 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_epoch": 17, 2023-12-11T17:44:32.239 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_epoch": 86, 2023-12-11T17:44:32.239 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_epoch_barrier": 0, 2023-12-11T17:44:32.239 INFO:teuthology.orchestra.run.smithi059.stdout: "blocklisted": false, 2023-12-11T17:44:32.239 INFO:teuthology.orchestra.run.smithi059.stdout: "fs_name": "cephfs" 2023-12-11T17:44:32.239 INFO:teuthology.orchestra.run.smithi059.stdout:} 2023-12-11T17:44:32.246 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-11T17:44:32.246 DEBUG:teuthology.orchestra.run.smithi059:> stat --file-system '--printf=%T 2023-12-11T17:44:32.247 DEBUG:teuthology.orchestra.run.smithi059:> ' -- /home/ubuntu/cephtest/mnt.admin 2023-12-11T17:44:32.264 INFO:teuthology.orchestra.run.smithi059.stdout:fuseblk 2023-12-11T17:44:32.264 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2023-12-11T17:44:32.264 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-11T17:44:32.264 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2023-12-11T17:44:32.333 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-11T17:44:32.334 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2023-12-11T17:44:32.440 INFO:teuthology.orchestra.run.smithi059.stdout:# file: . 2023-12-11T17:44:32.441 INFO:teuthology.orchestra.run.smithi059.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2023-12-11T17:44:32.441 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-12-11T17:44:32.441 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2023-12-11T17:44:32.441 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-11T17:44:32.459 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-11T17:44:32.625 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:32.622+0000 7f1c0d06b700 1 -- 172.21.15.59:0/1759602965 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c08100380 msgr2=0x7f1c081007a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:32.685 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:32.622+0000 7f1c0d06b700 1 --2- 172.21.15.59:0/1759602965 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c08100380 0x7f1c081007a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1bf80099c0 tx=0x7f1bf8009cd0 comp rx=0 tx=0).stop 2023-12-11T17:44:32.687 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:32.622+0000 7f1c0d06b700 1 -- 172.21.15.59:0/1759602965 shutdown_connections 2023-12-11T17:44:32.687 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:32.622+0000 7f1c0d06b700 1 --2- 172.21.15.59:0/1759602965 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1c08101f10 0x7f1c081087c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:32.687 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:32.622+0000 7f1c0d06b700 1 --2- 172.21.15.59:0/1759602965 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c08100380 0x7f1c081007a0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:32.687 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:32.622+0000 7f1c0d06b700 1 --2- 172.21.15.59:0/1759602965 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1c08101570 0x7f1c081019d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:32.687 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:32.622+0000 7f1c0d06b700 1 -- 172.21.15.59:0/1759602965 >> 172.21.15.59:0/1759602965 conn(0x7f1c080fb8f0 msgr2=0x7f1c080fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:44:32.688 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:32.622+0000 7f1c0d06b700 1 -- 172.21.15.59:0/1759602965 shutdown_connections 2023-12-11T17:44:32.688 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:32.622+0000 7f1c0d06b700 1 -- 172.21.15.59:0/1759602965 wait complete. 2023-12-11T17:44:32.688 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:32.623+0000 7f1c0d06b700 1 Processor -- start 2023-12-11T17:44:32.694 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:32.623+0000 7f1c0d06b700 1 -- start start 2023-12-11T17:44:32.694 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:32.623+0000 7f1c0d06b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1c08100380 0x7f1c081042c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:32.694 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:32.623+0000 7f1c0d06b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1c08101570 0x7f1c08102950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:32.694 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:32.623+0000 7f1c0d06b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c08101f10 0x7f1c08102e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:32.695 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:32.623+0000 7f1c0d06b700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c08103630 con 0x7f1c08101f10 2023-12-11T17:44:32.695 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:32.623+0000 7f1c0d06b700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c081037a0 con 0x7f1c08101570 2023-12-11T17:44:32.695 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:32.623+0000 7f1c0d06b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c081a1460 con 0x7f1c08100380 2023-12-11T17:44:32.695 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:32.623+0000 7f1c0759e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c08101f10 0x7f1c08102e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:32.695 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:32.623+0000 7f1c0759e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c08101f10 0x7f1c08102e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33076/0 (socket says 172.21.15.59:33076) 2023-12-11T17:44:32.695 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:32.623+0000 7f1c0759e700 1 -- 172.21.15.59:0/2303272100 learned_addr learned my addr 172.21.15.59:0/2303272100 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:44:32.695 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:32.623+0000 7f1c0659c700 1 --2- 172.21.15.59:0/2303272100 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1c08101570 0x7f1c08102950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:32.695 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:32.624+0000 7f1c06d9d700 1 --2- 172.21.15.59:0/2303272100 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1c08100380 0x7f1c081042c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:32.695 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:32.624+0000 7f1c0759e700 1 -- 172.21.15.59:0/2303272100 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1c08100380 msgr2=0x7f1c081042c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:32.695 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:32.624+0000 7f1c0759e700 1 --2- 172.21.15.59:0/2303272100 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1c08100380 0x7f1c081042c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:32.695 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:32.624+0000 7f1c0759e700 1 -- 172.21.15.59:0/2303272100 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1c08101570 msgr2=0x7f1c08102950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:32.696 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:32.624+0000 7f1c0759e700 1 --2- 172.21.15.59:0/2303272100 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1c08101570 0x7f1c08102950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:32.696 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:32.624+0000 7f1c0759e700 1 -- 172.21.15.59:0/2303272100 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bf80095f0 con 0x7f1c08101f10 2023-12-11T17:44:32.696 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:32.624+0000 7f1c0759e700 1 --2- 172.21.15.59:0/2303272100 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c08101f10 0x7f1c08102e90 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f1c00007ee0 tx=0x7f1c0000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:44:32.696 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:32.624+0000 7f1beffff700 1 -- 172.21.15.59:0/2303272100 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c00017070 con 0x7f1c08101f10 2023-12-11T17:44:32.696 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:32.624+0000 7f1beffff700 1 -- 172.21.15.59:0/2303272100 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1c0000f040 con 0x7f1c08101f10 2023-12-11T17:44:32.696 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:32.624+0000 7f1c0d06b700 1 -- 172.21.15.59:0/2303272100 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c081a16f0 con 0x7f1c08101f10 2023-12-11T17:44:32.696 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:32.624+0000 7f1beffff700 1 -- 172.21.15.59:0/2303272100 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c00005740 con 0x7f1c08101f10 2023-12-11T17:44:32.696 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:32.624+0000 7f1c0d06b700 1 -- 172.21.15.59:0/2303272100 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c081a1bb0 con 0x7f1c08101f10 2023-12-11T17:44:32.696 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:32.625+0000 7f1c0d06b700 1 -- 172.21.15.59:0/2303272100 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c08068230 con 0x7f1c08101f10 2023-12-11T17:44:32.696 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:32.626+0000 7f1beffff700 1 -- 172.21.15.59:0/2303272100 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f1c00004c40 con 0x7f1c08101f10 2023-12-11T17:44:32.696 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:32.627+0000 7f1beffff700 1 --2- 172.21.15.59:0/2303272100 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f1bf00a1480 0x7f1bf00a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:32.696 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:32.627+0000 7f1beffff700 1 -- 172.21.15.59:0/2303272100 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f1c000ccd80 con 0x7f1c08101f10 2023-12-11T17:44:32.696 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:32.627+0000 7f1c06d9d700 1 --2- 172.21.15.59:0/2303272100 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f1bf00a1480 0x7f1bf00a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:32.697 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:32.627+0000 7f1c06d9d700 1 --2- 172.21.15.59:0/2303272100 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f1bf00a1480 0x7f1bf00a3940 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1bf800b580 tx=0x7f1bf8005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:44:32.697 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:32.631+0000 7f1beffff700 1 -- 172.21.15.59:0/2303272100 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f1c000961f0 con 0x7f1c08101f10 2023-12-11T17:44:32.843 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:32.840+0000 7f1c0d06b700 1 -- 172.21.15.59:0/2303272100 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1c0810bd10 con 0x7f1c08101f10 2023-12-11T17:44:32.844 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:32.842+0000 7f1beffff700 1 -- 172.21.15.59:0/2303272100 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+23960 (secure 0 0 0) 0x7f1c00099810 con 0x7f1c08101f10 2023-12-11T17:44:32.845 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-12-11T17:44:32.845 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":86,"fsid":"d0a07eaa-984b-11ee-95a3-87774f69a715","created":"2023-12-11T17:37:14.526539+0000","modified":"2023-12-11T17:44:25.096094+0000","last_up_change":"2023-12-11T17:43:38.535503+0000","last_in_change":"2023-12-11T17:43:21.014888+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-11T17:40:26.596029+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-11T17:44:18.586290+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-11T17:44:19.432952+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-11T17:44:23.097308+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3c0b3218-4b2b-40f8-8bef-ed8cd5633607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":1040759398},{"type":"v1","addr":"172.21.15.59:6803","nonce":1040759398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":1040759398},{"type":"v1","addr":"172.21.15.59:6805","nonce":1040759398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":1040759398},{"type":"v1","addr":"172.21.15.59:6809","nonce":1040759398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":1040759398},{"type":"v1","addr":"172.21.15.59:6807","nonce":1040759398}]},"public_addr":"172.21.15.59:6803/1040759398","cluster_addr":"172.21.15.59:6805/1040759398","heartbeat_back_addr":"172.21.15.59:6809/1040759398","heartbeat_front_addr":"172.21.15.59:6807/1040759398","state":["exists","up"]},{"osd":1,"uuid":"2ead7578-765b-4f5a-a72c-863731ed69e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":2776354245},{"type":"v1","addr":"172.21.15.115:6801","nonce":2776354245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":2776354245},{"type":"v1","addr":"172.21.15.115:6803","nonce":2776354245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":2776354245},{"type":"v1","addr":"172.21.15.115:6807","nonce":2776354245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":2776354245},{"type":"v1","addr":"172.21.15.115:6805","nonce":2776354245}]},"public_addr":"172.21.15.115:6801/2776354245","cluster_addr":"172.21.15.115:6803/2776354245","heartbeat_back_addr":"172.21.15.115:6807/2776354245","heartbeat_front_addr":"172.21.15.115:6805/2776354245","state":["exists","up"]},{"osd":2,"uuid":"8cb1d7ec-53cc-4672-b0b6-70d73797c44f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4287675481},{"type":"v1","addr":"172.21.15.181:6801","nonce":4287675481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4287675481},{"type":"v1","addr":"172.21.15.181:6803","nonce":4287675481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4287675481},{"type":"v1","addr":"172.21.15.181:6807","nonce":4287675481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4287675481},{"type":"v1","addr":"172.21.15.181:6805","nonce":4287675481}]},"public_addr":"172.21.15.181:6801/4287675481","cluster_addr":"172.21.15.181:6803/4287675481","heartbeat_back_addr":"172.21.15.181:6807/4287675481","heartbeat_front_addr":"172.21.15.181:6805/4287675481","state":["exists","up"]},{"osd":3,"uuid":"5d64d9af-831b-4d0a-8ace-a452169351d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":219909995},{"type":"v1","addr":"172.21.15.59:6811","nonce":219909995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":219909995},{"type":"v1","addr":"172.21.15.59:6813","nonce":219909995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":219909995},{"type":"v1","addr":"172.21.15.59:6817","nonce":219909995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":219909995},{"type":"v1","addr":"172.21.15.59:6815","nonce":219909995}]},"public_addr":"172.21.15.59:6811/219909995","cluster_addr":"172.21.15.59:6813/219909995","heartbeat_back_addr":"172.21.15.59:6817/219909995","heartbeat_front_addr":"172.21.15.59:6815/219909995","state":["exists","up"]},{"osd":4,"uuid":"ef8fad28-1db5-4cba-b5f2-10e54e8e1549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":1021473826},{"type":"v1","addr":"172.21.15.115:6809","nonce":1021473826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":1021473826},{"type":"v1","addr":"172.21.15.115:6811","nonce":1021473826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":1021473826},{"type":"v1","addr":"172.21.15.115:6815","nonce":1021473826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":1021473826},{"type":"v1","addr":"172.21.15.115:6813","nonce":1021473826}]},"public_addr":"172.21.15.115:6809/1021473826","cluster_addr":"172.21.15.115:6811/1021473826","heartbeat_back_addr":"172.21.15.115:6815/1021473826","heartbeat_front_addr":"172.21.15.115:6813/1021473826","state":["exists","up"]},{"osd":5,"uuid":"4233a512-9e8f-4db8-aed2-6c3af706e589","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":2405172521},{"type":"v1","addr":"172.21.15.181:6809","nonce":2405172521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":2405172521},{"type":"v1","addr":"172.21.15.181:6811","nonce":2405172521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2405172521},{"type":"v1","addr":"172.21.15.181:6815","nonce":2405172521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":2405172521},{"type":"v1","addr":"172.21.15.181:6813","nonce":2405172521}]},"public_addr":"172.21.15.181:6809/2405172521","cluster_addr":"172.21.15.181:6811/2405172521","heartbeat_back_addr":"172.21.15.181:6815/2405172521","heartbeat_front_addr":"172.21.15.181:6813/2405172521","state":["exists","up"]},{"osd":6,"uuid":"8faab332-016e-4c77-8370-8bef95fc6f34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":815374853},{"type":"v1","addr":"172.21.15.59:6819","nonce":815374853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":815374853},{"type":"v1","addr":"172.21.15.59:6821","nonce":815374853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":815374853},{"type":"v1","addr":"172.21.15.59:6825","nonce":815374853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":815374853},{"type":"v1","addr":"172.21.15.59:6823","nonce":815374853}]},"public_addr":"172.21.15.59:6819/815374853","cluster_addr":"172.21.15.59:6821/815374853","heartbeat_back_addr":"172.21.15.59:6825/815374853","heartbeat_front_addr":"172.21.15.59:6823/815374853","state":["exists","up"]},{"osd":7,"uuid":"f20f470a-0d40-4ed3-8e0e-fd58675cf143","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":1314971870},{"type":"v1","addr":"172.21.15.115:6817","nonce":1314971870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":1314971870},{"type":"v1","addr":"172.21.15.115:6819","nonce":1314971870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":1314971870},{"type":"v1","addr":"172.21.15.115:6823","nonce":1314971870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":1314971870},{"type":"v1","addr":"172.21.15.115:6821","nonce":1314971870}]},"public_addr":"172.21.15.115:6817/1314971870","cluster_addr":"172.21.15.115:6819/1314971870","heartbeat_back_addr":"172.21.15.115:6823/1314971870","heartbeat_front_addr":"172.21.15.115:6821/1314971870","state":["exists","up"]},{"osd":8,"uuid":"eb657d32-1a95-4fd7-a293-d0a970837fc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":3954509873},{"type":"v1","addr":"172.21.15.181:6817","nonce":3954509873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":3954509873},{"type":"v1","addr":"172.21.15.181:6819","nonce":3954509873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":3954509873},{"type":"v1","addr":"172.21.15.181:6823","nonce":3954509873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":3954509873},{"type":"v1","addr":"172.21.15.181:6821","nonce":3954509873}]},"public_addr":"172.21.15.181:6817/3954509873","cluster_addr":"172.21.15.181:6819/3954509873","heartbeat_back_addr":"172.21.15.181:6823/3954509873","heartbeat_front_addr":"172.21.15.181:6821/3954509873","state":["exists","up"]},{"osd":9,"uuid":"8f7717d4-b6f2-4ec1-9b58-62feb0c9a146","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":1163095677},{"type":"v1","addr":"172.21.15.59:6827","nonce":1163095677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":1163095677},{"type":"v1","addr":"172.21.15.59:6829","nonce":1163095677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":1163095677},{"type":"v1","addr":"172.21.15.59:6833","nonce":1163095677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":1163095677},{"type":"v1","addr":"172.21.15.59:6831","nonce":1163095677}]},"public_addr":"172.21.15.59:6827/1163095677","cluster_addr":"172.21.15.59:6829/1163095677","heartbeat_back_addr":"172.21.15.59:6833/1163095677","heartbeat_front_addr":"172.21.15.59:6831/1163095677","state":["exists","up"]},{"osd":10,"uuid":"b75c179a-875d-43bc-b4c1-e8eb80fb64fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":2199610269},{"type":"v1","addr":"172.21.15.115:6825","nonce":2199610269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":2199610269},{"type":"v1","addr":"172.21.15.115:6827","nonce":2199610269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":2199610269},{"type":"v1","addr":"172.21.15.115:6831","nonce":2199610269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":2199610269},{"type":"v1","addr":"172.21.15.115:6829","nonce":2199610269}]},"public_addr":"172.21.15.115:6825/2199610269","cluster_addr":"172.21.15.115:6827/2199610269","heartbeat_back_addr":"172.21.15.115:6831/2199610269","heartbeat_front_addr":"172.21.15.115:6829/2199610269","state":["exists","up"]},{"osd":11,"uuid":"5e141dfe-8a00-4bbc-8204-2dc9be24e18d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":226506989},{"type":"v1","addr":"172.21.15.181:6825","nonce":226506989}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":226506989},{"type":"v1","addr":"172.21.15.181:6827","nonce":226506989}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":226506989},{"type":"v1","addr":"172.21.15.181:6831","nonce":226506989}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":226506989},{"type":"v1","addr":"172.21.15.181:6829","nonce":226506989}]},"public_addr":"172.21.15.181:6825/226506989","cluster_addr":"172.21.15.181:6827/226506989","heartbeat_back_addr":"172.21.15.181:6831/226506989","heartbeat_front_addr":"172.21.15.181:6829/226506989","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:39:42.687084+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:40:03.903448+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:40:24.899908+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:40:46.600251+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:41:07.690037+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:41:28.488855+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:41:49.459274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:42:10.863881+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:42:32.248391+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:42:54.112265+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:43:15.634872+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:43:36.922534+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.59:6800/1259916482":"2023-12-12T17:38:14.037922+0000","172.21.15.59:0/17457774":"2023-12-12T17:38:14.037922+0000","172.21.15.59:6801/1672530462":"2023-12-12T17:37:44.773925+0000","172.21.15.59:0/1053944547":"2023-12-12T17:38:14.037922+0000","172.21.15.59:6801/1259916482":"2023-12-12T17:38:14.037922+0000","172.21.15.59:6800/1672530462":"2023-12-12T17:37:44.773925+0000","172.21.15.59:0/3570767650":"2023-12-12T17:37:44.773925+0000","172.21.15.59:0/3853462899":"2023-12-12T17:37:44.773925+0000","172.21.15.59:0/2859415380":"2023-12-12T17:38:14.037922+0000","172.21.15.59:0/2192742819":"2023-12-12T17:37:44.773925+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-11T17:44:32.847 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:32.844+0000 7f1c0d06b700 1 -- 172.21.15.59:0/2303272100 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f1bf00a1480 msgr2=0x7f1bf00a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:32.847 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:32.844+0000 7f1c0d06b700 1 --2- 172.21.15.59:0/2303272100 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f1bf00a1480 0x7f1bf00a3940 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1bf800b580 tx=0x7f1bf8005fb0 comp rx=0 tx=0).stop 2023-12-11T17:44:32.847 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:32.844+0000 7f1c0d06b700 1 -- 172.21.15.59:0/2303272100 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c08101f10 msgr2=0x7f1c08102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:32.847 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:32.844+0000 7f1c0d06b700 1 --2- 172.21.15.59:0/2303272100 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c08101f10 0x7f1c08102e90 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f1c00007ee0 tx=0x7f1c0000d3b0 comp rx=0 tx=0).stop 2023-12-11T17:44:32.847 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:32.844+0000 7f1c0d06b700 1 -- 172.21.15.59:0/2303272100 shutdown_connections 2023-12-11T17:44:32.848 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:32.844+0000 7f1c0d06b700 1 --2- 172.21.15.59:0/2303272100 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f1bf00a1480 0x7f1bf00a3940 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:32.848 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:32.844+0000 7f1c0d06b700 1 --2- 172.21.15.59:0/2303272100 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c08101f10 0x7f1c08102e90 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:32.848 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:32.844+0000 7f1c0d06b700 1 --2- 172.21.15.59:0/2303272100 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1c08100380 0x7f1c081042c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:32.848 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:32.844+0000 7f1c0d06b700 1 --2- 172.21.15.59:0/2303272100 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1c08101570 0x7f1c08102950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:32.848 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:32.844+0000 7f1c0d06b700 1 -- 172.21.15.59:0/2303272100 >> 172.21.15.59:0/2303272100 conn(0x7f1c080fb8f0 msgr2=0x7f1c08068680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:44:32.848 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:32.844+0000 7f1c0d06b700 1 -- 172.21.15.59:0/2303272100 shutdown_connections 2023-12-11T17:44:32.848 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:32.844+0000 7f1c0d06b700 1 -- 172.21.15.59:0/2303272100 wait complete. 2023-12-11T17:44:32.859 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-11T17:40:26.596029+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-11T17:44:18.586290+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-11T17:44:19.432952+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-12-11T17:44:23.097308+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-12-11T17:44:32.860 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-11T17:44:32.993 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:32.988+0000 7f1861f7e700 1 -- 172.21.15.59:0/2123092417 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f185c100480 msgr2=0x7f185c1008a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:32.993 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:32.988+0000 7f1861f7e700 1 --2- 172.21.15.59:0/2123092417 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f185c100480 0x7f185c1008a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f184c0099c0 tx=0x7f184c009cd0 comp rx=0 tx=0).stop 2023-12-11T17:44:32.993 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:32.988+0000 7f1861f7e700 1 -- 172.21.15.59:0/2123092417 shutdown_connections 2023-12-11T17:44:32.994 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:32.988+0000 7f1861f7e700 1 --2- 172.21.15.59:0/2123092417 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f185c102010 0x7f185c1088c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:32.994 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:32.988+0000 7f1861f7e700 1 --2- 172.21.15.59:0/2123092417 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f185c100480 0x7f185c1008a0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:32.994 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:32.988+0000 7f1861f7e700 1 --2- 172.21.15.59:0/2123092417 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f185c101670 0x7f185c101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:32.994 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:32.988+0000 7f1861f7e700 1 -- 172.21.15.59:0/2123092417 >> 172.21.15.59:0/2123092417 conn(0x7f185c0fb9f0 msgr2=0x7f185c0fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:44:32.994 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:32.988+0000 7f1861f7e700 1 -- 172.21.15.59:0/2123092417 shutdown_connections 2023-12-11T17:44:32.994 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:32.988+0000 7f1861f7e700 1 -- 172.21.15.59:0/2123092417 wait complete. 2023-12-11T17:44:32.995 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:32.988+0000 7f1861f7e700 1 Processor -- start 2023-12-11T17:44:32.995 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:32.988+0000 7f1861f7e700 1 -- start start 2023-12-11T17:44:32.995 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:32.989+0000 7f1861f7e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f185c100480 0x7f185c1043c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:32.995 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:32.989+0000 7f1861f7e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f185c101670 0x7f185c102a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:32.995 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:32.989+0000 7f1861f7e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f185c102010 0x7f185c102f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:32.995 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:32.989+0000 7f1861f7e700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f185c103730 con 0x7f185c102010 2023-12-11T17:44:32.995 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:32.989+0000 7f1861f7e700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f185c1038a0 con 0x7f185c100480 2023-12-11T17:44:32.995 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:32.989+0000 7f1861f7e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f185c1a15a0 con 0x7f185c101670 2023-12-11T17:44:32.995 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:32.989+0000 7f185bfff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f185c102010 0x7f185c102f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:32.995 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:32.989+0000 7f185bfff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f185c102010 0x7f185c102f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33102/0 (socket says 172.21.15.59:33102) 2023-12-11T17:44:32.995 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:32.989+0000 7f185bfff700 1 -- 172.21.15.59:0/3141848061 learned_addr learned my addr 172.21.15.59:0/3141848061 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:44:32.995 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:32.989+0000 7f185b7fe700 1 --2- 172.21.15.59:0/3141848061 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f185c100480 0x7f185c1043c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:32.995 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:32.989+0000 7f1853fff700 1 --2- 172.21.15.59:0/3141848061 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f185c101670 0x7f185c102a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:32.995 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:32.989+0000 7f185bfff700 1 -- 172.21.15.59:0/3141848061 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f185c101670 msgr2=0x7f185c102a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:32.996 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:32.989+0000 7f185bfff700 1 --2- 172.21.15.59:0/3141848061 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f185c101670 0x7f185c102a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:32.996 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:32.989+0000 7f185bfff700 1 -- 172.21.15.59:0/3141848061 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f185c100480 msgr2=0x7f185c1043c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:32.996 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:32.989+0000 7f185bfff700 1 --2- 172.21.15.59:0/3141848061 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f185c100480 0x7f185c1043c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:32.996 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:32.989+0000 7f185bfff700 1 -- 172.21.15.59:0/3141848061 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f184c0095f0 con 0x7f185c102010 2023-12-11T17:44:32.996 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:32.989+0000 7f185bfff700 1 --2- 172.21.15.59:0/3141848061 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f185c102010 0x7f185c102f90 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f185400ada0 tx=0x7f185400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:44:32.996 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:32.989+0000 7f18597fa700 1 -- 172.21.15.59:0/3141848061 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f185400f070 con 0x7f185c102010 2023-12-11T17:44:32.996 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:32.989+0000 7f1861f7e700 1 -- 172.21.15.59:0/3141848061 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f185c1a1830 con 0x7f185c102010 2023-12-11T17:44:32.996 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:32.989+0000 7f18597fa700 1 -- 172.21.15.59:0/3141848061 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1854007b30 con 0x7f185c102010 2023-12-11T17:44:32.997 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:32.989+0000 7f18597fa700 1 -- 172.21.15.59:0/3141848061 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1854005310 con 0x7f185c102010 2023-12-11T17:44:32.997 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:32.989+0000 7f1861f7e700 1 -- 172.21.15.59:0/3141848061 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f185c1a1d40 con 0x7f185c102010 2023-12-11T17:44:32.997 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:32.990+0000 7f1861f7e700 1 -- 172.21.15.59:0/3141848061 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f185c068310 con 0x7f185c102010 2023-12-11T17:44:32.999 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:32.991+0000 7f18597fa700 1 -- 172.21.15.59:0/3141848061 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f185400c920 con 0x7f185c102010 2023-12-11T17:44:32.999 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:32.992+0000 7f18597fa700 1 --2- 172.21.15.59:0/3141848061 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f18480a1480 0x7f18480a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:32.999 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:32.992+0000 7f185b7fe700 1 --2- 172.21.15.59:0/3141848061 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f18480a1480 0x7f18480a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:32.999 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:32.992+0000 7f18597fa700 1 -- 172.21.15.59:0/3141848061 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f18540cc520 con 0x7f185c102010 2023-12-11T17:44:32.999 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:32.993+0000 7f185b7fe700 1 --2- 172.21.15.59:0/3141848061 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f18480a1480 0x7f18480a3940 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f184c003820 tx=0x7f184c005920 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:44:32.999 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:32.996+0000 7f18597fa700 1 -- 172.21.15.59:0/3141848061 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f1854095a20 con 0x7f185c102010 2023-12-11T17:44:33.200 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:33.198+0000 7f1861f7e700 1 -- 172.21.15.59:0/3141848061 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f185c1a20a0 con 0x7f185c102010 2023-12-11T17:44:33.201 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:33.198+0000 7f18597fa700 1 -- 172.21.15.59:0/3141848061 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v86) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1854099040 con 0x7f185c102010 2023-12-11T17:44:33.201 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2023-12-11T17:44:33.204 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:33.201+0000 7f1861f7e700 1 -- 172.21.15.59:0/3141848061 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f18480a1480 msgr2=0x7f18480a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:33.204 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:33.201+0000 7f1861f7e700 1 --2- 172.21.15.59:0/3141848061 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f18480a1480 0x7f18480a3940 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f184c003820 tx=0x7f184c005920 comp rx=0 tx=0).stop 2023-12-11T17:44:33.204 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:33.201+0000 7f1861f7e700 1 -- 172.21.15.59:0/3141848061 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f185c102010 msgr2=0x7f185c102f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:33.204 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:33.201+0000 7f1861f7e700 1 --2- 172.21.15.59:0/3141848061 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f185c102010 0x7f185c102f90 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f185400ada0 tx=0x7f185400c330 comp rx=0 tx=0).stop 2023-12-11T17:44:33.204 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:33.201+0000 7f1861f7e700 1 -- 172.21.15.59:0/3141848061 shutdown_connections 2023-12-11T17:44:33.204 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:33.201+0000 7f1861f7e700 1 --2- 172.21.15.59:0/3141848061 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f18480a1480 0x7f18480a3940 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:33.204 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:33.201+0000 7f1861f7e700 1 --2- 172.21.15.59:0/3141848061 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f185c102010 0x7f185c102f90 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:33.204 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:33.201+0000 7f1861f7e700 1 --2- 172.21.15.59:0/3141848061 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f185c100480 0x7f185c1043c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:33.204 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:33.201+0000 7f1861f7e700 1 --2- 172.21.15.59:0/3141848061 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f185c101670 0x7f185c102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:33.204 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:33.201+0000 7f1861f7e700 1 -- 172.21.15.59:0/3141848061 >> 172.21.15.59:0/3141848061 conn(0x7f185c0fb9f0 msgr2=0x7f185c068680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:44:33.204 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:33.201+0000 7f1861f7e700 1 -- 172.21.15.59:0/3141848061 shutdown_connections 2023-12-11T17:44:33.205 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:33.201+0000 7f1861f7e700 1 -- 172.21.15.59:0/3141848061 wait complete. 2023-12-11T17:44:33.216 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-11T17:44:33.345 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:33.342+0000 7ff56c43c700 1 -- 172.21.15.59:0/154674240 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff564102010 msgr2=0x7ff5641088c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:33.345 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:33.342+0000 7ff56c43c700 1 --2- 172.21.15.59:0/154674240 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff564102010 0x7ff5641088c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff55c00b0d0 tx=0x7ff55c00b3e0 comp rx=0 tx=0).stop 2023-12-11T17:44:33.345 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:33.342+0000 7ff56c43c700 1 -- 172.21.15.59:0/154674240 shutdown_connections 2023-12-11T17:44:33.345 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:33.342+0000 7ff56c43c700 1 --2- 172.21.15.59:0/154674240 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff564102010 0x7ff5641088c0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:33.345 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:33.342+0000 7ff56c43c700 1 --2- 172.21.15.59:0/154674240 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff564100480 0x7ff5641008a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:33.345 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:33.342+0000 7ff56c43c700 1 --2- 172.21.15.59:0/154674240 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff564101670 0x7ff564101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:33.345 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:33.342+0000 7ff56c43c700 1 -- 172.21.15.59:0/154674240 >> 172.21.15.59:0/154674240 conn(0x7ff5640fb9f0 msgr2=0x7ff5640fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:44:33.345 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:33.342+0000 7ff56c43c700 1 -- 172.21.15.59:0/154674240 shutdown_connections 2023-12-11T17:44:33.345 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:33.342+0000 7ff56c43c700 1 -- 172.21.15.59:0/154674240 wait complete. 2023-12-11T17:44:33.347 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:33.343+0000 7ff56c43c700 1 Processor -- start 2023-12-11T17:44:33.348 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:33.343+0000 7ff56c43c700 1 -- start start 2023-12-11T17:44:33.348 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:33.343+0000 7ff56c43c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff564100480 0x7ff564069150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:33.348 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:33.343+0000 7ff56c43c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff564101670 0x7ff56419c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:33.348 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:33.343+0000 7ff56c43c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff564102010 0x7ff56419e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:33.348 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:33.343+0000 7ff56c43c700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff56419f110 con 0x7ff564102010 2023-12-11T17:44:33.348 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:33.343+0000 7ff56c43c700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff56419f280 con 0x7ff564101670 2023-12-11T17:44:33.348 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:33.343+0000 7ff56c43c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff56419f440 con 0x7ff564100480 2023-12-11T17:44:33.348 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:33.343+0000 7ff56a9d9700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff564102010 0x7ff56419e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:33.348 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:33.343+0000 7ff56a9d9700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff564102010 0x7ff56419e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33116/0 (socket says 172.21.15.59:33116) 2023-12-11T17:44:33.348 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:33.343+0000 7ff56a9d9700 1 -- 172.21.15.59:0/2469657733 learned_addr learned my addr 172.21.15.59:0/2469657733 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:44:33.348 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:33.343+0000 7ff56a1d8700 1 --2- 172.21.15.59:0/2469657733 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff564100480 0x7ff564069150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:33.348 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:33.343+0000 7ff5699d7700 1 --2- 172.21.15.59:0/2469657733 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff564101670 0x7ff56419c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:33.349 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:33.343+0000 7ff56a9d9700 1 -- 172.21.15.59:0/2469657733 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff564100480 msgr2=0x7ff564069150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:33.349 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:33.343+0000 7ff56a9d9700 1 --2- 172.21.15.59:0/2469657733 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff564100480 0x7ff564069150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:33.349 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:33.343+0000 7ff56a9d9700 1 -- 172.21.15.59:0/2469657733 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff564101670 msgr2=0x7ff56419c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:33.349 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:33.343+0000 7ff56a9d9700 1 --2- 172.21.15.59:0/2469657733 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff564101670 0x7ff56419c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:33.349 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:33.343+0000 7ff56a9d9700 1 -- 172.21.15.59:0/2469657733 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff55c009d00 con 0x7ff564102010 2023-12-11T17:44:33.349 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:33.343+0000 7ff5699d7700 1 --2- 172.21.15.59:0/2469657733 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff564101670 0x7ff56419c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T17:44:33.349 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:33.343+0000 7ff56a9d9700 1 --2- 172.21.15.59:0/2469657733 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff564102010 0x7ff56419e920 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ff55c007bd0 tx=0x7ff55c009440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:44:33.349 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:33.344+0000 7ff5577fe700 1 -- 172.21.15.59:0/2469657733 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff55c00e040 con 0x7ff564102010 2023-12-11T17:44:33.349 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:33.344+0000 7ff5577fe700 1 -- 172.21.15.59:0/2469657733 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff55c01dd50 con 0x7ff564102010 2023-12-11T17:44:33.349 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:33.344+0000 7ff56c43c700 1 -- 172.21.15.59:0/2469657733 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff56419f660 con 0x7ff564102010 2023-12-11T17:44:33.349 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:33.344+0000 7ff5577fe700 1 -- 172.21.15.59:0/2469657733 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff55c01b450 con 0x7ff564102010 2023-12-11T17:44:33.349 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:33.344+0000 7ff56c43c700 1 -- 172.21.15.59:0/2469657733 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff56419fa90 con 0x7ff564102010 2023-12-11T17:44:33.349 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:33.345+0000 7ff56c43c700 1 -- 172.21.15.59:0/2469657733 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff564107830 con 0x7ff564102010 2023-12-11T17:44:33.350 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:33.346+0000 7ff5577fe700 1 -- 172.21.15.59:0/2469657733 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7ff55c01d890 con 0x7ff564102010 2023-12-11T17:44:33.350 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:33.346+0000 7ff5577fe700 1 --2- 172.21.15.59:0/2469657733 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7ff5500a1480 0x7ff5500a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:33.350 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:33.347+0000 7ff56a1d8700 1 --2- 172.21.15.59:0/2469657733 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7ff5500a1480 0x7ff5500a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:33.351 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:33.347+0000 7ff5577fe700 1 -- 172.21.15.59:0/2469657733 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7ff55c0ceee0 con 0x7ff564102010 2023-12-11T17:44:33.351 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:33.347+0000 7ff56a1d8700 1 --2- 172.21.15.59:0/2469657733 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7ff5500a1480 0x7ff5500a3940 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff5641014d0 tx=0x7ff560007400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:44:33.353 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:33.351+0000 7ff5577fe700 1 -- 172.21.15.59:0/2469657733 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7ff55c098380 con 0x7ff564102010 2023-12-11T17:44:33.553 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:33.550+0000 7ff56c43c700 1 -- 172.21.15.59:0/2469657733 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff564107210 con 0x7ff564102010 2023-12-11T17:44:33.553 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:33.551+0000 7ff5577fe700 1 -- 172.21.15.59:0/2469657733 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v86) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff55c09b9a0 con 0x7ff564102010 2023-12-11T17:44:33.554 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2023-12-11T17:44:33.556 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:33.553+0000 7ff56c43c700 1 -- 172.21.15.59:0/2469657733 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7ff5500a1480 msgr2=0x7ff5500a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:33.556 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:33.553+0000 7ff56c43c700 1 --2- 172.21.15.59:0/2469657733 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7ff5500a1480 0x7ff5500a3940 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff5641014d0 tx=0x7ff560007400 comp rx=0 tx=0).stop 2023-12-11T17:44:33.556 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:33.553+0000 7ff56c43c700 1 -- 172.21.15.59:0/2469657733 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff564102010 msgr2=0x7ff56419e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:33.556 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:33.553+0000 7ff56c43c700 1 --2- 172.21.15.59:0/2469657733 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff564102010 0x7ff56419e920 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ff55c007bd0 tx=0x7ff55c009440 comp rx=0 tx=0).stop 2023-12-11T17:44:33.556 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:33.553+0000 7ff56c43c700 1 -- 172.21.15.59:0/2469657733 shutdown_connections 2023-12-11T17:44:33.556 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:33.553+0000 7ff56c43c700 1 --2- 172.21.15.59:0/2469657733 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7ff5500a1480 0x7ff5500a3940 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:33.556 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:33.553+0000 7ff56c43c700 1 --2- 172.21.15.59:0/2469657733 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff564102010 0x7ff56419e920 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:33.556 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:33.553+0000 7ff56c43c700 1 --2- 172.21.15.59:0/2469657733 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff564100480 0x7ff564069150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:33.556 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:33.553+0000 7ff56c43c700 1 --2- 172.21.15.59:0/2469657733 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff564101670 0x7ff56419c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:33.557 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:33.553+0000 7ff56c43c700 1 -- 172.21.15.59:0/2469657733 >> 172.21.15.59:0/2469657733 conn(0x7ff5640fb9f0 msgr2=0x7ff564104970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:44:33.557 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:33.553+0000 7ff56c43c700 1 -- 172.21.15.59:0/2469657733 shutdown_connections 2023-12-11T17:44:33.557 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:33.553+0000 7ff56c43c700 1 -- 172.21.15.59:0/2469657733 wait complete. 2023-12-11T17:44:33.568 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-11T17:44:33.699 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:33.694+0000 7fa4ae497700 1 -- 172.21.15.59:0/735075212 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4a8103130 msgr2=0x7fa4a81099e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:33.699 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:33.694+0000 7fa4ae497700 1 --2- 172.21.15.59:0/735075212 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4a8103130 0x7fa4a81099e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa4a000b0d0 tx=0x7fa4a000b3e0 comp rx=0 tx=0).stop 2023-12-11T17:44:33.699 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:33.694+0000 7fa4ae497700 1 -- 172.21.15.59:0/735075212 shutdown_connections 2023-12-11T17:44:33.699 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:33.694+0000 7fa4ae497700 1 --2- 172.21.15.59:0/735075212 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4a8103130 0x7fa4a81099e0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:33.699 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:33.694+0000 7fa4ae497700 1 --2- 172.21.15.59:0/735075212 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa4a81015a0 0x7fa4a81019c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:33.699 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:33.694+0000 7fa4ae497700 1 --2- 172.21.15.59:0/735075212 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa4a8102790 0x7fa4a8102bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:33.699 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:33.694+0000 7fa4ae497700 1 -- 172.21.15.59:0/735075212 >> 172.21.15.59:0/735075212 conn(0x7fa4a80fcb10 msgr2=0x7fa4a80fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:44:33.699 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:33.694+0000 7fa4ae497700 1 -- 172.21.15.59:0/735075212 shutdown_connections 2023-12-11T17:44:33.699 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:33.694+0000 7fa4ae497700 1 -- 172.21.15.59:0/735075212 wait complete. 2023-12-11T17:44:33.699 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:33.695+0000 7fa4ae497700 1 Processor -- start 2023-12-11T17:44:33.699 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:33.695+0000 7fa4ae497700 1 -- start start 2023-12-11T17:44:33.700 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:33.695+0000 7fa4ae497700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa4a81015a0 0x7fa4a81975b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:33.700 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:33.695+0000 7fa4ae497700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa4a8102790 0x7fa4a8197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:33.700 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:33.695+0000 7fa4ae497700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4a8103130 0x7fa4a819eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:33.700 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:33.695+0000 7fa4ae497700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a819f0e0 con 0x7fa4a8103130 2023-12-11T17:44:33.700 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:33.695+0000 7fa4ae497700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a819f220 con 0x7fa4a81015a0 2023-12-11T17:44:33.700 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:33.695+0000 7fa4ae497700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a819f3b0 con 0x7fa4a8102790 2023-12-11T17:44:33.700 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:33.695+0000 7fa4aca34700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4a8103130 0x7fa4a819eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:33.700 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:33.695+0000 7fa4aca34700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4a8103130 0x7fa4a819eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33128/0 (socket says 172.21.15.59:33128) 2023-12-11T17:44:33.700 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:33.695+0000 7fa4aca34700 1 -- 172.21.15.59:0/3961654510 learned_addr learned my addr 172.21.15.59:0/3961654510 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:44:33.700 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:33.695+0000 7fa4a7fff700 1 --2- 172.21.15.59:0/3961654510 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa4a81015a0 0x7fa4a81975b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:33.700 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:33.695+0000 7fa4a77fe700 1 --2- 172.21.15.59:0/3961654510 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa4a8102790 0x7fa4a8197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:33.700 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:33.695+0000 7fa4aca34700 1 -- 172.21.15.59:0/3961654510 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa4a8102790 msgr2=0x7fa4a8197af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:33.700 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:33.695+0000 7fa4aca34700 1 --2- 172.21.15.59:0/3961654510 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa4a8102790 0x7fa4a8197af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:33.700 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:33.695+0000 7fa4aca34700 1 -- 172.21.15.59:0/3961654510 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa4a81015a0 msgr2=0x7fa4a81975b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:33.700 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:33.695+0000 7fa4aca34700 1 --2- 172.21.15.59:0/3961654510 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa4a81015a0 0x7fa4a81975b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:33.701 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:33.695+0000 7fa4aca34700 1 -- 172.21.15.59:0/3961654510 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4a0009d00 con 0x7fa4a8103130 2023-12-11T17:44:33.701 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:33.695+0000 7fa4a77fe700 1 --2- 172.21.15.59:0/3961654510 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa4a8102790 0x7fa4a8197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-11T17:44:33.701 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:33.695+0000 7fa4a7fff700 1 --2- 172.21.15.59:0/3961654510 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa4a81015a0 0x7fa4a81975b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T17:44:33.701 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:33.695+0000 7fa4aca34700 1 --2- 172.21.15.59:0/3961654510 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4a8103130 0x7fa4a819eb10 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fa4a0009fd0 tx=0x7fa4a0009510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:44:33.701 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:33.696+0000 7fa4a57fa700 1 -- 172.21.15.59:0/3961654510 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4a000e040 con 0x7fa4a8103130 2023-12-11T17:44:33.701 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:33.696+0000 7fa4a57fa700 1 -- 172.21.15.59:0/3961654510 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa4a0012a70 con 0x7fa4a8103130 2023-12-11T17:44:33.701 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:33.696+0000 7fa4a57fa700 1 -- 172.21.15.59:0/3961654510 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4a00056a0 con 0x7fa4a8103130 2023-12-11T17:44:33.701 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:33.696+0000 7fa4ae497700 1 -- 172.21.15.59:0/3961654510 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4a819f630 con 0x7fa4a8103130 2023-12-11T17:44:33.701 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:33.696+0000 7fa4ae497700 1 -- 172.21.15.59:0/3961654510 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4a819fa60 con 0x7fa4a8103130 2023-12-11T17:44:33.702 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:33.697+0000 7fa4ae497700 1 -- 172.21.15.59:0/3961654510 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4a8108950 con 0x7fa4a8103130 2023-12-11T17:44:33.702 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:33.698+0000 7fa4a57fa700 1 -- 172.21.15.59:0/3961654510 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7fa4a0012be0 con 0x7fa4a8103130 2023-12-11T17:44:33.702 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:33.698+0000 7fa4a57fa700 1 --2- 172.21.15.59:0/3961654510 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fa4900a1480 0x7fa4900a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:33.702 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:33.699+0000 7fa4a7fff700 1 --2- 172.21.15.59:0/3961654510 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fa4900a1480 0x7fa4900a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:33.702 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:33.699+0000 7fa4a57fa700 1 -- 172.21.15.59:0/3961654510 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fa4a00ce9e0 con 0x7fa4a8103130 2023-12-11T17:44:33.702 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:33.699+0000 7fa4a7fff700 1 --2- 172.21.15.59:0/3961654510 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fa4900a1480 0x7fa4900a3940 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fa4a81025f0 tx=0x7fa49800a680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:44:33.705 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:33.703+0000 7fa4a57fa700 1 -- 172.21.15.59:0/3961654510 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fa4a0097ee0 con 0x7fa4a8103130 2023-12-11T17:44:33.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:33 smithi059 ceph-mon[98847]: pgmap v265: 129 pgs: 129 active+clean; 579 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 383 B/s wr, 3 op/s 2023-12-11T17:44:33.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:33 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/2303272100' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-11T17:44:33.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:33 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/3141848061' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-11T17:44:33.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:33 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/2469657733' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-11T17:44:33.909 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:33.907+0000 7fa4ae497700 1 -- 172.21.15.59:0/3961654510 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa4a8108330 con 0x7fa4a8103130 2023-12-11T17:44:33.910 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:33.908+0000 7fa4a57fa700 1 -- 172.21.15.59:0/3961654510 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v86) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa4a009b500 con 0x7fa4a8103130 2023-12-11T17:44:33.910 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2023-12-11T17:44:33.912 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:33.910+0000 7fa4ae497700 1 -- 172.21.15.59:0/3961654510 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fa4900a1480 msgr2=0x7fa4900a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:33.912 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:33.910+0000 7fa4ae497700 1 --2- 172.21.15.59:0/3961654510 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fa4900a1480 0x7fa4900a3940 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fa4a81025f0 tx=0x7fa49800a680 comp rx=0 tx=0).stop 2023-12-11T17:44:33.912 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:33.910+0000 7fa4ae497700 1 -- 172.21.15.59:0/3961654510 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4a8103130 msgr2=0x7fa4a819eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:33.912 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:33.910+0000 7fa4ae497700 1 --2- 172.21.15.59:0/3961654510 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4a8103130 0x7fa4a819eb10 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fa4a0009fd0 tx=0x7fa4a0009510 comp rx=0 tx=0).stop 2023-12-11T17:44:33.913 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:33.910+0000 7fa4ae497700 1 -- 172.21.15.59:0/3961654510 shutdown_connections 2023-12-11T17:44:33.913 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:33.910+0000 7fa4ae497700 1 --2- 172.21.15.59:0/3961654510 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fa4900a1480 0x7fa4900a3940 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:33.913 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:33.910+0000 7fa4ae497700 1 --2- 172.21.15.59:0/3961654510 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4a8103130 0x7fa4a819eb10 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:33.913 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:33.910+0000 7fa4ae497700 1 --2- 172.21.15.59:0/3961654510 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa4a81015a0 0x7fa4a81975b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:33.913 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:33.910+0000 7fa4ae497700 1 --2- 172.21.15.59:0/3961654510 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa4a8102790 0x7fa4a8197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:33.913 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:33.910+0000 7fa4ae497700 1 -- 172.21.15.59:0/3961654510 >> 172.21.15.59:0/3961654510 conn(0x7fa4a80fcb10 msgr2=0x7fa4a8105a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:44:33.913 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:33.910+0000 7fa4ae497700 1 -- 172.21.15.59:0/3961654510 shutdown_connections 2023-12-11T17:44:33.913 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:33.910+0000 7fa4ae497700 1 -- 172.21.15.59:0/3961654510 wait complete. 2023-12-11T17:44:33.924 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-12-11T17:44:34.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:33 smithi115 ceph-mon[102748]: pgmap v265: 129 pgs: 129 active+clean; 579 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 383 B/s wr, 3 op/s 2023-12-11T17:44:34.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:33 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/2303272100' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-11T17:44:34.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:33 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/3141848061' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-11T17:44:34.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:33 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/2469657733' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-11T17:44:34.056 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.050+0000 7f049a793700 1 -- 172.21.15.59:0/2259973247 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0494101670 msgr2=0x7f0494101ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:34.056 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.050+0000 7f049a793700 1 --2- 172.21.15.59:0/2259973247 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0494101670 0x7f0494101ad0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f04880099c0 tx=0x7f0488009cd0 comp rx=0 tx=0).stop 2023-12-11T17:44:34.056 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.050+0000 7f049a793700 1 -- 172.21.15.59:0/2259973247 shutdown_connections 2023-12-11T17:44:34.056 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.050+0000 7f049a793700 1 --2- 172.21.15.59:0/2259973247 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0494102010 0x7f04941088c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:34.057 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.050+0000 7f049a793700 1 --2- 172.21.15.59:0/2259973247 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0494100480 0x7f04941008a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:34.057 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.050+0000 7f049a793700 1 --2- 172.21.15.59:0/2259973247 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0494101670 0x7f0494101ad0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:34.057 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.050+0000 7f049a793700 1 -- 172.21.15.59:0/2259973247 >> 172.21.15.59:0/2259973247 conn(0x7f04940fb9f0 msgr2=0x7f04940fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:44:34.057 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.050+0000 7f049a793700 1 -- 172.21.15.59:0/2259973247 shutdown_connections 2023-12-11T17:44:34.057 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.050+0000 7f049a793700 1 -- 172.21.15.59:0/2259973247 wait complete. 2023-12-11T17:44:34.057 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.050+0000 7f049a793700 1 Processor -- start 2023-12-11T17:44:34.057 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.051+0000 7f049a793700 1 -- start start 2023-12-11T17:44:34.057 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.051+0000 7f049a793700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0494100480 0x7f0494069130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:34.057 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.051+0000 7f049a793700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0494101670 0x7f049419c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:34.057 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.051+0000 7f049a793700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0494102010 0x7f049419e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:34.057 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.051+0000 7f049a793700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f049419f020 con 0x7f0494102010 2023-12-11T17:44:34.057 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.051+0000 7f049a793700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f049419f190 con 0x7f0494101670 2023-12-11T17:44:34.057 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.051+0000 7f049a793700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f049419f350 con 0x7f0494100480 2023-12-11T17:44:34.057 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.051+0000 7f0498d30700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0494102010 0x7f049419e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:34.058 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.051+0000 7f0498d30700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0494102010 0x7f049419e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33142/0 (socket says 172.21.15.59:33142) 2023-12-11T17:44:34.058 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.051+0000 7f0498d30700 1 -- 172.21.15.59:0/4070635125 learned_addr learned my addr 172.21.15.59:0/4070635125 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:44:34.058 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.051+0000 7f0493fff700 1 --2- 172.21.15.59:0/4070635125 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0494100480 0x7f0494069130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:34.058 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.051+0000 7f04937fe700 1 --2- 172.21.15.59:0/4070635125 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0494101670 0x7f049419c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:34.058 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.051+0000 7f0498d30700 1 -- 172.21.15.59:0/4070635125 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0494100480 msgr2=0x7f0494069130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:34.058 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.051+0000 7f0498d30700 1 --2- 172.21.15.59:0/4070635125 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0494100480 0x7f0494069130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:34.058 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.051+0000 7f0498d30700 1 -- 172.21.15.59:0/4070635125 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0494101670 msgr2=0x7f049419c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:34.058 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.051+0000 7f0498d30700 1 --2- 172.21.15.59:0/4070635125 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0494101670 0x7f049419c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:34.058 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.051+0000 7f0498d30700 1 -- 172.21.15.59:0/4070635125 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04880095f0 con 0x7f0494102010 2023-12-11T17:44:34.059 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.051+0000 7f04937fe700 1 --2- 172.21.15.59:0/4070635125 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0494101670 0x7f049419c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-11T17:44:34.059 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.051+0000 7f0498d30700 1 --2- 172.21.15.59:0/4070635125 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0494102010 0x7f049419e880 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f048c00ada0 tx=0x7f048c00c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:44:34.059 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.052+0000 7f04917fa700 1 -- 172.21.15.59:0/4070635125 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f048c00f070 con 0x7f0494102010 2023-12-11T17:44:34.059 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.052+0000 7f04917fa700 1 -- 172.21.15.59:0/4070635125 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f048c007b30 con 0x7f0494102010 2023-12-11T17:44:34.059 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.052+0000 7f049a793700 1 -- 172.21.15.59:0/4070635125 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f049419f630 con 0x7f0494102010 2023-12-11T17:44:34.059 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.052+0000 7f04917fa700 1 -- 172.21.15.59:0/4070635125 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f048c005440 con 0x7f0494102010 2023-12-11T17:44:34.059 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.052+0000 7f049a793700 1 -- 172.21.15.59:0/4070635125 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f049419fb40 con 0x7f0494102010 2023-12-11T17:44:34.059 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.053+0000 7f049a793700 1 -- 172.21.15.59:0/4070635125 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0494107900 con 0x7f0494102010 2023-12-11T17:44:34.059 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.054+0000 7f04917fa700 1 -- 172.21.15.59:0/4070635125 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f048c004ad0 con 0x7f0494102010 2023-12-11T17:44:34.060 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.054+0000 7f04917fa700 1 --2- 172.21.15.59:0/4070635125 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f047c0a1500 0x7f047c0a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:34.061 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.054+0000 7f0493fff700 1 --2- 172.21.15.59:0/4070635125 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f047c0a1500 0x7f047c0a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:34.061 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.054+0000 7f04917fa700 1 -- 172.21.15.59:0/4070635125 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f048c0cc520 con 0x7f0494102010 2023-12-11T17:44:34.061 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.055+0000 7f0493fff700 1 --2- 172.21.15.59:0/4070635125 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f047c0a1500 0x7f047c0a39c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0484005fd0 tx=0x7f048400a560 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:44:34.061 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.059+0000 7f04917fa700 1 -- 172.21.15.59:0/4070635125 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f048c095a20 con 0x7f0494102010 2023-12-11T17:44:34.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:33 smithi181 ceph-mon[103627]: pgmap v265: 129 pgs: 129 active+clean; 579 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 383 B/s wr, 3 op/s 2023-12-11T17:44:34.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:33 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/2303272100' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-11T17:44:34.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:33 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/3141848061' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-11T17:44:34.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:33 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/2469657733' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-11T17:44:34.265 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.263+0000 7f049a793700 1 -- 172.21.15.59:0/4070635125 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f049410be40 con 0x7f0494102010 2023-12-11T17:44:34.266 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.263+0000 7f04917fa700 1 -- 172.21.15.59:0/4070635125 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v86) v1 ==== 103+0+11 (secure 0 0 0) 0x7f048c099040 con 0x7f0494102010 2023-12-11T17:44:34.266 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2023-12-11T17:44:34.268 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.266+0000 7f049a793700 1 -- 172.21.15.59:0/4070635125 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f047c0a1500 msgr2=0x7f047c0a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:34.268 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.266+0000 7f049a793700 1 --2- 172.21.15.59:0/4070635125 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f047c0a1500 0x7f047c0a39c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0484005fd0 tx=0x7f048400a560 comp rx=0 tx=0).stop 2023-12-11T17:44:34.268 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.266+0000 7f049a793700 1 -- 172.21.15.59:0/4070635125 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0494102010 msgr2=0x7f049419e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:34.269 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.266+0000 7f049a793700 1 --2- 172.21.15.59:0/4070635125 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0494102010 0x7f049419e880 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f048c00ada0 tx=0x7f048c00c330 comp rx=0 tx=0).stop 2023-12-11T17:44:34.269 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.266+0000 7f049a793700 1 -- 172.21.15.59:0/4070635125 shutdown_connections 2023-12-11T17:44:34.269 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.266+0000 7f049a793700 1 --2- 172.21.15.59:0/4070635125 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f047c0a1500 0x7f047c0a39c0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:34.269 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.266+0000 7f049a793700 1 --2- 172.21.15.59:0/4070635125 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0494102010 0x7f049419e880 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:34.269 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.266+0000 7f049a793700 1 --2- 172.21.15.59:0/4070635125 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0494100480 0x7f0494069130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:34.269 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.266+0000 7f049a793700 1 --2- 172.21.15.59:0/4070635125 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0494101670 0x7f049419c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:34.269 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.266+0000 7f049a793700 1 -- 172.21.15.59:0/4070635125 >> 172.21.15.59:0/4070635125 conn(0x7f04940fb9f0 msgr2=0x7f0494104a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:44:34.269 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.266+0000 7f049a793700 1 -- 172.21.15.59:0/4070635125 shutdown_connections 2023-12-11T17:44:34.269 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.266+0000 7f049a793700 1 -- 172.21.15.59:0/4070635125 wait complete. 2023-12-11T17:44:34.281 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-11T17:44:34.409 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.404+0000 7f4ef32d5700 1 -- 172.21.15.59:0/941193605 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4eec101670 msgr2=0x7f4eec101ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:34.409 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.404+0000 7f4ef32d5700 1 --2- 172.21.15.59:0/941193605 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4eec101670 0x7f4eec101ad0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f4ee80099c0 tx=0x7f4ee8009cd0 comp rx=0 tx=0).stop 2023-12-11T17:44:34.409 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.404+0000 7f4ef32d5700 1 -- 172.21.15.59:0/941193605 shutdown_connections 2023-12-11T17:44:34.409 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.404+0000 7f4ef32d5700 1 --2- 172.21.15.59:0/941193605 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4eec102010 0x7f4eec1088c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:34.409 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.404+0000 7f4ef32d5700 1 --2- 172.21.15.59:0/941193605 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4eec100480 0x7f4eec1008a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:34.410 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.404+0000 7f4ef32d5700 1 --2- 172.21.15.59:0/941193605 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4eec101670 0x7f4eec101ad0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:34.410 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.404+0000 7f4ef32d5700 1 -- 172.21.15.59:0/941193605 >> 172.21.15.59:0/941193605 conn(0x7f4eec0fb9f0 msgr2=0x7f4eec0fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:44:34.410 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.404+0000 7f4ef32d5700 1 -- 172.21.15.59:0/941193605 shutdown_connections 2023-12-11T17:44:34.410 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.404+0000 7f4ef32d5700 1 -- 172.21.15.59:0/941193605 wait complete. 2023-12-11T17:44:34.410 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.405+0000 7f4ef32d5700 1 Processor -- start 2023-12-11T17:44:34.410 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.405+0000 7f4ef32d5700 1 -- start start 2023-12-11T17:44:34.410 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.405+0000 7f4ef32d5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4eec100480 0x7f4eec069130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:34.410 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.405+0000 7f4ef32d5700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4eec101670 0x7f4eec19c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:34.410 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.405+0000 7f4ef32d5700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4eec102010 0x7f4eec19e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:34.410 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.405+0000 7f4ef32d5700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eec19f010 con 0x7f4eec101670 2023-12-11T17:44:34.410 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.405+0000 7f4ef32d5700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eec19f180 con 0x7f4eec102010 2023-12-11T17:44:34.410 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.405+0000 7f4ef32d5700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eec19f340 con 0x7f4eec100480 2023-12-11T17:44:34.410 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.405+0000 7f4ef0870700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4eec101670 0x7f4eec19c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:34.411 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.405+0000 7f4ef1071700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4eec100480 0x7f4eec069130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:34.411 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.405+0000 7f4ef0870700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4eec101670 0x7f4eec19c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33154/0 (socket says 172.21.15.59:33154) 2023-12-11T17:44:34.411 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.405+0000 7f4ef0870700 1 -- 172.21.15.59:0/1026879538 learned_addr learned my addr 172.21.15.59:0/1026879538 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:44:34.411 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.405+0000 7f4ef1872700 1 --2- 172.21.15.59:0/1026879538 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4eec102010 0x7f4eec19e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:34.411 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.405+0000 7f4ef0870700 1 -- 172.21.15.59:0/1026879538 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4eec100480 msgr2=0x7f4eec069130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:34.411 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.405+0000 7f4ef0870700 1 --2- 172.21.15.59:0/1026879538 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4eec100480 0x7f4eec069130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:34.411 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.405+0000 7f4ef0870700 1 -- 172.21.15.59:0/1026879538 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4eec102010 msgr2=0x7f4eec19e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:34.411 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.405+0000 7f4ef0870700 1 --2- 172.21.15.59:0/1026879538 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4eec102010 0x7f4eec19e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:34.411 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.405+0000 7f4ef0870700 1 -- 172.21.15.59:0/1026879538 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ee80095f0 con 0x7f4eec101670 2023-12-11T17:44:34.411 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.406+0000 7f4ef0870700 1 --2- 172.21.15.59:0/1026879538 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4eec101670 0x7f4eec19c280 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f4ee8006010 tx=0x7f4ee8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:44:34.411 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.406+0000 7f4ede7fc700 1 -- 172.21.15.59:0/1026879538 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee801d070 con 0x7f4eec101670 2023-12-11T17:44:34.411 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.406+0000 7f4ede7fc700 1 -- 172.21.15.59:0/1026879538 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4ee8004050 con 0x7f4eec101670 2023-12-11T17:44:34.411 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.406+0000 7f4ef32d5700 1 -- 172.21.15.59:0/1026879538 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4eec19f5c0 con 0x7f4eec101670 2023-12-11T17:44:34.411 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.406+0000 7f4ede7fc700 1 -- 172.21.15.59:0/1026879538 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee8005290 con 0x7f4eec101670 2023-12-11T17:44:34.411 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.406+0000 7f4ef32d5700 1 -- 172.21.15.59:0/1026879538 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4eec19fa40 con 0x7f4eec101670 2023-12-11T17:44:34.412 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.407+0000 7f4ed7fff700 1 -- 172.21.15.59:0/1026879538 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4eec068310 con 0x7f4eec101670 2023-12-11T17:44:34.412 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.408+0000 7f4ede7fc700 1 -- 172.21.15.59:0/1026879538 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f4ee80048e0 con 0x7f4eec101670 2023-12-11T17:44:34.412 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.409+0000 7f4ede7fc700 1 --2- 172.21.15.59:0/1026879538 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f4ed80a1500 0x7f4ed80a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:34.412 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.409+0000 7f4ef1071700 1 --2- 172.21.15.59:0/1026879538 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f4ed80a1500 0x7f4ed80a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:34.413 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.409+0000 7f4ede7fc700 1 -- 172.21.15.59:0/1026879538 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f4ee80ce0a0 con 0x7f4eec101670 2023-12-11T17:44:34.413 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.409+0000 7f4ef1071700 1 --2- 172.21.15.59:0/1026879538 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f4ed80a1500 0x7f4ed80a39c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f4eec1014d0 tx=0x7f4ee000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:44:34.414 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.412+0000 7f4ede7fc700 1 -- 172.21.15.59:0/1026879538 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f4ee8097540 con 0x7f4eec101670 2023-12-11T17:44:34.633 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.631+0000 7f4ed7fff700 1 -- 172.21.15.59:0/1026879538 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f4eec10be50 con 0x7f4eec101670 2023-12-11T17:44:34.634 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.631+0000 7f4ede7fc700 1 -- 172.21.15.59:0/1026879538 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v86) v1 ==== 81+0+590 (secure 0 0 0) 0x7f4ee8026080 con 0x7f4eec101670 2023-12-11T17:44:34.634 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:6800/1259916482 2023-12-12T17:38:14.037922+0000 2023-12-11T17:44:34.634 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:0/17457774 2023-12-12T17:38:14.037922+0000 2023-12-11T17:44:34.634 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:6801/1672530462 2023-12-12T17:37:44.773925+0000 2023-12-11T17:44:34.634 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:0/1053944547 2023-12-12T17:38:14.037922+0000 2023-12-11T17:44:34.634 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:6801/1259916482 2023-12-12T17:38:14.037922+0000 2023-12-11T17:44:34.634 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:6800/1672530462 2023-12-12T17:37:44.773925+0000 2023-12-11T17:44:34.634 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:0/3570767650 2023-12-12T17:37:44.773925+0000 2023-12-11T17:44:34.634 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:0/3853462899 2023-12-12T17:37:44.773925+0000 2023-12-11T17:44:34.635 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:0/2859415380 2023-12-12T17:38:14.037922+0000 2023-12-11T17:44:34.635 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:0/2192742819 2023-12-12T17:37:44.773925+0000 2023-12-11T17:44:34.636 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.633+0000 7f4ed7fff700 1 -- 172.21.15.59:0/1026879538 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f4ed80a1500 msgr2=0x7f4ed80a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:34.637 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.633+0000 7f4ed7fff700 1 --2- 172.21.15.59:0/1026879538 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f4ed80a1500 0x7f4ed80a39c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f4eec1014d0 tx=0x7f4ee000b410 comp rx=0 tx=0).stop 2023-12-11T17:44:34.637 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.633+0000 7f4ed7fff700 1 -- 172.21.15.59:0/1026879538 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4eec101670 msgr2=0x7f4eec19c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:34.637 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.633+0000 7f4ed7fff700 1 --2- 172.21.15.59:0/1026879538 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4eec101670 0x7f4eec19c280 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f4ee8006010 tx=0x7f4ee8004730 comp rx=0 tx=0).stop 2023-12-11T17:44:34.637 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.633+0000 7f4ed7fff700 1 -- 172.21.15.59:0/1026879538 shutdown_connections 2023-12-11T17:44:34.637 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.633+0000 7f4ed7fff700 1 --2- 172.21.15.59:0/1026879538 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f4ed80a1500 0x7f4ed80a39c0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:34.637 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.633+0000 7f4ed7fff700 1 --2- 172.21.15.59:0/1026879538 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4eec102010 0x7f4eec19e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:34.637 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.633+0000 7f4ed7fff700 1 --2- 172.21.15.59:0/1026879538 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4eec100480 0x7f4eec069130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:34.637 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.633+0000 7f4ed7fff700 1 --2- 172.21.15.59:0/1026879538 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4eec101670 0x7f4eec19c280 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:34.637 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.633+0000 7f4ed7fff700 1 -- 172.21.15.59:0/1026879538 >> 172.21.15.59:0/1026879538 conn(0x7f4eec0fb9f0 msgr2=0x7f4eec104a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:44:34.637 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.633+0000 7f4ed7fff700 1 -- 172.21.15.59:0/1026879538 shutdown_connections 2023-12-11T17:44:34.637 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.634+0000 7f4ed7fff700 1 -- 172.21.15.59:0/1026879538 wait complete. 2023-12-11T17:44:34.638 INFO:teuthology.orchestra.run.smithi059.stderr:listed 10 entries 2023-12-11T17:44:34.648 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2023-12-11T17:44:34.648 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-11T17:44:34.668 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-11T17:44:34.836 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.831+0000 7f16602e6700 1 -- 172.21.15.59:0/3871632904 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f16580ffb10 msgr2=0x7f16580fff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:34.836 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.831+0000 7f16602e6700 1 --2- 172.21.15.59:0/3871632904 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f16580ffb10 0x7f16580fff30 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f16540099c0 tx=0x7f1654009cd0 comp rx=0 tx=0).stop 2023-12-11T17:44:34.836 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.831+0000 7f16602e6700 1 -- 172.21.15.59:0/3871632904 shutdown_connections 2023-12-11T17:44:34.836 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.831+0000 7f16602e6700 1 --2- 172.21.15.59:0/3871632904 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f16581016a0 0x7f1658107f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:34.836 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.831+0000 7f16602e6700 1 --2- 172.21.15.59:0/3871632904 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f16580ffb10 0x7f16580fff30 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:34.836 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.831+0000 7f16602e6700 1 --2- 172.21.15.59:0/3871632904 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1658100d00 0x7f1658101160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:34.836 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.831+0000 7f16602e6700 1 -- 172.21.15.59:0/3871632904 >> 172.21.15.59:0/3871632904 conn(0x7f16580fb0a0 msgr2=0x7f16580fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:44:34.836 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.831+0000 7f16602e6700 1 -- 172.21.15.59:0/3871632904 shutdown_connections 2023-12-11T17:44:34.836 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.831+0000 7f16602e6700 1 -- 172.21.15.59:0/3871632904 wait complete. 2023-12-11T17:44:34.836 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.831+0000 7f16602e6700 1 Processor -- start 2023-12-11T17:44:34.837 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.831+0000 7f16602e6700 1 -- start start 2023-12-11T17:44:34.837 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.831+0000 7f16602e6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f16580ffb10 0x7f16581969b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:34.837 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.831+0000 7f16602e6700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1658100d00 0x7f1658195040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:34.837 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.831+0000 7f16602e6700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f16581016a0 0x7f1658195580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:34.837 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.831+0000 7f16602e6700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1658195d20 con 0x7f1658100d00 2023-12-11T17:44:34.837 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.831+0000 7f16602e6700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1658195e90 con 0x7f16581016a0 2023-12-11T17:44:34.837 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.831+0000 7f16602e6700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16581a14d0 con 0x7f16580ffb10 2023-12-11T17:44:34.837 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.832+0000 7f165d881700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1658100d00 0x7f1658195040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:34.837 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.832+0000 7f165d881700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1658100d00 0x7f1658195040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33168/0 (socket says 172.21.15.59:33168) 2023-12-11T17:44:34.837 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.832+0000 7f165d881700 1 -- 172.21.15.59:0/1628065330 learned_addr learned my addr 172.21.15.59:0/1628065330 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:44:34.837 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.832+0000 7f165e883700 1 --2- 172.21.15.59:0/1628065330 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f16581016a0 0x7f1658195580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:34.837 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.832+0000 7f165e082700 1 --2- 172.21.15.59:0/1628065330 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f16580ffb10 0x7f16581969b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:34.837 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.832+0000 7f165d881700 1 -- 172.21.15.59:0/1628065330 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f16580ffb10 msgr2=0x7f16581969b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:34.837 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.832+0000 7f165d881700 1 --2- 172.21.15.59:0/1628065330 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f16580ffb10 0x7f16581969b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:34.838 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.832+0000 7f165d881700 1 -- 172.21.15.59:0/1628065330 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f16581016a0 msgr2=0x7f1658195580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:34.838 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.832+0000 7f165d881700 1 --2- 172.21.15.59:0/1628065330 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f16581016a0 0x7f1658195580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:34.838 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.832+0000 7f165d881700 1 -- 172.21.15.59:0/1628065330 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16540095f0 con 0x7f1658100d00 2023-12-11T17:44:34.838 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.832+0000 7f165d881700 1 --2- 172.21.15.59:0/1628065330 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1658100d00 0x7f1658195040 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f164800eab0 tx=0x7f164800edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:44:34.838 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.832+0000 7f164f7fe700 1 -- 172.21.15.59:0/1628065330 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f164800ca70 con 0x7f1658100d00 2023-12-11T17:44:34.838 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.832+0000 7f164f7fe700 1 -- 172.21.15.59:0/1628065330 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f164800cbd0 con 0x7f1658100d00 2023-12-11T17:44:34.838 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.832+0000 7f16602e6700 1 -- 172.21.15.59:0/1628065330 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16581a1760 con 0x7f1658100d00 2023-12-11T17:44:34.838 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.832+0000 7f164f7fe700 1 -- 172.21.15.59:0/1628065330 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f164800b5c0 con 0x7f1658100d00 2023-12-11T17:44:34.838 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.832+0000 7f16602e6700 1 -- 172.21.15.59:0/1628065330 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16581a1cc0 con 0x7f1658100d00 2023-12-11T17:44:34.838 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.833+0000 7f16602e6700 1 -- 172.21.15.59:0/1628065330 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1658106f80 con 0x7f1658100d00 2023-12-11T17:44:34.839 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.835+0000 7f164f7fe700 1 -- 172.21.15.59:0/1628065330 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f1648016410 con 0x7f1658100d00 2023-12-11T17:44:34.839 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.836+0000 7f164f7fe700 1 --2- 172.21.15.59:0/1628065330 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f16440a1500 0x7f16440a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:34.840 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.836+0000 7f165e082700 1 --2- 172.21.15.59:0/1628065330 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f16440a1500 0x7f16440a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:34.840 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.836+0000 7f164f7fe700 1 -- 172.21.15.59:0/1628065330 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f16480ccdd0 con 0x7f1658100d00 2023-12-11T17:44:34.840 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.836+0000 7f165e082700 1 --2- 172.21.15.59:0/1628065330 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f16440a1500 0x7f16440a39c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1654009990 tx=0x7f165401a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:44:34.842 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:34.840+0000 7f164f7fe700 1 -- 172.21.15.59:0/1628065330 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f16480961e0 con 0x7f1658100d00 2023-12-11T17:44:34.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:34 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/3961654510' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-11T17:44:34.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:34 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/4070635125' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-11T17:44:34.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:34 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/1026879538' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-12-11T17:44:35.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:34 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/3961654510' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-11T17:44:35.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:34 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/4070635125' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-11T17:44:35.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:34 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/1026879538' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-12-11T17:44:35.046 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.044+0000 7f16602e6700 1 -- 172.21.15.59:0/1628065330 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f16580679a0 con 0x7f1658100d00 2023-12-11T17:44:35.048 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.045+0000 7f164f7fe700 1 -- 172.21.15.59:0/1628065330 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+23960 (secure 0 0 0) 0x7f1648025090 con 0x7f1658100d00 2023-12-11T17:44:35.048 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-12-11T17:44:35.048 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":86,"fsid":"d0a07eaa-984b-11ee-95a3-87774f69a715","created":"2023-12-11T17:37:14.526539+0000","modified":"2023-12-11T17:44:25.096094+0000","last_up_change":"2023-12-11T17:43:38.535503+0000","last_in_change":"2023-12-11T17:43:21.014888+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-11T17:40:26.596029+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-11T17:44:18.586290+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-11T17:44:19.432952+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-11T17:44:23.097308+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3c0b3218-4b2b-40f8-8bef-ed8cd5633607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":1040759398},{"type":"v1","addr":"172.21.15.59:6803","nonce":1040759398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":1040759398},{"type":"v1","addr":"172.21.15.59:6805","nonce":1040759398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":1040759398},{"type":"v1","addr":"172.21.15.59:6809","nonce":1040759398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":1040759398},{"type":"v1","addr":"172.21.15.59:6807","nonce":1040759398}]},"public_addr":"172.21.15.59:6803/1040759398","cluster_addr":"172.21.15.59:6805/1040759398","heartbeat_back_addr":"172.21.15.59:6809/1040759398","heartbeat_front_addr":"172.21.15.59:6807/1040759398","state":["exists","up"]},{"osd":1,"uuid":"2ead7578-765b-4f5a-a72c-863731ed69e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":2776354245},{"type":"v1","addr":"172.21.15.115:6801","nonce":2776354245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":2776354245},{"type":"v1","addr":"172.21.15.115:6803","nonce":2776354245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":2776354245},{"type":"v1","addr":"172.21.15.115:6807","nonce":2776354245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":2776354245},{"type":"v1","addr":"172.21.15.115:6805","nonce":2776354245}]},"public_addr":"172.21.15.115:6801/2776354245","cluster_addr":"172.21.15.115:6803/2776354245","heartbeat_back_addr":"172.21.15.115:6807/2776354245","heartbeat_front_addr":"172.21.15.115:6805/2776354245","state":["exists","up"]},{"osd":2,"uuid":"8cb1d7ec-53cc-4672-b0b6-70d73797c44f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4287675481},{"type":"v1","addr":"172.21.15.181:6801","nonce":4287675481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4287675481},{"type":"v1","addr":"172.21.15.181:6803","nonce":4287675481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4287675481},{"type":"v1","addr":"172.21.15.181:6807","nonce":4287675481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4287675481},{"type":"v1","addr":"172.21.15.181:6805","nonce":4287675481}]},"public_addr":"172.21.15.181:6801/4287675481","cluster_addr":"172.21.15.181:6803/4287675481","heartbeat_back_addr":"172.21.15.181:6807/4287675481","heartbeat_front_addr":"172.21.15.181:6805/4287675481","state":["exists","up"]},{"osd":3,"uuid":"5d64d9af-831b-4d0a-8ace-a452169351d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":219909995},{"type":"v1","addr":"172.21.15.59:6811","nonce":219909995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":219909995},{"type":"v1","addr":"172.21.15.59:6813","nonce":219909995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":219909995},{"type":"v1","addr":"172.21.15.59:6817","nonce":219909995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":219909995},{"type":"v1","addr":"172.21.15.59:6815","nonce":219909995}]},"public_addr":"172.21.15.59:6811/219909995","cluster_addr":"172.21.15.59:6813/219909995","heartbeat_back_addr":"172.21.15.59:6817/219909995","heartbeat_front_addr":"172.21.15.59:6815/219909995","state":["exists","up"]},{"osd":4,"uuid":"ef8fad28-1db5-4cba-b5f2-10e54e8e1549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":1021473826},{"type":"v1","addr":"172.21.15.115:6809","nonce":1021473826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":1021473826},{"type":"v1","addr":"172.21.15.115:6811","nonce":1021473826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":1021473826},{"type":"v1","addr":"172.21.15.115:6815","nonce":1021473826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":1021473826},{"type":"v1","addr":"172.21.15.115:6813","nonce":1021473826}]},"public_addr":"172.21.15.115:6809/1021473826","cluster_addr":"172.21.15.115:6811/1021473826","heartbeat_back_addr":"172.21.15.115:6815/1021473826","heartbeat_front_addr":"172.21.15.115:6813/1021473826","state":["exists","up"]},{"osd":5,"uuid":"4233a512-9e8f-4db8-aed2-6c3af706e589","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":2405172521},{"type":"v1","addr":"172.21.15.181:6809","nonce":2405172521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":2405172521},{"type":"v1","addr":"172.21.15.181:6811","nonce":2405172521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2405172521},{"type":"v1","addr":"172.21.15.181:6815","nonce":2405172521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":2405172521},{"type":"v1","addr":"172.21.15.181:6813","nonce":2405172521}]},"public_addr":"172.21.15.181:6809/2405172521","cluster_addr":"172.21.15.181:6811/2405172521","heartbeat_back_addr":"172.21.15.181:6815/2405172521","heartbeat_front_addr":"172.21.15.181:6813/2405172521","state":["exists","up"]},{"osd":6,"uuid":"8faab332-016e-4c77-8370-8bef95fc6f34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":815374853},{"type":"v1","addr":"172.21.15.59:6819","nonce":815374853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":815374853},{"type":"v1","addr":"172.21.15.59:6821","nonce":815374853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":815374853},{"type":"v1","addr":"172.21.15.59:6825","nonce":815374853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":815374853},{"type":"v1","addr":"172.21.15.59:6823","nonce":815374853}]},"public_addr":"172.21.15.59:6819/815374853","cluster_addr":"172.21.15.59:6821/815374853","heartbeat_back_addr":"172.21.15.59:6825/815374853","heartbeat_front_addr":"172.21.15.59:6823/815374853","state":["exists","up"]},{"osd":7,"uuid":"f20f470a-0d40-4ed3-8e0e-fd58675cf143","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":1314971870},{"type":"v1","addr":"172.21.15.115:6817","nonce":1314971870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":1314971870},{"type":"v1","addr":"172.21.15.115:6819","nonce":1314971870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":1314971870},{"type":"v1","addr":"172.21.15.115:6823","nonce":1314971870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":1314971870},{"type":"v1","addr":"172.21.15.115:6821","nonce":1314971870}]},"public_addr":"172.21.15.115:6817/1314971870","cluster_addr":"172.21.15.115:6819/1314971870","heartbeat_back_addr":"172.21.15.115:6823/1314971870","heartbeat_front_addr":"172.21.15.115:6821/1314971870","state":["exists","up"]},{"osd":8,"uuid":"eb657d32-1a95-4fd7-a293-d0a970837fc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":3954509873},{"type":"v1","addr":"172.21.15.181:6817","nonce":3954509873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":3954509873},{"type":"v1","addr":"172.21.15.181:6819","nonce":3954509873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":3954509873},{"type":"v1","addr":"172.21.15.181:6823","nonce":3954509873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":3954509873},{"type":"v1","addr":"172.21.15.181:6821","nonce":3954509873}]},"public_addr":"172.21.15.181:6817/3954509873","cluster_addr":"172.21.15.181:6819/3954509873","heartbeat_back_addr":"172.21.15.181:6823/3954509873","heartbeat_front_addr":"172.21.15.181:6821/3954509873","state":["exists","up"]},{"osd":9,"uuid":"8f7717d4-b6f2-4ec1-9b58-62feb0c9a146","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":1163095677},{"type":"v1","addr":"172.21.15.59:6827","nonce":1163095677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":1163095677},{"type":"v1","addr":"172.21.15.59:6829","nonce":1163095677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":1163095677},{"type":"v1","addr":"172.21.15.59:6833","nonce":1163095677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":1163095677},{"type":"v1","addr":"172.21.15.59:6831","nonce":1163095677}]},"public_addr":"172.21.15.59:6827/1163095677","cluster_addr":"172.21.15.59:6829/1163095677","heartbeat_back_addr":"172.21.15.59:6833/1163095677","heartbeat_front_addr":"172.21.15.59:6831/1163095677","state":["exists","up"]},{"osd":10,"uuid":"b75c179a-875d-43bc-b4c1-e8eb80fb64fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":2199610269},{"type":"v1","addr":"172.21.15.115:6825","nonce":2199610269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":2199610269},{"type":"v1","addr":"172.21.15.115:6827","nonce":2199610269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":2199610269},{"type":"v1","addr":"172.21.15.115:6831","nonce":2199610269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":2199610269},{"type":"v1","addr":"172.21.15.115:6829","nonce":2199610269}]},"public_addr":"172.21.15.115:6825/2199610269","cluster_addr":"172.21.15.115:6827/2199610269","heartbeat_back_addr":"172.21.15.115:6831/2199610269","heartbeat_front_addr":"172.21.15.115:6829/2199610269","state":["exists","up"]},{"osd":11,"uuid":"5e141dfe-8a00-4bbc-8204-2dc9be24e18d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":226506989},{"type":"v1","addr":"172.21.15.181:6825","nonce":226506989}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":226506989},{"type":"v1","addr":"172.21.15.181:6827","nonce":226506989}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":226506989},{"type":"v1","addr":"172.21.15.181:6831","nonce":226506989}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":226506989},{"type":"v1","addr":"172.21.15.181:6829","nonce":226506989}]},"public_addr":"172.21.15.181:6825/226506989","cluster_addr":"172.21.15.181:6827/226506989","heartbeat_back_addr":"172.21.15.181:6831/226506989","heartbeat_front_addr":"172.21.15.181:6829/226506989","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:39:42.687084+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:40:03.903448+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:40:24.899908+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:40:46.600251+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:41:07.690037+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:41:28.488855+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:41:49.459274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:42:10.863881+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:42:32.248391+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:42:54.112265+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:43:15.634872+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:43:36.922534+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.59:6800/1259916482":"2023-12-12T17:38:14.037922+0000","172.21.15.59:0/17457774":"2023-12-12T17:38:14.037922+0000","172.21.15.59:6801/1672530462":"2023-12-12T17:37:44.773925+0000","172.21.15.59:0/1053944547":"2023-12-12T17:38:14.037922+0000","172.21.15.59:6801/1259916482":"2023-12-12T17:38:14.037922+0000","172.21.15.59:6800/1672530462":"2023-12-12T17:37:44.773925+0000","172.21.15.59:0/3570767650":"2023-12-12T17:37:44.773925+0000","172.21.15.59:0/3853462899":"2023-12-12T17:37:44.773925+0000","172.21.15.59:0/2859415380":"2023-12-12T17:38:14.037922+0000","172.21.15.59:0/2192742819":"2023-12-12T17:37:44.773925+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-11T17:44:35.050 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.047+0000 7f16602e6700 1 -- 172.21.15.59:0/1628065330 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f16440a1500 msgr2=0x7f16440a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:35.050 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.047+0000 7f16602e6700 1 --2- 172.21.15.59:0/1628065330 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f16440a1500 0x7f16440a39c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1654009990 tx=0x7f165401a040 comp rx=0 tx=0).stop 2023-12-11T17:44:35.050 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.047+0000 7f16602e6700 1 -- 172.21.15.59:0/1628065330 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1658100d00 msgr2=0x7f1658195040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:35.051 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.047+0000 7f16602e6700 1 --2- 172.21.15.59:0/1628065330 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1658100d00 0x7f1658195040 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f164800eab0 tx=0x7f164800edc0 comp rx=0 tx=0).stop 2023-12-11T17:44:35.051 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.048+0000 7f16602e6700 1 -- 172.21.15.59:0/1628065330 shutdown_connections 2023-12-11T17:44:35.051 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.048+0000 7f16602e6700 1 --2- 172.21.15.59:0/1628065330 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f16440a1500 0x7f16440a39c0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:35.051 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.048+0000 7f16602e6700 1 --2- 172.21.15.59:0/1628065330 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f16581016a0 0x7f1658195580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:35.051 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.048+0000 7f16602e6700 1 --2- 172.21.15.59:0/1628065330 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f16580ffb10 0x7f16581969b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:35.051 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.048+0000 7f16602e6700 1 --2- 172.21.15.59:0/1628065330 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1658100d00 0x7f1658195040 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:35.051 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.048+0000 7f16602e6700 1 -- 172.21.15.59:0/1628065330 >> 172.21.15.59:0/1628065330 conn(0x7f16580fb0a0 msgr2=0x7f16581040c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:44:35.051 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.048+0000 7f16602e6700 1 -- 172.21.15.59:0/1628065330 shutdown_connections 2023-12-11T17:44:35.051 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.048+0000 7f16602e6700 1 -- 172.21.15.59:0/1628065330 wait complete. 2023-12-11T17:44:35.063 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-11T17:40:26.596029+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-11T17:44:18.586290+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-11T17:44:19.432952+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-12-11T17:44:23.097308+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-12-11T17:44:35.063 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-11T17:44:35.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:34 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/3961654510' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-11T17:44:35.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:34 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/4070635125' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-11T17:44:35.146 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:34 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/1026879538' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-12-11T17:44:35.194 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.190+0000 7f146d273700 1 -- 172.21.15.59:0/1021405982 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f14681015a0 msgr2=0x7f14681019c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:35.194 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.190+0000 7f146d273700 1 --2- 172.21.15.59:0/1021405982 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f14681015a0 0x7f14681019c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f145c0099c0 tx=0x7f145c009cd0 comp rx=0 tx=0).stop 2023-12-11T17:44:35.194 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.190+0000 7f146d273700 1 -- 172.21.15.59:0/1021405982 shutdown_connections 2023-12-11T17:44:35.194 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.190+0000 7f146d273700 1 --2- 172.21.15.59:0/1021405982 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1468103130 0x7f14681099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:35.195 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.190+0000 7f146d273700 1 --2- 172.21.15.59:0/1021405982 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f14681015a0 0x7f14681019c0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:35.195 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.190+0000 7f146d273700 1 --2- 172.21.15.59:0/1021405982 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1468102790 0x7f1468102bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:35.195 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.190+0000 7f146d273700 1 -- 172.21.15.59:0/1021405982 >> 172.21.15.59:0/1021405982 conn(0x7f14680fcb10 msgr2=0x7f14680fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:44:35.195 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.190+0000 7f146d273700 1 -- 172.21.15.59:0/1021405982 shutdown_connections 2023-12-11T17:44:35.195 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.190+0000 7f146d273700 1 -- 172.21.15.59:0/1021405982 wait complete. 2023-12-11T17:44:35.195 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.190+0000 7f146d273700 1 Processor -- start 2023-12-11T17:44:35.195 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.191+0000 7f146d273700 1 -- start start 2023-12-11T17:44:35.195 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.191+0000 7f146d273700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f14681015a0 0x7f1468199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:35.195 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.191+0000 7f146d273700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1468102790 0x7f1468199cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:35.195 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.191+0000 7f146d273700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1468103130 0x7f14681a0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:35.195 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.191+0000 7f146d273700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14681a12a0 con 0x7f1468103130 2023-12-11T17:44:35.195 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.191+0000 7f146d273700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14681a13e0 con 0x7f1468102790 2023-12-11T17:44:35.195 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.191+0000 7f146d273700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14681a1570 con 0x7f14681015a0 2023-12-11T17:44:35.195 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.191+0000 7f14677fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1468103130 0x7f14681a0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:35.195 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.191+0000 7f14677fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1468103130 0x7f14681a0cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33184/0 (socket says 172.21.15.59:33184) 2023-12-11T17:44:35.196 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.191+0000 7f14677fe700 1 -- 172.21.15.59:0/1261458485 learned_addr learned my addr 172.21.15.59:0/1261458485 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:44:35.196 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.191+0000 7f1466ffd700 1 --2- 172.21.15.59:0/1261458485 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f14681015a0 0x7f1468199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:35.196 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.191+0000 7f14667fc700 1 --2- 172.21.15.59:0/1261458485 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1468102790 0x7f1468199cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:35.196 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.191+0000 7f14677fe700 1 -- 172.21.15.59:0/1261458485 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f14681015a0 msgr2=0x7f1468199770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:35.196 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.191+0000 7f14677fe700 1 --2- 172.21.15.59:0/1261458485 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f14681015a0 0x7f1468199770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:35.196 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.191+0000 7f14677fe700 1 -- 172.21.15.59:0/1261458485 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1468102790 msgr2=0x7f1468199cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:35.196 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.191+0000 7f14677fe700 1 --2- 172.21.15.59:0/1261458485 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1468102790 0x7f1468199cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:35.196 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.191+0000 7f14677fe700 1 -- 172.21.15.59:0/1261458485 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f145c0095f0 con 0x7f1468103130 2023-12-11T17:44:35.196 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.191+0000 7f14677fe700 1 --2- 172.21.15.59:0/1261458485 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1468103130 0x7f14681a0cd0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f1460007ee0 tx=0x7f146000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:44:35.196 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.191+0000 7f144ffff700 1 -- 172.21.15.59:0/1261458485 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1460017070 con 0x7f1468103130 2023-12-11T17:44:35.196 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.192+0000 7f144ffff700 1 -- 172.21.15.59:0/1261458485 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f146000f040 con 0x7f1468103130 2023-12-11T17:44:35.196 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.192+0000 7f146d273700 1 -- 172.21.15.59:0/1261458485 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14681a1850 con 0x7f1468103130 2023-12-11T17:44:35.196 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.192+0000 7f144ffff700 1 -- 172.21.15.59:0/1261458485 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1460005580 con 0x7f1468103130 2023-12-11T17:44:35.197 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.192+0000 7f146d273700 1 -- 172.21.15.59:0/1261458485 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14681a1d10 con 0x7f1468103130 2023-12-11T17:44:35.198 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.192+0000 7f146d273700 1 -- 172.21.15.59:0/1261458485 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1468068310 con 0x7f1468103130 2023-12-11T17:44:35.198 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.194+0000 7f144ffff700 1 -- 172.21.15.59:0/1261458485 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f146000db20 con 0x7f1468103130 2023-12-11T17:44:35.198 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.194+0000 7f144ffff700 1 --2- 172.21.15.59:0/1261458485 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f14500a1480 0x7f14500a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:35.198 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.194+0000 7f144ffff700 1 -- 172.21.15.59:0/1261458485 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f14600ccc50 con 0x7f1468103130 2023-12-11T17:44:35.198 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.194+0000 7f1466ffd700 1 --2- 172.21.15.59:0/1261458485 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f14500a1480 0x7f14500a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:35.198 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.195+0000 7f1466ffd700 1 --2- 172.21.15.59:0/1261458485 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f14500a1480 0x7f14500a3940 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f145c006010 tx=0x7f145c00b540 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:44:35.200 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.198+0000 7f144ffff700 1 -- 172.21.15.59:0/1261458485 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f14600960c0 con 0x7f1468103130 2023-12-11T17:44:35.401 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.399+0000 7f146d273700 1 -- 172.21.15.59:0/1261458485 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f14681a1fd0 con 0x7f1468103130 2023-12-11T17:44:35.402 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.399+0000 7f144ffff700 1 -- 172.21.15.59:0/1261458485 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v86) v1 ==== 93+0+10 (secure 0 0 0) 0x7f14600996e0 con 0x7f1468103130 2023-12-11T17:44:35.402 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2023-12-11T17:44:35.404 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.401+0000 7f146d273700 1 -- 172.21.15.59:0/1261458485 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f14500a1480 msgr2=0x7f14500a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:35.404 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.401+0000 7f146d273700 1 --2- 172.21.15.59:0/1261458485 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f14500a1480 0x7f14500a3940 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f145c006010 tx=0x7f145c00b540 comp rx=0 tx=0).stop 2023-12-11T17:44:35.404 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.401+0000 7f146d273700 1 -- 172.21.15.59:0/1261458485 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1468103130 msgr2=0x7f14681a0cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:35.404 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.401+0000 7f146d273700 1 --2- 172.21.15.59:0/1261458485 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1468103130 0x7f14681a0cd0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f1460007ee0 tx=0x7f146000d3b0 comp rx=0 tx=0).stop 2023-12-11T17:44:35.405 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.401+0000 7f146d273700 1 -- 172.21.15.59:0/1261458485 shutdown_connections 2023-12-11T17:44:35.405 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.401+0000 7f146d273700 1 --2- 172.21.15.59:0/1261458485 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f14500a1480 0x7f14500a3940 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:35.405 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.401+0000 7f146d273700 1 --2- 172.21.15.59:0/1261458485 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1468103130 0x7f14681a0cd0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:35.405 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.401+0000 7f146d273700 1 --2- 172.21.15.59:0/1261458485 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f14681015a0 0x7f1468199770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:35.405 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.401+0000 7f146d273700 1 --2- 172.21.15.59:0/1261458485 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1468102790 0x7f1468199cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:35.405 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.401+0000 7f146d273700 1 -- 172.21.15.59:0/1261458485 >> 172.21.15.59:0/1261458485 conn(0x7f14680fcb10 msgr2=0x7f1468107fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:44:35.405 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.402+0000 7f146d273700 1 -- 172.21.15.59:0/1261458485 shutdown_connections 2023-12-11T17:44:35.405 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.402+0000 7f146d273700 1 -- 172.21.15.59:0/1261458485 wait complete. 2023-12-11T17:44:35.416 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-11T17:44:35.546 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.543+0000 7f1718ae6700 1 -- 172.21.15.59:0/579038325 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1714104990 msgr2=0x7f1714106d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:35.546 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.543+0000 7f1718ae6700 1 --2- 172.21.15.59:0/579038325 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1714104990 0x7f1714106d80 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f1708009990 tx=0x7f1708009ca0 comp rx=0 tx=0).stop 2023-12-11T17:44:35.546 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.543+0000 7f1718ae6700 1 -- 172.21.15.59:0/579038325 shutdown_connections 2023-12-11T17:44:35.546 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.543+0000 7f1718ae6700 1 --2- 172.21.15.59:0/579038325 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f17141072c0 0x7f17141096d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:35.546 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.543+0000 7f1718ae6700 1 --2- 172.21.15.59:0/579038325 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1714102060 0x7f1714104450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:35.546 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.543+0000 7f1718ae6700 1 --2- 172.21.15.59:0/579038325 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1714104990 0x7f1714106d80 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:35.547 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.543+0000 7f1718ae6700 1 -- 172.21.15.59:0/579038325 >> 172.21.15.59:0/579038325 conn(0x7f17140fb9f0 msgr2=0x7f17140fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:44:35.547 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.543+0000 7f1718ae6700 1 -- 172.21.15.59:0/579038325 shutdown_connections 2023-12-11T17:44:35.547 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.543+0000 7f1718ae6700 1 -- 172.21.15.59:0/579038325 wait complete. 2023-12-11T17:44:35.547 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.544+0000 7f1718ae6700 1 Processor -- start 2023-12-11T17:44:35.547 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.544+0000 7f1718ae6700 1 -- start start 2023-12-11T17:44:35.549 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.544+0000 7f1718ae6700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1714102060 0x7f17141974d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:35.549 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.544+0000 7f1718ae6700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1714104990 0x7f1714197a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:35.549 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.544+0000 7f1718ae6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f17141072c0 0x7f171419ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:35.549 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.544+0000 7f1718ae6700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f171419f000 con 0x7f1714104990 2023-12-11T17:44:35.549 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.544+0000 7f1718ae6700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f171419f140 con 0x7f1714102060 2023-12-11T17:44:35.549 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.544+0000 7f1718ae6700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f171419f2d0 con 0x7f17141072c0 2023-12-11T17:44:35.549 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.544+0000 7f1711d9b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1714104990 0x7f1714197a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:35.549 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.544+0000 7f1711d9b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1714104990 0x7f1714197a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33196/0 (socket says 172.21.15.59:33196) 2023-12-11T17:44:35.549 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.544+0000 7f1711d9b700 1 -- 172.21.15.59:0/941343939 learned_addr learned my addr 172.21.15.59:0/941343939 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:44:35.549 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.544+0000 7f171259c700 1 --2- 172.21.15.59:0/941343939 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1714102060 0x7f17141974d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:35.549 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.544+0000 7f1712d9d700 1 --2- 172.21.15.59:0/941343939 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f17141072c0 0x7f171419ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:35.549 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.544+0000 7f1711d9b700 1 -- 172.21.15.59:0/941343939 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f17141072c0 msgr2=0x7f171419ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:35.549 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.544+0000 7f1711d9b700 1 --2- 172.21.15.59:0/941343939 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f17141072c0 0x7f171419ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:35.550 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.544+0000 7f1711d9b700 1 -- 172.21.15.59:0/941343939 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1714102060 msgr2=0x7f17141974d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:35.550 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.544+0000 7f1711d9b700 1 --2- 172.21.15.59:0/941343939 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1714102060 0x7f17141974d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:35.550 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.544+0000 7f1711d9b700 1 -- 172.21.15.59:0/941343939 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17080095f0 con 0x7f1714104990 2023-12-11T17:44:35.550 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.544+0000 7f1712d9d700 1 --2- 172.21.15.59:0/941343939 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f17141072c0 0x7f171419ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-11T17:44:35.550 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.545+0000 7f1711d9b700 1 --2- 172.21.15.59:0/941343939 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1714104990 0x7f1714197a10 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f17080095c0 tx=0x7f170800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:44:35.550 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.545+0000 7f17037fe700 1 -- 172.21.15.59:0/941343939 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f170801d070 con 0x7f1714104990 2023-12-11T17:44:35.550 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.545+0000 7f1718ae6700 1 -- 172.21.15.59:0/941343939 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f171419f550 con 0x7f1714104990 2023-12-11T17:44:35.550 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.545+0000 7f17037fe700 1 -- 172.21.15.59:0/941343939 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1708022470 con 0x7f1714104990 2023-12-11T17:44:35.550 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.545+0000 7f17037fe700 1 -- 172.21.15.59:0/941343939 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1708017650 con 0x7f1714104990 2023-12-11T17:44:35.550 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.545+0000 7f1718ae6700 1 -- 172.21.15.59:0/941343939 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f171419f970 con 0x7f1714104990 2023-12-11T17:44:35.550 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.546+0000 7f1718ae6700 1 -- 172.21.15.59:0/941343939 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17140fd570 con 0x7f1714104990 2023-12-11T17:44:35.551 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.548+0000 7f17037fe700 1 -- 172.21.15.59:0/941343939 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f1708022850 con 0x7f1714104990 2023-12-11T17:44:35.551 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.548+0000 7f17037fe700 1 --2- 172.21.15.59:0/941343939 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f16fc0a1480 0x7f16fc0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:35.551 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.548+0000 7f17037fe700 1 -- 172.21.15.59:0/941343939 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f17080ce910 con 0x7f1714104990 2023-12-11T17:44:35.551 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.548+0000 7f171259c700 1 --2- 172.21.15.59:0/941343939 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f16fc0a1480 0x7f16fc0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:35.551 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.549+0000 7f171259c700 1 --2- 172.21.15.59:0/941343939 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f16fc0a1480 0x7f16fc0a3940 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1704005950 tx=0x7f170400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:44:35.554 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.552+0000 7f17037fe700 1 -- 172.21.15.59:0/941343939 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f1708097f30 con 0x7f1714104990 2023-12-11T17:44:35.758 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.756+0000 7f1718ae6700 1 -- 172.21.15.59:0/941343939 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f171410ce90 con 0x7f1714104990 2023-12-11T17:44:35.759 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.757+0000 7f17037fe700 1 -- 172.21.15.59:0/941343939 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v86) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1708027020 con 0x7f1714104990 2023-12-11T17:44:35.759 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2023-12-11T17:44:35.761 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.759+0000 7f1718ae6700 1 -- 172.21.15.59:0/941343939 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f16fc0a1480 msgr2=0x7f16fc0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:35.761 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.759+0000 7f1718ae6700 1 --2- 172.21.15.59:0/941343939 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f16fc0a1480 0x7f16fc0a3940 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1704005950 tx=0x7f170400b410 comp rx=0 tx=0).stop 2023-12-11T17:44:35.762 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.759+0000 7f1718ae6700 1 -- 172.21.15.59:0/941343939 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1714104990 msgr2=0x7f1714197a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:35.762 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.759+0000 7f1718ae6700 1 --2- 172.21.15.59:0/941343939 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1714104990 0x7f1714197a10 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f17080095c0 tx=0x7f170800f690 comp rx=0 tx=0).stop 2023-12-11T17:44:35.762 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.759+0000 7f1718ae6700 1 -- 172.21.15.59:0/941343939 shutdown_connections 2023-12-11T17:44:35.762 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.759+0000 7f1718ae6700 1 --2- 172.21.15.59:0/941343939 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f16fc0a1480 0x7f16fc0a3940 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:35.762 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.759+0000 7f1718ae6700 1 --2- 172.21.15.59:0/941343939 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f17141072c0 0x7f171419ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:35.762 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.759+0000 7f1718ae6700 1 --2- 172.21.15.59:0/941343939 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1714102060 0x7f17141974d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:35.762 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.759+0000 7f1718ae6700 1 --2- 172.21.15.59:0/941343939 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1714104990 0x7f1714197a10 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:35.762 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.759+0000 7f1718ae6700 1 -- 172.21.15.59:0/941343939 >> 172.21.15.59:0/941343939 conn(0x7f17140fb9f0 msgr2=0x7f1714103f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:44:35.762 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.759+0000 7f1718ae6700 1 -- 172.21.15.59:0/941343939 shutdown_connections 2023-12-11T17:44:35.762 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.759+0000 7f1718ae6700 1 -- 172.21.15.59:0/941343939 wait complete. 2023-12-11T17:44:35.773 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-11T17:44:35.903 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.899+0000 7f1292786700 1 -- 172.21.15.59:0/3268160149 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f128c1016a0 msgr2=0x7f128c101b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:35.903 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.899+0000 7f1292786700 1 --2- 172.21.15.59:0/3268160149 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f128c1016a0 0x7f128c101b00 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f12800099c0 tx=0x7f1280009cd0 comp rx=0 tx=0).stop 2023-12-11T17:44:35.904 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.899+0000 7f1292786700 1 -- 172.21.15.59:0/3268160149 shutdown_connections 2023-12-11T17:44:35.904 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.899+0000 7f1292786700 1 --2- 172.21.15.59:0/3268160149 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f128c102040 0x7f128c1088f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:35.904 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.899+0000 7f1292786700 1 --2- 172.21.15.59:0/3268160149 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f128c1004b0 0x7f128c1008d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:35.904 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.899+0000 7f1292786700 1 --2- 172.21.15.59:0/3268160149 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f128c1016a0 0x7f128c101b00 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:35.904 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.899+0000 7f1292786700 1 -- 172.21.15.59:0/3268160149 >> 172.21.15.59:0/3268160149 conn(0x7f128c0fba20 msgr2=0x7f128c0fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:44:35.904 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.899+0000 7f1292786700 1 -- 172.21.15.59:0/3268160149 shutdown_connections 2023-12-11T17:44:35.904 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.899+0000 7f1292786700 1 -- 172.21.15.59:0/3268160149 wait complete. 2023-12-11T17:44:35.904 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.899+0000 7f1292786700 1 Processor -- start 2023-12-11T17:44:35.904 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.899+0000 7f1292786700 1 -- start start 2023-12-11T17:44:35.904 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.900+0000 7f1292786700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f128c1004b0 0x7f128c06a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:35.904 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.900+0000 7f1292786700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f128c1016a0 0x7f128c19c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:35.904 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.900+0000 7f1292786700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f128c102040 0x7f128c19e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:35.904 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.900+0000 7f1292786700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f128c19f040 con 0x7f128c1004b0 2023-12-11T17:44:35.904 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.900+0000 7f1292786700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f128c19f1b0 con 0x7f128c1016a0 2023-12-11T17:44:35.905 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.900+0000 7f1292786700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f128c19f370 con 0x7f128c102040 2023-12-11T17:44:35.905 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.900+0000 7f128bfff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f128c1004b0 0x7f128c06a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:35.905 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.900+0000 7f128bfff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f128c1004b0 0x7f128c06a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33212/0 (socket says 172.21.15.59:33212) 2023-12-11T17:44:35.905 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.900+0000 7f128bfff700 1 -- 172.21.15.59:0/3847513929 learned_addr learned my addr 172.21.15.59:0/3847513929 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:44:35.905 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.900+0000 7f1290d23700 1 --2- 172.21.15.59:0/3847513929 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f128c102040 0x7f128c19e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:35.905 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.900+0000 7f128b7fe700 1 --2- 172.21.15.59:0/3847513929 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f128c1016a0 0x7f128c19c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:35.905 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.900+0000 7f128bfff700 1 -- 172.21.15.59:0/3847513929 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f128c102040 msgr2=0x7f128c19e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:35.905 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.900+0000 7f128bfff700 1 --2- 172.21.15.59:0/3847513929 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f128c102040 0x7f128c19e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:35.905 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.900+0000 7f128bfff700 1 -- 172.21.15.59:0/3847513929 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f128c1016a0 msgr2=0x7f128c19c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:35.905 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.900+0000 7f128bfff700 1 --2- 172.21.15.59:0/3847513929 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f128c1016a0 0x7f128c19c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:35.905 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.900+0000 7f128bfff700 1 -- 172.21.15.59:0/3847513929 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12800095f0 con 0x7f128c1004b0 2023-12-11T17:44:35.905 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.900+0000 7f128bfff700 1 --2- 172.21.15.59:0/3847513929 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f128c1004b0 0x7f128c06a330 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f127c00d980 tx=0x7f127c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:44:35.905 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.901+0000 7f12897fa700 1 -- 172.21.15.59:0/3847513929 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f127c004020 con 0x7f128c1004b0 2023-12-11T17:44:35.905 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.901+0000 7f12897fa700 1 -- 172.21.15.59:0/3847513929 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f127c00f690 con 0x7f128c1004b0 2023-12-11T17:44:35.905 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.901+0000 7f1292786700 1 -- 172.21.15.59:0/3847513929 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f128c19f5f0 con 0x7f128c1004b0 2023-12-11T17:44:35.906 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.901+0000 7f12897fa700 1 -- 172.21.15.59:0/3847513929 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f127c003a60 con 0x7f128c1004b0 2023-12-11T17:44:35.906 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.901+0000 7f1292786700 1 -- 172.21.15.59:0/3847513929 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f128c19fad0 con 0x7f128c1004b0 2023-12-11T17:44:35.906 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.901+0000 7f1292786700 1 -- 172.21.15.59:0/3847513929 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f128c107900 con 0x7f128c1004b0 2023-12-11T17:44:35.906 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.903+0000 7f12897fa700 1 -- 172.21.15.59:0/3847513929 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f127c00fb00 con 0x7f128c1004b0 2023-12-11T17:44:35.907 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.903+0000 7f12897fa700 1 --2- 172.21.15.59:0/3847513929 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f12740a1480 0x7f12740a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:35.907 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.903+0000 7f12897fa700 1 -- 172.21.15.59:0/3847513929 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f127c0cce20 con 0x7f128c1004b0 2023-12-11T17:44:35.907 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.903+0000 7f128b7fe700 1 --2- 172.21.15.59:0/3847513929 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f12740a1480 0x7f12740a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:35.907 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.904+0000 7f128b7fe700 1 --2- 172.21.15.59:0/3847513929 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f12740a1480 0x7f12740a3940 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1280009990 tx=0x7f1280005ad0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:44:35.909 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:35.907+0000 7f12897fa700 1 -- 172.21.15.59:0/3847513929 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f127c096290 con 0x7f128c1004b0 2023-12-11T17:44:36.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:35 smithi115 ceph-mon[102748]: pgmap v266: 129 pgs: 129 active+clean; 579 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 433 B/s wr, 3 op/s 2023-12-11T17:44:36.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:35 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/1628065330' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-11T17:44:36.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:35 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/1261458485' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-11T17:44:36.110 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:36.107+0000 7f1292786700 1 -- 172.21.15.59:0/3847513929 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f128c10bcc0 con 0x7f128c1004b0 2023-12-11T17:44:36.110 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:36.107+0000 7f12897fa700 1 -- 172.21.15.59:0/3847513929 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v86) v1 ==== 100+0+11 (secure 0 0 0) 0x7f127c0998b0 con 0x7f128c1004b0 2023-12-11T17:44:36.110 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2023-12-11T17:44:36.112 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:36.110+0000 7f1292786700 1 -- 172.21.15.59:0/3847513929 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f12740a1480 msgr2=0x7f12740a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:36.113 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:36.110+0000 7f1292786700 1 --2- 172.21.15.59:0/3847513929 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f12740a1480 0x7f12740a3940 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1280009990 tx=0x7f1280005ad0 comp rx=0 tx=0).stop 2023-12-11T17:44:36.113 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:36.110+0000 7f1292786700 1 -- 172.21.15.59:0/3847513929 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f128c1004b0 msgr2=0x7f128c06a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:36.113 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:36.110+0000 7f1292786700 1 --2- 172.21.15.59:0/3847513929 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f128c1004b0 0x7f128c06a330 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f127c00d980 tx=0x7f127c00dc90 comp rx=0 tx=0).stop 2023-12-11T17:44:36.113 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:36.110+0000 7f1292786700 1 -- 172.21.15.59:0/3847513929 shutdown_connections 2023-12-11T17:44:36.113 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:36.110+0000 7f1292786700 1 --2- 172.21.15.59:0/3847513929 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f12740a1480 0x7f12740a3940 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:36.113 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:36.110+0000 7f1292786700 1 --2- 172.21.15.59:0/3847513929 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f128c102040 0x7f128c19e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:36.113 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:36.110+0000 7f1292786700 1 --2- 172.21.15.59:0/3847513929 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f128c1004b0 0x7f128c06a330 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:36.113 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:36.110+0000 7f1292786700 1 --2- 172.21.15.59:0/3847513929 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f128c1016a0 0x7f128c19c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:36.113 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:36.110+0000 7f1292786700 1 -- 172.21.15.59:0/3847513929 >> 172.21.15.59:0/3847513929 conn(0x7f128c0fba20 msgr2=0x7f128c104a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:44:36.113 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:36.110+0000 7f1292786700 1 -- 172.21.15.59:0/3847513929 shutdown_connections 2023-12-11T17:44:36.113 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:36.110+0000 7f1292786700 1 -- 172.21.15.59:0/3847513929 wait complete. 2023-12-11T17:44:36.124 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:35 smithi059 ceph-mon[98847]: pgmap v266: 129 pgs: 129 active+clean; 579 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 433 B/s wr, 3 op/s 2023-12-11T17:44:36.124 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:35 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/1628065330' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-11T17:44:36.124 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:35 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/1261458485' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-11T17:44:36.125 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-12-11T17:44:36.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:35 smithi181 ceph-mon[103627]: pgmap v266: 129 pgs: 129 active+clean; 579 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 433 B/s wr, 3 op/s 2023-12-11T17:44:36.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:35 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/1628065330' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-11T17:44:36.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:35 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/1261458485' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-11T17:44:36.254 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:36.251+0000 7f9eb51ba700 1 -- 172.21.15.59:0/550546549 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9eb0100480 msgr2=0x7f9eb01008a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:36.254 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:36.251+0000 7f9eb51ba700 1 --2- 172.21.15.59:0/550546549 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9eb0100480 0x7f9eb01008a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f9ea00099c0 tx=0x7f9ea0009cd0 comp rx=0 tx=0).stop 2023-12-11T17:44:36.254 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:36.251+0000 7f9eb51ba700 1 -- 172.21.15.59:0/550546549 shutdown_connections 2023-12-11T17:44:36.254 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:36.251+0000 7f9eb51ba700 1 --2- 172.21.15.59:0/550546549 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9eb0102010 0x7f9eb01088c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:36.254 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:36.251+0000 7f9eb51ba700 1 --2- 172.21.15.59:0/550546549 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9eb0100480 0x7f9eb01008a0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:36.254 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:36.251+0000 7f9eb51ba700 1 --2- 172.21.15.59:0/550546549 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9eb0101670 0x7f9eb0101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:36.254 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:36.251+0000 7f9eb51ba700 1 -- 172.21.15.59:0/550546549 >> 172.21.15.59:0/550546549 conn(0x7f9eb00fb9f0 msgr2=0x7f9eb00fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:44:36.254 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:36.251+0000 7f9eb51ba700 1 -- 172.21.15.59:0/550546549 shutdown_connections 2023-12-11T17:44:36.254 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:36.251+0000 7f9eb51ba700 1 -- 172.21.15.59:0/550546549 wait complete. 2023-12-11T17:44:36.256 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:36.252+0000 7f9eb51ba700 1 Processor -- start 2023-12-11T17:44:36.257 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:36.252+0000 7f9eb51ba700 1 -- start start 2023-12-11T17:44:36.257 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:36.252+0000 7f9eb51ba700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9eb0100480 0x7f9eb0196400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:36.257 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:36.252+0000 7f9eb51ba700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9eb0101670 0x7f9eb0196940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:36.257 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:36.252+0000 7f9eb51ba700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9eb0102010 0x7f9eb019d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:36.257 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:36.252+0000 7f9eb51ba700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb019df30 con 0x7f9eb0101670 2023-12-11T17:44:36.257 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:36.252+0000 7f9eb51ba700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb019e070 con 0x7f9eb0100480 2023-12-11T17:44:36.257 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:36.252+0000 7f9eb51ba700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb019e200 con 0x7f9eb0102010 2023-12-11T17:44:36.257 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:36.252+0000 7f9eae59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9eb0101670 0x7f9eb0196940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:36.257 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:36.252+0000 7f9eae59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9eb0101670 0x7f9eb0196940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33228/0 (socket says 172.21.15.59:33228) 2023-12-11T17:44:36.257 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:36.252+0000 7f9eae59c700 1 -- 172.21.15.59:0/62916994 learned_addr learned my addr 172.21.15.59:0/62916994 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:44:36.257 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:36.252+0000 7f9eaf59e700 1 --2- 172.21.15.59:0/62916994 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9eb0102010 0x7f9eb019d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:36.257 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:36.252+0000 7f9eaed9d700 1 --2- 172.21.15.59:0/62916994 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9eb0100480 0x7f9eb0196400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:36.257 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:36.253+0000 7f9eae59c700 1 -- 172.21.15.59:0/62916994 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9eb0102010 msgr2=0x7f9eb019d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:36.257 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:36.253+0000 7f9eae59c700 1 --2- 172.21.15.59:0/62916994 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9eb0102010 0x7f9eb019d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:36.258 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:36.253+0000 7f9eae59c700 1 -- 172.21.15.59:0/62916994 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9eb0100480 msgr2=0x7f9eb0196400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:36.258 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:36.253+0000 7f9eae59c700 1 --2- 172.21.15.59:0/62916994 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9eb0100480 0x7f9eb0196400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:36.258 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:36.253+0000 7f9eae59c700 1 -- 172.21.15.59:0/62916994 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ea00095f0 con 0x7f9eb0101670 2023-12-11T17:44:36.258 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:36.253+0000 7f9eae59c700 1 --2- 172.21.15.59:0/62916994 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9eb0101670 0x7f9eb0196940 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f9e9800c920 tx=0x7f9e9800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:44:36.258 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:36.253+0000 7f9ea7fff700 1 -- 172.21.15.59:0/62916994 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e98004a30 con 0x7f9eb0101670 2023-12-11T17:44:36.258 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:36.253+0000 7f9eb51ba700 1 -- 172.21.15.59:0/62916994 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9eb019e4e0 con 0x7f9eb0101670 2023-12-11T17:44:36.258 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:36.253+0000 7f9ea7fff700 1 -- 172.21.15.59:0/62916994 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9e980054e0 con 0x7f9eb0101670 2023-12-11T17:44:36.258 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:36.253+0000 7f9ea7fff700 1 -- 172.21.15.59:0/62916994 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e98007c10 con 0x7f9eb0101670 2023-12-11T17:44:36.258 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:36.253+0000 7f9eb51ba700 1 -- 172.21.15.59:0/62916994 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9eb019e9f0 con 0x7f9eb0101670 2023-12-11T17:44:36.259 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:36.254+0000 7f9eb51ba700 1 -- 172.21.15.59:0/62916994 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9eb0107950 con 0x7f9eb0101670 2023-12-11T17:44:36.259 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:36.255+0000 7f9ea7fff700 1 -- 172.21.15.59:0/62916994 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f9e98005000 con 0x7f9eb0101670 2023-12-11T17:44:36.260 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:36.256+0000 7f9ea7fff700 1 --2- 172.21.15.59:0/62916994 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f9e9c0a1480 0x7f9e9c0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:36.260 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:36.256+0000 7f9ea7fff700 1 -- 172.21.15.59:0/62916994 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f9e980ccdd0 con 0x7f9eb0101670 2023-12-11T17:44:36.260 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:36.256+0000 7f9eaed9d700 1 --2- 172.21.15.59:0/62916994 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f9e9c0a1480 0x7f9e9c0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:36.260 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:36.256+0000 7f9eaed9d700 1 --2- 172.21.15.59:0/62916994 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f9e9c0a1480 0x7f9e9c0a3940 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f9ea0003820 tx=0x7f9ea0005920 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:44:36.262 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:36.260+0000 7f9ea7fff700 1 -- 172.21.15.59:0/62916994 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f9e98096240 con 0x7f9eb0101670 2023-12-11T17:44:36.463 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:36.461+0000 7f9eb51ba700 1 -- 172.21.15.59:0/62916994 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f9eb0107330 con 0x7f9eb0101670 2023-12-11T17:44:36.464 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:36.461+0000 7f9ea7fff700 1 -- 172.21.15.59:0/62916994 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v86) v1 ==== 103+0+11 (secure 0 0 0) 0x7f9e9801d020 con 0x7f9eb0101670 2023-12-11T17:44:36.464 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2023-12-11T17:44:36.466 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:36.463+0000 7f9eb51ba700 1 -- 172.21.15.59:0/62916994 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f9e9c0a1480 msgr2=0x7f9e9c0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:36.466 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:36.463+0000 7f9eb51ba700 1 --2- 172.21.15.59:0/62916994 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f9e9c0a1480 0x7f9e9c0a3940 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f9ea0003820 tx=0x7f9ea0005920 comp rx=0 tx=0).stop 2023-12-11T17:44:36.466 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:36.463+0000 7f9eb51ba700 1 -- 172.21.15.59:0/62916994 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9eb0101670 msgr2=0x7f9eb0196940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:36.466 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:36.463+0000 7f9eb51ba700 1 --2- 172.21.15.59:0/62916994 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9eb0101670 0x7f9eb0196940 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f9e9800c920 tx=0x7f9e9800cc30 comp rx=0 tx=0).stop 2023-12-11T17:44:36.467 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:36.463+0000 7f9eb51ba700 1 -- 172.21.15.59:0/62916994 shutdown_connections 2023-12-11T17:44:36.467 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:36.464+0000 7f9eb51ba700 1 --2- 172.21.15.59:0/62916994 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f9e9c0a1480 0x7f9e9c0a3940 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:36.467 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:36.464+0000 7f9eb51ba700 1 --2- 172.21.15.59:0/62916994 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9eb0102010 0x7f9eb019d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:36.467 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:36.464+0000 7f9eb51ba700 1 --2- 172.21.15.59:0/62916994 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9eb0100480 0x7f9eb0196400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:36.467 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:36.464+0000 7f9eb51ba700 1 --2- 172.21.15.59:0/62916994 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9eb0101670 0x7f9eb0196940 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:36.467 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:36.464+0000 7f9eb51ba700 1 -- 172.21.15.59:0/62916994 >> 172.21.15.59:0/62916994 conn(0x7f9eb00fb9f0 msgr2=0x7f9eb0104a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:44:36.467 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:36.464+0000 7f9eb51ba700 1 -- 172.21.15.59:0/62916994 shutdown_connections 2023-12-11T17:44:36.467 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:36.464+0000 7f9eb51ba700 1 -- 172.21.15.59:0/62916994 wait complete. 2023-12-11T17:44:36.478 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-11T17:44:36.607 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:36.602+0000 7f2ba32f8700 1 -- 172.21.15.59:0/2719423208 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b9c102790 msgr2=0x7f2b9c102bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:36.607 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:36.602+0000 7f2ba32f8700 1 --2- 172.21.15.59:0/2719423208 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b9c102790 0x7f2b9c102bf0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f2b900099c0 tx=0x7f2b90009cd0 comp rx=0 tx=0).stop 2023-12-11T17:44:36.607 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:36.602+0000 7f2ba32f8700 1 -- 172.21.15.59:0/2719423208 shutdown_connections 2023-12-11T17:44:36.607 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:36.602+0000 7f2ba32f8700 1 --2- 172.21.15.59:0/2719423208 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2b9c103130 0x7f2b9c1099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:36.607 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:36.602+0000 7f2ba32f8700 1 --2- 172.21.15.59:0/2719423208 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2b9c1015a0 0x7f2b9c1019c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:36.607 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:36.602+0000 7f2ba32f8700 1 --2- 172.21.15.59:0/2719423208 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b9c102790 0x7f2b9c102bf0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:36.607 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:36.602+0000 7f2ba32f8700 1 -- 172.21.15.59:0/2719423208 >> 172.21.15.59:0/2719423208 conn(0x7f2b9c0fcb10 msgr2=0x7f2b9c0fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:44:36.607 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:36.602+0000 7f2ba32f8700 1 -- 172.21.15.59:0/2719423208 shutdown_connections 2023-12-11T17:44:36.607 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:36.602+0000 7f2ba32f8700 1 -- 172.21.15.59:0/2719423208 wait complete. 2023-12-11T17:44:36.607 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:36.603+0000 7f2ba32f8700 1 Processor -- start 2023-12-11T17:44:36.607 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:36.603+0000 7f2ba32f8700 1 -- start start 2023-12-11T17:44:36.608 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:36.603+0000 7f2ba32f8700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2b9c1015a0 0x7f2b9c197520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:36.608 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:36.603+0000 7f2ba32f8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2b9c102790 0x7f2b9c197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:36.608 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:36.603+0000 7f2ba32f8700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b9c103130 0x7f2b9c19ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:36.608 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:36.603+0000 7f2ba32f8700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b9c19f050 con 0x7f2b9c103130 2023-12-11T17:44:36.608 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:36.603+0000 7f2ba32f8700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b9c19f190 con 0x7f2b9c1015a0 2023-12-11T17:44:36.608 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:36.603+0000 7f2ba32f8700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b9c19f320 con 0x7f2b9c102790 2023-12-11T17:44:36.608 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:36.603+0000 7f2ba1895700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b9c103130 0x7f2b9c19ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:36.608 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:36.603+0000 7f2ba1895700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b9c103130 0x7f2b9c19ea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33244/0 (socket says 172.21.15.59:33244) 2023-12-11T17:44:36.608 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:36.603+0000 7f2ba1895700 1 -- 172.21.15.59:0/521987944 learned_addr learned my addr 172.21.15.59:0/521987944 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:44:36.608 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:36.603+0000 7f2ba1094700 1 --2- 172.21.15.59:0/521987944 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2b9c1015a0 0x7f2b9c197520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:36.608 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:36.603+0000 7f2ba0893700 1 --2- 172.21.15.59:0/521987944 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2b9c102790 0x7f2b9c197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:36.608 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:36.603+0000 7f2ba1895700 1 -- 172.21.15.59:0/521987944 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2b9c102790 msgr2=0x7f2b9c197a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:36.608 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:36.603+0000 7f2ba1895700 1 --2- 172.21.15.59:0/521987944 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2b9c102790 0x7f2b9c197a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:36.608 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:36.603+0000 7f2ba1895700 1 -- 172.21.15.59:0/521987944 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2b9c1015a0 msgr2=0x7f2b9c197520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:36.608 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:36.603+0000 7f2ba1895700 1 --2- 172.21.15.59:0/521987944 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2b9c1015a0 0x7f2b9c197520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:36.609 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:36.604+0000 7f2ba1895700 1 -- 172.21.15.59:0/521987944 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b900095f0 con 0x7f2b9c103130 2023-12-11T17:44:36.609 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:36.604+0000 7f2ba1895700 1 --2- 172.21.15.59:0/521987944 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b9c103130 0x7f2b9c19ea80 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f2b9400ada0 tx=0x7f2b9400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:44:36.609 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:36.604+0000 7f2b8e7fc700 1 -- 172.21.15.59:0/521987944 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b9400f070 con 0x7f2b9c103130 2023-12-11T17:44:36.609 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:36.604+0000 7f2b8e7fc700 1 -- 172.21.15.59:0/521987944 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2b94007b30 con 0x7f2b9c103130 2023-12-11T17:44:36.609 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:36.604+0000 7f2ba32f8700 1 -- 172.21.15.59:0/521987944 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b9c19f600 con 0x7f2b9c103130 2023-12-11T17:44:36.609 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:36.604+0000 7f2b8e7fc700 1 -- 172.21.15.59:0/521987944 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b94005280 con 0x7f2b9c103130 2023-12-11T17:44:36.609 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:36.604+0000 7f2ba32f8700 1 -- 172.21.15.59:0/521987944 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b9c19fb10 con 0x7f2b9c103130 2023-12-11T17:44:36.610 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:36.605+0000 7f2ba32f8700 1 -- 172.21.15.59:0/521987944 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b9c108a20 con 0x7f2b9c103130 2023-12-11T17:44:36.610 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:36.606+0000 7f2b8e7fc700 1 -- 172.21.15.59:0/521987944 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f2b9400c8d0 con 0x7f2b9c103130 2023-12-11T17:44:36.610 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:36.606+0000 7f2b8e7fc700 1 --2- 172.21.15.59:0/521987944 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f2b880a1500 0x7f2b880a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:36.610 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:36.607+0000 7f2b8e7fc700 1 -- 172.21.15.59:0/521987944 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f2b940cc370 con 0x7f2b9c103130 2023-12-11T17:44:36.610 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:36.607+0000 7f2ba1094700 1 --2- 172.21.15.59:0/521987944 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f2b880a1500 0x7f2b880a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:36.610 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:36.607+0000 7f2ba1094700 1 --2- 172.21.15.59:0/521987944 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f2b880a1500 0x7f2b880a39c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f2b9c1025f0 tx=0x7f2b9800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:44:36.613 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:36.611+0000 7f2b8e7fc700 1 -- 172.21.15.59:0/521987944 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f2b94099300 con 0x7f2b9c103130 2023-12-11T17:44:36.837 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:36.835+0000 7f2ba32f8700 1 -- 172.21.15.59:0/521987944 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f2b9c068310 con 0x7f2b9c103130 2023-12-11T17:44:36.837 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:36.835+0000 7f2b8e7fc700 1 -- 172.21.15.59:0/521987944 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v86) v1 ==== 81+0+590 (secure 0 0 0) 0x7f2b94098e90 con 0x7f2b9c103130 2023-12-11T17:44:36.838 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:6800/1259916482 2023-12-12T17:38:14.037922+0000 2023-12-11T17:44:36.838 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:0/17457774 2023-12-12T17:38:14.037922+0000 2023-12-11T17:44:36.838 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:6801/1672530462 2023-12-12T17:37:44.773925+0000 2023-12-11T17:44:36.838 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:0/1053944547 2023-12-12T17:38:14.037922+0000 2023-12-11T17:44:36.838 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:6801/1259916482 2023-12-12T17:38:14.037922+0000 2023-12-11T17:44:36.838 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:6800/1672530462 2023-12-12T17:37:44.773925+0000 2023-12-11T17:44:36.838 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:0/3570767650 2023-12-12T17:37:44.773925+0000 2023-12-11T17:44:36.838 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:0/3853462899 2023-12-12T17:37:44.773925+0000 2023-12-11T17:44:36.838 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:0/2859415380 2023-12-12T17:38:14.037922+0000 2023-12-11T17:44:36.838 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:0/2192742819 2023-12-12T17:37:44.773925+0000 2023-12-11T17:44:36.840 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:36.837+0000 7f2ba32f8700 1 -- 172.21.15.59:0/521987944 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f2b880a1500 msgr2=0x7f2b880a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:36.840 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:36.837+0000 7f2ba32f8700 1 --2- 172.21.15.59:0/521987944 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f2b880a1500 0x7f2b880a39c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f2b9c1025f0 tx=0x7f2b9800b410 comp rx=0 tx=0).stop 2023-12-11T17:44:36.840 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:36.837+0000 7f2ba32f8700 1 -- 172.21.15.59:0/521987944 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b9c103130 msgr2=0x7f2b9c19ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:36.840 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:36.837+0000 7f2ba32f8700 1 --2- 172.21.15.59:0/521987944 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b9c103130 0x7f2b9c19ea80 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f2b9400ada0 tx=0x7f2b9400c330 comp rx=0 tx=0).stop 2023-12-11T17:44:36.840 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:36.837+0000 7f2ba32f8700 1 -- 172.21.15.59:0/521987944 shutdown_connections 2023-12-11T17:44:36.841 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:36.837+0000 7f2ba32f8700 1 --2- 172.21.15.59:0/521987944 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f2b880a1500 0x7f2b880a39c0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:36.841 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:36.837+0000 7f2ba32f8700 1 --2- 172.21.15.59:0/521987944 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b9c103130 0x7f2b9c19ea80 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:36.841 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:36.837+0000 7f2ba32f8700 1 --2- 172.21.15.59:0/521987944 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2b9c1015a0 0x7f2b9c197520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:36.841 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:36.837+0000 7f2ba32f8700 1 --2- 172.21.15.59:0/521987944 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2b9c102790 0x7f2b9c197a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:36.841 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:36.837+0000 7f2ba32f8700 1 -- 172.21.15.59:0/521987944 >> 172.21.15.59:0/521987944 conn(0x7f2b9c0fcb10 msgr2=0x7f2b9c105b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:44:36.841 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:36.837+0000 7f2ba32f8700 1 -- 172.21.15.59:0/521987944 shutdown_connections 2023-12-11T17:44:36.841 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:36.837+0000 7f2ba32f8700 1 -- 172.21.15.59:0/521987944 wait complete. 2023-12-11T17:44:36.841 INFO:teuthology.orchestra.run.smithi059.stderr:listed 10 entries 2023-12-11T17:44:36.850 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:36 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/941343939' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-11T17:44:36.851 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:36 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/3847513929' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-11T17:44:36.851 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:36 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/62916994' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-11T17:44:36.852 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi059.front.sepia.ceph.com... 2023-12-11T17:44:36.852 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-11T17:44:36.852 DEBUG:teuthology.orchestra.run.smithi059:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2023-12-11T17:44:36.895 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi059.stderr:ceph-fuse[153949]: fuse finished with error 0 and tester_r 0 2023-12-11T17:44:36.897 INFO:teuthology.orchestra.run:waiting for 300 2023-12-11T17:44:37.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:36 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/941343939' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-11T17:44:37.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:36 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/3847513929' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-11T17:44:37.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:36 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/62916994' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-11T17:44:37.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:36 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/941343939' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-11T17:44:37.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:36 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/3847513929' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-11T17:44:37.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:36 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/62916994' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-11T17:44:38.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:37 smithi115 ceph-mon[102748]: pgmap v267: 129 pgs: 129 active+clean; 579 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 98 B/s wr, 0 op/s 2023-12-11T17:44:38.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:37 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/521987944' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-12-11T17:44:38.146 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:37 smithi181 ceph-mon[103627]: pgmap v267: 129 pgs: 129 active+clean; 579 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 98 B/s wr, 0 op/s 2023-12-11T17:44:38.146 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:37 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/521987944' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-12-11T17:44:38.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:37 smithi059 ceph-mon[98847]: pgmap v267: 129 pgs: 129 active+clean; 579 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 98 B/s wr, 0 op/s 2023-12-11T17:44:38.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:37 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/521987944' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-12-11T17:44:40.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:39 smithi115 ceph-mon[102748]: pgmap v268: 129 pgs: 129 active+clean; 581 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-12-11T17:44:40.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:39 smithi181 ceph-mon[103627]: pgmap v268: 129 pgs: 129 active+clean; 581 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-12-11T17:44:40.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:39 smithi059 ceph-mon[98847]: pgmap v268: 129 pgs: 129 active+clean; 581 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-12-11T17:44:42.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:41 smithi115 ceph-mon[102748]: pgmap v269: 129 pgs: 129 active+clean; 581 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-12-11T17:44:42.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:41 smithi181 ceph-mon[103627]: pgmap v269: 129 pgs: 129 active+clean; 581 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-12-11T17:44:42.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:41 smithi059 ceph-mon[98847]: pgmap v269: 129 pgs: 129 active+clean; 581 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-12-11T17:44:42.999 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi059.front.sepia.ceph.com 2023-12-11T17:44:42.999 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-11T17:44:42.999 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2023-12-11T17:44:43.035 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-11T17:44:43.216 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.212+0000 7fee19d3b700 1 -- 172.21.15.59:0/2637687907 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee14104830 msgr2=0x7fee14106c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:43.216 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.212+0000 7fee19d3b700 1 --2- 172.21.15.59:0/2637687907 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee14104830 0x7fee14106c20 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fedfc009990 tx=0x7fedfc009ca0 comp rx=0 tx=0).stop 2023-12-11T17:44:43.216 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.212+0000 7fee19d3b700 1 -- 172.21.15.59:0/2637687907 shutdown_connections 2023-12-11T17:44:43.216 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.212+0000 7fee19d3b700 1 --2- 172.21.15.59:0/2637687907 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee14107160 0x7fee141095d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:43.216 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.212+0000 7fee19d3b700 1 --2- 172.21.15.59:0/2637687907 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fee14101f00 0x7fee141042f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:43.216 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.212+0000 7fee19d3b700 1 --2- 172.21.15.59:0/2637687907 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee14104830 0x7fee14106c20 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:43.216 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.212+0000 7fee19d3b700 1 -- 172.21.15.59:0/2637687907 >> 172.21.15.59:0/2637687907 conn(0x7fee140fb8f0 msgr2=0x7fee140fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:44:43.217 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.212+0000 7fee19d3b700 1 -- 172.21.15.59:0/2637687907 shutdown_connections 2023-12-11T17:44:43.217 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.212+0000 7fee19d3b700 1 -- 172.21.15.59:0/2637687907 wait complete. 2023-12-11T17:44:43.217 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.213+0000 7fee19d3b700 1 Processor -- start 2023-12-11T17:44:43.219 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.213+0000 7fee19d3b700 1 -- start start 2023-12-11T17:44:43.219 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.213+0000 7fee19d3b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee14101f00 0x7fee14197390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:43.219 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.213+0000 7fee19d3b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee14104830 0x7fee141978d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:43.219 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.213+0000 7fee19d3b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fee14107160 0x7fee1419e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:43.219 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.213+0000 7fee19d3b700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee1419eec0 con 0x7fee14104830 2023-12-11T17:44:43.219 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.213+0000 7fee19d3b700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee1419f000 con 0x7fee14107160 2023-12-11T17:44:43.219 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.213+0000 7fee19d3b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee1419f190 con 0x7fee14101f00 2023-12-11T17:44:43.219 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.213+0000 7fee12ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee14104830 0x7fee141978d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:43.219 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.213+0000 7fee12ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee14104830 0x7fee141978d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33258/0 (socket says 172.21.15.59:33258) 2023-12-11T17:44:43.219 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.213+0000 7fee12ffd700 1 -- 172.21.15.59:0/563982075 learned_addr learned my addr 172.21.15.59:0/563982075 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:44:43.219 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.214+0000 7fee137fe700 1 --2- 172.21.15.59:0/563982075 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee14101f00 0x7fee14197390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:43.219 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.214+0000 7fee13fff700 1 --2- 172.21.15.59:0/563982075 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fee14107160 0x7fee1419e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:43.220 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.214+0000 7fee12ffd700 1 -- 172.21.15.59:0/563982075 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee14101f00 msgr2=0x7fee14197390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:43.220 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.214+0000 7fee12ffd700 1 --2- 172.21.15.59:0/563982075 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee14101f00 0x7fee14197390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:43.220 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.214+0000 7fee12ffd700 1 -- 172.21.15.59:0/563982075 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fee14107160 msgr2=0x7fee1419e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:43.220 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.214+0000 7fee12ffd700 1 --2- 172.21.15.59:0/563982075 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fee14107160 0x7fee1419e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:43.220 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.214+0000 7fee12ffd700 1 -- 172.21.15.59:0/563982075 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedfc0095f0 con 0x7fee14104830 2023-12-11T17:44:43.220 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.214+0000 7fee12ffd700 1 --2- 172.21.15.59:0/563982075 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee14104830 0x7fee141978d0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fedfc0095c0 tx=0x7fedfc00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:44:43.220 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.214+0000 7fee10ff9700 1 -- 172.21.15.59:0/563982075 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedfc01d070 con 0x7fee14104830 2023-12-11T17:44:43.220 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.214+0000 7fee10ff9700 1 -- 172.21.15.59:0/563982075 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fedfc00fb80 con 0x7fee14104830 2023-12-11T17:44:43.220 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.214+0000 7fee19d3b700 1 -- 172.21.15.59:0/563982075 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee1419f410 con 0x7fee14104830 2023-12-11T17:44:43.220 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.214+0000 7fee10ff9700 1 -- 172.21.15.59:0/563982075 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedfc017650 con 0x7fee14104830 2023-12-11T17:44:43.220 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.214+0000 7fee19d3b700 1 -- 172.21.15.59:0/563982075 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee1419f830 con 0x7fee14104830 2023-12-11T17:44:43.220 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.215+0000 7fee19d3b700 1 -- 172.21.15.59:0/563982075 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee140fd470 con 0x7fee14104830 2023-12-11T17:44:43.221 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.216+0000 7fee10ff9700 1 -- 172.21.15.59:0/563982075 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7fedfc00fcf0 con 0x7fee14104830 2023-12-11T17:44:43.221 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.217+0000 7fee10ff9700 1 --2- 172.21.15.59:0/563982075 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fee000a1480 0x7fee000a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:43.221 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.217+0000 7fee137fe700 1 --2- 172.21.15.59:0/563982075 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fee000a1480 0x7fee000a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:43.221 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.217+0000 7fee10ff9700 1 -- 172.21.15.59:0/563982075 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fedfc0ce460 con 0x7fee14104830 2023-12-11T17:44:43.221 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.217+0000 7fee137fe700 1 --2- 172.21.15.59:0/563982075 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fee000a1480 0x7fee000a3940 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fee04005950 tx=0x7fee040058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:44:43.223 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.220+0000 7fee10ff9700 1 -- 172.21.15.59:0/563982075 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fedfc097900 con 0x7fee14104830 2023-12-11T17:44:43.430 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.426+0000 7fee19d3b700 1 -- 172.21.15.59:0/563982075 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fee1410cca0 con 0x7fee14104830 2023-12-11T17:44:43.431 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.428+0000 7fee10ff9700 1 -- 172.21.15.59:0/563982075 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+23960 (secure 0 0 0) 0x7fedfc09af20 con 0x7fee14104830 2023-12-11T17:44:43.432 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-12-11T17:44:43.432 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":86,"fsid":"d0a07eaa-984b-11ee-95a3-87774f69a715","created":"2023-12-11T17:37:14.526539+0000","modified":"2023-12-11T17:44:25.096094+0000","last_up_change":"2023-12-11T17:43:38.535503+0000","last_in_change":"2023-12-11T17:43:21.014888+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-11T17:40:26.596029+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-11T17:44:18.586290+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-11T17:44:19.432952+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-11T17:44:23.097308+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3c0b3218-4b2b-40f8-8bef-ed8cd5633607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":1040759398},{"type":"v1","addr":"172.21.15.59:6803","nonce":1040759398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":1040759398},{"type":"v1","addr":"172.21.15.59:6805","nonce":1040759398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":1040759398},{"type":"v1","addr":"172.21.15.59:6809","nonce":1040759398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":1040759398},{"type":"v1","addr":"172.21.15.59:6807","nonce":1040759398}]},"public_addr":"172.21.15.59:6803/1040759398","cluster_addr":"172.21.15.59:6805/1040759398","heartbeat_back_addr":"172.21.15.59:6809/1040759398","heartbeat_front_addr":"172.21.15.59:6807/1040759398","state":["exists","up"]},{"osd":1,"uuid":"2ead7578-765b-4f5a-a72c-863731ed69e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":2776354245},{"type":"v1","addr":"172.21.15.115:6801","nonce":2776354245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":2776354245},{"type":"v1","addr":"172.21.15.115:6803","nonce":2776354245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":2776354245},{"type":"v1","addr":"172.21.15.115:6807","nonce":2776354245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":2776354245},{"type":"v1","addr":"172.21.15.115:6805","nonce":2776354245}]},"public_addr":"172.21.15.115:6801/2776354245","cluster_addr":"172.21.15.115:6803/2776354245","heartbeat_back_addr":"172.21.15.115:6807/2776354245","heartbeat_front_addr":"172.21.15.115:6805/2776354245","state":["exists","up"]},{"osd":2,"uuid":"8cb1d7ec-53cc-4672-b0b6-70d73797c44f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4287675481},{"type":"v1","addr":"172.21.15.181:6801","nonce":4287675481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4287675481},{"type":"v1","addr":"172.21.15.181:6803","nonce":4287675481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4287675481},{"type":"v1","addr":"172.21.15.181:6807","nonce":4287675481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4287675481},{"type":"v1","addr":"172.21.15.181:6805","nonce":4287675481}]},"public_addr":"172.21.15.181:6801/4287675481","cluster_addr":"172.21.15.181:6803/4287675481","heartbeat_back_addr":"172.21.15.181:6807/4287675481","heartbeat_front_addr":"172.21.15.181:6805/4287675481","state":["exists","up"]},{"osd":3,"uuid":"5d64d9af-831b-4d0a-8ace-a452169351d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":219909995},{"type":"v1","addr":"172.21.15.59:6811","nonce":219909995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":219909995},{"type":"v1","addr":"172.21.15.59:6813","nonce":219909995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":219909995},{"type":"v1","addr":"172.21.15.59:6817","nonce":219909995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":219909995},{"type":"v1","addr":"172.21.15.59:6815","nonce":219909995}]},"public_addr":"172.21.15.59:6811/219909995","cluster_addr":"172.21.15.59:6813/219909995","heartbeat_back_addr":"172.21.15.59:6817/219909995","heartbeat_front_addr":"172.21.15.59:6815/219909995","state":["exists","up"]},{"osd":4,"uuid":"ef8fad28-1db5-4cba-b5f2-10e54e8e1549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":1021473826},{"type":"v1","addr":"172.21.15.115:6809","nonce":1021473826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":1021473826},{"type":"v1","addr":"172.21.15.115:6811","nonce":1021473826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":1021473826},{"type":"v1","addr":"172.21.15.115:6815","nonce":1021473826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":1021473826},{"type":"v1","addr":"172.21.15.115:6813","nonce":1021473826}]},"public_addr":"172.21.15.115:6809/1021473826","cluster_addr":"172.21.15.115:6811/1021473826","heartbeat_back_addr":"172.21.15.115:6815/1021473826","heartbeat_front_addr":"172.21.15.115:6813/1021473826","state":["exists","up"]},{"osd":5,"uuid":"4233a512-9e8f-4db8-aed2-6c3af706e589","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":2405172521},{"type":"v1","addr":"172.21.15.181:6809","nonce":2405172521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":2405172521},{"type":"v1","addr":"172.21.15.181:6811","nonce":2405172521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2405172521},{"type":"v1","addr":"172.21.15.181:6815","nonce":2405172521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":2405172521},{"type":"v1","addr":"172.21.15.181:6813","nonce":2405172521}]},"public_addr":"172.21.15.181:6809/2405172521","cluster_addr":"172.21.15.181:6811/2405172521","heartbeat_back_addr":"172.21.15.181:6815/2405172521","heartbeat_front_addr":"172.21.15.181:6813/2405172521","state":["exists","up"]},{"osd":6,"uuid":"8faab332-016e-4c77-8370-8bef95fc6f34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":815374853},{"type":"v1","addr":"172.21.15.59:6819","nonce":815374853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":815374853},{"type":"v1","addr":"172.21.15.59:6821","nonce":815374853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":815374853},{"type":"v1","addr":"172.21.15.59:6825","nonce":815374853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":815374853},{"type":"v1","addr":"172.21.15.59:6823","nonce":815374853}]},"public_addr":"172.21.15.59:6819/815374853","cluster_addr":"172.21.15.59:6821/815374853","heartbeat_back_addr":"172.21.15.59:6825/815374853","heartbeat_front_addr":"172.21.15.59:6823/815374853","state":["exists","up"]},{"osd":7,"uuid":"f20f470a-0d40-4ed3-8e0e-fd58675cf143","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":1314971870},{"type":"v1","addr":"172.21.15.115:6817","nonce":1314971870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":1314971870},{"type":"v1","addr":"172.21.15.115:6819","nonce":1314971870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":1314971870},{"type":"v1","addr":"172.21.15.115:6823","nonce":1314971870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":1314971870},{"type":"v1","addr":"172.21.15.115:6821","nonce":1314971870}]},"public_addr":"172.21.15.115:6817/1314971870","cluster_addr":"172.21.15.115:6819/1314971870","heartbeat_back_addr":"172.21.15.115:6823/1314971870","heartbeat_front_addr":"172.21.15.115:6821/1314971870","state":["exists","up"]},{"osd":8,"uuid":"eb657d32-1a95-4fd7-a293-d0a970837fc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":3954509873},{"type":"v1","addr":"172.21.15.181:6817","nonce":3954509873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":3954509873},{"type":"v1","addr":"172.21.15.181:6819","nonce":3954509873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":3954509873},{"type":"v1","addr":"172.21.15.181:6823","nonce":3954509873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":3954509873},{"type":"v1","addr":"172.21.15.181:6821","nonce":3954509873}]},"public_addr":"172.21.15.181:6817/3954509873","cluster_addr":"172.21.15.181:6819/3954509873","heartbeat_back_addr":"172.21.15.181:6823/3954509873","heartbeat_front_addr":"172.21.15.181:6821/3954509873","state":["exists","up"]},{"osd":9,"uuid":"8f7717d4-b6f2-4ec1-9b58-62feb0c9a146","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":1163095677},{"type":"v1","addr":"172.21.15.59:6827","nonce":1163095677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":1163095677},{"type":"v1","addr":"172.21.15.59:6829","nonce":1163095677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":1163095677},{"type":"v1","addr":"172.21.15.59:6833","nonce":1163095677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":1163095677},{"type":"v1","addr":"172.21.15.59:6831","nonce":1163095677}]},"public_addr":"172.21.15.59:6827/1163095677","cluster_addr":"172.21.15.59:6829/1163095677","heartbeat_back_addr":"172.21.15.59:6833/1163095677","heartbeat_front_addr":"172.21.15.59:6831/1163095677","state":["exists","up"]},{"osd":10,"uuid":"b75c179a-875d-43bc-b4c1-e8eb80fb64fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":2199610269},{"type":"v1","addr":"172.21.15.115:6825","nonce":2199610269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":2199610269},{"type":"v1","addr":"172.21.15.115:6827","nonce":2199610269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":2199610269},{"type":"v1","addr":"172.21.15.115:6831","nonce":2199610269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":2199610269},{"type":"v1","addr":"172.21.15.115:6829","nonce":2199610269}]},"public_addr":"172.21.15.115:6825/2199610269","cluster_addr":"172.21.15.115:6827/2199610269","heartbeat_back_addr":"172.21.15.115:6831/2199610269","heartbeat_front_addr":"172.21.15.115:6829/2199610269","state":["exists","up"]},{"osd":11,"uuid":"5e141dfe-8a00-4bbc-8204-2dc9be24e18d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":226506989},{"type":"v1","addr":"172.21.15.181:6825","nonce":226506989}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":226506989},{"type":"v1","addr":"172.21.15.181:6827","nonce":226506989}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":226506989},{"type":"v1","addr":"172.21.15.181:6831","nonce":226506989}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":226506989},{"type":"v1","addr":"172.21.15.181:6829","nonce":226506989}]},"public_addr":"172.21.15.181:6825/226506989","cluster_addr":"172.21.15.181:6827/226506989","heartbeat_back_addr":"172.21.15.181:6831/226506989","heartbeat_front_addr":"172.21.15.181:6829/226506989","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:39:42.687084+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:40:03.903448+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:40:24.899908+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:40:46.600251+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:41:07.690037+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:41:28.488855+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:41:49.459274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:42:10.863881+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:42:32.248391+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:42:54.112265+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:43:15.634872+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:43:36.922534+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.59:6800/1259916482":"2023-12-12T17:38:14.037922+0000","172.21.15.59:0/17457774":"2023-12-12T17:38:14.037922+0000","172.21.15.59:6801/1672530462":"2023-12-12T17:37:44.773925+0000","172.21.15.59:0/1053944547":"2023-12-12T17:38:14.037922+0000","172.21.15.59:6801/1259916482":"2023-12-12T17:38:14.037922+0000","172.21.15.59:6800/1672530462":"2023-12-12T17:37:44.773925+0000","172.21.15.59:0/3570767650":"2023-12-12T17:37:44.773925+0000","172.21.15.59:0/3853462899":"2023-12-12T17:37:44.773925+0000","172.21.15.59:0/2859415380":"2023-12-12T17:38:14.037922+0000","172.21.15.59:0/2192742819":"2023-12-12T17:37:44.773925+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-11T17:44:43.434 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.430+0000 7fee19d3b700 1 -- 172.21.15.59:0/563982075 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fee000a1480 msgr2=0x7fee000a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:43.434 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.430+0000 7fee19d3b700 1 --2- 172.21.15.59:0/563982075 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fee000a1480 0x7fee000a3940 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fee04005950 tx=0x7fee040058e0 comp rx=0 tx=0).stop 2023-12-11T17:44:43.434 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.430+0000 7fee19d3b700 1 -- 172.21.15.59:0/563982075 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee14104830 msgr2=0x7fee141978d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:43.434 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.430+0000 7fee19d3b700 1 --2- 172.21.15.59:0/563982075 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee14104830 0x7fee141978d0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fedfc0095c0 tx=0x7fedfc00f690 comp rx=0 tx=0).stop 2023-12-11T17:44:43.434 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.430+0000 7fee19d3b700 1 -- 172.21.15.59:0/563982075 shutdown_connections 2023-12-11T17:44:43.434 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.430+0000 7fee19d3b700 1 --2- 172.21.15.59:0/563982075 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fee000a1480 0x7fee000a3940 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:43.434 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.430+0000 7fee19d3b700 1 --2- 172.21.15.59:0/563982075 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fee14107160 0x7fee1419e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:43.434 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.430+0000 7fee19d3b700 1 --2- 172.21.15.59:0/563982075 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee14101f00 0x7fee14197390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:43.434 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.430+0000 7fee19d3b700 1 --2- 172.21.15.59:0/563982075 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee14104830 0x7fee141978d0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:43.435 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.430+0000 7fee19d3b700 1 -- 172.21.15.59:0/563982075 >> 172.21.15.59:0/563982075 conn(0x7fee140fb8f0 msgr2=0x7fee14102860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:44:43.435 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.430+0000 7fee19d3b700 1 -- 172.21.15.59:0/563982075 shutdown_connections 2023-12-11T17:44:43.435 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.430+0000 7fee19d3b700 1 -- 172.21.15.59:0/563982075 wait complete. 2023-12-11T17:44:43.446 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-11T17:44:43.576 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.570+0000 7f1a79b9d700 1 -- 172.21.15.59:0/3920359809 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1a74101560 msgr2=0x7f1a741019c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:43.576 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.570+0000 7f1a79b9d700 1 --2- 172.21.15.59:0/3920359809 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1a74101560 0x7f1a741019c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f1a600099c0 tx=0x7f1a60009cd0 comp rx=0 tx=0).stop 2023-12-11T17:44:43.576 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.570+0000 7f1a79b9d700 1 -- 172.21.15.59:0/3920359809 shutdown_connections 2023-12-11T17:44:43.576 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.570+0000 7f1a79b9d700 1 --2- 172.21.15.59:0/3920359809 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a74101f00 0x7f1a741087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:43.576 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.570+0000 7f1a79b9d700 1 --2- 172.21.15.59:0/3920359809 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1a74100370 0x7f1a74100790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:43.576 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.570+0000 7f1a79b9d700 1 --2- 172.21.15.59:0/3920359809 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1a74101560 0x7f1a741019c0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:43.577 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.570+0000 7f1a79b9d700 1 -- 172.21.15.59:0/3920359809 >> 172.21.15.59:0/3920359809 conn(0x7f1a740fb8e0 msgr2=0x7f1a740fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:44:43.577 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.570+0000 7f1a79b9d700 1 -- 172.21.15.59:0/3920359809 shutdown_connections 2023-12-11T17:44:43.577 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.570+0000 7f1a79b9d700 1 -- 172.21.15.59:0/3920359809 wait complete. 2023-12-11T17:44:43.577 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.571+0000 7f1a79b9d700 1 Processor -- start 2023-12-11T17:44:43.577 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.571+0000 7f1a79b9d700 1 -- start start 2023-12-11T17:44:43.577 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.571+0000 7f1a79b9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1a74100370 0x7f1a7406a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:43.577 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.571+0000 7f1a79b9d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1a74101560 0x7f1a7419c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:43.577 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.571+0000 7f1a79b9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a74101f00 0x7f1a7419e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:43.577 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.571+0000 7f1a79b9d700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a7419eea0 con 0x7f1a74100370 2023-12-11T17:44:43.577 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.571+0000 7f1a79b9d700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a7419f010 con 0x7f1a74101560 2023-12-11T17:44:43.578 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.571+0000 7f1a79b9d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a7419f1d0 con 0x7f1a74101f00 2023-12-11T17:44:43.578 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.571+0000 7f1a78b9b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1a74100370 0x7f1a7406a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:43.578 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.571+0000 7f1a78b9b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1a74100370 0x7f1a7406a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33270/0 (socket says 172.21.15.59:33270) 2023-12-11T17:44:43.578 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.571+0000 7f1a78b9b700 1 -- 172.21.15.59:0/2862784590 learned_addr learned my addr 172.21.15.59:0/2862784590 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:44:43.578 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.571+0000 7f1a73fff700 1 --2- 172.21.15.59:0/2862784590 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1a74101560 0x7f1a7419c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:43.578 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.571+0000 7f1a7939c700 1 --2- 172.21.15.59:0/2862784590 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a74101f00 0x7f1a7419e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:43.578 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.571+0000 7f1a78b9b700 1 -- 172.21.15.59:0/2862784590 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a74101f00 msgr2=0x7f1a7419e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:43.578 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.571+0000 7f1a78b9b700 1 --2- 172.21.15.59:0/2862784590 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a74101f00 0x7f1a7419e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:43.578 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.571+0000 7f1a78b9b700 1 -- 172.21.15.59:0/2862784590 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1a74101560 msgr2=0x7f1a7419c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:43.578 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.571+0000 7f1a78b9b700 1 --2- 172.21.15.59:0/2862784590 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1a74101560 0x7f1a7419c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:43.578 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.572+0000 7f1a78b9b700 1 -- 172.21.15.59:0/2862784590 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a600095f0 con 0x7f1a74100370 2023-12-11T17:44:43.578 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.572+0000 7f1a7939c700 1 --2- 172.21.15.59:0/2862784590 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a74101f00 0x7f1a7419e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T17:44:43.578 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.572+0000 7f1a78b9b700 1 --2- 172.21.15.59:0/2862784590 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1a74100370 0x7f1a7406a500 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f1a6800d980 tx=0x7f1a6800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:44:43.579 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.572+0000 7f1a71ffb700 1 -- 172.21.15.59:0/2862784590 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a68004020 con 0x7f1a74100370 2023-12-11T17:44:43.579 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.572+0000 7f1a71ffb700 1 -- 172.21.15.59:0/2862784590 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1a68009730 con 0x7f1a74100370 2023-12-11T17:44:43.579 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.572+0000 7f1a71ffb700 1 -- 172.21.15.59:0/2862784590 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a68003a80 con 0x7f1a74100370 2023-12-11T17:44:43.579 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.572+0000 7f1a79b9d700 1 -- 172.21.15.59:0/2862784590 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a7419f4b0 con 0x7f1a74100370 2023-12-11T17:44:43.579 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.572+0000 7f1a79b9d700 1 -- 172.21.15.59:0/2862784590 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a7419f990 con 0x7f1a74100370 2023-12-11T17:44:43.580 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.573+0000 7f1a79b9d700 1 -- 172.21.15.59:0/2862784590 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a74107740 con 0x7f1a74100370 2023-12-11T17:44:43.580 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.574+0000 7f1a71ffb700 1 -- 172.21.15.59:0/2862784590 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f1a680098a0 con 0x7f1a74100370 2023-12-11T17:44:43.580 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.575+0000 7f1a71ffb700 1 --2- 172.21.15.59:0/2862784590 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f1a640a1500 0x7f1a640a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:43.581 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.575+0000 7f1a73fff700 1 --2- 172.21.15.59:0/2862784590 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f1a640a1500 0x7f1a640a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:43.581 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.575+0000 7f1a71ffb700 1 -- 172.21.15.59:0/2862784590 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f1a680ccb80 con 0x7f1a74100370 2023-12-11T17:44:43.581 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.575+0000 7f1a73fff700 1 --2- 172.21.15.59:0/2862784590 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f1a640a1500 0x7f1a640a39c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f1a60009990 tx=0x7f1a60005ad0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:44:43.582 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.579+0000 7f1a71ffb700 1 -- 172.21.15.59:0/2862784590 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f1a68095ff0 con 0x7f1a74100370 2023-12-11T17:44:43.788 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.785+0000 7f1a79b9d700 1 -- 172.21.15.59:0/2862784590 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1a7410bb90 con 0x7f1a74100370 2023-12-11T17:44:43.790 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.786+0000 7f1a71ffb700 1 -- 172.21.15.59:0/2862784590 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+23960 (secure 0 0 0) 0x7f1a68099610 con 0x7f1a74100370 2023-12-11T17:44:43.790 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-12-11T17:44:43.791 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":86,"fsid":"d0a07eaa-984b-11ee-95a3-87774f69a715","created":"2023-12-11T17:37:14.526539+0000","modified":"2023-12-11T17:44:25.096094+0000","last_up_change":"2023-12-11T17:43:38.535503+0000","last_in_change":"2023-12-11T17:43:21.014888+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-11T17:40:26.596029+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-11T17:44:18.586290+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-11T17:44:19.432952+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-11T17:44:23.097308+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3c0b3218-4b2b-40f8-8bef-ed8cd5633607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":1040759398},{"type":"v1","addr":"172.21.15.59:6803","nonce":1040759398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":1040759398},{"type":"v1","addr":"172.21.15.59:6805","nonce":1040759398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":1040759398},{"type":"v1","addr":"172.21.15.59:6809","nonce":1040759398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":1040759398},{"type":"v1","addr":"172.21.15.59:6807","nonce":1040759398}]},"public_addr":"172.21.15.59:6803/1040759398","cluster_addr":"172.21.15.59:6805/1040759398","heartbeat_back_addr":"172.21.15.59:6809/1040759398","heartbeat_front_addr":"172.21.15.59:6807/1040759398","state":["exists","up"]},{"osd":1,"uuid":"2ead7578-765b-4f5a-a72c-863731ed69e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":2776354245},{"type":"v1","addr":"172.21.15.115:6801","nonce":2776354245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":2776354245},{"type":"v1","addr":"172.21.15.115:6803","nonce":2776354245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":2776354245},{"type":"v1","addr":"172.21.15.115:6807","nonce":2776354245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":2776354245},{"type":"v1","addr":"172.21.15.115:6805","nonce":2776354245}]},"public_addr":"172.21.15.115:6801/2776354245","cluster_addr":"172.21.15.115:6803/2776354245","heartbeat_back_addr":"172.21.15.115:6807/2776354245","heartbeat_front_addr":"172.21.15.115:6805/2776354245","state":["exists","up"]},{"osd":2,"uuid":"8cb1d7ec-53cc-4672-b0b6-70d73797c44f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4287675481},{"type":"v1","addr":"172.21.15.181:6801","nonce":4287675481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4287675481},{"type":"v1","addr":"172.21.15.181:6803","nonce":4287675481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4287675481},{"type":"v1","addr":"172.21.15.181:6807","nonce":4287675481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4287675481},{"type":"v1","addr":"172.21.15.181:6805","nonce":4287675481}]},"public_addr":"172.21.15.181:6801/4287675481","cluster_addr":"172.21.15.181:6803/4287675481","heartbeat_back_addr":"172.21.15.181:6807/4287675481","heartbeat_front_addr":"172.21.15.181:6805/4287675481","state":["exists","up"]},{"osd":3,"uuid":"5d64d9af-831b-4d0a-8ace-a452169351d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":219909995},{"type":"v1","addr":"172.21.15.59:6811","nonce":219909995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":219909995},{"type":"v1","addr":"172.21.15.59:6813","nonce":219909995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":219909995},{"type":"v1","addr":"172.21.15.59:6817","nonce":219909995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":219909995},{"type":"v1","addr":"172.21.15.59:6815","nonce":219909995}]},"public_addr":"172.21.15.59:6811/219909995","cluster_addr":"172.21.15.59:6813/219909995","heartbeat_back_addr":"172.21.15.59:6817/219909995","heartbeat_front_addr":"172.21.15.59:6815/219909995","state":["exists","up"]},{"osd":4,"uuid":"ef8fad28-1db5-4cba-b5f2-10e54e8e1549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":1021473826},{"type":"v1","addr":"172.21.15.115:6809","nonce":1021473826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":1021473826},{"type":"v1","addr":"172.21.15.115:6811","nonce":1021473826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":1021473826},{"type":"v1","addr":"172.21.15.115:6815","nonce":1021473826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":1021473826},{"type":"v1","addr":"172.21.15.115:6813","nonce":1021473826}]},"public_addr":"172.21.15.115:6809/1021473826","cluster_addr":"172.21.15.115:6811/1021473826","heartbeat_back_addr":"172.21.15.115:6815/1021473826","heartbeat_front_addr":"172.21.15.115:6813/1021473826","state":["exists","up"]},{"osd":5,"uuid":"4233a512-9e8f-4db8-aed2-6c3af706e589","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":2405172521},{"type":"v1","addr":"172.21.15.181:6809","nonce":2405172521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":2405172521},{"type":"v1","addr":"172.21.15.181:6811","nonce":2405172521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2405172521},{"type":"v1","addr":"172.21.15.181:6815","nonce":2405172521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":2405172521},{"type":"v1","addr":"172.21.15.181:6813","nonce":2405172521}]},"public_addr":"172.21.15.181:6809/2405172521","cluster_addr":"172.21.15.181:6811/2405172521","heartbeat_back_addr":"172.21.15.181:6815/2405172521","heartbeat_front_addr":"172.21.15.181:6813/2405172521","state":["exists","up"]},{"osd":6,"uuid":"8faab332-016e-4c77-8370-8bef95fc6f34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":815374853},{"type":"v1","addr":"172.21.15.59:6819","nonce":815374853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":815374853},{"type":"v1","addr":"172.21.15.59:6821","nonce":815374853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":815374853},{"type":"v1","addr":"172.21.15.59:6825","nonce":815374853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":815374853},{"type":"v1","addr":"172.21.15.59:6823","nonce":815374853}]},"public_addr":"172.21.15.59:6819/815374853","cluster_addr":"172.21.15.59:6821/815374853","heartbeat_back_addr":"172.21.15.59:6825/815374853","heartbeat_front_addr":"172.21.15.59:6823/815374853","state":["exists","up"]},{"osd":7,"uuid":"f20f470a-0d40-4ed3-8e0e-fd58675cf143","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":1314971870},{"type":"v1","addr":"172.21.15.115:6817","nonce":1314971870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":1314971870},{"type":"v1","addr":"172.21.15.115:6819","nonce":1314971870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":1314971870},{"type":"v1","addr":"172.21.15.115:6823","nonce":1314971870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":1314971870},{"type":"v1","addr":"172.21.15.115:6821","nonce":1314971870}]},"public_addr":"172.21.15.115:6817/1314971870","cluster_addr":"172.21.15.115:6819/1314971870","heartbeat_back_addr":"172.21.15.115:6823/1314971870","heartbeat_front_addr":"172.21.15.115:6821/1314971870","state":["exists","up"]},{"osd":8,"uuid":"eb657d32-1a95-4fd7-a293-d0a970837fc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":3954509873},{"type":"v1","addr":"172.21.15.181:6817","nonce":3954509873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":3954509873},{"type":"v1","addr":"172.21.15.181:6819","nonce":3954509873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":3954509873},{"type":"v1","addr":"172.21.15.181:6823","nonce":3954509873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":3954509873},{"type":"v1","addr":"172.21.15.181:6821","nonce":3954509873}]},"public_addr":"172.21.15.181:6817/3954509873","cluster_addr":"172.21.15.181:6819/3954509873","heartbeat_back_addr":"172.21.15.181:6823/3954509873","heartbeat_front_addr":"172.21.15.181:6821/3954509873","state":["exists","up"]},{"osd":9,"uuid":"8f7717d4-b6f2-4ec1-9b58-62feb0c9a146","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":1163095677},{"type":"v1","addr":"172.21.15.59:6827","nonce":1163095677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":1163095677},{"type":"v1","addr":"172.21.15.59:6829","nonce":1163095677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":1163095677},{"type":"v1","addr":"172.21.15.59:6833","nonce":1163095677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":1163095677},{"type":"v1","addr":"172.21.15.59:6831","nonce":1163095677}]},"public_addr":"172.21.15.59:6827/1163095677","cluster_addr":"172.21.15.59:6829/1163095677","heartbeat_back_addr":"172.21.15.59:6833/1163095677","heartbeat_front_addr":"172.21.15.59:6831/1163095677","state":["exists","up"]},{"osd":10,"uuid":"b75c179a-875d-43bc-b4c1-e8eb80fb64fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":2199610269},{"type":"v1","addr":"172.21.15.115:6825","nonce":2199610269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":2199610269},{"type":"v1","addr":"172.21.15.115:6827","nonce":2199610269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":2199610269},{"type":"v1","addr":"172.21.15.115:6831","nonce":2199610269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":2199610269},{"type":"v1","addr":"172.21.15.115:6829","nonce":2199610269}]},"public_addr":"172.21.15.115:6825/2199610269","cluster_addr":"172.21.15.115:6827/2199610269","heartbeat_back_addr":"172.21.15.115:6831/2199610269","heartbeat_front_addr":"172.21.15.115:6829/2199610269","state":["exists","up"]},{"osd":11,"uuid":"5e141dfe-8a00-4bbc-8204-2dc9be24e18d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":226506989},{"type":"v1","addr":"172.21.15.181:6825","nonce":226506989}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":226506989},{"type":"v1","addr":"172.21.15.181:6827","nonce":226506989}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":226506989},{"type":"v1","addr":"172.21.15.181:6831","nonce":226506989}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":226506989},{"type":"v1","addr":"172.21.15.181:6829","nonce":226506989}]},"public_addr":"172.21.15.181:6825/226506989","cluster_addr":"172.21.15.181:6827/226506989","heartbeat_back_addr":"172.21.15.181:6831/226506989","heartbeat_front_addr":"172.21.15.181:6829/226506989","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:39:42.687084+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:40:03.903448+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:40:24.899908+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:40:46.600251+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:41:07.690037+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:41:28.488855+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:41:49.459274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:42:10.863881+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:42:32.248391+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:42:54.112265+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:43:15.634872+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:43:36.922534+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.59:6800/1259916482":"2023-12-12T17:38:14.037922+0000","172.21.15.59:0/17457774":"2023-12-12T17:38:14.037922+0000","172.21.15.59:6801/1672530462":"2023-12-12T17:37:44.773925+0000","172.21.15.59:0/1053944547":"2023-12-12T17:38:14.037922+0000","172.21.15.59:6801/1259916482":"2023-12-12T17:38:14.037922+0000","172.21.15.59:6800/1672530462":"2023-12-12T17:37:44.773925+0000","172.21.15.59:0/3570767650":"2023-12-12T17:37:44.773925+0000","172.21.15.59:0/3853462899":"2023-12-12T17:37:44.773925+0000","172.21.15.59:0/2859415380":"2023-12-12T17:38:14.037922+0000","172.21.15.59:0/2192742819":"2023-12-12T17:37:44.773925+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-11T17:44:43.792 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.789+0000 7f1a79b9d700 1 -- 172.21.15.59:0/2862784590 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f1a640a1500 msgr2=0x7f1a640a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:43.792 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.789+0000 7f1a79b9d700 1 --2- 172.21.15.59:0/2862784590 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f1a640a1500 0x7f1a640a39c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f1a60009990 tx=0x7f1a60005ad0 comp rx=0 tx=0).stop 2023-12-11T17:44:43.792 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.789+0000 7f1a79b9d700 1 -- 172.21.15.59:0/2862784590 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1a74100370 msgr2=0x7f1a7406a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:43.793 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.789+0000 7f1a79b9d700 1 --2- 172.21.15.59:0/2862784590 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1a74100370 0x7f1a7406a500 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f1a6800d980 tx=0x7f1a6800dc90 comp rx=0 tx=0).stop 2023-12-11T17:44:43.793 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.789+0000 7f1a79b9d700 1 -- 172.21.15.59:0/2862784590 shutdown_connections 2023-12-11T17:44:43.793 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.789+0000 7f1a79b9d700 1 --2- 172.21.15.59:0/2862784590 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f1a640a1500 0x7f1a640a39c0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:43.793 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.789+0000 7f1a79b9d700 1 --2- 172.21.15.59:0/2862784590 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a74101f00 0x7f1a7419e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:43.793 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.789+0000 7f1a79b9d700 1 --2- 172.21.15.59:0/2862784590 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1a74100370 0x7f1a7406a500 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:43.793 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.789+0000 7f1a79b9d700 1 --2- 172.21.15.59:0/2862784590 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1a74101560 0x7f1a7419c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:43.793 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.789+0000 7f1a79b9d700 1 -- 172.21.15.59:0/2862784590 >> 172.21.15.59:0/2862784590 conn(0x7f1a740fb8e0 msgr2=0x7f1a74104880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:44:43.793 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.789+0000 7f1a79b9d700 1 -- 172.21.15.59:0/2862784590 shutdown_connections 2023-12-11T17:44:43.793 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.789+0000 7f1a79b9d700 1 -- 172.21.15.59:0/2862784590 wait complete. 2023-12-11T17:44:43.805 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-12-11T17:44:43.937 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.930+0000 7f3f98765700 1 -- 172.21.15.59:0/1528333996 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3f901016a0 msgr2=0x7f3f90101b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:43.937 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.930+0000 7f3f98765700 1 --2- 172.21.15.59:0/1528333996 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3f901016a0 0x7f3f90101b00 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f3f800099c0 tx=0x7f3f80009cd0 comp rx=0 tx=0).stop 2023-12-11T17:44:43.937 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.931+0000 7f3f98765700 1 -- 172.21.15.59:0/1528333996 shutdown_connections 2023-12-11T17:44:43.937 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.931+0000 7f3f98765700 1 --2- 172.21.15.59:0/1528333996 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f90102040 0x7f3f901088f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:43.937 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.931+0000 7f3f98765700 1 --2- 172.21.15.59:0/1528333996 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3f901004b0 0x7f3f901008d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:43.937 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.931+0000 7f3f98765700 1 --2- 172.21.15.59:0/1528333996 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3f901016a0 0x7f3f90101b00 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:43.937 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.931+0000 7f3f98765700 1 -- 172.21.15.59:0/1528333996 >> 172.21.15.59:0/1528333996 conn(0x7f3f900fba20 msgr2=0x7f3f900fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:44:43.937 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.931+0000 7f3f98765700 1 -- 172.21.15.59:0/1528333996 shutdown_connections 2023-12-11T17:44:43.937 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.931+0000 7f3f98765700 1 -- 172.21.15.59:0/1528333996 wait complete. 2023-12-11T17:44:43.937 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.931+0000 7f3f98765700 1 Processor -- start 2023-12-11T17:44:43.937 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.931+0000 7f3f98765700 1 -- start start 2023-12-11T17:44:43.938 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.931+0000 7f3f98765700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3f901004b0 0x7f3f90197480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:43.938 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.931+0000 7f3f98765700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f901016a0 0x7f3f901979c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:43.938 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.931+0000 7f3f98765700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3f90102040 0x7f3f9019e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:43.938 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.931+0000 7f3f98765700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f9019efb0 con 0x7f3f901004b0 2023-12-11T17:44:43.938 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.931+0000 7f3f98765700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f9019f0f0 con 0x7f3f90102040 2023-12-11T17:44:43.938 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.931+0000 7f3f98765700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f9019f280 con 0x7f3f901016a0 2023-12-11T17:44:43.938 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.932+0000 7f3f96501700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3f901004b0 0x7f3f90197480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:43.938 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.932+0000 7f3f96501700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3f901004b0 0x7f3f90197480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33284/0 (socket says 172.21.15.59:33284) 2023-12-11T17:44:43.938 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.932+0000 7f3f96501700 1 -- 172.21.15.59:0/2485806995 learned_addr learned my addr 172.21.15.59:0/2485806995 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:44:43.938 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.932+0000 7f3f95d00700 1 --2- 172.21.15.59:0/2485806995 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f901016a0 0x7f3f901979c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:43.938 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.932+0000 7f3f96501700 1 -- 172.21.15.59:0/2485806995 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f901016a0 msgr2=0x7f3f901979c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:43.938 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.932+0000 7f3f96501700 1 --2- 172.21.15.59:0/2485806995 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f901016a0 0x7f3f901979c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:43.939 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.932+0000 7f3f96d02700 1 --2- 172.21.15.59:0/2485806995 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3f90102040 0x7f3f9019e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:43.939 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.932+0000 7f3f96501700 1 -- 172.21.15.59:0/2485806995 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3f90102040 msgr2=0x7f3f9019e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:43.939 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.932+0000 7f3f96501700 1 --2- 172.21.15.59:0/2485806995 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3f90102040 0x7f3f9019e9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:43.939 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.932+0000 7f3f96501700 1 -- 172.21.15.59:0/2485806995 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f800095f0 con 0x7f3f901004b0 2023-12-11T17:44:43.939 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.932+0000 7f3f95d00700 1 --2- 172.21.15.59:0/2485806995 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f901016a0 0x7f3f901979c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T17:44:43.939 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.932+0000 7f3f96501700 1 --2- 172.21.15.59:0/2485806995 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3f901004b0 0x7f3f90197480 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f3f8c00e970 tx=0x7f3f8c00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:44:43.939 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.932+0000 7f3f877fe700 1 -- 172.21.15.59:0/2485806995 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f8c009790 con 0x7f3f901004b0 2023-12-11T17:44:43.939 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.932+0000 7f3f877fe700 1 -- 172.21.15.59:0/2485806995 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3f8c0044e0 con 0x7f3f901004b0 2023-12-11T17:44:43.939 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.932+0000 7f3f98765700 1 -- 172.21.15.59:0/2485806995 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f9019f560 con 0x7f3f901004b0 2023-12-11T17:44:43.939 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.932+0000 7f3f877fe700 1 -- 172.21.15.59:0/2485806995 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f8c003e30 con 0x7f3f901004b0 2023-12-11T17:44:43.939 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.932+0000 7f3f98765700 1 -- 172.21.15.59:0/2485806995 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f9019fa70 con 0x7f3f901004b0 2023-12-11T17:44:43.939 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.933+0000 7f3f98765700 1 -- 172.21.15.59:0/2485806995 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f90067a80 con 0x7f3f901004b0 2023-12-11T17:44:43.940 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.934+0000 7f3f877fe700 1 -- 172.21.15.59:0/2485806995 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f3f8c003f90 con 0x7f3f901004b0 2023-12-11T17:44:43.940 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.935+0000 7f3f877fe700 1 --2- 172.21.15.59:0/2485806995 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f3f7c0a1480 0x7f3f7c0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:43.940 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.935+0000 7f3f877fe700 1 -- 172.21.15.59:0/2485806995 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f3f8c014070 con 0x7f3f901004b0 2023-12-11T17:44:43.940 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.935+0000 7f3f95d00700 1 --2- 172.21.15.59:0/2485806995 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f3f7c0a1480 0x7f3f7c0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:43.941 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.935+0000 7f3f95d00700 1 --2- 172.21.15.59:0/2485806995 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f3f7c0a1480 0x7f3f7c0a3940 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3f8000b580 tx=0x7f3f8001a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:44:43.943 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:43.939+0000 7f3f877fe700 1 -- 172.21.15.59:0/2485806995 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f3f8c0966b0 con 0x7f3f901004b0 2023-12-11T17:44:44.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:43 smithi115 ceph-mon[102748]: pgmap v270: 129 pgs: 129 active+clean; 581 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-12-11T17:44:44.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:43 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/563982075' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-11T17:44:44.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:43 smithi181 ceph-mon[103627]: pgmap v270: 129 pgs: 129 active+clean; 581 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-12-11T17:44:44.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:43 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/563982075' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-11T17:44:44.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:43 smithi059 ceph-mon[98847]: pgmap v270: 129 pgs: 129 active+clean; 581 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-12-11T17:44:44.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:43 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/563982075' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-11T17:44:44.193 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:44.190+0000 7f3f98765700 1 -- 172.21.15.59:0/2485806995 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f3f9010bcc0 con 0x7f3f901004b0 2023-12-11T17:44:44.740 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:44.736+0000 7f3f877fe700 1 -- 172.21.15.59:0/2485806995 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) v1 ==== 118+0+0 (secure 0 0 0) 0x7f3f8c099cd0 con 0x7f3f901004b0 2023-12-11T17:44:44.743 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:44.739+0000 7f3f98765700 1 -- 172.21.15.59:0/2485806995 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f3f7c0a1480 msgr2=0x7f3f7c0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:44.743 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:44.739+0000 7f3f98765700 1 --2- 172.21.15.59:0/2485806995 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f3f7c0a1480 0x7f3f7c0a3940 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3f8000b580 tx=0x7f3f8001a040 comp rx=0 tx=0).stop 2023-12-11T17:44:44.743 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:44.739+0000 7f3f98765700 1 -- 172.21.15.59:0/2485806995 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3f901004b0 msgr2=0x7f3f90197480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:44.743 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:44.739+0000 7f3f98765700 1 --2- 172.21.15.59:0/2485806995 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3f901004b0 0x7f3f90197480 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f3f8c00e970 tx=0x7f3f8c00ed30 comp rx=0 tx=0).stop 2023-12-11T17:44:44.743 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:44.739+0000 7f3f98765700 1 -- 172.21.15.59:0/2485806995 shutdown_connections 2023-12-11T17:44:44.743 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:44.739+0000 7f3f98765700 1 --2- 172.21.15.59:0/2485806995 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f3f7c0a1480 0x7f3f7c0a3940 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:44.744 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:44.739+0000 7f3f98765700 1 --2- 172.21.15.59:0/2485806995 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3f90102040 0x7f3f9019e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:44.744 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:44.739+0000 7f3f98765700 1 --2- 172.21.15.59:0/2485806995 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3f901004b0 0x7f3f90197480 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:44.744 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:44.739+0000 7f3f98765700 1 --2- 172.21.15.59:0/2485806995 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f901016a0 0x7f3f901979c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:44.744 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:44.739+0000 7f3f98765700 1 -- 172.21.15.59:0/2485806995 >> 172.21.15.59:0/2485806995 conn(0x7f3f900fba20 msgr2=0x7f3f900fd500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:44:44.744 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:44.740+0000 7f3f98765700 1 -- 172.21.15.59:0/2485806995 shutdown_connections 2023-12-11T17:44:44.744 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:44.740+0000 7f3f98765700 1 -- 172.21.15.59:0/2485806995 wait complete. 2023-12-11T17:44:44.761 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2023-12-11T17:44:44.761 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2023-12-11T17:44:44.898 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:44.893+0000 7f7da8cf9700 1 -- 172.21.15.59:0/1763670171 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7da4101670 msgr2=0x7f7da4101ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:44.899 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:44.893+0000 7f7da8cf9700 1 --2- 172.21.15.59:0/1763670171 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7da4101670 0x7f7da4101ad0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f7d980099c0 tx=0x7f7d98009cd0 comp rx=0 tx=0).stop 2023-12-11T17:44:44.899 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:44.893+0000 7f7da8cf9700 1 -- 172.21.15.59:0/1763670171 shutdown_connections 2023-12-11T17:44:44.899 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:44.893+0000 7f7da8cf9700 1 --2- 172.21.15.59:0/1763670171 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7da4102010 0x7f7da41088c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:44.899 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:44.893+0000 7f7da8cf9700 1 --2- 172.21.15.59:0/1763670171 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7da4100480 0x7f7da41008a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:44.899 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:44.893+0000 7f7da8cf9700 1 --2- 172.21.15.59:0/1763670171 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7da4101670 0x7f7da4101ad0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:44.899 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:44.893+0000 7f7da8cf9700 1 -- 172.21.15.59:0/1763670171 >> 172.21.15.59:0/1763670171 conn(0x7f7da40fb9f0 msgr2=0x7f7da40fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:44:44.899 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:44.893+0000 7f7da8cf9700 1 -- 172.21.15.59:0/1763670171 shutdown_connections 2023-12-11T17:44:44.901 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:44.893+0000 7f7da8cf9700 1 -- 172.21.15.59:0/1763670171 wait complete. 2023-12-11T17:44:44.901 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:44.893+0000 7f7da8cf9700 1 Processor -- start 2023-12-11T17:44:44.901 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:44.893+0000 7f7da8cf9700 1 -- start start 2023-12-11T17:44:44.901 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:44.894+0000 7f7da8cf9700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7da4100480 0x7f7da4196380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:44.901 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:44.894+0000 7f7da8cf9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7da4101670 0x7f7da41968c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:44.902 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:44.894+0000 7f7da8cf9700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7da4102010 0x7f7da419d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:44.902 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:44.894+0000 7f7da8cf9700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7da419deb0 con 0x7f7da4100480 2023-12-11T17:44:44.902 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:44.894+0000 7f7da8cf9700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7da419dff0 con 0x7f7da4102010 2023-12-11T17:44:44.902 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:44.894+0000 7f7da8cf9700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7da419e180 con 0x7f7da4101670 2023-12-11T17:44:44.902 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:44.894+0000 7f7da259c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7da4100480 0x7f7da4196380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:44.902 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:44.894+0000 7f7da259c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7da4100480 0x7f7da4196380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33298/0 (socket says 172.21.15.59:33298) 2023-12-11T17:44:44.902 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:44.894+0000 7f7da259c700 1 -- 172.21.15.59:0/2135285527 learned_addr learned my addr 172.21.15.59:0/2135285527 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:44:44.902 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:44.894+0000 7f7da1d9b700 1 --2- 172.21.15.59:0/2135285527 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7da4101670 0x7f7da41968c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:44.902 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:44.894+0000 7f7da2d9d700 1 --2- 172.21.15.59:0/2135285527 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7da4102010 0x7f7da419d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:44.902 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:44.894+0000 7f7da259c700 1 -- 172.21.15.59:0/2135285527 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7da4101670 msgr2=0x7f7da41968c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:44.902 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:44.894+0000 7f7da259c700 1 --2- 172.21.15.59:0/2135285527 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7da4101670 0x7f7da41968c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:44.902 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:44.894+0000 7f7da259c700 1 -- 172.21.15.59:0/2135285527 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7da4102010 msgr2=0x7f7da419d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:44.903 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:44.894+0000 7f7da259c700 1 --2- 172.21.15.59:0/2135285527 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7da4102010 0x7f7da419d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:44.903 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:44.894+0000 7f7da259c700 1 -- 172.21.15.59:0/2135285527 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d980095f0 con 0x7f7da4100480 2023-12-11T17:44:44.903 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:44.894+0000 7f7da2d9d700 1 --2- 172.21.15.59:0/2135285527 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7da4102010 0x7f7da419d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-11T17:44:44.903 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:44.894+0000 7f7da259c700 1 --2- 172.21.15.59:0/2135285527 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7da4100480 0x7f7da4196380 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f7d9400c970 tx=0x7f7d9400cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:44:44.903 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:44.894+0000 7f7d937fe700 1 -- 172.21.15.59:0/2135285527 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d94004220 con 0x7f7da4100480 2023-12-11T17:44:44.903 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:44.894+0000 7f7da8cf9700 1 -- 172.21.15.59:0/2135285527 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7da419e460 con 0x7f7da4100480 2023-12-11T17:44:44.903 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:44.894+0000 7f7d937fe700 1 -- 172.21.15.59:0/2135285527 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7d94007d60 con 0x7f7da4100480 2023-12-11T17:44:44.903 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:44.894+0000 7f7d937fe700 1 -- 172.21.15.59:0/2135285527 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d94003d00 con 0x7f7da4100480 2023-12-11T17:44:44.903 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:44.894+0000 7f7da8cf9700 1 -- 172.21.15.59:0/2135285527 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7da419e9c0 con 0x7f7da4100480 2023-12-11T17:44:44.903 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:44.895+0000 7f7da8cf9700 1 -- 172.21.15.59:0/2135285527 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7da41078b0 con 0x7f7da4100480 2023-12-11T17:44:44.904 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:44.896+0000 7f7d937fe700 1 -- 172.21.15.59:0/2135285527 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f7d94004380 con 0x7f7da4100480 2023-12-11T17:44:44.904 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:44.897+0000 7f7d937fe700 1 --2- 172.21.15.59:0/2135285527 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f7d8c0a1480 0x7f7d8c0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:44.904 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:44.897+0000 7f7d937fe700 1 -- 172.21.15.59:0/2135285527 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f7d940cce70 con 0x7f7da4100480 2023-12-11T17:44:44.904 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:44.897+0000 7f7da1d9b700 1 --2- 172.21.15.59:0/2135285527 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f7d8c0a1480 0x7f7d8c0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:44.904 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:44.898+0000 7f7da1d9b700 1 --2- 172.21.15.59:0/2135285527 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f7d8c0a1480 0x7f7d8c0a3940 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f7d98000c00 tx=0x7f7d98005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:44:44.905 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:44.901+0000 7f7d937fe700 1 -- 172.21.15.59:0/2135285527 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f7d940962e0 con 0x7f7da4100480 2023-12-11T17:44:45.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:44 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/2862784590' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-11T17:44:45.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:44 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/2485806995' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-12-11T17:44:45.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:44 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/2862784590' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-11T17:44:45.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:44 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/2485806995' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-12-11T17:44:45.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:44 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/2862784590' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-11T17:44:45.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:44 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/2485806995' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-12-11T17:44:45.155 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:45.152+0000 7f7da8cf9700 1 -- 172.21.15.59:0/2135285527 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) v1 -- 0x7f7da40682e0 con 0x7f7da4100480 2023-12-11T17:44:45.749 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:45.745+0000 7f7d937fe700 1 -- 172.21.15.59:0/2135285527 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v19) v1 ==== 105+0+0 (secure 0 0 0) 0x7f7d9401d0b0 con 0x7f7da4100480 2023-12-11T17:44:45.752 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:45.748+0000 7f7da8cf9700 1 -- 172.21.15.59:0/2135285527 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f7d8c0a1480 msgr2=0x7f7d8c0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:45.752 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:45.748+0000 7f7da8cf9700 1 --2- 172.21.15.59:0/2135285527 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f7d8c0a1480 0x7f7d8c0a3940 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f7d98000c00 tx=0x7f7d98005fd0 comp rx=0 tx=0).stop 2023-12-11T17:44:45.752 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:45.748+0000 7f7da8cf9700 1 -- 172.21.15.59:0/2135285527 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7da4100480 msgr2=0x7f7da4196380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:45.752 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:45.748+0000 7f7da8cf9700 1 --2- 172.21.15.59:0/2135285527 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7da4100480 0x7f7da4196380 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f7d9400c970 tx=0x7f7d9400cd30 comp rx=0 tx=0).stop 2023-12-11T17:44:45.752 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:45.749+0000 7f7da8cf9700 1 -- 172.21.15.59:0/2135285527 shutdown_connections 2023-12-11T17:44:45.753 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:45.749+0000 7f7da8cf9700 1 --2- 172.21.15.59:0/2135285527 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f7d8c0a1480 0x7f7d8c0a3940 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:45.753 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:45.749+0000 7f7da8cf9700 1 --2- 172.21.15.59:0/2135285527 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7da4102010 0x7f7da419d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:45.753 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:45.749+0000 7f7da8cf9700 1 --2- 172.21.15.59:0/2135285527 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7da4100480 0x7f7da4196380 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:45.753 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:45.749+0000 7f7da8cf9700 1 --2- 172.21.15.59:0/2135285527 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7da4101670 0x7f7da41968c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:45.753 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:45.749+0000 7f7da8cf9700 1 -- 172.21.15.59:0/2135285527 >> 172.21.15.59:0/2135285527 conn(0x7f7da40fb9f0 msgr2=0x7f7da41049f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:44:45.753 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:45.749+0000 7f7da8cf9700 1 -- 172.21.15.59:0/2135285527 shutdown_connections 2023-12-11T17:44:45.753 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:45.749+0000 7f7da8cf9700 1 -- 172.21.15.59:0/2135285527 wait complete. 2023-12-11T17:44:45.771 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2023-12-11T17:44:45.909 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:45.903+0000 7f5be4953700 1 -- 172.21.15.59:0/3184861016 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5be0101070 msgr2=0x7f5be0108910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:45.909 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:45.904+0000 7f5be4953700 1 --2- 172.21.15.59:0/3184861016 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5be0101070 0x7f5be0108910 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f5bd800b0a0 tx=0x7f5bd800b3b0 comp rx=0 tx=0).stop 2023-12-11T17:44:45.909 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:45.904+0000 7f5be4953700 1 -- 172.21.15.59:0/3184861016 shutdown_connections 2023-12-11T17:44:45.909 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:45.904+0000 7f5be4953700 1 --2- 172.21.15.59:0/3184861016 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5be0101070 0x7f5be0108910 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:45.910 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:45.904+0000 7f5be4953700 1 --2- 172.21.15.59:0/3184861016 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5be0101a60 0x7f5be0101e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:45.910 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:45.904+0000 7f5be4953700 1 --2- 172.21.15.59:0/3184861016 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5be01006b0 0x7f5be0100b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:45.910 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:45.904+0000 7f5be4953700 1 -- 172.21.15.59:0/3184861016 >> 172.21.15.59:0/3184861016 conn(0x7f5be00fb1a0 msgr2=0x7f5be00fd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:44:45.910 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:45.904+0000 7f5be4953700 1 -- 172.21.15.59:0/3184861016 shutdown_connections 2023-12-11T17:44:45.910 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:45.904+0000 7f5be4953700 1 -- 172.21.15.59:0/3184861016 wait complete. 2023-12-11T17:44:45.910 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:45.904+0000 7f5be4953700 1 Processor -- start 2023-12-11T17:44:45.910 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:45.904+0000 7f5be4953700 1 -- start start 2023-12-11T17:44:45.910 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:45.905+0000 7f5be4953700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5be01006b0 0x7f5be01000e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:45.910 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:45.905+0000 7f5be4953700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5be0101070 0x7f5be019c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:45.910 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:45.905+0000 7f5be4953700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5be0101a60 0x7f5be019e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:45.910 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:45.905+0000 7f5be4953700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5be019f0e0 con 0x7f5be0101070 2023-12-11T17:44:45.910 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:45.905+0000 7f5be4953700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5be019f250 con 0x7f5be01006b0 2023-12-11T17:44:45.910 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:45.905+0000 7f5be4953700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5be019f410 con 0x7f5be0101a60 2023-12-11T17:44:45.910 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:45.905+0000 7f5bddd9b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5be0101070 0x7f5be019c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:45.911 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:45.905+0000 7f5bddd9b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5be0101070 0x7f5be019c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33314/0 (socket says 172.21.15.59:33314) 2023-12-11T17:44:45.911 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:45.905+0000 7f5bddd9b700 1 -- 172.21.15.59:0/3500119668 learned_addr learned my addr 172.21.15.59:0/3500119668 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:44:45.911 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:45.905+0000 7f5bde59c700 1 --2- 172.21.15.59:0/3500119668 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5be01006b0 0x7f5be01000e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:45.911 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:45.905+0000 7f5bded9d700 1 --2- 172.21.15.59:0/3500119668 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5be0101a60 0x7f5be019e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:45.911 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:45.905+0000 7f5bddd9b700 1 -- 172.21.15.59:0/3500119668 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5be0101a60 msgr2=0x7f5be019e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:45.911 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:45.905+0000 7f5bddd9b700 1 --2- 172.21.15.59:0/3500119668 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5be0101a60 0x7f5be019e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:45.911 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:45.905+0000 7f5bddd9b700 1 -- 172.21.15.59:0/3500119668 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5be01006b0 msgr2=0x7f5be01000e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:45.911 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:45.905+0000 7f5bddd9b700 1 --2- 172.21.15.59:0/3500119668 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5be01006b0 0x7f5be01000e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:45.911 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:45.905+0000 7f5bddd9b700 1 -- 172.21.15.59:0/3500119668 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bd8009d00 con 0x7f5be0101070 2023-12-11T17:44:45.911 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:45.905+0000 7f5bded9d700 1 --2- 172.21.15.59:0/3500119668 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5be0101a60 0x7f5be019e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-11T17:44:45.911 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:45.905+0000 7f5bddd9b700 1 --2- 172.21.15.59:0/3500119668 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5be0101070 0x7f5be019c2d0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f5bd400e9a0 tx=0x7f5bd400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:44:45.911 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:45.905+0000 7f5bcf7fe700 1 -- 172.21.15.59:0/3500119668 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bd4009790 con 0x7f5be0101070 2023-12-11T17:44:45.911 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:45.905+0000 7f5bcf7fe700 1 -- 172.21.15.59:0/3500119668 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5bd400cbd0 con 0x7f5be0101070 2023-12-11T17:44:45.911 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:45.905+0000 7f5be4953700 1 -- 172.21.15.59:0/3500119668 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5be019f690 con 0x7f5be0101070 2023-12-11T17:44:45.911 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:45.905+0000 7f5bcf7fe700 1 -- 172.21.15.59:0/3500119668 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bd4018680 con 0x7f5be0101070 2023-12-11T17:44:45.912 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:45.906+0000 7f5be4953700 1 -- 172.21.15.59:0/3500119668 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5be019fba0 con 0x7f5be0101070 2023-12-11T17:44:45.912 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:45.906+0000 7f5be4953700 1 -- 172.21.15.59:0/3500119668 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5be0107820 con 0x7f5be0101070 2023-12-11T17:44:45.912 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:45.907+0000 7f5bcf7fe700 1 -- 172.21.15.59:0/3500119668 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f5bd400cd40 con 0x7f5be0101070 2023-12-11T17:44:45.912 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:45.908+0000 7f5bcf7fe700 1 --2- 172.21.15.59:0/3500119668 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f5bc80a1480 0x7f5bc80a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:45.912 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:45.908+0000 7f5bde59c700 1 --2- 172.21.15.59:0/3500119668 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f5bc80a1480 0x7f5bc80a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:45.913 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:45.908+0000 7f5bcf7fe700 1 -- 172.21.15.59:0/3500119668 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f5bd4014070 con 0x7f5be0101070 2023-12-11T17:44:45.913 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:45.909+0000 7f5bde59c700 1 --2- 172.21.15.59:0/3500119668 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f5bc80a1480 0x7f5bc80a3940 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5bd000ac50 tx=0x7f5bd000a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:44:45.915 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:45.911+0000 7f5bcf7fe700 1 -- 172.21.15.59:0/3500119668 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f5bd40968b0 con 0x7f5be0101070 2023-12-11T17:44:46.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:45 smithi115 ceph-mon[102748]: pgmap v271: 129 pgs: 129 active+clean; 581 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-12-11T17:44:46.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:45 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/2485806995' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-12-11T17:44:46.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:45 smithi115 ceph-mon[102748]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-12-11T17:44:46.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:45 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/2135285527' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2023-12-11T17:44:46.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:45 smithi181 ceph-mon[103627]: pgmap v271: 129 pgs: 129 active+clean; 581 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-12-11T17:44:46.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:45 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/2485806995' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-12-11T17:44:46.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:45 smithi181 ceph-mon[103627]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-12-11T17:44:46.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:45 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/2135285527' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2023-12-11T17:44:46.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:45 smithi059 ceph-mon[98847]: pgmap v271: 129 pgs: 129 active+clean; 581 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-12-11T17:44:46.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:45 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/2485806995' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-12-11T17:44:46.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:45 smithi059 ceph-mon[98847]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-12-11T17:44:46.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:45 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/2135285527' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2023-12-11T17:44:46.169 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:46.166+0000 7f5be4953700 1 -- 172.21.15.59:0/3500119668 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7f5be0068760 con 0x7f5be0101070 2023-12-11T17:44:46.761 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:46.758+0000 7f5bcf7fe700 1 -- 172.21.15.59:0/3500119668 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v21) v1 ==== 121+0+0 (secure 0 0 0) 0x7f5bd4099ed0 con 0x7f5be0101070 2023-12-11T17:44:46.765 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:46.761+0000 7f5be4953700 1 -- 172.21.15.59:0/3500119668 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f5bc80a1480 msgr2=0x7f5bc80a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:46.765 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:46.761+0000 7f5be4953700 1 --2- 172.21.15.59:0/3500119668 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f5bc80a1480 0x7f5bc80a3940 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5bd000ac50 tx=0x7f5bd000a380 comp rx=0 tx=0).stop 2023-12-11T17:44:46.765 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:46.761+0000 7f5be4953700 1 -- 172.21.15.59:0/3500119668 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5be0101070 msgr2=0x7f5be019c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:46.765 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:46.761+0000 7f5be4953700 1 --2- 172.21.15.59:0/3500119668 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5be0101070 0x7f5be019c2d0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f5bd400e9a0 tx=0x7f5bd400ecb0 comp rx=0 tx=0).stop 2023-12-11T17:44:46.765 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:46.761+0000 7f5be4953700 1 -- 172.21.15.59:0/3500119668 shutdown_connections 2023-12-11T17:44:46.765 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:46.761+0000 7f5be4953700 1 --2- 172.21.15.59:0/3500119668 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f5bc80a1480 0x7f5bc80a3940 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:46.765 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:46.761+0000 7f5be4953700 1 --2- 172.21.15.59:0/3500119668 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5be0101a60 0x7f5be019e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:46.765 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:46.761+0000 7f5be4953700 1 --2- 172.21.15.59:0/3500119668 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5be01006b0 0x7f5be01000e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:46.766 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:46.761+0000 7f5be4953700 1 --2- 172.21.15.59:0/3500119668 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5be0101070 0x7f5be019c2d0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:46.766 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:46.761+0000 7f5be4953700 1 -- 172.21.15.59:0/3500119668 >> 172.21.15.59:0/3500119668 conn(0x7f5be00fb1a0 msgr2=0x7f5be0104960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:44:46.766 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:46.762+0000 7f5be4953700 1 -- 172.21.15.59:0/3500119668 shutdown_connections 2023-12-11T17:44:46.766 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:46.762+0000 7f5be4953700 1 -- 172.21.15.59:0/3500119668 wait complete. 2023-12-11T17:44:46.785 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2023-12-11T17:44:46.923 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:46.918+0000 7ff10022d700 1 -- 172.21.15.59:0/1864552656 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff0f8102010 msgr2=0x7ff0f81088c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:46.924 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:46.918+0000 7ff10022d700 1 --2- 172.21.15.59:0/1864552656 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff0f8102010 0x7ff0f81088c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ff0f000b0d0 tx=0x7ff0f000b3e0 comp rx=0 tx=0).stop 2023-12-11T17:44:46.924 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:46.918+0000 7ff10022d700 1 -- 172.21.15.59:0/1864552656 shutdown_connections 2023-12-11T17:44:46.924 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:46.918+0000 7ff10022d700 1 --2- 172.21.15.59:0/1864552656 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff0f8102010 0x7ff0f81088c0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:46.924 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:46.918+0000 7ff10022d700 1 --2- 172.21.15.59:0/1864552656 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff0f8100480 0x7ff0f81008a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:46.924 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:46.918+0000 7ff10022d700 1 --2- 172.21.15.59:0/1864552656 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff0f8101670 0x7ff0f8101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:46.924 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:46.918+0000 7ff10022d700 1 -- 172.21.15.59:0/1864552656 >> 172.21.15.59:0/1864552656 conn(0x7ff0f80fb9f0 msgr2=0x7ff0f80fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:44:46.924 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:46.918+0000 7ff10022d700 1 -- 172.21.15.59:0/1864552656 shutdown_connections 2023-12-11T17:44:46.924 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:46.918+0000 7ff10022d700 1 -- 172.21.15.59:0/1864552656 wait complete. 2023-12-11T17:44:46.924 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:46.918+0000 7ff10022d700 1 Processor -- start 2023-12-11T17:44:46.924 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:46.918+0000 7ff10022d700 1 -- start start 2023-12-11T17:44:46.924 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:46.919+0000 7ff10022d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff0f8100480 0x7ff0f8069190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:46.924 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:46.919+0000 7ff10022d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff0f8101670 0x7ff0f819c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:46.925 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:46.919+0000 7ff10022d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff0f8102010 0x7ff0f819e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:46.925 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:46.919+0000 7ff10022d700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0f819f020 con 0x7ff0f8100480 2023-12-11T17:44:46.925 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:46.919+0000 7ff10022d700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0f819f190 con 0x7ff0f8102010 2023-12-11T17:44:46.925 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:46.919+0000 7ff10022d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0f819f350 con 0x7ff0f8101670 2023-12-11T17:44:46.925 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:46.919+0000 7ff0fdfc9700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff0f8100480 0x7ff0f8069190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:46.925 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:46.919+0000 7ff0fdfc9700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff0f8100480 0x7ff0f8069190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33326/0 (socket says 172.21.15.59:33326) 2023-12-11T17:44:46.925 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:46.919+0000 7ff0fdfc9700 1 -- 172.21.15.59:0/208837848 learned_addr learned my addr 172.21.15.59:0/208837848 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:44:46.925 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:46.919+0000 7ff0fd7c8700 1 --2- 172.21.15.59:0/208837848 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff0f8101670 0x7ff0f819c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:46.925 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:46.919+0000 7ff0fe7ca700 1 --2- 172.21.15.59:0/208837848 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff0f8102010 0x7ff0f819e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:46.925 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:46.919+0000 7ff0fdfc9700 1 -- 172.21.15.59:0/208837848 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff0f8101670 msgr2=0x7ff0f819c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:46.926 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:46.919+0000 7ff0fdfc9700 1 --2- 172.21.15.59:0/208837848 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff0f8101670 0x7ff0f819c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:46.926 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:46.919+0000 7ff0fdfc9700 1 -- 172.21.15.59:0/208837848 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff0f8102010 msgr2=0x7ff0f819e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:46.926 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:46.919+0000 7ff0fdfc9700 1 --2- 172.21.15.59:0/208837848 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff0f8102010 0x7ff0f819e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:46.926 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:46.919+0000 7ff0fdfc9700 1 -- 172.21.15.59:0/208837848 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0f0009d00 con 0x7ff0f8100480 2023-12-11T17:44:46.926 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:46.919+0000 7ff0fe7ca700 1 --2- 172.21.15.59:0/208837848 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff0f8102010 0x7ff0f819e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-11T17:44:46.926 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:46.919+0000 7ff0fdfc9700 1 --2- 172.21.15.59:0/208837848 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff0f8100480 0x7ff0f8069190 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ff0f400d980 tx=0x7ff0f400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:44:46.926 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:46.920+0000 7ff0eeffd700 1 -- 172.21.15.59:0/208837848 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0f4004020 con 0x7ff0f8100480 2023-12-11T17:44:46.926 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:46.920+0000 7ff0eeffd700 1 -- 172.21.15.59:0/208837848 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff0f4009730 con 0x7ff0f8100480 2023-12-11T17:44:46.926 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:46.920+0000 7ff10022d700 1 -- 172.21.15.59:0/208837848 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0f819f630 con 0x7ff0f8100480 2023-12-11T17:44:46.926 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:46.920+0000 7ff0eeffd700 1 -- 172.21.15.59:0/208837848 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0f4003a80 con 0x7ff0f8100480 2023-12-11T17:44:46.926 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:46.920+0000 7ff10022d700 1 -- 172.21.15.59:0/208837848 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0f819fb40 con 0x7ff0f8100480 2023-12-11T17:44:46.927 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:46.921+0000 7ff10022d700 1 -- 172.21.15.59:0/208837848 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0f8107830 con 0x7ff0f8100480 2023-12-11T17:44:46.927 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:46.922+0000 7ff0eeffd700 1 -- 172.21.15.59:0/208837848 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7ff0f40098a0 con 0x7ff0f8100480 2023-12-11T17:44:46.927 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:46.922+0000 7ff0eeffd700 1 --2- 172.21.15.59:0/208837848 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7ff0e40a1480 0x7ff0e40a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:46.927 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:46.922+0000 7ff0fd7c8700 1 --2- 172.21.15.59:0/208837848 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7ff0e40a1480 0x7ff0e40a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:46.927 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:46.922+0000 7ff0eeffd700 1 -- 172.21.15.59:0/208837848 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7ff0f40ccf90 con 0x7ff0f8100480 2023-12-11T17:44:46.927 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:46.923+0000 7ff0fd7c8700 1 --2- 172.21.15.59:0/208837848 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7ff0e40a1480 0x7ff0e40a3940 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff0f80696f0 tx=0x7ff0e800a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:44:46.929 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:46.926+0000 7ff0eeffd700 1 -- 172.21.15.59:0/208837848 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7ff0f40963a0 con 0x7ff0f8100480 2023-12-11T17:44:47.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:46 smithi115 ceph-mon[102748]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-12-11T17:44:47.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:46 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/2135285527' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2023-12-11T17:44:47.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:46 smithi115 ceph-mon[102748]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-12-11T17:44:47.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:46 smithi115 ceph-mon[102748]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-12-11T17:44:47.034 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:46 smithi115 ceph-mon[102748]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2023-12-11T17:44:47.034 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:46 smithi115 ceph-mon[102748]: daemon mds.i is now active in filesystem cephfs as rank 1 2023-12-11T17:44:47.034 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:46 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/3500119668' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-12-11T17:44:47.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:46 smithi181 ceph-mon[103627]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-12-11T17:44:47.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:46 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/2135285527' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2023-12-11T17:44:47.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:46 smithi181 ceph-mon[103627]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-12-11T17:44:47.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:46 smithi181 ceph-mon[103627]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-12-11T17:44:47.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:46 smithi181 ceph-mon[103627]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2023-12-11T17:44:47.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:46 smithi181 ceph-mon[103627]: daemon mds.i is now active in filesystem cephfs as rank 1 2023-12-11T17:44:47.146 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:46 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/3500119668' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-12-11T17:44:47.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:46 smithi059 ceph-mon[98847]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-12-11T17:44:47.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:46 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/2135285527' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2023-12-11T17:44:47.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:46 smithi059 ceph-mon[98847]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-12-11T17:44:47.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:46 smithi059 ceph-mon[98847]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-12-11T17:44:47.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:46 smithi059 ceph-mon[98847]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2023-12-11T17:44:47.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:46 smithi059 ceph-mon[98847]: daemon mds.i is now active in filesystem cephfs as rank 1 2023-12-11T17:44:47.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:46 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/3500119668' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-12-11T17:44:47.180 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:47.177+0000 7ff10022d700 1 -- 172.21.15.59:0/208837848 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7ff0f80682e0 con 0x7ff0f8100480 2023-12-11T17:44:47.769 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:47.765+0000 7ff0eeffd700 1 -- 172.21.15.59:0/208837848 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v23) v1 ==== 115+0+0 (secure 0 0 0) 0x7ff0f40999c0 con 0x7ff0f8100480 2023-12-11T17:44:47.771 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:47.768+0000 7ff10022d700 1 -- 172.21.15.59:0/208837848 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7ff0e40a1480 msgr2=0x7ff0e40a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:47.772 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:47.768+0000 7ff10022d700 1 --2- 172.21.15.59:0/208837848 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7ff0e40a1480 0x7ff0e40a3940 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff0f80696f0 tx=0x7ff0e800a380 comp rx=0 tx=0).stop 2023-12-11T17:44:47.772 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:47.768+0000 7ff10022d700 1 -- 172.21.15.59:0/208837848 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff0f8100480 msgr2=0x7ff0f8069190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:47.772 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:47.768+0000 7ff10022d700 1 --2- 172.21.15.59:0/208837848 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff0f8100480 0x7ff0f8069190 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ff0f400d980 tx=0x7ff0f400dc90 comp rx=0 tx=0).stop 2023-12-11T17:44:47.772 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:47.768+0000 7ff10022d700 1 -- 172.21.15.59:0/208837848 shutdown_connections 2023-12-11T17:44:47.772 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:47.768+0000 7ff10022d700 1 --2- 172.21.15.59:0/208837848 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7ff0e40a1480 0x7ff0e40a3940 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:47.772 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:47.768+0000 7ff10022d700 1 --2- 172.21.15.59:0/208837848 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff0f8102010 0x7ff0f819e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:47.772 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:47.768+0000 7ff10022d700 1 --2- 172.21.15.59:0/208837848 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff0f8100480 0x7ff0f8069190 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:47.772 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:47.768+0000 7ff10022d700 1 --2- 172.21.15.59:0/208837848 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff0f8101670 0x7ff0f819c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:47.772 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:47.768+0000 7ff10022d700 1 -- 172.21.15.59:0/208837848 >> 172.21.15.59:0/208837848 conn(0x7ff0f80fb9f0 msgr2=0x7ff0f8104970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:44:47.772 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:47.768+0000 7ff10022d700 1 -- 172.21.15.59:0/208837848 shutdown_connections 2023-12-11T17:44:47.772 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:47.768+0000 7ff10022d700 1 -- 172.21.15.59:0/208837848 wait complete. 2023-12-11T17:44:47.788 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--size 25000000000'} subvols for filesystem 'cephfs' 2023-12-11T17:44:47.788 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 '' 2023-12-11T17:44:47.927 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:47.922+0000 7fa5a3427700 1 -- 172.21.15.59:0/1022337490 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa59c102010 msgr2=0x7fa59c1088c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:47.927 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:47.922+0000 7fa5a3427700 1 --2- 172.21.15.59:0/1022337490 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa59c102010 0x7fa59c1088c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fa59400b0d0 tx=0x7fa59400b3e0 comp rx=0 tx=0).stop 2023-12-11T17:44:47.927 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:47.922+0000 7fa5a3427700 1 -- 172.21.15.59:0/1022337490 shutdown_connections 2023-12-11T17:44:47.927 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:47.922+0000 7fa5a3427700 1 --2- 172.21.15.59:0/1022337490 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa59c102010 0x7fa59c1088c0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:47.927 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:47.922+0000 7fa5a3427700 1 --2- 172.21.15.59:0/1022337490 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa59c100480 0x7fa59c1008a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:47.927 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:47.922+0000 7fa5a3427700 1 --2- 172.21.15.59:0/1022337490 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa59c101670 0x7fa59c101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:47.928 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:47.922+0000 7fa5a3427700 1 -- 172.21.15.59:0/1022337490 >> 172.21.15.59:0/1022337490 conn(0x7fa59c0fb9f0 msgr2=0x7fa59c0fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:44:47.928 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:47.922+0000 7fa5a3427700 1 -- 172.21.15.59:0/1022337490 shutdown_connections 2023-12-11T17:44:47.928 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:47.922+0000 7fa5a3427700 1 -- 172.21.15.59:0/1022337490 wait complete. 2023-12-11T17:44:47.928 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:47.922+0000 7fa5a3427700 1 Processor -- start 2023-12-11T17:44:47.928 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:47.922+0000 7fa5a3427700 1 -- start start 2023-12-11T17:44:47.928 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:47.922+0000 7fa5a3427700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa59c100480 0x7fa59c069190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:47.928 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:47.922+0000 7fa5a3427700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa59c101670 0x7fa59c19c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:47.928 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:47.922+0000 7fa5a3427700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa59c102010 0x7fa59c19e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:47.928 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:47.922+0000 7fa5a3427700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa59c19f0b0 con 0x7fa59c102010 2023-12-11T17:44:47.928 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:47.923+0000 7fa5a3427700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa59c19f220 con 0x7fa59c101670 2023-12-11T17:44:47.928 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:47.923+0000 7fa5a3427700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa59c19f3e0 con 0x7fa59c100480 2023-12-11T17:44:47.928 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:47.923+0000 7fa5a19c4700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa59c102010 0x7fa59c19e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:47.928 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:47.923+0000 7fa5a19c4700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa59c102010 0x7fa59c19e910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33344/0 (socket says 172.21.15.59:33344) 2023-12-11T17:44:47.928 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:47.923+0000 7fa5a19c4700 1 -- 172.21.15.59:0/3652101685 learned_addr learned my addr 172.21.15.59:0/3652101685 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:44:47.928 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:47.923+0000 7fa5a11c3700 1 --2- 172.21.15.59:0/3652101685 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa59c100480 0x7fa59c069190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:47.929 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:47.923+0000 7fa5a19c4700 1 -- 172.21.15.59:0/3652101685 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa59c100480 msgr2=0x7fa59c069190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:47.929 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:47.923+0000 7fa5a19c4700 1 --2- 172.21.15.59:0/3652101685 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa59c100480 0x7fa59c069190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:47.929 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:47.923+0000 7fa5a09c2700 1 --2- 172.21.15.59:0/3652101685 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa59c101670 0x7fa59c19c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:47.929 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:47.923+0000 7fa5a19c4700 1 -- 172.21.15.59:0/3652101685 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa59c101670 msgr2=0x7fa59c19c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:47.929 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:47.923+0000 7fa5a19c4700 1 --2- 172.21.15.59:0/3652101685 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa59c101670 0x7fa59c19c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:47.929 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:47.923+0000 7fa5a19c4700 1 -- 172.21.15.59:0/3652101685 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa594009d00 con 0x7fa59c102010 2023-12-11T17:44:47.929 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:47.923+0000 7fa5a19c4700 1 --2- 172.21.15.59:0/3652101685 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa59c102010 0x7fa59c19e910 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fa594007bd0 tx=0x7fa594009440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:44:47.929 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:47.923+0000 7fa5927fc700 1 -- 172.21.15.59:0/3652101685 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa59400e040 con 0x7fa59c102010 2023-12-11T17:44:47.929 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:47.923+0000 7fa5927fc700 1 -- 172.21.15.59:0/3652101685 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa59401dd50 con 0x7fa59c102010 2023-12-11T17:44:47.929 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:47.923+0000 7fa5a3427700 1 -- 172.21.15.59:0/3652101685 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa59c19f660 con 0x7fa59c102010 2023-12-11T17:44:47.929 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:47.923+0000 7fa5927fc700 1 -- 172.21.15.59:0/3652101685 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa59401b450 con 0x7fa59c102010 2023-12-11T17:44:47.929 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:47.923+0000 7fa5a3427700 1 -- 172.21.15.59:0/3652101685 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa59c19fb30 con 0x7fa59c102010 2023-12-11T17:44:47.930 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:47.924+0000 7fa5a3427700 1 -- 172.21.15.59:0/3652101685 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa59c107830 con 0x7fa59c102010 2023-12-11T17:44:47.930 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:47.925+0000 7fa5927fc700 1 -- 172.21.15.59:0/3652101685 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7fa59401d890 con 0x7fa59c102010 2023-12-11T17:44:47.930 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:47.926+0000 7fa5927fc700 1 --2- 172.21.15.59:0/3652101685 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fa5880a1500 0x7fa5880a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:47.930 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:47.926+0000 7fa5927fc700 1 -- 172.21.15.59:0/3652101685 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fa5940cf530 con 0x7fa59c102010 2023-12-11T17:44:47.931 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:47.926+0000 7fa5a11c3700 1 --2- 172.21.15.59:0/3652101685 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fa5880a1500 0x7fa5880a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:47.931 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:47.926+0000 7fa5a11c3700 1 --2- 172.21.15.59:0/3652101685 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fa5880a1500 0x7fa5880a39c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa59c1014d0 tx=0x7fa598007400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:44:47.933 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:47.929+0000 7fa5927fc700 1 -- 172.21.15.59:0/3652101685 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fa594098a30 con 0x7fa59c102010 2023-12-11T17:44:48.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:47 smithi115 ceph-mon[102748]: pgmap v272: 129 pgs: 129 active+clean; 581 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2023-12-11T17:44:48.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:47 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/3500119668' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-12-11T17:44:48.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:47 smithi115 ceph-mon[102748]: mds.? [v2:172.21.15.181:6838/2214384806,v1:172.21.15.181:6839/2214384806] up:active 2023-12-11T17:44:48.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:47 smithi115 ceph-mon[102748]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-12-11T17:44:48.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:47 smithi115 ceph-mon[102748]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-12-11T17:44:48.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:47 smithi115 ceph-mon[102748]: Cluster is now healthy 2023-12-11T17:44:48.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:47 smithi115 ceph-mon[102748]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2023-12-11T17:44:48.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:47 smithi115 ceph-mon[102748]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2023-12-11T17:44:48.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:47 smithi115 ceph-mon[102748]: daemon mds.h is now active in filesystem cephfs as rank 2 2023-12-11T17:44:48.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:47 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/208837848' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-12-11T17:44:48.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:47 smithi181 ceph-mon[103627]: pgmap v272: 129 pgs: 129 active+clean; 581 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2023-12-11T17:44:48.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:47 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/3500119668' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-12-11T17:44:48.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:47 smithi181 ceph-mon[103627]: mds.? [v2:172.21.15.181:6838/2214384806,v1:172.21.15.181:6839/2214384806] up:active 2023-12-11T17:44:48.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:47 smithi181 ceph-mon[103627]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-12-11T17:44:48.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:47 smithi181 ceph-mon[103627]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-12-11T17:44:48.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:47 smithi181 ceph-mon[103627]: Cluster is now healthy 2023-12-11T17:44:48.146 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:47 smithi181 ceph-mon[103627]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2023-12-11T17:44:48.146 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:47 smithi181 ceph-mon[103627]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2023-12-11T17:44:48.146 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:47 smithi181 ceph-mon[103627]: daemon mds.h is now active in filesystem cephfs as rank 2 2023-12-11T17:44:48.146 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:47 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/208837848' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-12-11T17:44:48.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:47 smithi059 ceph-mon[98847]: pgmap v272: 129 pgs: 129 active+clean; 581 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2023-12-11T17:44:48.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:47 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/3500119668' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-12-11T17:44:48.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:47 smithi059 ceph-mon[98847]: mds.? [v2:172.21.15.181:6838/2214384806,v1:172.21.15.181:6839/2214384806] up:active 2023-12-11T17:44:48.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:47 smithi059 ceph-mon[98847]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-12-11T17:44:48.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:47 smithi059 ceph-mon[98847]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-12-11T17:44:48.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:47 smithi059 ceph-mon[98847]: Cluster is now healthy 2023-12-11T17:44:48.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:47 smithi059 ceph-mon[98847]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2023-12-11T17:44:48.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:47 smithi059 ceph-mon[98847]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2023-12-11T17:44:48.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:47 smithi059 ceph-mon[98847]: daemon mds.h is now active in filesystem cephfs as rank 2 2023-12-11T17:44:48.155 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:47 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/208837848' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-12-11T17:44:48.168 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:48.164+0000 7fa5a3427700 1 -- 172.21.15.59:0/3652101685 --> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}) v1 -- 0x7fa59c0682e0 con 0x7fa5880a1500 2023-12-11T17:44:48.213 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:48.209+0000 7fa5927fc700 1 -- 172.21.15.59:0/3652101685 <== mgr.14150 v2:172.21.15.59:6800/3627728353 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fa59c0682e0 con 0x7fa5880a1500 2023-12-11T17:44:48.215 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:48.211+0000 7fa5a3427700 1 -- 172.21.15.59:0/3652101685 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fa5880a1500 msgr2=0x7fa5880a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:48.215 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:48.211+0000 7fa5a3427700 1 --2- 172.21.15.59:0/3652101685 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fa5880a1500 0x7fa5880a39c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa59c1014d0 tx=0x7fa598007400 comp rx=0 tx=0).stop 2023-12-11T17:44:48.215 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:48.211+0000 7fa5a3427700 1 -- 172.21.15.59:0/3652101685 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa59c102010 msgr2=0x7fa59c19e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:48.216 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:48.211+0000 7fa5a3427700 1 --2- 172.21.15.59:0/3652101685 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa59c102010 0x7fa59c19e910 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fa594007bd0 tx=0x7fa594009440 comp rx=0 tx=0).stop 2023-12-11T17:44:48.216 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:48.211+0000 7fa5a3427700 1 -- 172.21.15.59:0/3652101685 shutdown_connections 2023-12-11T17:44:48.216 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:48.211+0000 7fa5a3427700 1 --2- 172.21.15.59:0/3652101685 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fa5880a1500 0x7fa5880a39c0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:48.216 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:48.211+0000 7fa5a3427700 1 --2- 172.21.15.59:0/3652101685 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa59c102010 0x7fa59c19e910 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:48.216 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:48.211+0000 7fa5a3427700 1 --2- 172.21.15.59:0/3652101685 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa59c100480 0x7fa59c069190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:48.216 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:48.211+0000 7fa5a3427700 1 --2- 172.21.15.59:0/3652101685 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa59c101670 0x7fa59c19c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:48.216 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:48.211+0000 7fa5a3427700 1 -- 172.21.15.59:0/3652101685 >> 172.21.15.59:0/3652101685 conn(0x7fa59c0fb9f0 msgr2=0x7fa59c104970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:44:48.216 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:48.212+0000 7fa5a3427700 1 -- 172.21.15.59:0/3652101685 shutdown_connections 2023-12-11T17:44:48.216 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:48.212+0000 7fa5a3427700 1 -- 172.21.15.59:0/3652101685 wait complete. 2023-12-11T17:44:48.227 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv_0 2023-12-11T17:44:48.357 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:48.353+0000 7f5271b39700 1 -- 172.21.15.59:0/428374083 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f526c100480 msgr2=0x7f526c1008a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:48.357 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:48.353+0000 7f5271b39700 1 --2- 172.21.15.59:0/428374083 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f526c100480 0x7f526c1008a0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f52600099c0 tx=0x7f5260009cd0 comp rx=0 tx=0).stop 2023-12-11T17:44:48.357 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:48.353+0000 7f5271b39700 1 -- 172.21.15.59:0/428374083 shutdown_connections 2023-12-11T17:44:48.358 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:48.353+0000 7f5271b39700 1 --2- 172.21.15.59:0/428374083 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f526c102010 0x7f526c1088c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:48.358 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:48.353+0000 7f5271b39700 1 --2- 172.21.15.59:0/428374083 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f526c100480 0x7f526c1008a0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:48.358 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:48.353+0000 7f5271b39700 1 --2- 172.21.15.59:0/428374083 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f526c101670 0x7f526c101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:48.358 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:48.353+0000 7f5271b39700 1 -- 172.21.15.59:0/428374083 >> 172.21.15.59:0/428374083 conn(0x7f526c0fb9f0 msgr2=0x7f526c0fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:44:48.358 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:48.353+0000 7f5271b39700 1 -- 172.21.15.59:0/428374083 shutdown_connections 2023-12-11T17:44:48.358 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:48.353+0000 7f5271b39700 1 -- 172.21.15.59:0/428374083 wait complete. 2023-12-11T17:44:48.358 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:48.354+0000 7f5271b39700 1 Processor -- start 2023-12-11T17:44:48.360 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:48.354+0000 7f5271b39700 1 -- start start 2023-12-11T17:44:48.360 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:48.355+0000 7f5271b39700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f526c100480 0x7f526c196420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:48.360 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:48.355+0000 7f5271b39700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f526c101670 0x7f526c196960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:48.360 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:48.355+0000 7f5271b39700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f526c102010 0x7f526c19d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:48.360 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:48.355+0000 7f5271b39700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f526c19df50 con 0x7f526c101670 2023-12-11T17:44:48.360 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:48.355+0000 7f5271b39700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f526c19e090 con 0x7f526c100480 2023-12-11T17:44:48.361 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:48.355+0000 7f5271b39700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f526c19e220 con 0x7f526c102010 2023-12-11T17:44:48.361 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:48.355+0000 7f526affd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f526c101670 0x7f526c196960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:48.361 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:48.355+0000 7f526affd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f526c101670 0x7f526c196960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33376/0 (socket says 172.21.15.59:33376) 2023-12-11T17:44:48.361 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:48.355+0000 7f526affd700 1 -- 172.21.15.59:0/2539072390 learned_addr learned my addr 172.21.15.59:0/2539072390 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:44:48.361 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:48.355+0000 7f526bfff700 1 --2- 172.21.15.59:0/2539072390 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f526c102010 0x7f526c19d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:48.361 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:48.355+0000 7f526b7fe700 1 --2- 172.21.15.59:0/2539072390 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f526c100480 0x7f526c196420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:48.361 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:48.355+0000 7f526affd700 1 -- 172.21.15.59:0/2539072390 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f526c102010 msgr2=0x7f526c19d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:48.361 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:48.355+0000 7f526affd700 1 --2- 172.21.15.59:0/2539072390 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f526c102010 0x7f526c19d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:48.361 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:48.355+0000 7f526affd700 1 -- 172.21.15.59:0/2539072390 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f526c100480 msgr2=0x7f526c196420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:48.361 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:48.355+0000 7f526affd700 1 --2- 172.21.15.59:0/2539072390 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f526c100480 0x7f526c196420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:48.361 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:48.355+0000 7f526affd700 1 -- 172.21.15.59:0/2539072390 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52600095f0 con 0x7f526c101670 2023-12-11T17:44:48.361 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:48.355+0000 7f526b7fe700 1 --2- 172.21.15.59:0/2539072390 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f526c100480 0x7f526c196420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-11T17:44:48.361 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:48.355+0000 7f526affd700 1 --2- 172.21.15.59:0/2539072390 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f526c101670 0x7f526c196960 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f525c00d980 tx=0x7f525c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:44:48.361 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:48.356+0000 7f5268ff9700 1 -- 172.21.15.59:0/2539072390 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f525c004220 con 0x7f526c101670 2023-12-11T17:44:48.362 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:48.356+0000 7f5268ff9700 1 -- 172.21.15.59:0/2539072390 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f525c007640 con 0x7f526c101670 2023-12-11T17:44:48.362 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:48.356+0000 7f5271b39700 1 -- 172.21.15.59:0/2539072390 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f526c19e500 con 0x7f526c101670 2023-12-11T17:44:48.362 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:48.356+0000 7f5268ff9700 1 -- 172.21.15.59:0/2539072390 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f525c005610 con 0x7f526c101670 2023-12-11T17:44:48.362 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:48.356+0000 7f5271b39700 1 -- 172.21.15.59:0/2539072390 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f526c19ea10 con 0x7f526c101670 2023-12-11T17:44:48.362 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:48.357+0000 7f5271b39700 1 -- 172.21.15.59:0/2539072390 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f526c107920 con 0x7f526c101670 2023-12-11T17:44:48.363 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:48.358+0000 7f5268ff9700 1 -- 172.21.15.59:0/2539072390 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f525c004750 con 0x7f526c101670 2023-12-11T17:44:48.363 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:48.358+0000 7f5268ff9700 1 --2- 172.21.15.59:0/2539072390 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f52540a1480 0x7f52540a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:48.363 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:48.358+0000 7f526b7fe700 1 --2- 172.21.15.59:0/2539072390 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f52540a1480 0x7f52540a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:48.363 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:48.358+0000 7f5268ff9700 1 -- 172.21.15.59:0/2539072390 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f525c0cd1e0 con 0x7f526c101670 2023-12-11T17:44:48.363 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:48.359+0000 7f526b7fe700 1 --2- 172.21.15.59:0/2539072390 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f52540a1480 0x7f52540a3940 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5260006010 tx=0x7f5260009530 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:44:48.365 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:48.362+0000 7f5268ff9700 1 -- 172.21.15.59:0/2539072390 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f525c0966f0 con 0x7f526c101670 2023-12-11T17:44:48.601 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:48.598+0000 7f5271b39700 1 -- 172.21.15.59:0/2539072390 --> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) v1 -- 0x7f526c061a10 con 0x7f52540a1480 2023-12-11T17:44:48.606 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:48.603+0000 7f5268ff9700 1 -- 172.21.15.59:0/2539072390 <== mgr.14150 v2:172.21.15.59:6800/3627728353 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+60 (secure 0 0 0) 0x7f526c061a10 con 0x7f52540a1480 2023-12-11T17:44:48.606 INFO:teuthology.orchestra.run.smithi059.stdout:/volumes/_nogroup/sv_0/7738f1b1-39d7-4e23-9988-6363623e5c2d 2023-12-11T17:44:48.608 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:48.605+0000 7f5271b39700 1 -- 172.21.15.59:0/2539072390 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f52540a1480 msgr2=0x7f52540a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:48.608 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:48.605+0000 7f5271b39700 1 --2- 172.21.15.59:0/2539072390 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f52540a1480 0x7f52540a3940 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5260006010 tx=0x7f5260009530 comp rx=0 tx=0).stop 2023-12-11T17:44:48.608 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:48.605+0000 7f5271b39700 1 -- 172.21.15.59:0/2539072390 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f526c101670 msgr2=0x7f526c196960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:48.609 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:48.605+0000 7f5271b39700 1 --2- 172.21.15.59:0/2539072390 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f526c101670 0x7f526c196960 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f525c00d980 tx=0x7f525c00dc90 comp rx=0 tx=0).stop 2023-12-11T17:44:48.609 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:48.605+0000 7f5271b39700 1 -- 172.21.15.59:0/2539072390 shutdown_connections 2023-12-11T17:44:48.609 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:48.605+0000 7f5271b39700 1 --2- 172.21.15.59:0/2539072390 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f52540a1480 0x7f52540a3940 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:48.609 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:48.605+0000 7f5271b39700 1 --2- 172.21.15.59:0/2539072390 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f526c102010 0x7f526c19d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:48.609 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:48.605+0000 7f5271b39700 1 --2- 172.21.15.59:0/2539072390 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f526c100480 0x7f526c196420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:48.609 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:48.605+0000 7f5271b39700 1 --2- 172.21.15.59:0/2539072390 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f526c101670 0x7f526c196960 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:48.609 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:48.605+0000 7f5271b39700 1 -- 172.21.15.59:0/2539072390 >> 172.21.15.59:0/2539072390 conn(0x7f526c0fb9f0 msgr2=0x7f526c104a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:44:48.609 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:48.605+0000 7f5271b39700 1 -- 172.21.15.59:0/2539072390 shutdown_connections 2023-12-11T17:44:48.609 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:48.605+0000 7f5271b39700 1 -- 172.21.15.59:0/2539072390 wait complete. 2023-12-11T17:44:48.620 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:44:48 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T17:44:48.170+0000 7fab23093700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-11T17:44:48.620 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:44:48 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T17:44:48.170+0000 7fab23093700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-11T17:44:48.620 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:44:48 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T17:44:48.170+0000 7fab23093700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-11T17:44:48.620 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:44:48 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T17:44:48.170+0000 7fab23093700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-11T17:44:48.620 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:44:48 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T17:44:48.170+0000 7fab23093700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-11T17:44:48.621 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 '' 2023-12-11T17:44:48.755 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:48.749+0000 7f538ac8c700 1 -- 172.21.15.59:0/2936447061 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5384104990 msgr2=0x7f5384106d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:48.755 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:48.749+0000 7f538ac8c700 1 --2- 172.21.15.59:0/2936447061 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5384104990 0x7f5384106d80 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f5370009990 tx=0x7f5370009ca0 comp rx=0 tx=0).stop 2023-12-11T17:44:48.755 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:48.749+0000 7f538ac8c700 1 -- 172.21.15.59:0/2936447061 shutdown_connections 2023-12-11T17:44:48.755 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:48.749+0000 7f538ac8c700 1 --2- 172.21.15.59:0/2936447061 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53841072c0 0x7f53841096d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:48.755 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:48.749+0000 7f538ac8c700 1 --2- 172.21.15.59:0/2936447061 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5384102060 0x7f5384104450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:48.755 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:48.749+0000 7f538ac8c700 1 --2- 172.21.15.59:0/2936447061 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5384104990 0x7f5384106d80 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:48.755 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:48.749+0000 7f538ac8c700 1 -- 172.21.15.59:0/2936447061 >> 172.21.15.59:0/2936447061 conn(0x7f53840fb9f0 msgr2=0x7f53840fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:44:48.755 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:48.749+0000 7f538ac8c700 1 -- 172.21.15.59:0/2936447061 shutdown_connections 2023-12-11T17:44:48.755 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:48.749+0000 7f538ac8c700 1 -- 172.21.15.59:0/2936447061 wait complete. 2023-12-11T17:44:48.755 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:48.750+0000 7f538ac8c700 1 Processor -- start 2023-12-11T17:44:48.755 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:48.750+0000 7f538ac8c700 1 -- start start 2023-12-11T17:44:48.756 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:48.750+0000 7f538ac8c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5384102060 0x7f53841974f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:48.756 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:48.750+0000 7f538ac8c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5384104990 0x7f5384197a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:48.756 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:48.750+0000 7f538ac8c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f53841072c0 0x7f538419ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:48.756 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:48.750+0000 7f538ac8c700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f538419f020 con 0x7f53841072c0 2023-12-11T17:44:48.756 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:48.750+0000 7f538ac8c700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f538419f160 con 0x7f5384102060 2023-12-11T17:44:48.756 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:48.750+0000 7f538ac8c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f538419f2f0 con 0x7f5384104990 2023-12-11T17:44:48.756 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:48.750+0000 7f5389229700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f53841072c0 0x7f538419ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:48.756 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:48.750+0000 7f5389229700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f53841072c0 0x7f538419ea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33392/0 (socket says 172.21.15.59:33392) 2023-12-11T17:44:48.756 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:48.750+0000 7f5389229700 1 -- 172.21.15.59:0/2318230440 learned_addr learned my addr 172.21.15.59:0/2318230440 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:44:48.756 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:48.750+0000 7f5388a28700 1 --2- 172.21.15.59:0/2318230440 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5384102060 0x7f53841974f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:48.756 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:48.750+0000 7f5383fff700 1 --2- 172.21.15.59:0/2318230440 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5384104990 0x7f5384197a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:48.756 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:48.750+0000 7f5389229700 1 -- 172.21.15.59:0/2318230440 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5384104990 msgr2=0x7f5384197a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:48.756 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:48.750+0000 7f5389229700 1 --2- 172.21.15.59:0/2318230440 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5384104990 0x7f5384197a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:48.756 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:48.750+0000 7f5389229700 1 -- 172.21.15.59:0/2318230440 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5384102060 msgr2=0x7f53841974f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:48.756 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:48.750+0000 7f5389229700 1 --2- 172.21.15.59:0/2318230440 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5384102060 0x7f53841974f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:48.757 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:48.751+0000 7f5389229700 1 -- 172.21.15.59:0/2318230440 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53700095f0 con 0x7f53841072c0 2023-12-11T17:44:48.757 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:48.751+0000 7f5389229700 1 --2- 172.21.15.59:0/2318230440 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f53841072c0 0x7f538419ea50 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f537c007ee0 tx=0x7f537c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:44:48.757 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:48.751+0000 7f5381ffb700 1 -- 172.21.15.59:0/2318230440 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f537c00dd20 con 0x7f53841072c0 2023-12-11T17:44:48.757 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:48.751+0000 7f5381ffb700 1 -- 172.21.15.59:0/2318230440 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f537c00f040 con 0x7f53841072c0 2023-12-11T17:44:48.757 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:48.751+0000 7f538ac8c700 1 -- 172.21.15.59:0/2318230440 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f538419f5d0 con 0x7f53841072c0 2023-12-11T17:44:48.757 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:48.751+0000 7f5381ffb700 1 -- 172.21.15.59:0/2318230440 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f537c0048a0 con 0x7f53841072c0 2023-12-11T17:44:48.757 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:48.751+0000 7f538ac8c700 1 -- 172.21.15.59:0/2318230440 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f538419fb30 con 0x7f53841072c0 2023-12-11T17:44:48.758 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:48.752+0000 7f538ac8c700 1 -- 172.21.15.59:0/2318230440 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53840fd570 con 0x7f53841072c0 2023-12-11T17:44:48.758 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:48.753+0000 7f5381ffb700 1 -- 172.21.15.59:0/2318230440 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f537c003a00 con 0x7f53841072c0 2023-12-11T17:44:48.758 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:48.754+0000 7f5381ffb700 1 --2- 172.21.15.59:0/2318230440 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f53740a1500 0x7f53740a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:48.759 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:48.754+0000 7f5381ffb700 1 -- 172.21.15.59:0/2318230440 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f537c0cc6c0 con 0x7f53841072c0 2023-12-11T17:44:48.759 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:48.754+0000 7f5388a28700 1 --2- 172.21.15.59:0/2318230440 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f53740a1500 0x7f53740a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:48.759 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:48.754+0000 7f5388a28700 1 --2- 172.21.15.59:0/2318230440 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f53740a1500 0x7f53740a39c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f5378009c60 tx=0x7f5378009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:44:48.761 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:48.758+0000 7f5381ffb700 1 -- 172.21.15.59:0/2318230440 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f537c04f2f0 con 0x7f53841072c0 2023-12-11T17:44:48.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:48 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/208837848' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-12-11T17:44:48.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:48 smithi059 ceph-mon[98847]: mds.? [v2:172.21.15.115:6836/1209365460,v1:172.21.15.115:6837/1209365460] up:active 2023-12-11T17:44:48.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:48 smithi059 ceph-mon[98847]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2023-12-11T17:44:48.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:48 smithi059 ceph-mon[98847]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-12-11T17:44:48.997 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:48.993+0000 7f538ac8c700 1 -- 172.21.15.59:0/2318230440 --> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}) v1 -- 0x7f53840682e0 con 0x7f53740a1500 2023-12-11T17:44:49.020 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.017+0000 7f5381ffb700 1 -- 172.21.15.59:0/2318230440 <== mgr.14150 v2:172.21.15.59:6800/3627728353 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f53840682e0 con 0x7f53740a1500 2023-12-11T17:44:49.022 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.019+0000 7f538ac8c700 1 -- 172.21.15.59:0/2318230440 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f53740a1500 msgr2=0x7f53740a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:49.023 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.019+0000 7f538ac8c700 1 --2- 172.21.15.59:0/2318230440 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f53740a1500 0x7f53740a39c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f5378009c60 tx=0x7f5378009260 comp rx=0 tx=0).stop 2023-12-11T17:44:49.023 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.019+0000 7f538ac8c700 1 -- 172.21.15.59:0/2318230440 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f53841072c0 msgr2=0x7f538419ea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:49.023 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.019+0000 7f538ac8c700 1 --2- 172.21.15.59:0/2318230440 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f53841072c0 0x7f538419ea50 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f537c007ee0 tx=0x7f537c00d3b0 comp rx=0 tx=0).stop 2023-12-11T17:44:49.023 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.019+0000 7f538ac8c700 1 -- 172.21.15.59:0/2318230440 shutdown_connections 2023-12-11T17:44:49.023 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.019+0000 7f538ac8c700 1 --2- 172.21.15.59:0/2318230440 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f53740a1500 0x7f53740a39c0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:49.023 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.019+0000 7f538ac8c700 1 --2- 172.21.15.59:0/2318230440 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f53841072c0 0x7f538419ea50 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:49.023 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.019+0000 7f538ac8c700 1 --2- 172.21.15.59:0/2318230440 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5384102060 0x7f53841974f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:49.023 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.019+0000 7f538ac8c700 1 --2- 172.21.15.59:0/2318230440 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5384104990 0x7f5384197a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:49.023 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.019+0000 7f538ac8c700 1 -- 172.21.15.59:0/2318230440 >> 172.21.15.59:0/2318230440 conn(0x7f53840fb9f0 msgr2=0x7f5384102aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:44:49.023 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.019+0000 7f538ac8c700 1 -- 172.21.15.59:0/2318230440 shutdown_connections 2023-12-11T17:44:49.023 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.019+0000 7f538ac8c700 1 -- 172.21.15.59:0/2318230440 wait complete. 2023-12-11T17:44:49.032 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:48 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/208837848' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-12-11T17:44:49.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:48 smithi115 ceph-mon[102748]: mds.? [v2:172.21.15.115:6836/1209365460,v1:172.21.15.115:6837/1209365460] up:active 2023-12-11T17:44:49.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:48 smithi115 ceph-mon[102748]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2023-12-11T17:44:49.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:48 smithi115 ceph-mon[102748]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-12-11T17:44:49.035 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv_1 2023-12-11T17:44:49.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:48 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/208837848' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-12-11T17:44:49.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:48 smithi181 ceph-mon[103627]: mds.? [v2:172.21.15.115:6836/1209365460,v1:172.21.15.115:6837/1209365460] up:active 2023-12-11T17:44:49.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:48 smithi181 ceph-mon[103627]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2023-12-11T17:44:49.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:48 smithi181 ceph-mon[103627]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-12-11T17:44:49.164 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.160+0000 7f33e4b18700 1 -- 172.21.15.59:0/2153186853 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f33e0105340 msgr2=0x7f33e010bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:49.164 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.160+0000 7f33e4b18700 1 --2- 172.21.15.59:0/2153186853 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f33e0105340 0x7f33e010bbf0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f33d800b0d0 tx=0x7f33d800b3e0 comp rx=0 tx=0).stop 2023-12-11T17:44:49.164 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.160+0000 7f33e4b18700 1 -- 172.21.15.59:0/2153186853 shutdown_connections 2023-12-11T17:44:49.164 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.160+0000 7f33e4b18700 1 --2- 172.21.15.59:0/2153186853 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f33e0105340 0x7f33e010bbf0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:49.164 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.160+0000 7f33e4b18700 1 --2- 172.21.15.59:0/2153186853 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33e01037b0 0x7f33e0103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:49.165 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.160+0000 7f33e4b18700 1 --2- 172.21.15.59:0/2153186853 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33e01049a0 0x7f33e0104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:49.165 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.160+0000 7f33e4b18700 1 -- 172.21.15.59:0/2153186853 >> 172.21.15.59:0/2153186853 conn(0x7f33e00fed40 msgr2=0x7f33e0101160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:44:49.165 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.160+0000 7f33e4b18700 1 -- 172.21.15.59:0/2153186853 shutdown_connections 2023-12-11T17:44:49.165 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.160+0000 7f33e4b18700 1 -- 172.21.15.59:0/2153186853 wait complete. 2023-12-11T17:44:49.167 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.161+0000 7f33e4b18700 1 Processor -- start 2023-12-11T17:44:49.167 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.161+0000 7f33e4b18700 1 -- start start 2023-12-11T17:44:49.167 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.161+0000 7f33e4b18700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f33e01037b0 0x7f33e0199820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:49.167 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.161+0000 7f33e4b18700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33e01049a0 0x7f33e0199d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:49.167 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.161+0000 7f33e4b18700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33e0105340 0x7f33e01a0d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:49.167 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.161+0000 7f33e4b18700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33e01a1350 con 0x7f33e01037b0 2023-12-11T17:44:49.167 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.161+0000 7f33e4b18700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33e01a1490 con 0x7f33e0105340 2023-12-11T17:44:49.167 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.161+0000 7f33e4b18700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33e01a1620 con 0x7f33e01049a0 2023-12-11T17:44:49.167 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.161+0000 7f33de59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f33e01037b0 0x7f33e0199820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:49.168 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.161+0000 7f33de59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f33e01037b0 0x7f33e0199820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33404/0 (socket says 172.21.15.59:33404) 2023-12-11T17:44:49.168 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.161+0000 7f33de59c700 1 -- 172.21.15.59:0/1685245018 learned_addr learned my addr 172.21.15.59:0/1685245018 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:44:49.168 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.162+0000 7f33ddd9b700 1 --2- 172.21.15.59:0/1685245018 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33e01049a0 0x7f33e0199d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:49.168 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.162+0000 7f33ded9d700 1 --2- 172.21.15.59:0/1685245018 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33e0105340 0x7f33e01a0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:49.168 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.162+0000 7f33de59c700 1 -- 172.21.15.59:0/1685245018 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33e01049a0 msgr2=0x7f33e0199d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:49.168 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.162+0000 7f33de59c700 1 --2- 172.21.15.59:0/1685245018 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33e01049a0 0x7f33e0199d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:49.168 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.162+0000 7f33de59c700 1 -- 172.21.15.59:0/1685245018 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33e0105340 msgr2=0x7f33e01a0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:49.168 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.162+0000 7f33de59c700 1 --2- 172.21.15.59:0/1685245018 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33e0105340 0x7f33e01a0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:49.168 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.162+0000 7f33de59c700 1 -- 172.21.15.59:0/1685245018 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33d8009d00 con 0x7f33e01037b0 2023-12-11T17:44:49.168 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.162+0000 7f33de59c700 1 --2- 172.21.15.59:0/1685245018 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f33e01037b0 0x7f33e0199820 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f33d400d980 tx=0x7f33d400dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:44:49.168 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.162+0000 7f33cf7fe700 1 -- 172.21.15.59:0/1685245018 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33d400bb60 con 0x7f33e01037b0 2023-12-11T17:44:49.168 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.162+0000 7f33cf7fe700 1 -- 172.21.15.59:0/1685245018 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f33d4007620 con 0x7f33e01037b0 2023-12-11T17:44:49.168 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.162+0000 7f33e4b18700 1 -- 172.21.15.59:0/1685245018 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33e01a1900 con 0x7f33e01037b0 2023-12-11T17:44:49.168 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.162+0000 7f33cf7fe700 1 -- 172.21.15.59:0/1685245018 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33d400f620 con 0x7f33e01037b0 2023-12-11T17:44:49.169 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.162+0000 7f33e4b18700 1 -- 172.21.15.59:0/1685245018 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33e01a1e10 con 0x7f33e01037b0 2023-12-11T17:44:49.169 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.163+0000 7f33e4b18700 1 -- 172.21.15.59:0/1685245018 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33e010abe0 con 0x7f33e01037b0 2023-12-11T17:44:49.170 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.164+0000 7f33cf7fe700 1 -- 172.21.15.59:0/1685245018 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f33d4010460 con 0x7f33e01037b0 2023-12-11T17:44:49.170 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.165+0000 7f33cf7fe700 1 --2- 172.21.15.59:0/1685245018 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f33c80a1480 0x7f33c80a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:49.170 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.165+0000 7f33ddd9b700 1 --2- 172.21.15.59:0/1685245018 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f33c80a1480 0x7f33c80a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:49.170 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.165+0000 7f33cf7fe700 1 -- 172.21.15.59:0/1685245018 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f33d40ccf70 con 0x7f33e01037b0 2023-12-11T17:44:49.170 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.165+0000 7f33ddd9b700 1 --2- 172.21.15.59:0/1685245018 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f33c80a1480 0x7f33c80a3940 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f33d000dad0 tx=0x7f33d000d4b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:44:49.172 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.169+0000 7f33cf7fe700 1 -- 172.21.15.59:0/1685245018 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f33d4096380 con 0x7f33e01037b0 2023-12-11T17:44:49.408 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.405+0000 7f33e4b18700 1 -- 172.21.15.59:0/1685245018 --> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f33e002d380 con 0x7f33c80a1480 2023-12-11T17:44:49.413 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.409+0000 7f33cf7fe700 1 -- 172.21.15.59:0/1685245018 <== mgr.14150 v2:172.21.15.59:6800/3627728353 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+60 (secure 0 0 0) 0x7f33e002d380 con 0x7f33c80a1480 2023-12-11T17:44:49.413 INFO:teuthology.orchestra.run.smithi059.stdout:/volumes/_nogroup/sv_1/b5d81e8c-41e7-4beb-9cc4-c365f0e4a5a8 2023-12-11T17:44:49.415 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.412+0000 7f33e4b18700 1 -- 172.21.15.59:0/1685245018 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f33c80a1480 msgr2=0x7f33c80a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:49.415 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.412+0000 7f33e4b18700 1 --2- 172.21.15.59:0/1685245018 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f33c80a1480 0x7f33c80a3940 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f33d000dad0 tx=0x7f33d000d4b0 comp rx=0 tx=0).stop 2023-12-11T17:44:49.416 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.412+0000 7f33e4b18700 1 -- 172.21.15.59:0/1685245018 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f33e01037b0 msgr2=0x7f33e0199820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:49.416 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.412+0000 7f33e4b18700 1 --2- 172.21.15.59:0/1685245018 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f33e01037b0 0x7f33e0199820 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f33d400d980 tx=0x7f33d400dd40 comp rx=0 tx=0).stop 2023-12-11T17:44:49.416 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.412+0000 7f33e4b18700 1 -- 172.21.15.59:0/1685245018 shutdown_connections 2023-12-11T17:44:49.416 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.412+0000 7f33e4b18700 1 --2- 172.21.15.59:0/1685245018 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f33c80a1480 0x7f33c80a3940 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:49.416 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.412+0000 7f33e4b18700 1 --2- 172.21.15.59:0/1685245018 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33e0105340 0x7f33e01a0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:49.416 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.412+0000 7f33e4b18700 1 --2- 172.21.15.59:0/1685245018 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f33e01037b0 0x7f33e0199820 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:49.416 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.412+0000 7f33e4b18700 1 --2- 172.21.15.59:0/1685245018 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33e01049a0 0x7f33e0199d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:49.416 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.412+0000 7f33e4b18700 1 -- 172.21.15.59:0/1685245018 >> 172.21.15.59:0/1685245018 conn(0x7f33e00fed40 msgr2=0x7f33e0107d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:44:49.416 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.412+0000 7f33e4b18700 1 -- 172.21.15.59:0/1685245018 shutdown_connections 2023-12-11T17:44:49.416 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.412+0000 7f33e4b18700 1 -- 172.21.15.59:0/1685245018 wait complete. 2023-12-11T17:44:49.428 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-11T17:44:49.556 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.550+0000 7fe4f4379700 1 -- 172.21.15.59:0/2410415817 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4ec107180 msgr2=0x7fe4ec1095f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:49.556 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.550+0000 7fe4f4379700 1 --2- 172.21.15.59:0/2410415817 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4ec107180 0x7fe4ec1095f0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fe4e400b0a0 tx=0x7fe4e400b3b0 comp rx=0 tx=0).stop 2023-12-11T17:44:49.557 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.550+0000 7fe4f4379700 1 -- 172.21.15.59:0/2410415817 shutdown_connections 2023-12-11T17:44:49.557 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.550+0000 7fe4f4379700 1 --2- 172.21.15.59:0/2410415817 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4ec107180 0x7fe4ec1095f0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:49.557 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.550+0000 7fe4f4379700 1 --2- 172.21.15.59:0/2410415817 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe4ec101f20 0x7fe4ec104310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:49.557 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.550+0000 7fe4f4379700 1 --2- 172.21.15.59:0/2410415817 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4ec104850 0x7fe4ec106c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:49.557 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.550+0000 7fe4f4379700 1 -- 172.21.15.59:0/2410415817 >> 172.21.15.59:0/2410415817 conn(0x7fe4ec0fb910 msgr2=0x7fe4ec0fdd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:44:49.557 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.550+0000 7fe4f4379700 1 -- 172.21.15.59:0/2410415817 shutdown_connections 2023-12-11T17:44:49.557 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.550+0000 7fe4f4379700 1 -- 172.21.15.59:0/2410415817 wait complete. 2023-12-11T17:44:49.558 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.551+0000 7fe4f4379700 1 Processor -- start 2023-12-11T17:44:49.558 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.551+0000 7fe4f4379700 1 -- start start 2023-12-11T17:44:49.558 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.551+0000 7fe4f4379700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe4ec101f20 0x7fe4ec1996e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:49.558 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.551+0000 7fe4f4379700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4ec104850 0x7fe4ec199c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:49.558 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.551+0000 7fe4f4379700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4ec107180 0x7fe4ec1a0c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:49.558 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.551+0000 7fe4f4379700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4ec1a1210 con 0x7fe4ec104850 2023-12-11T17:44:49.558 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.551+0000 7fe4f4379700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4ec1a1350 con 0x7fe4ec101f20 2023-12-11T17:44:49.558 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.551+0000 7fe4f4379700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4ec1a14e0 con 0x7fe4ec107180 2023-12-11T17:44:49.558 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.551+0000 7fe4f1914700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4ec104850 0x7fe4ec199c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:49.558 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.551+0000 7fe4f1914700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4ec104850 0x7fe4ec199c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33420/0 (socket says 172.21.15.59:33420) 2023-12-11T17:44:49.558 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.551+0000 7fe4f1914700 1 -- 172.21.15.59:0/1062268508 learned_addr learned my addr 172.21.15.59:0/1062268508 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:44:49.558 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.551+0000 7fe4f2115700 1 --2- 172.21.15.59:0/1062268508 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe4ec101f20 0x7fe4ec1996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:49.558 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.551+0000 7fe4f2916700 1 --2- 172.21.15.59:0/1062268508 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4ec107180 0x7fe4ec1a0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:49.559 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.551+0000 7fe4f1914700 1 -- 172.21.15.59:0/1062268508 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4ec107180 msgr2=0x7fe4ec1a0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:49.559 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.551+0000 7fe4f1914700 1 --2- 172.21.15.59:0/1062268508 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4ec107180 0x7fe4ec1a0c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:49.559 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.551+0000 7fe4f1914700 1 -- 172.21.15.59:0/1062268508 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe4ec101f20 msgr2=0x7fe4ec1996e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:49.559 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.551+0000 7fe4f1914700 1 --2- 172.21.15.59:0/1062268508 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe4ec101f20 0x7fe4ec1996e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:49.559 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.551+0000 7fe4f1914700 1 -- 172.21.15.59:0/1062268508 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4e4009d00 con 0x7fe4ec104850 2023-12-11T17:44:49.559 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.552+0000 7fe4f1914700 1 --2- 172.21.15.59:0/1062268508 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4ec104850 0x7fe4ec199c20 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fe4e800d950 tx=0x7fe4e800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:44:49.559 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.552+0000 7fe4e37fe700 1 -- 172.21.15.59:0/1062268508 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e800f890 con 0x7fe4ec104850 2023-12-11T17:44:49.559 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.552+0000 7fe4e37fe700 1 -- 172.21.15.59:0/1062268508 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe4e8010460 con 0x7fe4ec104850 2023-12-11T17:44:49.559 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.552+0000 7fe4f4379700 1 -- 172.21.15.59:0/1062268508 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4ec1a17c0 con 0x7fe4ec104850 2023-12-11T17:44:49.560 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.552+0000 7fe4e37fe700 1 -- 172.21.15.59:0/1062268508 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e800e610 con 0x7fe4ec104850 2023-12-11T17:44:49.560 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.552+0000 7fe4f4379700 1 -- 172.21.15.59:0/1062268508 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4ec1a1d20 con 0x7fe4ec104850 2023-12-11T17:44:49.560 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.553+0000 7fe4f4379700 1 -- 172.21.15.59:0/1062268508 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4ec108d00 con 0x7fe4ec104850 2023-12-11T17:44:49.561 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.554+0000 7fe4e37fe700 1 -- 172.21.15.59:0/1062268508 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7fe4e80108d0 con 0x7fe4ec104850 2023-12-11T17:44:49.561 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.555+0000 7fe4e37fe700 1 --2- 172.21.15.59:0/1062268508 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fe4d80a1500 0x7fe4d80a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:49.561 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.555+0000 7fe4f2115700 1 --2- 172.21.15.59:0/1062268508 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fe4d80a1500 0x7fe4d80a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:49.561 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.555+0000 7fe4e37fe700 1 -- 172.21.15.59:0/1062268508 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fe4e80ccc40 con 0x7fe4ec104850 2023-12-11T17:44:49.561 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.555+0000 7fe4f2115700 1 --2- 172.21.15.59:0/1062268508 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fe4d80a1500 0x7fe4d80a39c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc005950 tx=0x7fe4dc00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:44:49.562 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.558+0000 7fe4e37fe700 1 -- 172.21.15.59:0/1062268508 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fe4e80960b0 con 0x7fe4ec104850 2023-12-11T17:44:49.814 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.808+0000 7fe4f4379700 1 -- 172.21.15.59:0/1062268508 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe4ec0679a0 con 0x7fe4ec104850 2023-12-11T17:44:49.814 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.808+0000 7fe4e37fe700 1 -- 172.21.15.59:0/1062268508 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+11325 (secure 0 0 0) 0x7fe4e8020090 con 0x7fe4ec104850 2023-12-11T17:44:49.814 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-12-11T17:44:49.814 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/2184151593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":2184151593},{"type":"v1","addr":"172.21.15.59:6835","nonce":2184151593}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/2817614886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2817614886},{"type":"v1","addr":"172.21.15.59:6837","nonce":2817614886}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6839/229873490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":229873490},{"type":"v1","addr":"172.21.15.59:6839","nonce":229873490}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6841/967998755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":967998755},{"type":"v1","addr":"172.21.15.59:6841","nonce":967998755}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/395684042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":395684042},{"type":"v1","addr":"172.21.15.181:6833","nonce":395684042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6833/3214606191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3214606191},{"type":"v1","addr":"172.21.15.115:6833","nonce":3214606191}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-12-11T17:44:20.076660+0000","modified":"2023-12-11T17:44:47.770695+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24521,"mds_1":24484,"mds_2":24482},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/1874159657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1874159657},{"type":"v1","addr":"172.21.15.115:6835","nonce":1874159657}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24457":{"gid":24457,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/4221183620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":4221183620},{"type":"v1","addr":"172.21.15.181:6835","nonce":4221183620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24466":{"gid":24466,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6837/3580309407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":3580309407},{"type":"v1","addr":"172.21.15.181:6837","nonce":3580309407}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24482":{"gid":24482,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.115:6837/1209365460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":1209365460},{"type":"v1","addr":"172.21.15.115:6837","nonce":1209365460}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.181:6839/2214384806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":2214384806},{"type":"v1","addr":"172.21.15.181:6839","nonce":2214384806}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.115:6839/4122922866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":4122922866},{"type":"v1","addr":"172.21.15.115:6839","nonce":4122922866}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-11T17:44:49.815 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.810+0000 7fe4f4379700 1 -- 172.21.15.59:0/1062268508 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fe4d80a1500 msgr2=0x7fe4d80a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:49.815 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.810+0000 7fe4f4379700 1 --2- 172.21.15.59:0/1062268508 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fe4d80a1500 0x7fe4d80a39c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc005950 tx=0x7fe4dc00b410 comp rx=0 tx=0).stop 2023-12-11T17:44:49.815 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.810+0000 7fe4f4379700 1 -- 172.21.15.59:0/1062268508 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4ec104850 msgr2=0x7fe4ec199c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:49.816 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.810+0000 7fe4f4379700 1 --2- 172.21.15.59:0/1062268508 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4ec104850 0x7fe4ec199c20 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fe4e800d950 tx=0x7fe4e800dc60 comp rx=0 tx=0).stop 2023-12-11T17:44:49.816 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.810+0000 7fe4f4379700 1 -- 172.21.15.59:0/1062268508 shutdown_connections 2023-12-11T17:44:49.816 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.810+0000 7fe4f4379700 1 --2- 172.21.15.59:0/1062268508 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fe4d80a1500 0x7fe4d80a39c0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:49.816 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.810+0000 7fe4f4379700 1 --2- 172.21.15.59:0/1062268508 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4ec107180 0x7fe4ec1a0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:49.816 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.810+0000 7fe4f4379700 1 --2- 172.21.15.59:0/1062268508 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe4ec101f20 0x7fe4ec1996e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:49.816 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.810+0000 7fe4f4379700 1 --2- 172.21.15.59:0/1062268508 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4ec104850 0x7fe4ec199c20 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:49.816 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.811+0000 7fe4f4379700 1 -- 172.21.15.59:0/1062268508 >> 172.21.15.59:0/1062268508 conn(0x7fe4ec0fb910 msgr2=0x7fe4ec101620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:44:49.816 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.811+0000 7fe4f4379700 1 -- 172.21.15.59:0/1062268508 shutdown_connections 2023-12-11T17:44:49.816 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.811+0000 7fe4f4379700 1 -- 172.21.15.59:0/1062268508 wait complete. 2023-12-11T17:44:49.816 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 24 2023-12-11T17:44:49.827 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-11T17:44:49.952 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.949+0000 7f6e41106700 1 -- 172.21.15.59:0/2607024003 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6e3c100380 msgr2=0x7f6e3c1007a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:49.953 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.949+0000 7f6e41106700 1 --2- 172.21.15.59:0/2607024003 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6e3c100380 0x7f6e3c1007a0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f6e2c0099c0 tx=0x7f6e2c009cd0 comp rx=0 tx=0).stop 2023-12-11T17:44:49.953 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.949+0000 7f6e41106700 1 -- 172.21.15.59:0/2607024003 shutdown_connections 2023-12-11T17:44:49.953 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.949+0000 7f6e41106700 1 --2- 172.21.15.59:0/2607024003 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6e3c101f10 0x7f6e3c1087c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:49.953 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.949+0000 7f6e41106700 1 --2- 172.21.15.59:0/2607024003 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6e3c100380 0x7f6e3c1007a0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:49.953 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.949+0000 7f6e41106700 1 --2- 172.21.15.59:0/2607024003 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6e3c101570 0x7f6e3c1019d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:49.953 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.949+0000 7f6e41106700 1 -- 172.21.15.59:0/2607024003 >> 172.21.15.59:0/2607024003 conn(0x7f6e3c0fb8f0 msgr2=0x7f6e3c0fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:44:49.953 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.949+0000 7f6e41106700 1 -- 172.21.15.59:0/2607024003 shutdown_connections 2023-12-11T17:44:49.953 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.949+0000 7f6e41106700 1 -- 172.21.15.59:0/2607024003 wait complete. 2023-12-11T17:44:49.955 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.949+0000 7f6e41106700 1 Processor -- start 2023-12-11T17:44:49.955 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.950+0000 7f6e41106700 1 -- start start 2023-12-11T17:44:49.955 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.950+0000 7f6e41106700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6e3c100380 0x7f6e3c069190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:49.955 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.950+0000 7f6e41106700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6e3c101570 0x7f6e3c19c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:49.955 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.950+0000 7f6e41106700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6e3c101f10 0x7f6e3c19e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:49.955 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.950+0000 7f6e41106700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e3c19ee70 con 0x7f6e3c101570 2023-12-11T17:44:49.956 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.950+0000 7f6e41106700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e3c19efe0 con 0x7f6e3c101f10 2023-12-11T17:44:49.956 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.950+0000 7f6e41106700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e3c19f1a0 con 0x7f6e3c100380 2023-12-11T17:44:49.956 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.950+0000 7f6e3a59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6e3c101570 0x7f6e3c19c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:49.956 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.950+0000 7f6e3a59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6e3c101570 0x7f6e3c19c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33434/0 (socket says 172.21.15.59:33434) 2023-12-11T17:44:49.956 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.950+0000 7f6e3a59c700 1 -- 172.21.15.59:0/2579014155 learned_addr learned my addr 172.21.15.59:0/2579014155 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:44:49.956 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.950+0000 7f6e3b59e700 1 --2- 172.21.15.59:0/2579014155 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6e3c101f10 0x7f6e3c19e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:49.956 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.950+0000 7f6e3ad9d700 1 --2- 172.21.15.59:0/2579014155 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6e3c100380 0x7f6e3c069190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:49.956 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.950+0000 7f6e3a59c700 1 -- 172.21.15.59:0/2579014155 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6e3c100380 msgr2=0x7f6e3c069190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:49.956 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.950+0000 7f6e3a59c700 1 --2- 172.21.15.59:0/2579014155 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6e3c100380 0x7f6e3c069190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:49.956 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.950+0000 7f6e3a59c700 1 -- 172.21.15.59:0/2579014155 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6e3c101f10 msgr2=0x7f6e3c19e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:49.956 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.950+0000 7f6e3a59c700 1 --2- 172.21.15.59:0/2579014155 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6e3c101f10 0x7f6e3c19e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:49.956 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.950+0000 7f6e3a59c700 1 -- 172.21.15.59:0/2579014155 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e2c0095f0 con 0x7f6e3c101570 2023-12-11T17:44:49.956 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.951+0000 7f6e3a59c700 1 --2- 172.21.15.59:0/2579014155 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6e3c101570 0x7f6e3c19c080 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f6e2400d950 tx=0x7f6e2400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:44:49.956 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.951+0000 7f6e33fff700 1 -- 172.21.15.59:0/2579014155 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e2400f890 con 0x7f6e3c101570 2023-12-11T17:44:49.956 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.951+0000 7f6e41106700 1 -- 172.21.15.59:0/2579014155 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e3c19f480 con 0x7f6e3c101570 2023-12-11T17:44:49.957 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.951+0000 7f6e33fff700 1 -- 172.21.15.59:0/2579014155 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6e24010460 con 0x7f6e3c101570 2023-12-11T17:44:49.957 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.951+0000 7f6e33fff700 1 -- 172.21.15.59:0/2579014155 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e2400e610 con 0x7f6e3c101570 2023-12-11T17:44:49.957 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.951+0000 7f6e41106700 1 -- 172.21.15.59:0/2579014155 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e3c19f990 con 0x7f6e3c101570 2023-12-11T17:44:49.957 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.952+0000 7f6e41106700 1 -- 172.21.15.59:0/2579014155 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e3c1077b0 con 0x7f6e3c101570 2023-12-11T17:44:49.957 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.953+0000 7f6e33fff700 1 -- 172.21.15.59:0/2579014155 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f6e240108d0 con 0x7f6e3c101570 2023-12-11T17:44:49.957 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.953+0000 7f6e33fff700 1 --2- 172.21.15.59:0/2579014155 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f6e280a1480 0x7f6e280a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:49.958 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.954+0000 7f6e3ad9d700 1 --2- 172.21.15.59:0/2579014155 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f6e280a1480 0x7f6e280a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:49.958 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.954+0000 7f6e33fff700 1 -- 172.21.15.59:0/2579014155 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f6e240ccc80 con 0x7f6e3c101570 2023-12-11T17:44:49.958 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.954+0000 7f6e3ad9d700 1 --2- 172.21.15.59:0/2579014155 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f6e280a1480 0x7f6e280a3940 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f6e2c003820 tx=0x7f6e2c005920 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:44:49.961 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:49.958+0000 7f6e33fff700 1 -- 172.21.15.59:0/2579014155 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f6e240960f0 con 0x7f6e3c101570 2023-12-11T17:44:50.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:49 smithi115 ceph-mon[102748]: pgmap v273: 129 pgs: 129 active+clean; 582 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 3 op/s 2023-12-11T17:44:50.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:49 smithi115 ceph-mon[102748]: from='client.14847 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T17:44:50.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:49 smithi115 ceph-mon[102748]: from='client.14859 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T17:44:50.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:49 smithi115 ceph-mon[102748]: from='client.14865 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T17:44:50.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:49 smithi181 ceph-mon[103627]: pgmap v273: 129 pgs: 129 active+clean; 582 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 3 op/s 2023-12-11T17:44:50.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:49 smithi181 ceph-mon[103627]: from='client.14847 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T17:44:50.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:49 smithi181 ceph-mon[103627]: from='client.14859 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T17:44:50.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:49 smithi181 ceph-mon[103627]: from='client.14865 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T17:44:50.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:49 smithi059 ceph-mon[98847]: pgmap v273: 129 pgs: 129 active+clean; 582 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 3 op/s 2023-12-11T17:44:50.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:49 smithi059 ceph-mon[98847]: from='client.14847 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T17:44:50.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:49 smithi059 ceph-mon[98847]: from='client.14859 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T17:44:50.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:49 smithi059 ceph-mon[98847]: from='client.14865 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T17:44:50.164 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:50.161+0000 7f6e41106700 1 -- 172.21.15.59:0/2579014155 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6e3c02d310 con 0x7f6e3c101570 2023-12-11T17:44:50.166 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:50.162+0000 7f6e33fff700 1 -- 172.21.15.59:0/2579014155 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+23960 (secure 0 0 0) 0x7f6e24016090 con 0x7f6e3c101570 2023-12-11T17:44:50.166 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-12-11T17:44:50.166 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":86,"fsid":"d0a07eaa-984b-11ee-95a3-87774f69a715","created":"2023-12-11T17:37:14.526539+0000","modified":"2023-12-11T17:44:25.096094+0000","last_up_change":"2023-12-11T17:43:38.535503+0000","last_in_change":"2023-12-11T17:43:21.014888+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-11T17:40:26.596029+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-11T17:44:18.586290+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-11T17:44:19.432952+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-11T17:44:23.097308+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3c0b3218-4b2b-40f8-8bef-ed8cd5633607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":1040759398},{"type":"v1","addr":"172.21.15.59:6803","nonce":1040759398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":1040759398},{"type":"v1","addr":"172.21.15.59:6805","nonce":1040759398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":1040759398},{"type":"v1","addr":"172.21.15.59:6809","nonce":1040759398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":1040759398},{"type":"v1","addr":"172.21.15.59:6807","nonce":1040759398}]},"public_addr":"172.21.15.59:6803/1040759398","cluster_addr":"172.21.15.59:6805/1040759398","heartbeat_back_addr":"172.21.15.59:6809/1040759398","heartbeat_front_addr":"172.21.15.59:6807/1040759398","state":["exists","up"]},{"osd":1,"uuid":"2ead7578-765b-4f5a-a72c-863731ed69e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":2776354245},{"type":"v1","addr":"172.21.15.115:6801","nonce":2776354245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":2776354245},{"type":"v1","addr":"172.21.15.115:6803","nonce":2776354245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":2776354245},{"type":"v1","addr":"172.21.15.115:6807","nonce":2776354245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":2776354245},{"type":"v1","addr":"172.21.15.115:6805","nonce":2776354245}]},"public_addr":"172.21.15.115:6801/2776354245","cluster_addr":"172.21.15.115:6803/2776354245","heartbeat_back_addr":"172.21.15.115:6807/2776354245","heartbeat_front_addr":"172.21.15.115:6805/2776354245","state":["exists","up"]},{"osd":2,"uuid":"8cb1d7ec-53cc-4672-b0b6-70d73797c44f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4287675481},{"type":"v1","addr":"172.21.15.181:6801","nonce":4287675481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4287675481},{"type":"v1","addr":"172.21.15.181:6803","nonce":4287675481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4287675481},{"type":"v1","addr":"172.21.15.181:6807","nonce":4287675481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4287675481},{"type":"v1","addr":"172.21.15.181:6805","nonce":4287675481}]},"public_addr":"172.21.15.181:6801/4287675481","cluster_addr":"172.21.15.181:6803/4287675481","heartbeat_back_addr":"172.21.15.181:6807/4287675481","heartbeat_front_addr":"172.21.15.181:6805/4287675481","state":["exists","up"]},{"osd":3,"uuid":"5d64d9af-831b-4d0a-8ace-a452169351d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":219909995},{"type":"v1","addr":"172.21.15.59:6811","nonce":219909995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":219909995},{"type":"v1","addr":"172.21.15.59:6813","nonce":219909995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":219909995},{"type":"v1","addr":"172.21.15.59:6817","nonce":219909995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":219909995},{"type":"v1","addr":"172.21.15.59:6815","nonce":219909995}]},"public_addr":"172.21.15.59:6811/219909995","cluster_addr":"172.21.15.59:6813/219909995","heartbeat_back_addr":"172.21.15.59:6817/219909995","heartbeat_front_addr":"172.21.15.59:6815/219909995","state":["exists","up"]},{"osd":4,"uuid":"ef8fad28-1db5-4cba-b5f2-10e54e8e1549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":1021473826},{"type":"v1","addr":"172.21.15.115:6809","nonce":1021473826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":1021473826},{"type":"v1","addr":"172.21.15.115:6811","nonce":1021473826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":1021473826},{"type":"v1","addr":"172.21.15.115:6815","nonce":1021473826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":1021473826},{"type":"v1","addr":"172.21.15.115:6813","nonce":1021473826}]},"public_addr":"172.21.15.115:6809/1021473826","cluster_addr":"172.21.15.115:6811/1021473826","heartbeat_back_addr":"172.21.15.115:6815/1021473826","heartbeat_front_addr":"172.21.15.115:6813/1021473826","state":["exists","up"]},{"osd":5,"uuid":"4233a512-9e8f-4db8-aed2-6c3af706e589","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":2405172521},{"type":"v1","addr":"172.21.15.181:6809","nonce":2405172521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":2405172521},{"type":"v1","addr":"172.21.15.181:6811","nonce":2405172521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2405172521},{"type":"v1","addr":"172.21.15.181:6815","nonce":2405172521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":2405172521},{"type":"v1","addr":"172.21.15.181:6813","nonce":2405172521}]},"public_addr":"172.21.15.181:6809/2405172521","cluster_addr":"172.21.15.181:6811/2405172521","heartbeat_back_addr":"172.21.15.181:6815/2405172521","heartbeat_front_addr":"172.21.15.181:6813/2405172521","state":["exists","up"]},{"osd":6,"uuid":"8faab332-016e-4c77-8370-8bef95fc6f34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":815374853},{"type":"v1","addr":"172.21.15.59:6819","nonce":815374853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":815374853},{"type":"v1","addr":"172.21.15.59:6821","nonce":815374853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":815374853},{"type":"v1","addr":"172.21.15.59:6825","nonce":815374853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":815374853},{"type":"v1","addr":"172.21.15.59:6823","nonce":815374853}]},"public_addr":"172.21.15.59:6819/815374853","cluster_addr":"172.21.15.59:6821/815374853","heartbeat_back_addr":"172.21.15.59:6825/815374853","heartbeat_front_addr":"172.21.15.59:6823/815374853","state":["exists","up"]},{"osd":7,"uuid":"f20f470a-0d40-4ed3-8e0e-fd58675cf143","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":1314971870},{"type":"v1","addr":"172.21.15.115:6817","nonce":1314971870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":1314971870},{"type":"v1","addr":"172.21.15.115:6819","nonce":1314971870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":1314971870},{"type":"v1","addr":"172.21.15.115:6823","nonce":1314971870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":1314971870},{"type":"v1","addr":"172.21.15.115:6821","nonce":1314971870}]},"public_addr":"172.21.15.115:6817/1314971870","cluster_addr":"172.21.15.115:6819/1314971870","heartbeat_back_addr":"172.21.15.115:6823/1314971870","heartbeat_front_addr":"172.21.15.115:6821/1314971870","state":["exists","up"]},{"osd":8,"uuid":"eb657d32-1a95-4fd7-a293-d0a970837fc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":3954509873},{"type":"v1","addr":"172.21.15.181:6817","nonce":3954509873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":3954509873},{"type":"v1","addr":"172.21.15.181:6819","nonce":3954509873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":3954509873},{"type":"v1","addr":"172.21.15.181:6823","nonce":3954509873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":3954509873},{"type":"v1","addr":"172.21.15.181:6821","nonce":3954509873}]},"public_addr":"172.21.15.181:6817/3954509873","cluster_addr":"172.21.15.181:6819/3954509873","heartbeat_back_addr":"172.21.15.181:6823/3954509873","heartbeat_front_addr":"172.21.15.181:6821/3954509873","state":["exists","up"]},{"osd":9,"uuid":"8f7717d4-b6f2-4ec1-9b58-62feb0c9a146","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":1163095677},{"type":"v1","addr":"172.21.15.59:6827","nonce":1163095677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":1163095677},{"type":"v1","addr":"172.21.15.59:6829","nonce":1163095677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":1163095677},{"type":"v1","addr":"172.21.15.59:6833","nonce":1163095677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":1163095677},{"type":"v1","addr":"172.21.15.59:6831","nonce":1163095677}]},"public_addr":"172.21.15.59:6827/1163095677","cluster_addr":"172.21.15.59:6829/1163095677","heartbeat_back_addr":"172.21.15.59:6833/1163095677","heartbeat_front_addr":"172.21.15.59:6831/1163095677","state":["exists","up"]},{"osd":10,"uuid":"b75c179a-875d-43bc-b4c1-e8eb80fb64fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":2199610269},{"type":"v1","addr":"172.21.15.115:6825","nonce":2199610269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":2199610269},{"type":"v1","addr":"172.21.15.115:6827","nonce":2199610269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":2199610269},{"type":"v1","addr":"172.21.15.115:6831","nonce":2199610269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":2199610269},{"type":"v1","addr":"172.21.15.115:6829","nonce":2199610269}]},"public_addr":"172.21.15.115:6825/2199610269","cluster_addr":"172.21.15.115:6827/2199610269","heartbeat_back_addr":"172.21.15.115:6831/2199610269","heartbeat_front_addr":"172.21.15.115:6829/2199610269","state":["exists","up"]},{"osd":11,"uuid":"5e141dfe-8a00-4bbc-8204-2dc9be24e18d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":226506989},{"type":"v1","addr":"172.21.15.181:6825","nonce":226506989}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":226506989},{"type":"v1","addr":"172.21.15.181:6827","nonce":226506989}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":226506989},{"type":"v1","addr":"172.21.15.181:6831","nonce":226506989}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":226506989},{"type":"v1","addr":"172.21.15.181:6829","nonce":226506989}]},"public_addr":"172.21.15.181:6825/226506989","cluster_addr":"172.21.15.181:6827/226506989","heartbeat_back_addr":"172.21.15.181:6831/226506989","heartbeat_front_addr":"172.21.15.181:6829/226506989","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:39:42.687084+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:40:03.903448+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:40:24.899908+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:40:46.600251+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:41:07.690037+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:41:28.488855+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:41:49.459274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:42:10.863881+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:42:32.248391+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:42:54.112265+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:43:15.634872+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:43:36.922534+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.59:6800/1259916482":"2023-12-12T17:38:14.037922+0000","172.21.15.59:0/17457774":"2023-12-12T17:38:14.037922+0000","172.21.15.59:6801/1672530462":"2023-12-12T17:37:44.773925+0000","172.21.15.59:0/1053944547":"2023-12-12T17:38:14.037922+0000","172.21.15.59:6801/1259916482":"2023-12-12T17:38:14.037922+0000","172.21.15.59:6800/1672530462":"2023-12-12T17:37:44.773925+0000","172.21.15.59:0/3570767650":"2023-12-12T17:37:44.773925+0000","172.21.15.59:0/3853462899":"2023-12-12T17:37:44.773925+0000","172.21.15.59:0/2859415380":"2023-12-12T17:38:14.037922+0000","172.21.15.59:0/2192742819":"2023-12-12T17:37:44.773925+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-11T17:44:50.168 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:50.164+0000 7f6e41106700 1 -- 172.21.15.59:0/2579014155 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f6e280a1480 msgr2=0x7f6e280a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:50.168 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:50.164+0000 7f6e41106700 1 --2- 172.21.15.59:0/2579014155 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f6e280a1480 0x7f6e280a3940 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f6e2c003820 tx=0x7f6e2c005920 comp rx=0 tx=0).stop 2023-12-11T17:44:50.168 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:50.164+0000 7f6e41106700 1 -- 172.21.15.59:0/2579014155 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6e3c101570 msgr2=0x7f6e3c19c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:50.168 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:50.164+0000 7f6e41106700 1 --2- 172.21.15.59:0/2579014155 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6e3c101570 0x7f6e3c19c080 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f6e2400d950 tx=0x7f6e2400dc60 comp rx=0 tx=0).stop 2023-12-11T17:44:50.168 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:50.164+0000 7f6e41106700 1 -- 172.21.15.59:0/2579014155 shutdown_connections 2023-12-11T17:44:50.168 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:50.164+0000 7f6e41106700 1 --2- 172.21.15.59:0/2579014155 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f6e280a1480 0x7f6e280a3940 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:50.168 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:50.165+0000 7f6e41106700 1 --2- 172.21.15.59:0/2579014155 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6e3c101f10 0x7f6e3c19e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:50.168 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:50.165+0000 7f6e41106700 1 --2- 172.21.15.59:0/2579014155 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6e3c100380 0x7f6e3c069190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:50.168 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:50.165+0000 7f6e41106700 1 --2- 172.21.15.59:0/2579014155 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6e3c101570 0x7f6e3c19c080 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:50.169 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:50.165+0000 7f6e41106700 1 -- 172.21.15.59:0/2579014155 >> 172.21.15.59:0/2579014155 conn(0x7f6e3c0fb8f0 msgr2=0x7f6e3c1048f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:44:50.169 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:50.165+0000 7f6e41106700 1 -- 172.21.15.59:0/2579014155 shutdown_connections 2023-12-11T17:44:50.169 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:50.165+0000 7f6e41106700 1 -- 172.21.15.59:0/2579014155 wait complete. 2023-12-11T17:44:50.183 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-12-11T17:44:50.183 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-11T17:44:50.311 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:50.306+0000 7f0e540f0700 1 -- 172.21.15.59:0/4105501267 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e4c100380 msgr2=0x7f0e4c1007a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:50.311 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:50.306+0000 7f0e540f0700 1 --2- 172.21.15.59:0/4105501267 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e4c100380 0x7f0e4c1007a0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f0e480099c0 tx=0x7f0e48009cd0 comp rx=0 tx=0).stop 2023-12-11T17:44:50.311 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:50.306+0000 7f0e540f0700 1 -- 172.21.15.59:0/4105501267 shutdown_connections 2023-12-11T17:44:50.312 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:50.306+0000 7f0e540f0700 1 --2- 172.21.15.59:0/4105501267 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0e4c101f10 0x7f0e4c1087c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:50.312 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:50.306+0000 7f0e540f0700 1 --2- 172.21.15.59:0/4105501267 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e4c100380 0x7f0e4c1007a0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:50.312 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:50.306+0000 7f0e540f0700 1 --2- 172.21.15.59:0/4105501267 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0e4c101570 0x7f0e4c1019d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:50.312 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:50.306+0000 7f0e540f0700 1 -- 172.21.15.59:0/4105501267 >> 172.21.15.59:0/4105501267 conn(0x7f0e4c0fb8f0 msgr2=0x7f0e4c0fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:44:50.312 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:50.306+0000 7f0e540f0700 1 -- 172.21.15.59:0/4105501267 shutdown_connections 2023-12-11T17:44:50.312 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:50.306+0000 7f0e540f0700 1 -- 172.21.15.59:0/4105501267 wait complete. 2023-12-11T17:44:50.312 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:50.306+0000 7f0e540f0700 1 Processor -- start 2023-12-11T17:44:50.312 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:50.306+0000 7f0e540f0700 1 -- start start 2023-12-11T17:44:50.312 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:50.307+0000 7f0e540f0700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e4c100380 0x7f0e4c196290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:50.312 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:50.307+0000 7f0e540f0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0e4c101570 0x7f0e4c1967d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:50.312 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:50.307+0000 7f0e540f0700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0e4c101f10 0x7f0e4c19d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:50.312 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:50.307+0000 7f0e540f0700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e4c19ddc0 con 0x7f0e4c100380 2023-12-11T17:44:50.312 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:50.307+0000 7f0e540f0700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e4c19df00 con 0x7f0e4c101f10 2023-12-11T17:44:50.312 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:50.307+0000 7f0e540f0700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e4c19e090 con 0x7f0e4c101570 2023-12-11T17:44:50.313 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:50.307+0000 7f0e51e8c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e4c100380 0x7f0e4c196290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:50.313 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:50.307+0000 7f0e51e8c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e4c100380 0x7f0e4c196290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33446/0 (socket says 172.21.15.59:33446) 2023-12-11T17:44:50.313 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:50.307+0000 7f0e51e8c700 1 -- 172.21.15.59:0/97018211 learned_addr learned my addr 172.21.15.59:0/97018211 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:44:50.313 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:50.307+0000 7f0e5168b700 1 --2- 172.21.15.59:0/97018211 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0e4c101570 0x7f0e4c1967d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:50.313 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:50.307+0000 7f0e51e8c700 1 -- 172.21.15.59:0/97018211 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0e4c101570 msgr2=0x7f0e4c1967d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:50.313 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:50.307+0000 7f0e5268d700 1 --2- 172.21.15.59:0/97018211 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0e4c101f10 0x7f0e4c19d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:50.313 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:50.307+0000 7f0e51e8c700 1 --2- 172.21.15.59:0/97018211 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0e4c101570 0x7f0e4c1967d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:50.313 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:50.307+0000 7f0e51e8c700 1 -- 172.21.15.59:0/97018211 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0e4c101f10 msgr2=0x7f0e4c19d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:50.313 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:50.307+0000 7f0e51e8c700 1 --2- 172.21.15.59:0/97018211 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0e4c101f10 0x7f0e4c19d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:50.313 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:50.307+0000 7f0e51e8c700 1 -- 172.21.15.59:0/97018211 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e480095f0 con 0x7f0e4c100380 2023-12-11T17:44:50.313 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:50.307+0000 7f0e5268d700 1 --2- 172.21.15.59:0/97018211 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0e4c101f10 0x7f0e4c19d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-11T17:44:50.313 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:50.307+0000 7f0e51e8c700 1 --2- 172.21.15.59:0/97018211 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e4c100380 0x7f0e4c196290 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f0e48009fd0 tx=0x7f0e48005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:44:50.313 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:50.307+0000 7f0e42ffd700 1 -- 172.21.15.59:0/97018211 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e4801d070 con 0x7f0e4c100380 2023-12-11T17:44:50.313 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:50.307+0000 7f0e42ffd700 1 -- 172.21.15.59:0/97018211 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0e480059a0 con 0x7f0e4c100380 2023-12-11T17:44:50.313 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:50.307+0000 7f0e540f0700 1 -- 172.21.15.59:0/97018211 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e4c19e310 con 0x7f0e4c100380 2023-12-11T17:44:50.314 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:50.307+0000 7f0e42ffd700 1 -- 172.21.15.59:0/97018211 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e48017650 con 0x7f0e4c100380 2023-12-11T17:44:50.314 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:50.308+0000 7f0e540f0700 1 -- 172.21.15.59:0/97018211 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e4c19e7c0 con 0x7f0e4c100380 2023-12-11T17:44:50.314 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:50.308+0000 7f0e540f0700 1 -- 172.21.15.59:0/97018211 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e4c1077b0 con 0x7f0e4c100380 2023-12-11T17:44:50.315 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:50.309+0000 7f0e42ffd700 1 -- 172.21.15.59:0/97018211 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f0e48022470 con 0x7f0e4c100380 2023-12-11T17:44:50.315 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:50.310+0000 7f0e42ffd700 1 --2- 172.21.15.59:0/97018211 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f0e380a1480 0x7f0e380a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:50.315 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:50.310+0000 7f0e5168b700 1 --2- 172.21.15.59:0/97018211 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f0e380a1480 0x7f0e380a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:50.315 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:50.310+0000 7f0e42ffd700 1 -- 172.21.15.59:0/97018211 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f0e480ceb10 con 0x7f0e4c100380 2023-12-11T17:44:50.315 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:50.311+0000 7f0e5168b700 1 --2- 172.21.15.59:0/97018211 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f0e380a1480 0x7f0e380a3940 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c009bf0 tx=0x7f0e3c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:44:50.317 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:50.313+0000 7f0e42ffd700 1 -- 172.21.15.59:0/97018211 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f0e48097fb0 con 0x7f0e4c100380 2023-12-11T17:44:50.518 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:50.514+0000 7f0e540f0700 1 -- 172.21.15.59:0/97018211 --> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0e4c02d3b0 con 0x7f0e380a1480 2023-12-11T17:44:50.533 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:50.530+0000 7f0e42ffd700 1 -- 172.21.15.59:0/97018211 <== mgr.14150 v2:172.21.15.59:6800/3627728353 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+422599 (secure 0 0 0) 0x7f0e4c02d3b0 con 0x7f0e380a1480 2023-12-11T17:44:50.539 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-12-11T17:44:50.543 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:50.534+0000 7f0e540f0700 1 -- 172.21.15.59:0/97018211 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f0e380a1480 msgr2=0x7f0e380a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:50.543 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:50.534+0000 7f0e540f0700 1 --2- 172.21.15.59:0/97018211 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f0e380a1480 0x7f0e380a3940 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c009bf0 tx=0x7f0e3c009260 comp rx=0 tx=0).stop 2023-12-11T17:44:50.543 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:50.534+0000 7f0e540f0700 1 -- 172.21.15.59:0/97018211 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e4c100380 msgr2=0x7f0e4c196290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:50.543 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:50.534+0000 7f0e540f0700 1 --2- 172.21.15.59:0/97018211 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e4c100380 0x7f0e4c196290 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f0e48009fd0 tx=0x7f0e48005f10 comp rx=0 tx=0).stop 2023-12-11T17:44:50.543 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:50.534+0000 7f0e540f0700 1 -- 172.21.15.59:0/97018211 shutdown_connections 2023-12-11T17:44:50.543 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:50.534+0000 7f0e540f0700 1 --2- 172.21.15.59:0/97018211 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f0e380a1480 0x7f0e380a3940 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:50.543 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:50.534+0000 7f0e540f0700 1 --2- 172.21.15.59:0/97018211 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0e4c101f10 0x7f0e4c19d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:50.543 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:50.534+0000 7f0e540f0700 1 --2- 172.21.15.59:0/97018211 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e4c100380 0x7f0e4c196290 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:50.543 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:50.534+0000 7f0e540f0700 1 --2- 172.21.15.59:0/97018211 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0e4c101570 0x7f0e4c1967d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:50.543 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:50.534+0000 7f0e540f0700 1 -- 172.21.15.59:0/97018211 >> 172.21.15.59:0/97018211 conn(0x7f0e4c0fb8f0 msgr2=0x7f0e4c1048f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:44:50.543 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:50.534+0000 7f0e540f0700 1 -- 172.21.15.59:0/97018211 shutdown_connections 2023-12-11T17:44:50.543 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:50.534+0000 7f0e540f0700 1 -- 172.21.15.59:0/97018211 wait complete. 2023-12-11T17:44:50.543 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2023-12-11T17:44:50.550 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":274,"stamp":"2023-12-11T17:44:50.118430+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":597496,"num_objects":51,"num_object_clones":0,"num_object_copies":153,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":148,"num_read_kb":129,"num_write":293,"num_write_kb":2973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":32,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":236,"ondisk_log_size":236,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":335448,"kb_used_data":11244,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124606376,"statfs":{"total":1151940427776,"available":1151596929024,"internally_reserved":0,"allocated":11513856,"data_stored":7325246,"data_compressed":248031,"data_compressed_allocated":331776,"data_compressed_original":2162688,"omap_allocated":19142,"internal_metadata":331855162},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2489,"num_objects":27,"num_object_clones":0,"num_object_copies":81,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":63,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003835"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.675453+0000","last_change":"2023-12-11T17:44:20.624256+0000","last_active":"2023-12-11T17:44:24.675453+0000","last_peered":"2023-12-11T17:44:24.675453+0000","last_clean":"2023-12-11T17:44:24.675453+0000","last_became_active":"2023-12-11T17:44:20.623947+0000","last_became_peered":"2023-12-11T17:44:20.623947+0000","last_unstale":"2023-12-11T17:44:24.675453+0000","last_undegraded":"2023-12-11T17:44:24.675453+0000","last_fullsized":"2023-12-11T17:44:24.675453+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T18:12:43.669849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.683681+0000","last_change":"2023-12-11T17:44:20.620606+0000","last_active":"2023-12-11T17:44:24.683681+0000","last_peered":"2023-12-11T17:44:24.683681+0000","last_clean":"2023-12-11T17:44:24.683681+0000","last_became_active":"2023-12-11T17:44:20.620265+0000","last_became_peered":"2023-12-11T17:44:20.620265+0000","last_unstale":"2023-12-11T17:44:24.683681+0000","last_undegraded":"2023-12-11T17:44:24.683681+0000","last_fullsized":"2023-12-11T17:44:24.683681+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T19:53:35.349842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.680469+0000","last_change":"2023-12-11T17:44:20.620702+0000","last_active":"2023-12-11T17:44:24.680469+0000","last_peered":"2023-12-11T17:44:24.680469+0000","last_clean":"2023-12-11T17:44:24.680469+0000","last_became_active":"2023-12-11T17:44:20.620401+0000","last_became_peered":"2023-12-11T17:44:20.620401+0000","last_unstale":"2023-12-11T17:44:24.680469+0000","last_undegraded":"2023-12-11T17:44:24.680469+0000","last_fullsized":"2023-12-11T17:44:24.680469+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T21:14:21.156270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.687595+0000","last_change":"2023-12-11T17:44:20.614263+0000","last_active":"2023-12-11T17:44:24.687595+0000","last_peered":"2023-12-11T17:44:24.687595+0000","last_clean":"2023-12-11T17:44:24.687595+0000","last_became_active":"2023-12-11T17:44:20.613585+0000","last_became_peered":"2023-12-11T17:44:20.613585+0000","last_unstale":"2023-12-11T17:44:24.687595+0000","last_undegraded":"2023-12-11T17:44:24.687595+0000","last_fullsized":"2023-12-11T17:44:24.687595+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T23:03:26.514011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.680122+0000","last_change":"2023-12-11T17:44:20.615553+0000","last_active":"2023-12-11T17:44:24.680122+0000","last_peered":"2023-12-11T17:44:24.680122+0000","last_clean":"2023-12-11T17:44:24.680122+0000","last_became_active":"2023-12-11T17:44:20.615272+0000","last_became_peered":"2023-12-11T17:44:20.615272+0000","last_unstale":"2023-12-11T17:44:24.680122+0000","last_undegraded":"2023-12-11T17:44:24.680122+0000","last_fullsized":"2023-12-11T17:44:24.680122+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T20:37:02.178411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.695048+0000","last_change":"2023-12-11T17:44:20.612431+0000","last_active":"2023-12-11T17:44:24.695048+0000","last_peered":"2023-12-11T17:44:24.695048+0000","last_clean":"2023-12-11T17:44:24.695048+0000","last_became_active":"2023-12-11T17:44:20.611444+0000","last_became_peered":"2023-12-11T17:44:20.611444+0000","last_unstale":"2023-12-11T17:44:24.695048+0000","last_undegraded":"2023-12-11T17:44:24.695048+0000","last_fullsized":"2023-12-11T17:44:24.695048+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T22:45:12.892966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.686018+0000","last_change":"2023-12-11T17:44:20.623828+0000","last_active":"2023-12-11T17:44:24.686018+0000","last_peered":"2023-12-11T17:44:24.686018+0000","last_clean":"2023-12-11T17:44:24.686018+0000","last_became_active":"2023-12-11T17:44:20.623510+0000","last_became_peered":"2023-12-11T17:44:20.623510+0000","last_unstale":"2023-12-11T17:44:24.686018+0000","last_undegraded":"2023-12-11T17:44:24.686018+0000","last_fullsized":"2023-12-11T17:44:24.686018+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T22:04:32.298900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.680583+0000","last_change":"2023-12-11T17:44:20.620831+0000","last_active":"2023-12-11T17:44:24.680583+0000","last_peered":"2023-12-11T17:44:24.680583+0000","last_clean":"2023-12-11T17:44:24.680583+0000","last_became_active":"2023-12-11T17:44:20.620481+0000","last_became_peered":"2023-12-11T17:44:20.620481+0000","last_unstale":"2023-12-11T17:44:24.680583+0000","last_undegraded":"2023-12-11T17:44:24.680583+0000","last_fullsized":"2023-12-11T17:44:24.680583+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T22:14:28.838192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.679798+0000","last_change":"2023-12-11T17:44:20.619515+0000","last_active":"2023-12-11T17:44:24.679798+0000","last_peered":"2023-12-11T17:44:24.679798+0000","last_clean":"2023-12-11T17:44:24.679798+0000","last_became_active":"2023-12-11T17:44:20.619219+0000","last_became_peered":"2023-12-11T17:44:20.619219+0000","last_unstale":"2023-12-11T17:44:24.679798+0000","last_undegraded":"2023-12-11T17:44:24.679798+0000","last_fullsized":"2023-12-11T17:44:24.679798+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T23:08:55.112752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.695462+0000","last_change":"2023-12-11T17:44:20.612645+0000","last_active":"2023-12-11T17:44:24.695462+0000","last_peered":"2023-12-11T17:44:24.695462+0000","last_clean":"2023-12-11T17:44:24.695462+0000","last_became_active":"2023-12-11T17:44:20.611336+0000","last_became_peered":"2023-12-11T17:44:20.611336+0000","last_unstale":"2023-12-11T17:44:24.695462+0000","last_undegraded":"2023-12-11T17:44:24.695462+0000","last_fullsized":"2023-12-11T17:44:24.695462+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T02:43:07.447653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.688266+0000","last_change":"2023-12-11T17:44:20.621198+0000","last_active":"2023-12-11T17:44:24.688266+0000","last_peered":"2023-12-11T17:44:24.688266+0000","last_clean":"2023-12-11T17:44:24.688266+0000","last_became_active":"2023-12-11T17:44:20.620693+0000","last_became_peered":"2023-12-11T17:44:20.620693+0000","last_unstale":"2023-12-11T17:44:24.688266+0000","last_undegraded":"2023-12-11T17:44:24.688266+0000","last_fullsized":"2023-12-11T17:44:24.688266+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T22:10:23.396031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.687515+0000","last_change":"2023-12-11T17:44:20.614457+0000","last_active":"2023-12-11T17:44:24.687515+0000","last_peered":"2023-12-11T17:44:24.687515+0000","last_clean":"2023-12-11T17:44:24.687515+0000","last_became_active":"2023-12-11T17:44:20.613984+0000","last_became_peered":"2023-12-11T17:44:20.613984+0000","last_unstale":"2023-12-11T17:44:24.687515+0000","last_undegraded":"2023-12-11T17:44:24.687515+0000","last_fullsized":"2023-12-11T17:44:24.687515+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T00:50:01.101398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.675193+0000","last_change":"2023-12-11T17:44:20.619874+0000","last_active":"2023-12-11T17:44:24.675193+0000","last_peered":"2023-12-11T17:44:24.675193+0000","last_clean":"2023-12-11T17:44:24.675193+0000","last_became_active":"2023-12-11T17:44:20.618537+0000","last_became_peered":"2023-12-11T17:44:20.618537+0000","last_unstale":"2023-12-11T17:44:24.675193+0000","last_undegraded":"2023-12-11T17:44:24.675193+0000","last_fullsized":"2023-12-11T17:44:24.675193+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T18:42:43.587245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.681021+0000","last_change":"2023-12-11T17:44:20.616306+0000","last_active":"2023-12-11T17:44:24.681021+0000","last_peered":"2023-12-11T17:44:24.681021+0000","last_clean":"2023-12-11T17:44:24.681021+0000","last_became_active":"2023-12-11T17:44:20.615930+0000","last_became_peered":"2023-12-11T17:44:20.615930+0000","last_unstale":"2023-12-11T17:44:24.681021+0000","last_undegraded":"2023-12-11T17:44:24.681021+0000","last_fullsized":"2023-12-11T17:44:24.681021+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T05:40:24.495052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.680343+0000","last_change":"2023-12-11T17:44:20.624339+0000","last_active":"2023-12-11T17:44:24.680343+0000","last_peered":"2023-12-11T17:44:24.680343+0000","last_clean":"2023-12-11T17:44:24.680343+0000","last_became_active":"2023-12-11T17:44:20.624045+0000","last_became_peered":"2023-12-11T17:44:20.624045+0000","last_unstale":"2023-12-11T17:44:24.680343+0000","last_undegraded":"2023-12-11T17:44:24.680343+0000","last_fullsized":"2023-12-11T17:44:24.680343+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T01:32:30.792940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.679095+0000","last_change":"2023-12-11T17:44:20.620750+0000","last_active":"2023-12-11T17:44:24.679095+0000","last_peered":"2023-12-11T17:44:24.679095+0000","last_clean":"2023-12-11T17:44:24.679095+0000","last_became_active":"2023-12-11T17:44:20.620464+0000","last_became_peered":"2023-12-11T17:44:20.620464+0000","last_unstale":"2023-12-11T17:44:24.679095+0000","last_undegraded":"2023-12-11T17:44:24.679095+0000","last_fullsized":"2023-12-11T17:44:24.679095+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T05:43:33.680889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.675193+0000","last_change":"2023-12-11T17:44:20.624391+0000","last_active":"2023-12-11T17:44:24.675193+0000","last_peered":"2023-12-11T17:44:24.675193+0000","last_clean":"2023-12-11T17:44:24.675193+0000","last_became_active":"2023-12-11T17:44:20.624217+0000","last_became_peered":"2023-12-11T17:44:20.624217+0000","last_unstale":"2023-12-11T17:44:24.675193+0000","last_undegraded":"2023-12-11T17:44:24.675193+0000","last_fullsized":"2023-12-11T17:44:24.675193+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T01:06:41.660726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.696138+0000","last_change":"2023-12-11T17:44:20.612429+0000","last_active":"2023-12-11T17:44:24.696138+0000","last_peered":"2023-12-11T17:44:24.696138+0000","last_clean":"2023-12-11T17:44:24.696138+0000","last_became_active":"2023-12-11T17:44:20.612082+0000","last_became_peered":"2023-12-11T17:44:20.612082+0000","last_unstale":"2023-12-11T17:44:24.696138+0000","last_undegraded":"2023-12-11T17:44:24.696138+0000","last_fullsized":"2023-12-11T17:44:24.696138+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T02:34:42.364876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.688363+0000","last_change":"2023-12-11T17:44:20.614907+0000","last_active":"2023-12-11T17:44:24.688363+0000","last_peered":"2023-12-11T17:44:24.688363+0000","last_clean":"2023-12-11T17:44:24.688363+0000","last_became_active":"2023-12-11T17:44:20.614611+0000","last_became_peered":"2023-12-11T17:44:20.614611+0000","last_unstale":"2023-12-11T17:44:24.688363+0000","last_undegraded":"2023-12-11T17:44:24.688363+0000","last_fullsized":"2023-12-11T17:44:24.688363+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T05:29:50.095253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T17:44:25.781615+0000","last_change":"2023-12-11T17:44:24.690974+0000","last_active":"2023-12-11T17:44:25.781615+0000","last_peered":"2023-12-11T17:44:25.781615+0000","last_clean":"2023-12-11T17:44:25.781615+0000","last_became_active":"2023-12-11T17:44:24.690579+0000","last_became_peered":"2023-12-11T17:44:24.690579+0000","last_unstale":"2023-12-11T17:44:25.781615+0000","last_undegraded":"2023-12-11T17:44:25.781615+0000","last_fullsized":"2023-12-11T17:44:25.781615+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_clean_scrub_stamp":"2023-12-11T17:44:23.663161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:31:59.830486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'9","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T17:44:46.775802+0000","last_change":"2023-12-11T17:44:19.659110+0000","last_active":"2023-12-11T17:44:46.775802+0000","last_peered":"2023-12-11T17:44:46.775802+0000","last_clean":"2023-12-11T17:44:46.775802+0000","last_became_active":"2023-12-11T17:44:19.658780+0000","last_became_peered":"2023-12-11T17:44:19.658780+0000","last_unstale":"2023-12-11T17:44:46.775802+0000","last_undegraded":"2023-12-11T17:44:46.775802+0000","last_fullsized":"2023-12-11T17:44:46.775802+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_clean_scrub_stamp":"2023-12-11T17:44:18.636443+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T00:08:58.425711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.680643+0000","last_change":"2023-12-11T17:44:20.621928+0000","last_active":"2023-12-11T17:44:24.680643+0000","last_peered":"2023-12-11T17:44:24.680643+0000","last_clean":"2023-12-11T17:44:24.680643+0000","last_became_active":"2023-12-11T17:44:20.621575+0000","last_became_peered":"2023-12-11T17:44:20.621575+0000","last_unstale":"2023-12-11T17:44:24.680643+0000","last_undegraded":"2023-12-11T17:44:24.680643+0000","last_fullsized":"2023-12-11T17:44:24.680643+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T17:57:44.860200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T17:44:25.676954+0000","last_change":"2023-12-11T17:44:24.704170+0000","last_active":"2023-12-11T17:44:25.676954+0000","last_peered":"2023-12-11T17:44:25.676954+0000","last_clean":"2023-12-11T17:44:25.676954+0000","last_became_active":"2023-12-11T17:44:24.703800+0000","last_became_peered":"2023-12-11T17:44:24.703800+0000","last_unstale":"2023-12-11T17:44:25.676954+0000","last_undegraded":"2023-12-11T17:44:25.676954+0000","last_fullsized":"2023-12-11T17:44:25.676954+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_clean_scrub_stamp":"2023-12-11T17:44:23.663161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T01:26:29.578910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T17:44:25.677720+0000","last_change":"2023-12-11T17:44:19.659660+0000","last_active":"2023-12-11T17:44:25.677720+0000","last_peered":"2023-12-11T17:44:25.677720+0000","last_clean":"2023-12-11T17:44:25.677720+0000","last_became_active":"2023-12-11T17:44:19.659283+0000","last_became_peered":"2023-12-11T17:44:19.659283+0000","last_unstale":"2023-12-11T17:44:25.677720+0000","last_undegraded":"2023-12-11T17:44:25.677720+0000","last_fullsized":"2023-12-11T17:44:25.677720+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_clean_scrub_stamp":"2023-12-11T17:44:18.636443+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T22:29:43.427794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.682875+0000","last_change":"2023-12-11T17:44:20.624962+0000","last_active":"2023-12-11T17:44:24.682875+0000","last_peered":"2023-12-11T17:44:24.682875+0000","last_clean":"2023-12-11T17:44:24.682875+0000","last_became_active":"2023-12-11T17:44:20.624640+0000","last_became_peered":"2023-12-11T17:44:20.624640+0000","last_unstale":"2023-12-11T17:44:24.682875+0000","last_undegraded":"2023-12-11T17:44:24.682875+0000","last_fullsized":"2023-12-11T17:44:24.682875+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T23:37:43.252062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T17:44:25.677149+0000","last_change":"2023-12-11T17:44:24.698866+0000","last_active":"2023-12-11T17:44:25.677149+0000","last_peered":"2023-12-11T17:44:25.677149+0000","last_clean":"2023-12-11T17:44:25.677149+0000","last_became_active":"2023-12-11T17:44:24.698481+0000","last_became_peered":"2023-12-11T17:44:24.698481+0000","last_unstale":"2023-12-11T17:44:25.677149+0000","last_undegraded":"2023-12-11T17:44:25.677149+0000","last_fullsized":"2023-12-11T17:44:25.677149+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_clean_scrub_stamp":"2023-12-11T17:44:23.663161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T20:50:19.571616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T17:44:46.774408+0000","last_change":"2023-12-11T17:44:19.662637+0000","last_active":"2023-12-11T17:44:46.774408+0000","last_peered":"2023-12-11T17:44:46.774408+0000","last_clean":"2023-12-11T17:44:46.774408+0000","last_became_active":"2023-12-11T17:44:19.662355+0000","last_became_peered":"2023-12-11T17:44:19.662355+0000","last_unstale":"2023-12-11T17:44:46.774408+0000","last_undegraded":"2023-12-11T17:44:46.774408+0000","last_fullsized":"2023-12-11T17:44:46.774408+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_clean_scrub_stamp":"2023-12-11T17:44:18.636443+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T17:56:37.436037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.679261+0000","last_change":"2023-12-11T17:44:20.615119+0000","last_active":"2023-12-11T17:44:24.679261+0000","last_peered":"2023-12-11T17:44:24.679261+0000","last_clean":"2023-12-11T17:44:24.679261+0000","last_became_active":"2023-12-11T17:44:20.614775+0000","last_became_peered":"2023-12-11T17:44:20.614775+0000","last_unstale":"2023-12-11T17:44:24.679261+0000","last_undegraded":"2023-12-11T17:44:24.679261+0000","last_fullsized":"2023-12-11T17:44:24.679261+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T20:45:47.162890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T17:44:25.677063+0000","last_change":"2023-12-11T17:44:24.707718+0000","last_active":"2023-12-11T17:44:25.677063+0000","last_peered":"2023-12-11T17:44:25.677063+0000","last_clean":"2023-12-11T17:44:25.677063+0000","last_became_active":"2023-12-11T17:44:24.707404+0000","last_became_peered":"2023-12-11T17:44:24.707404+0000","last_unstale":"2023-12-11T17:44:25.677063+0000","last_undegraded":"2023-12-11T17:44:25.677063+0000","last_fullsized":"2023-12-11T17:44:25.677063+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_clean_scrub_stamp":"2023-12-11T17:44:23.663161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T22:53:53.918649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"81'2","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.682642+0000","last_change":"2023-12-11T17:44:19.665282+0000","last_active":"2023-12-11T17:44:24.682642+0000","last_peered":"2023-12-11T17:44:24.682642+0000","last_clean":"2023-12-11T17:44:24.682642+0000","last_became_active":"2023-12-11T17:44:19.664972+0000","last_became_peered":"2023-12-11T17:44:19.664972+0000","last_unstale":"2023-12-11T17:44:24.682642+0000","last_undegraded":"2023-12-11T17:44:24.682642+0000","last_fullsized":"2023-12-11T17:44:24.682642+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_clean_scrub_stamp":"2023-12-11T17:44:18.636443+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T01:59:50.809386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.684473+0000","last_change":"2023-12-11T17:44:20.615894+0000","last_active":"2023-12-11T17:44:24.684473+0000","last_peered":"2023-12-11T17:44:24.684473+0000","last_clean":"2023-12-11T17:44:24.684473+0000","last_became_active":"2023-12-11T17:44:20.615582+0000","last_became_peered":"2023-12-11T17:44:20.615582+0000","last_unstale":"2023-12-11T17:44:24.684473+0000","last_undegraded":"2023-12-11T17:44:24.684473+0000","last_fullsized":"2023-12-11T17:44:24.684473+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T19:04:50.725088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T17:44:25.781958+0000","last_change":"2023-12-11T17:44:24.691459+0000","last_active":"2023-12-11T17:44:25.781958+0000","last_peered":"2023-12-11T17:44:25.781958+0000","last_clean":"2023-12-11T17:44:25.781958+0000","last_became_active":"2023-12-11T17:44:24.690301+0000","last_became_peered":"2023-12-11T17:44:24.690301+0000","last_unstale":"2023-12-11T17:44:25.781958+0000","last_undegraded":"2023-12-11T17:44:25.781958+0000","last_fullsized":"2023-12-11T17:44:25.781958+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_clean_scrub_stamp":"2023-12-11T17:44:23.663161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T17:58:04.685785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"86'6","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T17:44:48.782361+0000","last_change":"2023-12-11T17:44:19.661360+0000","last_active":"2023-12-11T17:44:48.782361+0000","last_peered":"2023-12-11T17:44:48.782361+0000","last_clean":"2023-12-11T17:44:48.782361+0000","last_became_active":"2023-12-11T17:44:19.661056+0000","last_became_peered":"2023-12-11T17:44:19.661056+0000","last_unstale":"2023-12-11T17:44:48.782361+0000","last_undegraded":"2023-12-11T17:44:48.782361+0000","last_fullsized":"2023-12-11T17:44:48.782361+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_clean_scrub_stamp":"2023-12-11T17:44:18.636443+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:21:19.212002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1011,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.680386+0000","last_change":"2023-12-11T17:44:20.608280+0000","last_active":"2023-12-11T17:44:24.680386+0000","last_peered":"2023-12-11T17:44:24.680386+0000","last_clean":"2023-12-11T17:44:24.680386+0000","last_became_active":"2023-12-11T17:44:20.607980+0000","last_became_peered":"2023-12-11T17:44:20.607980+0000","last_unstale":"2023-12-11T17:44:24.680386+0000","last_undegraded":"2023-12-11T17:44:24.680386+0000","last_fullsized":"2023-12-11T17:44:24.680386+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T04:45:56.108916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T17:44:25.781523+0000","last_change":"2023-12-11T17:44:24.703629+0000","last_active":"2023-12-11T17:44:25.781523+0000","last_peered":"2023-12-11T17:44:25.781523+0000","last_clean":"2023-12-11T17:44:25.781523+0000","last_became_active":"2023-12-11T17:44:24.703219+0000","last_became_peered":"2023-12-11T17:44:24.703219+0000","last_unstale":"2023-12-11T17:44:25.781523+0000","last_undegraded":"2023-12-11T17:44:25.781523+0000","last_fullsized":"2023-12-11T17:44:25.781523+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_clean_scrub_stamp":"2023-12-11T17:44:23.663161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T19:57:44.998039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T17:44:46.777497+0000","last_change":"2023-12-11T17:44:19.659311+0000","last_active":"2023-12-11T17:44:46.777497+0000","last_peered":"2023-12-11T17:44:46.777497+0000","last_clean":"2023-12-11T17:44:46.777497+0000","last_became_active":"2023-12-11T17:44:19.658994+0000","last_became_peered":"2023-12-11T17:44:19.658994+0000","last_unstale":"2023-12-11T17:44:46.777497+0000","last_undegraded":"2023-12-11T17:44:46.777497+0000","last_fullsized":"2023-12-11T17:44:46.777497+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_clean_scrub_stamp":"2023-12-11T17:44:18.636443+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T17:46:35.701179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.684251+0000","last_change":"2023-12-11T17:44:20.615684+0000","last_active":"2023-12-11T17:44:24.684251+0000","last_peered":"2023-12-11T17:44:24.684251+0000","last_clean":"2023-12-11T17:44:24.684251+0000","last_became_active":"2023-12-11T17:44:20.615373+0000","last_became_peered":"2023-12-11T17:44:20.615373+0000","last_unstale":"2023-12-11T17:44:24.684251+0000","last_undegraded":"2023-12-11T17:44:24.684251+0000","last_fullsized":"2023-12-11T17:44:24.684251+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T19:45:59.980253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T17:44:25.782048+0000","last_change":"2023-12-11T17:44:24.699085+0000","last_active":"2023-12-11T17:44:25.782048+0000","last_peered":"2023-12-11T17:44:25.782048+0000","last_clean":"2023-12-11T17:44:25.782048+0000","last_became_active":"2023-12-11T17:44:24.698717+0000","last_became_peered":"2023-12-11T17:44:24.698717+0000","last_unstale":"2023-12-11T17:44:25.782048+0000","last_undegraded":"2023-12-11T17:44:25.782048+0000","last_fullsized":"2023-12-11T17:44:25.782048+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_clean_scrub_stamp":"2023-12-11T17:44:23.663161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T05:00:52.341938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.675120+0000","last_change":"2023-12-11T17:44:19.660392+0000","last_active":"2023-12-11T17:44:24.675120+0000","last_peered":"2023-12-11T17:44:24.675120+0000","last_clean":"2023-12-11T17:44:24.675120+0000","last_became_active":"2023-12-11T17:44:19.660099+0000","last_became_peered":"2023-12-11T17:44:19.660099+0000","last_unstale":"2023-12-11T17:44:24.675120+0000","last_undegraded":"2023-12-11T17:44:24.675120+0000","last_fullsized":"2023-12-11T17:44:24.675120+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_clean_scrub_stamp":"2023-12-11T17:44:18.636443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T00:38:37.279230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.680701+0000","last_change":"2023-12-11T17:44:20.615629+0000","last_active":"2023-12-11T17:44:24.680701+0000","last_peered":"2023-12-11T17:44:24.680701+0000","last_clean":"2023-12-11T17:44:24.680701+0000","last_became_active":"2023-12-11T17:44:20.615193+0000","last_became_peered":"2023-12-11T17:44:20.615193+0000","last_unstale":"2023-12-11T17:44:24.680701+0000","last_undegraded":"2023-12-11T17:44:24.680701+0000","last_fullsized":"2023-12-11T17:44:24.680701+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:09:56.237298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T17:44:25.781425+0000","last_change":"2023-12-11T17:44:24.705845+0000","last_active":"2023-12-11T17:44:25.781425+0000","last_peered":"2023-12-11T17:44:25.781425+0000","last_clean":"2023-12-11T17:44:25.781425+0000","last_became_active":"2023-12-11T17:44:24.705434+0000","last_became_peered":"2023-12-11T17:44:24.705434+0000","last_unstale":"2023-12-11T17:44:25.781425+0000","last_undegraded":"2023-12-11T17:44:25.781425+0000","last_fullsized":"2023-12-11T17:44:25.781425+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_clean_scrub_stamp":"2023-12-11T17:44:23.663161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T02:37:25.663753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T17:44:25.782676+0000","last_change":"2023-12-11T17:44:19.661317+0000","last_active":"2023-12-11T17:44:25.782676+0000","last_peered":"2023-12-11T17:44:25.782676+0000","last_clean":"2023-12-11T17:44:25.782676+0000","last_became_active":"2023-12-11T17:44:19.660993+0000","last_became_peered":"2023-12-11T17:44:19.660993+0000","last_unstale":"2023-12-11T17:44:25.782676+0000","last_undegraded":"2023-12-11T17:44:25.782676+0000","last_fullsized":"2023-12-11T17:44:25.782676+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_clean_scrub_stamp":"2023-12-11T17:44:18.636443+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T23:00:07.345964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.695205+0000","last_change":"2023-12-11T17:44:20.623681+0000","last_active":"2023-12-11T17:44:24.695205+0000","last_peered":"2023-12-11T17:44:24.695205+0000","last_clean":"2023-12-11T17:44:24.695205+0000","last_became_active":"2023-12-11T17:44:20.623288+0000","last_became_peered":"2023-12-11T17:44:20.623288+0000","last_unstale":"2023-12-11T17:44:24.695205+0000","last_undegraded":"2023-12-11T17:44:24.695205+0000","last_fullsized":"2023-12-11T17:44:24.695205+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T23:47:55.348480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T17:44:25.677106+0000","last_change":"2023-12-11T17:44:24.687613+0000","last_active":"2023-12-11T17:44:25.677106+0000","last_peered":"2023-12-11T17:44:25.677106+0000","last_clean":"2023-12-11T17:44:25.677106+0000","last_became_active":"2023-12-11T17:44:24.687240+0000","last_became_peered":"2023-12-11T17:44:24.687240+0000","last_unstale":"2023-12-11T17:44:25.677106+0000","last_undegraded":"2023-12-11T17:44:25.677106+0000","last_fullsized":"2023-12-11T17:44:25.677106+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_clean_scrub_stamp":"2023-12-11T17:44:23.663161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T20:54:25.409631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"86'3","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T17:44:45.765254+0000","last_change":"2023-12-11T17:44:19.653571+0000","last_active":"2023-12-11T17:44:45.765254+0000","last_peered":"2023-12-11T17:44:45.765254+0000","last_clean":"2023-12-11T17:44:45.765254+0000","last_became_active":"2023-12-11T17:44:19.653035+0000","last_became_peered":"2023-12-11T17:44:19.653035+0000","last_unstale":"2023-12-11T17:44:45.765254+0000","last_undegraded":"2023-12-11T17:44:45.765254+0000","last_fullsized":"2023-12-11T17:44:45.765254+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_clean_scrub_stamp":"2023-12-11T17:44:18.636443+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T02:29:38.310132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.679683+0000","last_change":"2023-12-11T17:44:20.623777+0000","last_active":"2023-12-11T17:44:24.679683+0000","last_peered":"2023-12-11T17:44:24.679683+0000","last_clean":"2023-12-11T17:44:24.679683+0000","last_became_active":"2023-12-11T17:44:20.623449+0000","last_became_peered":"2023-12-11T17:44:20.623449+0000","last_unstale":"2023-12-11T17:44:24.679683+0000","last_undegraded":"2023-12-11T17:44:24.679683+0000","last_fullsized":"2023-12-11T17:44:24.679683+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T05:23:03.493677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T17:44:25.782574+0000","last_change":"2023-12-11T17:44:24.693893+0000","last_active":"2023-12-11T17:44:25.782574+0000","last_peered":"2023-12-11T17:44:25.782574+0000","last_clean":"2023-12-11T17:44:25.782574+0000","last_became_active":"2023-12-11T17:44:24.693441+0000","last_became_peered":"2023-12-11T17:44:24.693441+0000","last_unstale":"2023-12-11T17:44:25.782574+0000","last_undegraded":"2023-12-11T17:44:25.782574+0000","last_fullsized":"2023-12-11T17:44:25.782574+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_clean_scrub_stamp":"2023-12-11T17:44:23.663161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T23:25:06.457176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T17:44:45.763408+0000","last_change":"2023-12-11T17:44:19.662988+0000","last_active":"2023-12-11T17:44:45.763408+0000","last_peered":"2023-12-11T17:44:45.763408+0000","last_clean":"2023-12-11T17:44:45.763408+0000","last_became_active":"2023-12-11T17:44:19.662660+0000","last_became_peered":"2023-12-11T17:44:19.662660+0000","last_unstale":"2023-12-11T17:44:45.763408+0000","last_undegraded":"2023-12-11T17:44:45.763408+0000","last_fullsized":"2023-12-11T17:44:45.763408+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_clean_scrub_stamp":"2023-12-11T17:44:18.636443+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T19:08:50.703161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.682688+0000","last_change":"2023-12-11T17:44:20.622993+0000","last_active":"2023-12-11T17:44:24.682688+0000","last_peered":"2023-12-11T17:44:24.682688+0000","last_clean":"2023-12-11T17:44:24.682688+0000","last_became_active":"2023-12-11T17:44:20.621050+0000","last_became_peered":"2023-12-11T17:44:20.621050+0000","last_unstale":"2023-12-11T17:44:24.682688+0000","last_undegraded":"2023-12-11T17:44:24.682688+0000","last_fullsized":"2023-12-11T17:44:24.682688+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T22:28:03.230596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T17:44:25.677339+0000","last_change":"2023-12-11T17:44:24.698715+0000","last_active":"2023-12-11T17:44:25.677339+0000","last_peered":"2023-12-11T17:44:25.677339+0000","last_clean":"2023-12-11T17:44:25.677339+0000","last_became_active":"2023-12-11T17:44:24.698359+0000","last_became_peered":"2023-12-11T17:44:24.698359+0000","last_unstale":"2023-12-11T17:44:25.677339+0000","last_undegraded":"2023-12-11T17:44:25.677339+0000","last_fullsized":"2023-12-11T17:44:25.677339+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_clean_scrub_stamp":"2023-12-11T17:44:23.663161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T01:07:23.870922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T17:44:45.763733+0000","last_change":"2023-12-11T17:44:19.655914+0000","last_active":"2023-12-11T17:44:45.763733+0000","last_peered":"2023-12-11T17:44:45.763733+0000","last_clean":"2023-12-11T17:44:45.763733+0000","last_became_active":"2023-12-11T17:44:19.655593+0000","last_became_peered":"2023-12-11T17:44:19.655593+0000","last_unstale":"2023-12-11T17:44:45.763733+0000","last_undegraded":"2023-12-11T17:44:45.763733+0000","last_fullsized":"2023-12-11T17:44:45.763733+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_clean_scrub_stamp":"2023-12-11T17:44:18.636443+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T01:54:50.246100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.679438+0000","last_change":"2023-12-11T17:44:20.618469+0000","last_active":"2023-12-11T17:44:24.679438+0000","last_peered":"2023-12-11T17:44:24.679438+0000","last_clean":"2023-12-11T17:44:24.679438+0000","last_became_active":"2023-12-11T17:44:20.618186+0000","last_became_peered":"2023-12-11T17:44:20.618186+0000","last_unstale":"2023-12-11T17:44:24.679438+0000","last_undegraded":"2023-12-11T17:44:24.679438+0000","last_fullsized":"2023-12-11T17:44:24.679438+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T20:53:22.405100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T17:44:25.781421+0000","last_change":"2023-12-11T17:44:24.697048+0000","last_active":"2023-12-11T17:44:25.781421+0000","last_peered":"2023-12-11T17:44:25.781421+0000","last_clean":"2023-12-11T17:44:25.781421+0000","last_became_active":"2023-12-11T17:44:24.696660+0000","last_became_peered":"2023-12-11T17:44:24.696660+0000","last_unstale":"2023-12-11T17:44:25.781421+0000","last_undegraded":"2023-12-11T17:44:25.781421+0000","last_fullsized":"2023-12-11T17:44:25.781421+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_clean_scrub_stamp":"2023-12-11T17:44:23.663161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T21:50:51.018386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"86'12","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T17:44:45.764204+0000","last_change":"2023-12-11T17:44:19.653528+0000","last_active":"2023-12-11T17:44:45.764204+0000","last_peered":"2023-12-11T17:44:45.764204+0000","last_clean":"2023-12-11T17:44:45.764204+0000","last_became_active":"2023-12-11T17:44:19.653215+0000","last_became_peered":"2023-12-11T17:44:19.653215+0000","last_unstale":"2023-12-11T17:44:45.764204+0000","last_undegraded":"2023-12-11T17:44:45.764204+0000","last_fullsized":"2023-12-11T17:44:45.764204+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_clean_scrub_stamp":"2023-12-11T17:44:18.636443+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T21:19:12.024008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3841,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.680635+0000","last_change":"2023-12-11T17:44:20.614085+0000","last_active":"2023-12-11T17:44:24.680635+0000","last_peered":"2023-12-11T17:44:24.680635+0000","last_clean":"2023-12-11T17:44:24.680635+0000","last_became_active":"2023-12-11T17:44:20.613768+0000","last_became_peered":"2023-12-11T17:44:20.613768+0000","last_unstale":"2023-12-11T17:44:24.680635+0000","last_undegraded":"2023-12-11T17:44:24.680635+0000","last_fullsized":"2023-12-11T17:44:24.680635+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T23:42:52.133207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T17:44:25.782452+0000","last_change":"2023-12-11T17:44:24.702039+0000","last_active":"2023-12-11T17:44:25.782452+0000","last_peered":"2023-12-11T17:44:25.782452+0000","last_clean":"2023-12-11T17:44:25.782452+0000","last_became_active":"2023-12-11T17:44:24.701539+0000","last_became_peered":"2023-12-11T17:44:24.701539+0000","last_unstale":"2023-12-11T17:44:25.782452+0000","last_undegraded":"2023-12-11T17:44:25.782452+0000","last_fullsized":"2023-12-11T17:44:25.782452+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_clean_scrub_stamp":"2023-12-11T17:44:23.663161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T02:23:55.393595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T17:44:46.773873+0000","last_change":"2023-12-11T17:44:19.662118+0000","last_active":"2023-12-11T17:44:46.773873+0000","last_peered":"2023-12-11T17:44:46.773873+0000","last_clean":"2023-12-11T17:44:46.773873+0000","last_became_active":"2023-12-11T17:44:19.661796+0000","last_became_peered":"2023-12-11T17:44:19.661796+0000","last_unstale":"2023-12-11T17:44:46.773873+0000","last_undegraded":"2023-12-11T17:44:46.773873+0000","last_fullsized":"2023-12-11T17:44:46.773873+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_clean_scrub_stamp":"2023-12-11T17:44:18.636443+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T21:08:02.842544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.675150+0000","last_change":"2023-12-11T17:44:20.615403+0000","last_active":"2023-12-11T17:44:24.675150+0000","last_peered":"2023-12-11T17:44:24.675150+0000","last_clean":"2023-12-11T17:44:24.675150+0000","last_became_active":"2023-12-11T17:44:20.615106+0000","last_became_peered":"2023-12-11T17:44:20.615106+0000","last_unstale":"2023-12-11T17:44:24.675150+0000","last_undegraded":"2023-12-11T17:44:24.675150+0000","last_fullsized":"2023-12-11T17:44:24.675150+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T02:10:24.047720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T17:44:25.781715+0000","last_change":"2023-12-11T17:44:24.699737+0000","last_active":"2023-12-11T17:44:25.781715+0000","last_peered":"2023-12-11T17:44:25.781715+0000","last_clean":"2023-12-11T17:44:25.781715+0000","last_became_active":"2023-12-11T17:44:24.699345+0000","last_became_peered":"2023-12-11T17:44:24.699345+0000","last_unstale":"2023-12-11T17:44:25.781715+0000","last_undegraded":"2023-12-11T17:44:25.781715+0000","last_fullsized":"2023-12-11T17:44:25.781715+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_clean_scrub_stamp":"2023-12-11T17:44:23.663161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T01:30:25.471636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T17:44:45.764021+0000","last_change":"2023-12-11T17:44:19.661325+0000","last_active":"2023-12-11T17:44:45.764021+0000","last_peered":"2023-12-11T17:44:45.764021+0000","last_clean":"2023-12-11T17:44:45.764021+0000","last_became_active":"2023-12-11T17:44:19.661012+0000","last_became_peered":"2023-12-11T17:44:19.661012+0000","last_unstale":"2023-12-11T17:44:45.764021+0000","last_undegraded":"2023-12-11T17:44:45.764021+0000","last_fullsized":"2023-12-11T17:44:45.764021+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_clean_scrub_stamp":"2023-12-11T17:44:18.636443+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T19:33:08.258019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.686029+0000","last_change":"2023-12-11T17:44:20.614795+0000","last_active":"2023-12-11T17:44:24.686029+0000","last_peered":"2023-12-11T17:44:24.686029+0000","last_clean":"2023-12-11T17:44:24.686029+0000","last_became_active":"2023-12-11T17:44:20.613826+0000","last_became_peered":"2023-12-11T17:44:20.613826+0000","last_unstale":"2023-12-11T17:44:24.686029+0000","last_undegraded":"2023-12-11T17:44:24.686029+0000","last_fullsized":"2023-12-11T17:44:24.686029+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T21:32:19.029040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T17:44:25.781716+0000","last_change":"2023-12-11T17:44:24.701569+0000","last_active":"2023-12-11T17:44:25.781716+0000","last_peered":"2023-12-11T17:44:25.781716+0000","last_clean":"2023-12-11T17:44:25.781716+0000","last_became_active":"2023-12-11T17:44:24.701173+0000","last_became_peered":"2023-12-11T17:44:24.701173+0000","last_unstale":"2023-12-11T17:44:25.781716+0000","last_undegraded":"2023-12-11T17:44:25.781716+0000","last_fullsized":"2023-12-11T17:44:25.781716+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_clean_scrub_stamp":"2023-12-11T17:44:23.663161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T23:02:45.843827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.684158+0000","last_change":"2023-12-11T17:44:19.659643+0000","last_active":"2023-12-11T17:44:24.684158+0000","last_peered":"2023-12-11T17:44:24.684158+0000","last_clean":"2023-12-11T17:44:24.684158+0000","last_became_active":"2023-12-11T17:44:19.659320+0000","last_became_peered":"2023-12-11T17:44:19.659320+0000","last_unstale":"2023-12-11T17:44:24.684158+0000","last_undegraded":"2023-12-11T17:44:24.684158+0000","last_fullsized":"2023-12-11T17:44:24.684158+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_clean_scrub_stamp":"2023-12-11T17:44:18.636443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T20:02:03.633439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.685932+0000","last_change":"2023-12-11T17:44:20.619445+0000","last_active":"2023-12-11T17:44:24.685932+0000","last_peered":"2023-12-11T17:44:24.685932+0000","last_clean":"2023-12-11T17:44:24.685932+0000","last_became_active":"2023-12-11T17:44:20.619094+0000","last_became_peered":"2023-12-11T17:44:20.619094+0000","last_unstale":"2023-12-11T17:44:24.685932+0000","last_undegraded":"2023-12-11T17:44:24.685932+0000","last_fullsized":"2023-12-11T17:44:24.685932+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T04:03:21.917498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T17:44:25.781832+0000","last_change":"2023-12-11T17:44:24.701821+0000","last_active":"2023-12-11T17:44:25.781832+0000","last_peered":"2023-12-11T17:44:25.781832+0000","last_clean":"2023-12-11T17:44:25.781832+0000","last_became_active":"2023-12-11T17:44:24.700590+0000","last_became_peered":"2023-12-11T17:44:24.700590+0000","last_unstale":"2023-12-11T17:44:25.781832+0000","last_undegraded":"2023-12-11T17:44:25.781832+0000","last_fullsized":"2023-12-11T17:44:25.781832+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_clean_scrub_stamp":"2023-12-11T17:44:23.663161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T19:58:28.422648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":33,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T17:44:46.775889+0000","last_change":"2023-12-11T17:44:19.662540+0000","last_active":"2023-12-11T17:44:46.775889+0000","last_peered":"2023-12-11T17:44:46.775889+0000","last_clean":"2023-12-11T17:44:46.775889+0000","last_became_active":"2023-12-11T17:44:19.662231+0000","last_became_peered":"2023-12-11T17:44:19.662231+0000","last_unstale":"2023-12-11T17:44:46.775889+0000","last_undegraded":"2023-12-11T17:44:46.775889+0000","last_fullsized":"2023-12-11T17:44:46.775889+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_clean_scrub_stamp":"2023-12-11T17:44:18.636443+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T21:56:45.241138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.683927+0000","last_change":"2023-12-11T17:44:20.620457+0000","last_active":"2023-12-11T17:44:24.683927+0000","last_peered":"2023-12-11T17:44:24.683927+0000","last_clean":"2023-12-11T17:44:24.683927+0000","last_became_active":"2023-12-11T17:44:20.619949+0000","last_became_peered":"2023-12-11T17:44:20.619949+0000","last_unstale":"2023-12-11T17:44:24.683927+0000","last_undegraded":"2023-12-11T17:44:24.683927+0000","last_fullsized":"2023-12-11T17:44:24.683927+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T22:59:14.328817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T17:44:25.782384+0000","last_change":"2023-12-11T17:44:24.691573+0000","last_active":"2023-12-11T17:44:25.782384+0000","last_peered":"2023-12-11T17:44:25.782384+0000","last_clean":"2023-12-11T17:44:25.782384+0000","last_became_active":"2023-12-11T17:44:24.691182+0000","last_became_peered":"2023-12-11T17:44:24.691182+0000","last_unstale":"2023-12-11T17:44:25.782384+0000","last_undegraded":"2023-12-11T17:44:25.782384+0000","last_fullsized":"2023-12-11T17:44:25.782384+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_clean_scrub_stamp":"2023-12-11T17:44:23.663161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T02:18:36.582108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"81'1","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T17:44:25.677343+0000","last_change":"2023-12-11T17:44:19.653470+0000","last_active":"2023-12-11T17:44:25.677343+0000","last_peered":"2023-12-11T17:44:25.677343+0000","last_clean":"2023-12-11T17:44:25.677343+0000","last_became_active":"2023-12-11T17:44:19.653148+0000","last_became_peered":"2023-12-11T17:44:19.653148+0000","last_unstale":"2023-12-11T17:44:25.677343+0000","last_undegraded":"2023-12-11T17:44:25.677343+0000","last_fullsized":"2023-12-11T17:44:25.677343+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_clean_scrub_stamp":"2023-12-11T17:44:18.636443+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T05:26:12.017814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.674922+0000","last_change":"2023-12-11T17:44:20.617019+0000","last_active":"2023-12-11T17:44:24.674922+0000","last_peered":"2023-12-11T17:44:24.674922+0000","last_clean":"2023-12-11T17:44:24.674922+0000","last_became_active":"2023-12-11T17:44:20.616770+0000","last_became_peered":"2023-12-11T17:44:20.616770+0000","last_unstale":"2023-12-11T17:44:24.674922+0000","last_undegraded":"2023-12-11T17:44:24.674922+0000","last_fullsized":"2023-12-11T17:44:24.674922+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T18:32:15.050179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T17:44:25.676860+0000","last_change":"2023-12-11T17:44:24.694843+0000","last_active":"2023-12-11T17:44:25.676860+0000","last_peered":"2023-12-11T17:44:25.676860+0000","last_clean":"2023-12-11T17:44:25.676860+0000","last_became_active":"2023-12-11T17:44:24.694423+0000","last_became_peered":"2023-12-11T17:44:24.694423+0000","last_unstale":"2023-12-11T17:44:25.676860+0000","last_undegraded":"2023-12-11T17:44:25.676860+0000","last_fullsized":"2023-12-11T17:44:25.676860+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_clean_scrub_stamp":"2023-12-11T17:44:23.663161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T18:42:35.281782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.695692+0000","last_change":"2023-12-11T17:44:19.655676+0000","last_active":"2023-12-11T17:44:24.695692+0000","last_peered":"2023-12-11T17:44:24.695692+0000","last_clean":"2023-12-11T17:44:24.695692+0000","last_became_active":"2023-12-11T17:44:19.655331+0000","last_became_peered":"2023-12-11T17:44:19.655331+0000","last_unstale":"2023-12-11T17:44:24.695692+0000","last_undegraded":"2023-12-11T17:44:24.695692+0000","last_fullsized":"2023-12-11T17:44:24.695692+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_clean_scrub_stamp":"2023-12-11T17:44:18.636443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T00:01:27.963540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.695557+0000","last_change":"2023-12-11T17:44:20.617396+0000","last_active":"2023-12-11T17:44:24.695557+0000","last_peered":"2023-12-11T17:44:24.695557+0000","last_clean":"2023-12-11T17:44:24.695557+0000","last_became_active":"2023-12-11T17:44:20.617086+0000","last_became_peered":"2023-12-11T17:44:20.617086+0000","last_unstale":"2023-12-11T17:44:24.695557+0000","last_undegraded":"2023-12-11T17:44:24.695557+0000","last_fullsized":"2023-12-11T17:44:24.695557+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T02:31:55.763125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T17:44:25.781340+0000","last_change":"2023-12-11T17:44:24.703283+0000","last_active":"2023-12-11T17:44:25.781340+0000","last_peered":"2023-12-11T17:44:25.781340+0000","last_clean":"2023-12-11T17:44:25.781340+0000","last_became_active":"2023-12-11T17:44:24.701703+0000","last_became_peered":"2023-12-11T17:44:24.701703+0000","last_unstale":"2023-12-11T17:44:25.781340+0000","last_undegraded":"2023-12-11T17:44:25.781340+0000","last_fullsized":"2023-12-11T17:44:25.781340+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_clean_scrub_stamp":"2023-12-11T17:44:23.663161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T18:52:50.570036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T17:44:25.781972+0000","last_change":"2023-12-11T17:44:19.662851+0000","last_active":"2023-12-11T17:44:25.781972+0000","last_peered":"2023-12-11T17:44:25.781972+0000","last_clean":"2023-12-11T17:44:25.781972+0000","last_became_active":"2023-12-11T17:44:19.662548+0000","last_became_peered":"2023-12-11T17:44:19.662548+0000","last_unstale":"2023-12-11T17:44:25.781972+0000","last_undegraded":"2023-12-11T17:44:25.781972+0000","last_fullsized":"2023-12-11T17:44:25.781972+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_clean_scrub_stamp":"2023-12-11T17:44:18.636443+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T02:34:14.914687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.695033+0000","last_change":"2023-12-11T17:44:20.612750+0000","last_active":"2023-12-11T17:44:24.695033+0000","last_peered":"2023-12-11T17:44:24.695033+0000","last_clean":"2023-12-11T17:44:24.695033+0000","last_became_active":"2023-12-11T17:44:20.612479+0000","last_became_peered":"2023-12-11T17:44:20.612479+0000","last_unstale":"2023-12-11T17:44:24.695033+0000","last_undegraded":"2023-12-11T17:44:24.695033+0000","last_fullsized":"2023-12-11T17:44:24.695033+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:45:03.217844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T17:44:25.782005+0000","last_change":"2023-12-11T17:44:24.703050+0000","last_active":"2023-12-11T17:44:25.782005+0000","last_peered":"2023-12-11T17:44:25.782005+0000","last_clean":"2023-12-11T17:44:25.782005+0000","last_became_active":"2023-12-11T17:44:24.701408+0000","last_became_peered":"2023-12-11T17:44:24.701408+0000","last_unstale":"2023-12-11T17:44:25.782005+0000","last_undegraded":"2023-12-11T17:44:25.782005+0000","last_fullsized":"2023-12-11T17:44:25.782005+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_clean_scrub_stamp":"2023-12-11T17:44:23.663161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T01:02:44.726831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.680820+0000","last_change":"2023-12-11T17:44:20.616004+0000","last_active":"2023-12-11T17:44:24.680820+0000","last_peered":"2023-12-11T17:44:24.680820+0000","last_clean":"2023-12-11T17:44:24.680820+0000","last_became_active":"2023-12-11T17:44:20.614985+0000","last_became_peered":"2023-12-11T17:44:20.614985+0000","last_unstale":"2023-12-11T17:44:24.680820+0000","last_undegraded":"2023-12-11T17:44:24.680820+0000","last_fullsized":"2023-12-11T17:44:24.680820+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T05:31:13.414818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'1","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T17:44:25.782150+0000","last_change":"2023-12-11T17:44:19.663092+0000","last_active":"2023-12-11T17:44:25.782150+0000","last_peered":"2023-12-11T17:44:25.782150+0000","last_clean":"2023-12-11T17:44:25.782150+0000","last_became_active":"2023-12-11T17:44:19.662728+0000","last_became_peered":"2023-12-11T17:44:19.662728+0000","last_unstale":"2023-12-11T17:44:25.782150+0000","last_undegraded":"2023-12-11T17:44:25.782150+0000","last_fullsized":"2023-12-11T17:44:25.782150+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_clean_scrub_stamp":"2023-12-11T17:44:18.636443+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T20:05:23.952470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T17:44:25.782684+0000","last_change":"2023-12-11T17:44:24.694194+0000","last_active":"2023-12-11T17:44:25.782684+0000","last_peered":"2023-12-11T17:44:25.782684+0000","last_clean":"2023-12-11T17:44:25.782684+0000","last_became_active":"2023-12-11T17:44:24.693860+0000","last_became_peered":"2023-12-11T17:44:24.693860+0000","last_unstale":"2023-12-11T17:44:25.782684+0000","last_undegraded":"2023-12-11T17:44:25.782684+0000","last_fullsized":"2023-12-11T17:44:25.782684+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_clean_scrub_stamp":"2023-12-11T17:44:23.663161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T01:16:04.877531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.679992+0000","last_change":"2023-12-11T17:44:20.615445+0000","last_active":"2023-12-11T17:44:24.679992+0000","last_peered":"2023-12-11T17:44:24.679992+0000","last_clean":"2023-12-11T17:44:24.679992+0000","last_became_active":"2023-12-11T17:44:20.614973+0000","last_became_peered":"2023-12-11T17:44:20.614973+0000","last_unstale":"2023-12-11T17:44:24.679992+0000","last_undegraded":"2023-12-11T17:44:24.679992+0000","last_fullsized":"2023-12-11T17:44:24.679992+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T21:51:31.923310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.687462+0000","last_change":"2023-12-11T17:44:19.651980+0000","last_active":"2023-12-11T17:44:24.687462+0000","last_peered":"2023-12-11T17:44:24.687462+0000","last_clean":"2023-12-11T17:44:24.687462+0000","last_became_active":"2023-12-11T17:44:19.651659+0000","last_became_peered":"2023-12-11T17:44:19.651659+0000","last_unstale":"2023-12-11T17:44:24.687462+0000","last_undegraded":"2023-12-11T17:44:24.687462+0000","last_fullsized":"2023-12-11T17:44:24.687462+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_clean_scrub_stamp":"2023-12-11T17:44:18.636443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T19:29:21.453758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":320,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.685586+0000","last_change":"2023-12-11T17:43:21.021919+0000","last_active":"2023-12-11T17:44:24.685586+0000","last_peered":"2023-12-11T17:44:24.685586+0000","last_clean":"2023-12-11T17:44:24.685586+0000","last_became_active":"2023-12-11T17:43:21.021489+0000","last_became_peered":"2023-12-11T17:43:21.021489+0000","last_unstale":"2023-12-11T17:44:24.685586+0000","last_undegraded":"2023-12-11T17:44:24.685586+0000","last_fullsized":"2023-12-11T17:44:24.685586+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:40:27.026768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:40:27.026768+0000","last_clean_scrub_stamp":"2023-12-11T17:40:27.026768+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T22:30:14.730794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T17:44:25.781461+0000","last_change":"2023-12-11T17:44:24.701996+0000","last_active":"2023-12-11T17:44:25.781461+0000","last_peered":"2023-12-11T17:44:25.781461+0000","last_clean":"2023-12-11T17:44:25.781461+0000","last_became_active":"2023-12-11T17:44:24.701574+0000","last_became_peered":"2023-12-11T17:44:24.701574+0000","last_unstale":"2023-12-11T17:44:25.781461+0000","last_undegraded":"2023-12-11T17:44:25.781461+0000","last_fullsized":"2023-12-11T17:44:25.781461+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_clean_scrub_stamp":"2023-12-11T17:44:23.663161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T19:49:01.458432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.674905+0000","last_change":"2023-12-11T17:44:20.614765+0000","last_active":"2023-12-11T17:44:24.674905+0000","last_peered":"2023-12-11T17:44:24.674905+0000","last_clean":"2023-12-11T17:44:24.674905+0000","last_became_active":"2023-12-11T17:44:20.614462+0000","last_became_peered":"2023-12-11T17:44:20.614462+0000","last_unstale":"2023-12-11T17:44:24.674905+0000","last_undegraded":"2023-12-11T17:44:24.674905+0000","last_fullsized":"2023-12-11T17:44:24.674905+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:08:20.784124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T17:44:46.774838+0000","last_change":"2023-12-11T17:44:19.661528+0000","last_active":"2023-12-11T17:44:46.774838+0000","last_peered":"2023-12-11T17:44:46.774838+0000","last_clean":"2023-12-11T17:44:46.774838+0000","last_became_active":"2023-12-11T17:44:19.661208+0000","last_became_peered":"2023-12-11T17:44:19.661208+0000","last_unstale":"2023-12-11T17:44:46.774838+0000","last_undegraded":"2023-12-11T17:44:46.774838+0000","last_fullsized":"2023-12-11T17:44:46.774838+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_clean_scrub_stamp":"2023-12-11T17:44:18.636443+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T18:31:23.295193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T17:44:25.781318+0000","last_change":"2023-12-11T17:44:24.688218+0000","last_active":"2023-12-11T17:44:25.781318+0000","last_peered":"2023-12-11T17:44:25.781318+0000","last_clean":"2023-12-11T17:44:25.781318+0000","last_became_active":"2023-12-11T17:44:24.687809+0000","last_became_peered":"2023-12-11T17:44:24.687809+0000","last_unstale":"2023-12-11T17:44:25.781318+0000","last_undegraded":"2023-12-11T17:44:25.781318+0000","last_fullsized":"2023-12-11T17:44:25.781318+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_clean_scrub_stamp":"2023-12-11T17:44:23.663161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T02:12:50.975614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T17:44:45.762595+0000","last_change":"2023-12-11T17:44:19.657797+0000","last_active":"2023-12-11T17:44:45.762595+0000","last_peered":"2023-12-11T17:44:45.762595+0000","last_clean":"2023-12-11T17:44:45.762595+0000","last_became_active":"2023-12-11T17:44:19.657483+0000","last_became_peered":"2023-12-11T17:44:19.657483+0000","last_unstale":"2023-12-11T17:44:45.762595+0000","last_undegraded":"2023-12-11T17:44:45.762595+0000","last_fullsized":"2023-12-11T17:44:45.762595+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_clean_scrub_stamp":"2023-12-11T17:44:18.636443+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T22:43:35.785943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.676427+0000","last_change":"2023-12-11T17:44:20.610002+0000","last_active":"2023-12-11T17:44:24.676427+0000","last_peered":"2023-12-11T17:44:24.676427+0000","last_clean":"2023-12-11T17:44:24.676427+0000","last_became_active":"2023-12-11T17:44:20.609579+0000","last_became_peered":"2023-12-11T17:44:20.609579+0000","last_unstale":"2023-12-11T17:44:24.676427+0000","last_undegraded":"2023-12-11T17:44:24.676427+0000","last_fullsized":"2023-12-11T17:44:24.676427+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:18:25.670403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T17:44:25.676923+0000","last_change":"2023-12-11T17:44:24.694707+0000","last_active":"2023-12-11T17:44:25.676923+0000","last_peered":"2023-12-11T17:44:25.676923+0000","last_clean":"2023-12-11T17:44:25.676923+0000","last_became_active":"2023-12-11T17:44:24.694275+0000","last_became_peered":"2023-12-11T17:44:24.694275+0000","last_unstale":"2023-12-11T17:44:25.676923+0000","last_undegraded":"2023-12-11T17:44:25.676923+0000","last_fullsized":"2023-12-11T17:44:25.676923+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_clean_scrub_stamp":"2023-12-11T17:44:23.663161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T18:28:12.922991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"86'3","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T17:44:46.775636+0000","last_change":"2023-12-11T17:44:19.658825+0000","last_active":"2023-12-11T17:44:46.775636+0000","last_peered":"2023-12-11T17:44:46.775636+0000","last_clean":"2023-12-11T17:44:46.775636+0000","last_became_active":"2023-12-11T17:44:19.658425+0000","last_became_peered":"2023-12-11T17:44:19.658425+0000","last_unstale":"2023-12-11T17:44:46.775636+0000","last_undegraded":"2023-12-11T17:44:46.775636+0000","last_fullsized":"2023-12-11T17:44:46.775636+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_clean_scrub_stamp":"2023-12-11T17:44:18.636443+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T04:06:15.289454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.680643+0000","last_change":"2023-12-11T17:44:20.617557+0000","last_active":"2023-12-11T17:44:24.680643+0000","last_peered":"2023-12-11T17:44:24.680643+0000","last_clean":"2023-12-11T17:44:24.680643+0000","last_became_active":"2023-12-11T17:44:20.616879+0000","last_became_peered":"2023-12-11T17:44:20.616879+0000","last_unstale":"2023-12-11T17:44:24.680643+0000","last_undegraded":"2023-12-11T17:44:24.680643+0000","last_fullsized":"2023-12-11T17:44:24.680643+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T19:34:04.954862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T17:44:25.781207+0000","last_change":"2023-12-11T17:44:24.691471+0000","last_active":"2023-12-11T17:44:25.781207+0000","last_peered":"2023-12-11T17:44:25.781207+0000","last_clean":"2023-12-11T17:44:25.781207+0000","last_became_active":"2023-12-11T17:44:24.690390+0000","last_became_peered":"2023-12-11T17:44:24.690390+0000","last_unstale":"2023-12-11T17:44:25.781207+0000","last_undegraded":"2023-12-11T17:44:25.781207+0000","last_fullsized":"2023-12-11T17:44:25.781207+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_clean_scrub_stamp":"2023-12-11T17:44:23.663161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T00:26:20.141717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"81'2","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.685191+0000","last_change":"2023-12-11T17:44:19.661004+0000","last_active":"2023-12-11T17:44:24.685191+0000","last_peered":"2023-12-11T17:44:24.685191+0000","last_clean":"2023-12-11T17:44:24.685191+0000","last_became_active":"2023-12-11T17:44:19.660691+0000","last_became_peered":"2023-12-11T17:44:19.660691+0000","last_unstale":"2023-12-11T17:44:24.685191+0000","last_undegraded":"2023-12-11T17:44:24.685191+0000","last_fullsized":"2023-12-11T17:44:24.685191+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_clean_scrub_stamp":"2023-12-11T17:44:18.636443+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T17:54:53.956473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.683776+0000","last_change":"2023-12-11T17:44:20.621437+0000","last_active":"2023-12-11T17:44:24.683776+0000","last_peered":"2023-12-11T17:44:24.683776+0000","last_clean":"2023-12-11T17:44:24.683776+0000","last_became_active":"2023-12-11T17:44:20.621113+0000","last_became_peered":"2023-12-11T17:44:20.621113+0000","last_unstale":"2023-12-11T17:44:24.683776+0000","last_undegraded":"2023-12-11T17:44:24.683776+0000","last_fullsized":"2023-12-11T17:44:24.683776+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T21:05:50.097605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T17:44:25.677532+0000","last_change":"2023-12-11T17:44:24.697432+0000","last_active":"2023-12-11T17:44:25.677532+0000","last_peered":"2023-12-11T17:44:25.677532+0000","last_clean":"2023-12-11T17:44:25.677532+0000","last_became_active":"2023-12-11T17:44:24.696924+0000","last_became_peered":"2023-12-11T17:44:24.696924+0000","last_unstale":"2023-12-11T17:44:25.677532+0000","last_undegraded":"2023-12-11T17:44:25.677532+0000","last_fullsized":"2023-12-11T17:44:25.677532+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_clean_scrub_stamp":"2023-12-11T17:44:23.663161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T21:00:01.967894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T17:44:45.763470+0000","last_change":"2023-12-11T17:44:19.665080+0000","last_active":"2023-12-11T17:44:45.763470+0000","last_peered":"2023-12-11T17:44:45.763470+0000","last_clean":"2023-12-11T17:44:45.763470+0000","last_became_active":"2023-12-11T17:44:19.664862+0000","last_became_peered":"2023-12-11T17:44:19.664862+0000","last_unstale":"2023-12-11T17:44:45.763470+0000","last_undegraded":"2023-12-11T17:44:45.763470+0000","last_fullsized":"2023-12-11T17:44:45.763470+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_clean_scrub_stamp":"2023-12-11T17:44:18.636443+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T20:22:05.909035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.679991+0000","last_change":"2023-12-11T17:44:20.614140+0000","last_active":"2023-12-11T17:44:24.679991+0000","last_peered":"2023-12-11T17:44:24.679991+0000","last_clean":"2023-12-11T17:44:24.679991+0000","last_became_active":"2023-12-11T17:44:20.613808+0000","last_became_peered":"2023-12-11T17:44:20.613808+0000","last_unstale":"2023-12-11T17:44:24.679991+0000","last_undegraded":"2023-12-11T17:44:24.679991+0000","last_fullsized":"2023-12-11T17:44:24.679991+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T20:55:57.572836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T17:44:25.781960+0000","last_change":"2023-12-11T17:44:24.697388+0000","last_active":"2023-12-11T17:44:25.781960+0000","last_peered":"2023-12-11T17:44:25.781960+0000","last_clean":"2023-12-11T17:44:25.781960+0000","last_became_active":"2023-12-11T17:44:24.696964+0000","last_became_peered":"2023-12-11T17:44:24.696964+0000","last_unstale":"2023-12-11T17:44:25.781960+0000","last_undegraded":"2023-12-11T17:44:25.781960+0000","last_fullsized":"2023-12-11T17:44:25.781960+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_clean_scrub_stamp":"2023-12-11T17:44:23.663161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T18:08:04.162075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T17:44:45.764439+0000","last_change":"2023-12-11T17:44:19.660327+0000","last_active":"2023-12-11T17:44:45.764439+0000","last_peered":"2023-12-11T17:44:45.764439+0000","last_clean":"2023-12-11T17:44:45.764439+0000","last_became_active":"2023-12-11T17:44:19.659999+0000","last_became_peered":"2023-12-11T17:44:19.659999+0000","last_unstale":"2023-12-11T17:44:45.764439+0000","last_undegraded":"2023-12-11T17:44:45.764439+0000","last_fullsized":"2023-12-11T17:44:45.764439+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_clean_scrub_stamp":"2023-12-11T17:44:18.636443+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T23:51:24.580854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.674733+0000","last_change":"2023-12-11T17:44:20.624126+0000","last_active":"2023-12-11T17:44:24.674733+0000","last_peered":"2023-12-11T17:44:24.674733+0000","last_clean":"2023-12-11T17:44:24.674733+0000","last_became_active":"2023-12-11T17:44:20.623836+0000","last_became_peered":"2023-12-11T17:44:20.623836+0000","last_unstale":"2023-12-11T17:44:24.674733+0000","last_undegraded":"2023-12-11T17:44:24.674733+0000","last_fullsized":"2023-12-11T17:44:24.674733+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T21:43:39.123299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T17:44:25.677271+0000","last_change":"2023-12-11T17:44:24.698994+0000","last_active":"2023-12-11T17:44:25.677271+0000","last_peered":"2023-12-11T17:44:25.677271+0000","last_clean":"2023-12-11T17:44:25.677271+0000","last_became_active":"2023-12-11T17:44:24.698583+0000","last_became_peered":"2023-12-11T17:44:24.698583+0000","last_unstale":"2023-12-11T17:44:25.677271+0000","last_undegraded":"2023-12-11T17:44:25.677271+0000","last_fullsized":"2023-12-11T17:44:25.677271+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_clean_scrub_stamp":"2023-12-11T17:44:23.663161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T00:03:20.178331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.686821+0000","last_change":"2023-12-11T17:44:19.662505+0000","last_active":"2023-12-11T17:44:24.686821+0000","last_peered":"2023-12-11T17:44:24.686821+0000","last_clean":"2023-12-11T17:44:24.686821+0000","last_became_active":"2023-12-11T17:44:19.661478+0000","last_became_peered":"2023-12-11T17:44:19.661478+0000","last_unstale":"2023-12-11T17:44:24.686821+0000","last_undegraded":"2023-12-11T17:44:24.686821+0000","last_fullsized":"2023-12-11T17:44:24.686821+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_clean_scrub_stamp":"2023-12-11T17:44:18.636443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T23:28:09.345693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.680169+0000","last_change":"2023-12-11T17:44:20.615026+0000","last_active":"2023-12-11T17:44:24.680169+0000","last_peered":"2023-12-11T17:44:24.680169+0000","last_clean":"2023-12-11T17:44:24.680169+0000","last_became_active":"2023-12-11T17:44:20.614669+0000","last_became_peered":"2023-12-11T17:44:20.614669+0000","last_unstale":"2023-12-11T17:44:24.680169+0000","last_undegraded":"2023-12-11T17:44:24.680169+0000","last_fullsized":"2023-12-11T17:44:24.680169+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T20:52:20.455326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T17:44:25.677459+0000","last_change":"2023-12-11T17:44:24.694681+0000","last_active":"2023-12-11T17:44:25.677459+0000","last_peered":"2023-12-11T17:44:25.677459+0000","last_clean":"2023-12-11T17:44:25.677459+0000","last_became_active":"2023-12-11T17:44:24.694172+0000","last_became_peered":"2023-12-11T17:44:24.694172+0000","last_unstale":"2023-12-11T17:44:25.677459+0000","last_undegraded":"2023-12-11T17:44:25.677459+0000","last_fullsized":"2023-12-11T17:44:25.677459+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_clean_scrub_stamp":"2023-12-11T17:44:23.663161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T20:54:46.367735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T17:44:45.763580+0000","last_change":"2023-12-11T17:44:19.664254+0000","last_active":"2023-12-11T17:44:45.763580+0000","last_peered":"2023-12-11T17:44:45.763580+0000","last_clean":"2023-12-11T17:44:45.763580+0000","last_became_active":"2023-12-11T17:44:19.663814+0000","last_became_peered":"2023-12-11T17:44:19.663814+0000","last_unstale":"2023-12-11T17:44:45.763580+0000","last_undegraded":"2023-12-11T17:44:45.763580+0000","last_fullsized":"2023-12-11T17:44:45.763580+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_clean_scrub_stamp":"2023-12-11T17:44:18.636443+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T20:11:23.717547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.686362+0000","last_change":"2023-12-11T17:44:20.621879+0000","last_active":"2023-12-11T17:44:24.686362+0000","last_peered":"2023-12-11T17:44:24.686362+0000","last_clean":"2023-12-11T17:44:24.686362+0000","last_became_active":"2023-12-11T17:44:20.621430+0000","last_became_peered":"2023-12-11T17:44:20.621430+0000","last_unstale":"2023-12-11T17:44:24.686362+0000","last_undegraded":"2023-12-11T17:44:24.686362+0000","last_fullsized":"2023-12-11T17:44:24.686362+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T18:08:46.359629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T17:44:25.782485+0000","last_change":"2023-12-11T17:44:24.700599+0000","last_active":"2023-12-11T17:44:25.782485+0000","last_peered":"2023-12-11T17:44:25.782485+0000","last_clean":"2023-12-11T17:44:25.782485+0000","last_became_active":"2023-12-11T17:44:24.700121+0000","last_became_peered":"2023-12-11T17:44:24.700121+0000","last_unstale":"2023-12-11T17:44:25.782485+0000","last_undegraded":"2023-12-11T17:44:25.782485+0000","last_fullsized":"2023-12-11T17:44:25.782485+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_clean_scrub_stamp":"2023-12-11T17:44:23.663161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T01:08:18.000642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T17:44:46.775265+0000","last_change":"2023-12-11T17:44:19.662414+0000","last_active":"2023-12-11T17:44:46.775265+0000","last_peered":"2023-12-11T17:44:46.775265+0000","last_clean":"2023-12-11T17:44:46.775265+0000","last_became_active":"2023-12-11T17:44:19.661753+0000","last_became_peered":"2023-12-11T17:44:19.661753+0000","last_unstale":"2023-12-11T17:44:46.775265+0000","last_undegraded":"2023-12-11T17:44:46.775265+0000","last_fullsized":"2023-12-11T17:44:46.775265+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_clean_scrub_stamp":"2023-12-11T17:44:18.636443+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T00:49:11.713347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.688473+0000","last_change":"2023-12-11T17:44:20.620973+0000","last_active":"2023-12-11T17:44:24.688473+0000","last_peered":"2023-12-11T17:44:24.688473+0000","last_clean":"2023-12-11T17:44:24.688473+0000","last_became_active":"2023-12-11T17:44:20.620284+0000","last_became_peered":"2023-12-11T17:44:20.620284+0000","last_unstale":"2023-12-11T17:44:24.688473+0000","last_undegraded":"2023-12-11T17:44:24.688473+0000","last_fullsized":"2023-12-11T17:44:24.688473+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:00:52.529062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T17:44:25.781009+0000","last_change":"2023-12-11T17:44:24.696888+0000","last_active":"2023-12-11T17:44:25.781009+0000","last_peered":"2023-12-11T17:44:25.781009+0000","last_clean":"2023-12-11T17:44:25.781009+0000","last_became_active":"2023-12-11T17:44:24.696475+0000","last_became_peered":"2023-12-11T17:44:24.696475+0000","last_unstale":"2023-12-11T17:44:25.781009+0000","last_undegraded":"2023-12-11T17:44:25.781009+0000","last_fullsized":"2023-12-11T17:44:25.781009+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_clean_scrub_stamp":"2023-12-11T17:44:23.663161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T21:42:17.287557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"81'1","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T17:44:25.782089+0000","last_change":"2023-12-11T17:44:19.659094+0000","last_active":"2023-12-11T17:44:25.782089+0000","last_peered":"2023-12-11T17:44:25.782089+0000","last_clean":"2023-12-11T17:44:25.782089+0000","last_became_active":"2023-12-11T17:44:19.658794+0000","last_became_peered":"2023-12-11T17:44:19.658794+0000","last_unstale":"2023-12-11T17:44:25.782089+0000","last_undegraded":"2023-12-11T17:44:25.782089+0000","last_fullsized":"2023-12-11T17:44:25.782089+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_clean_scrub_stamp":"2023-12-11T17:44:18.636443+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T01:39:41.151915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.687828+0000","last_change":"2023-12-11T17:44:20.623342+0000","last_active":"2023-12-11T17:44:24.687828+0000","last_peered":"2023-12-11T17:44:24.687828+0000","last_clean":"2023-12-11T17:44:24.687828+0000","last_became_active":"2023-12-11T17:44:20.622957+0000","last_became_peered":"2023-12-11T17:44:20.622957+0000","last_unstale":"2023-12-11T17:44:24.687828+0000","last_undegraded":"2023-12-11T17:44:24.687828+0000","last_fullsized":"2023-12-11T17:44:24.687828+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:25:00.910645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T17:44:25.782300+0000","last_change":"2023-12-11T17:44:24.690746+0000","last_active":"2023-12-11T17:44:25.782300+0000","last_peered":"2023-12-11T17:44:25.782300+0000","last_clean":"2023-12-11T17:44:25.782300+0000","last_became_active":"2023-12-11T17:44:24.690290+0000","last_became_peered":"2023-12-11T17:44:24.690290+0000","last_unstale":"2023-12-11T17:44:25.782300+0000","last_undegraded":"2023-12-11T17:44:25.782300+0000","last_fullsized":"2023-12-11T17:44:25.782300+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_clean_scrub_stamp":"2023-12-11T17:44:23.663161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T01:26:14.409832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"81'2","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.676412+0000","last_change":"2023-12-11T17:44:19.655775+0000","last_active":"2023-12-11T17:44:24.676412+0000","last_peered":"2023-12-11T17:44:24.676412+0000","last_clean":"2023-12-11T17:44:24.676412+0000","last_became_active":"2023-12-11T17:44:19.654360+0000","last_became_peered":"2023-12-11T17:44:19.654360+0000","last_unstale":"2023-12-11T17:44:24.676412+0000","last_undegraded":"2023-12-11T17:44:24.676412+0000","last_fullsized":"2023-12-11T17:44:24.676412+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_clean_scrub_stamp":"2023-12-11T17:44:18.636443+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T00:07:08.678389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.688216+0000","last_change":"2023-12-11T17:44:20.614898+0000","last_active":"2023-12-11T17:44:24.688216+0000","last_peered":"2023-12-11T17:44:24.688216+0000","last_clean":"2023-12-11T17:44:24.688216+0000","last_became_active":"2023-12-11T17:44:20.613972+0000","last_became_peered":"2023-12-11T17:44:20.613972+0000","last_unstale":"2023-12-11T17:44:24.688216+0000","last_undegraded":"2023-12-11T17:44:24.688216+0000","last_fullsized":"2023-12-11T17:44:24.688216+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T04:28:51.576955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.674777+0000","last_change":"2023-12-11T17:44:20.617502+0000","last_active":"2023-12-11T17:44:24.674777+0000","last_peered":"2023-12-11T17:44:24.674777+0000","last_clean":"2023-12-11T17:44:24.674777+0000","last_became_active":"2023-12-11T17:44:20.617252+0000","last_became_peered":"2023-12-11T17:44:20.617252+0000","last_unstale":"2023-12-11T17:44:24.674777+0000","last_undegraded":"2023-12-11T17:44:24.674777+0000","last_fullsized":"2023-12-11T17:44:24.674777+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T19:23:37.133765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.686677+0000","last_change":"2023-12-11T17:44:20.619325+0000","last_active":"2023-12-11T17:44:24.686677+0000","last_peered":"2023-12-11T17:44:24.686677+0000","last_clean":"2023-12-11T17:44:24.686677+0000","last_became_active":"2023-12-11T17:44:20.619016+0000","last_became_peered":"2023-12-11T17:44:20.619016+0000","last_unstale":"2023-12-11T17:44:24.686677+0000","last_undegraded":"2023-12-11T17:44:24.686677+0000","last_fullsized":"2023-12-11T17:44:24.686677+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T20:21:24.877379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.686636+0000","last_change":"2023-12-11T17:44:20.624432+0000","last_active":"2023-12-11T17:44:24.686636+0000","last_peered":"2023-12-11T17:44:24.686636+0000","last_clean":"2023-12-11T17:44:24.686636+0000","last_became_active":"2023-12-11T17:44:20.624148+0000","last_became_peered":"2023-12-11T17:44:20.624148+0000","last_unstale":"2023-12-11T17:44:24.686636+0000","last_undegraded":"2023-12-11T17:44:24.686636+0000","last_fullsized":"2023-12-11T17:44:24.686636+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T19:15:46.054968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.681296+0000","last_change":"2023-12-11T17:44:20.624134+0000","last_active":"2023-12-11T17:44:24.681296+0000","last_peered":"2023-12-11T17:44:24.681296+0000","last_clean":"2023-12-11T17:44:24.681296+0000","last_became_active":"2023-12-11T17:44:20.623858+0000","last_became_peered":"2023-12-11T17:44:20.623858+0000","last_unstale":"2023-12-11T17:44:24.681296+0000","last_undegraded":"2023-12-11T17:44:24.681296+0000","last_fullsized":"2023-12-11T17:44:24.681296+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:24:48.946619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.677112+0000","last_change":"2023-12-11T17:44:20.620407+0000","last_active":"2023-12-11T17:44:24.677112+0000","last_peered":"2023-12-11T17:44:24.677112+0000","last_clean":"2023-12-11T17:44:24.677112+0000","last_became_active":"2023-12-11T17:44:20.620106+0000","last_became_peered":"2023-12-11T17:44:20.620106+0000","last_unstale":"2023-12-11T17:44:24.677112+0000","last_undegraded":"2023-12-11T17:44:24.677112+0000","last_fullsized":"2023-12-11T17:44:24.677112+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T22:18:36.780037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.675040+0000","last_change":"2023-12-11T17:44:20.618426+0000","last_active":"2023-12-11T17:44:24.675040+0000","last_peered":"2023-12-11T17:44:24.675040+0000","last_clean":"2023-12-11T17:44:24.675040+0000","last_became_active":"2023-12-11T17:44:20.618136+0000","last_became_peered":"2023-12-11T17:44:20.618136+0000","last_unstale":"2023-12-11T17:44:24.675040+0000","last_undegraded":"2023-12-11T17:44:24.675040+0000","last_fullsized":"2023-12-11T17:44:24.675040+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T05:02:04.096866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.687279+0000","last_change":"2023-12-11T17:44:20.619609+0000","last_active":"2023-12-11T17:44:24.687279+0000","last_peered":"2023-12-11T17:44:24.687279+0000","last_clean":"2023-12-11T17:44:24.687279+0000","last_became_active":"2023-12-11T17:44:20.619345+0000","last_became_peered":"2023-12-11T17:44:20.619345+0000","last_unstale":"2023-12-11T17:44:24.687279+0000","last_undegraded":"2023-12-11T17:44:24.687279+0000","last_fullsized":"2023-12-11T17:44:24.687279+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T19:56:00.839819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.680739+0000","last_change":"2023-12-11T17:44:20.614115+0000","last_active":"2023-12-11T17:44:24.680739+0000","last_peered":"2023-12-11T17:44:24.680739+0000","last_clean":"2023-12-11T17:44:24.680739+0000","last_became_active":"2023-12-11T17:44:20.613800+0000","last_became_peered":"2023-12-11T17:44:20.613800+0000","last_unstale":"2023-12-11T17:44:24.680739+0000","last_undegraded":"2023-12-11T17:44:24.680739+0000","last_fullsized":"2023-12-11T17:44:24.680739+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T20:33:37.170611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.680726+0000","last_change":"2023-12-11T17:44:20.621737+0000","last_active":"2023-12-11T17:44:24.680726+0000","last_peered":"2023-12-11T17:44:24.680726+0000","last_clean":"2023-12-11T17:44:24.680726+0000","last_became_active":"2023-12-11T17:44:20.621320+0000","last_became_peered":"2023-12-11T17:44:20.621320+0000","last_unstale":"2023-12-11T17:44:24.680726+0000","last_undegraded":"2023-12-11T17:44:24.680726+0000","last_fullsized":"2023-12-11T17:44:24.680726+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T22:37:02.316528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.680302+0000","last_change":"2023-12-11T17:44:20.617195+0000","last_active":"2023-12-11T17:44:24.680302+0000","last_peered":"2023-12-11T17:44:24.680302+0000","last_clean":"2023-12-11T17:44:24.680302+0000","last_became_active":"2023-12-11T17:44:20.616529+0000","last_became_peered":"2023-12-11T17:44:20.616529+0000","last_unstale":"2023-12-11T17:44:24.680302+0000","last_undegraded":"2023-12-11T17:44:24.680302+0000","last_fullsized":"2023-12-11T17:44:24.680302+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T23:53:50.558483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.679469+0000","last_change":"2023-12-11T17:44:20.618290+0000","last_active":"2023-12-11T17:44:24.679469+0000","last_peered":"2023-12-11T17:44:24.679469+0000","last_clean":"2023-12-11T17:44:24.679469+0000","last_became_active":"2023-12-11T17:44:20.617982+0000","last_became_peered":"2023-12-11T17:44:20.617982+0000","last_unstale":"2023-12-11T17:44:24.679469+0000","last_undegraded":"2023-12-11T17:44:24.679469+0000","last_fullsized":"2023-12-11T17:44:24.679469+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T21:58:14.444462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.675268+0000","last_change":"2023-12-11T17:44:20.619958+0000","last_active":"2023-12-11T17:44:24.675268+0000","last_peered":"2023-12-11T17:44:24.675268+0000","last_clean":"2023-12-11T17:44:24.675268+0000","last_became_active":"2023-12-11T17:44:20.619504+0000","last_became_peered":"2023-12-11T17:44:20.619504+0000","last_unstale":"2023-12-11T17:44:24.675268+0000","last_undegraded":"2023-12-11T17:44:24.675268+0000","last_fullsized":"2023-12-11T17:44:24.675268+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T19:36:41.193000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.675091+0000","last_change":"2023-12-11T17:44:20.612953+0000","last_active":"2023-12-11T17:44:24.675091+0000","last_peered":"2023-12-11T17:44:24.675091+0000","last_clean":"2023-12-11T17:44:24.675091+0000","last_became_active":"2023-12-11T17:44:20.612618+0000","last_became_peered":"2023-12-11T17:44:20.612618+0000","last_unstale":"2023-12-11T17:44:24.675091+0000","last_undegraded":"2023-12-11T17:44:24.675091+0000","last_fullsized":"2023-12-11T17:44:24.675091+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T04:25:39.528227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":12288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":3542208,"data_compressed":211312,"data_compressed_allocated":270336,"data_compressed_original":3506176,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":7128,"num_objects":49,"num_object_clones":0,"num_object_copies":147,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":112,"num_write_kb":35,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":32,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":208896,"data_stored":20642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":99,"ondisk_log_size":99,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":77,"seq":330712481809,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":933888,"data_stored":464187,"data_compressed":11728,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 11 17:44:38 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.72599999999999998}]},{"osd":1,"last update":"Mon Dec 11 17:44:38 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.55000000000000004}]},{"osd":2,"last update":"Mon Dec 11 17:44:38 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.50900000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48599999999999999}]},{"osd":4,"last update":"Mon Dec 11 17:44:38 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.49199999999999999}]},{"osd":5,"last update":"Mon Dec 11 17:44:38 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.745}]},{"osd":6,"last update":"Mon Dec 11 17:44:38 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.448,"5min":1.448,"15min":1.448},"last":0.77200000000000002}]},{"osd":7,"last update":"Mon Dec 11 17:44:38 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.385,"5min":1.385,"15min":1.385},"last":0.61399999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80600000000000005}]},{"osd":9,"last update":"Mon Dec 11 17:44:38 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.5089999999999999,"5min":1.5089999999999999,"15min":1.5089999999999999},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.86899999999999999}]},{"osd":10,"last update":"Mon Dec 11 17:44:38 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.4870000000000001,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.5309999999999999,"5min":1.5309999999999999,"15min":1.5309999999999999},"last":0.85299999999999998}]}]},{"osd":10,"up_from":70,"seq":300647710739,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1019904,"data_stored":1055671,"data_compressed":47493,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36499999999999999}]},{"osd":1,"last update":"Mon Dec 11 17:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.59299999999999997}]},{"osd":2,"last update":"Mon Dec 11 17:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.63200000000000001}]},{"osd":3,"last update":"Mon Dec 11 17:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.49299999999999999}]},{"osd":4,"last update":"Mon Dec 11 17:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.46200000000000002}]},{"osd":5,"last update":"Mon Dec 11 17:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.68999999999999995}]},{"osd":6,"last update":"Mon Dec 11 17:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.70399999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56799999999999995}]},{"osd":8,"last update":"Mon Dec 11 17:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.327,"5min":1.327,"15min":1.327},"last":0.67200000000000004}]},{"osd":9,"last update":"Mon Dec 11 17:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.3979999999999999,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.71999999999999997}]},{"osd":11,"last update":"Mon Dec 11 17:44:40 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.745},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.75600000000000001}]}]},{"osd":9,"up_from":65,"seq":279172874265,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":461033,"data_compressed":11728,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 11 17:43:57 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.45300000000000001}]},{"osd":1,"last update":"Mon Dec 11 17:43:57 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.54800000000000004}]},{"osd":2,"last update":"Mon Dec 11 17:43:57 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.5},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.42599999999999999}]},{"osd":3,"last update":"Mon Dec 11 17:43:57 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.40799999999999997}]},{"osd":4,"last update":"Mon Dec 11 17:43:57 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.157,"5min":1.157,"15min":1.157},"last":0.68700000000000006}]},{"osd":5,"last update":"Mon Dec 11 17:43:57 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.748},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.442}]},{"osd":6,"last update":"Mon Dec 11 17:43:57 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.5289999999999999,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.44800000000000001}]},{"osd":7,"last update":"Mon Dec 11 17:43:57 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.224,"5min":1.224,"15min":1.224},"last":0.60999999999999999}]},{"osd":8,"last update":"Mon Dec 11 17:43:57 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.77600000000000002}]},{"osd":10,"last update":"Mon Dec 11 17:44:18 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.622},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.252,"5min":1.252,"15min":1.252},"last":0.79600000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81699999999999995}]}]},{"osd":8,"up_from":57,"seq":244813135902,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":954368,"data_stored":465049,"data_compressed":11728,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 11 17:44:37 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.54300000000000004,"15min":0.53600000000000003},"min":{"1min":0.36499999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.437},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.56200000000000006,"15min":0.56299999999999994},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.78000000000000003,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.46200000000000002}]},{"osd":1,"last update":"Mon Dec 11 17:44:37 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.57799999999999996,"15min":0.57599999999999996},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.94999999999999996,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.62,"5min":0.59699999999999998,"15min":0.59299999999999997},"min":{"1min":0.42799999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.69699999999999995}]},{"osd":2,"last update":"Mon Dec 11 17:44:37 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.437,"15min":0.42999999999999999},"min":{"1min":0.32000000000000001,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.503},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.47899999999999998,"15min":0.48199999999999998},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.68899999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.53300000000000003}]},{"osd":3,"last update":"Mon Dec 11 17:44:37 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.59599999999999997,"15min":0.59999999999999998},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.73099999999999998,"5min":1.077,"15min":1.077},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.57899999999999996,"15min":0.57599999999999996},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.80200000000000005,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.56899999999999995}]},{"osd":4,"last update":"Mon Dec 11 17:44:37 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.628,"15min":0.625},"min":{"1min":0.47299999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.87,"5min":1.141,"15min":1.141},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.621,"15min":0.61099999999999999},"min":{"1min":0.53000000000000003,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.0069999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.85199999999999998}]},{"osd":5,"last update":"Mon Dec 11 17:44:37 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.56299999999999994,"15min":0.56499999999999995},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.72799999999999998,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.59199999999999997,"15min":0.59799999999999998},"min":{"1min":0.35599999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.76300000000000001,"5min":1.482,"15min":1.482},"last":0.622}]},{"osd":6,"last update":"Mon Dec 11 17:44:37 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.64300000000000002,"15min":0.63400000000000001},"min":{"1min":0.45000000000000001,"5min":0.37,"15min":0.37},"max":{"1min":0.84599999999999997,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.751},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.68500000000000005,"15min":0.68300000000000005},"min":{"1min":0.53600000000000003,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.87,"5min":1.008,"15min":1.008},"last":0.77100000000000002}]},{"osd":7,"last update":"Mon Dec 11 17:44:37 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.68000000000000005,"15min":0.67300000000000004},"min":{"1min":0.48699999999999999,"5min":0.432,"15min":0.432},"max":{"1min":0.93300000000000005,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.73099999999999998,"15min":0.72499999999999998},"min":{"1min":0.55400000000000005,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.546,"5min":1.546,"15min":1.546},"last":0.81699999999999995}]},{"osd":9,"last update":"Mon Dec 11 17:43:59 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.80500000000000005}]},{"osd":10,"last update":"Mon Dec 11 17:44:17 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.88}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79200000000000004}]}]},{"osd":7,"up_from":50,"seq":214748364833,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":995328,"data_stored":1052375,"data_compressed":47493,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 11 17:44:13 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.54700000000000004,"15min":0.54300000000000004},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.44},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.55200000000000005,"15min":0.54800000000000004},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.42399999999999999}]},{"osd":1,"last update":"Mon Dec 11 17:44:13 2023","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.40999999999999998,"15min":0.40999999999999998},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.56799999999999995,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.441},{"interface":"front","average":{"1min":0.436,"5min":0.44500000000000001,"15min":0.44700000000000001},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.60399999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.55500000000000005}]},{"osd":2,"last update":"Mon Dec 11 17:44:13 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.41299999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.627,"5min":0.59899999999999998,"15min":0.59399999999999997},"min":{"1min":0.46200000000000002,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.69299999999999995}]},{"osd":3,"last update":"Mon Dec 11 17:44:13 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.56999999999999995,"15min":0.56399999999999995},"min":{"1min":0.38700000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.60699999999999998,"15min":0.60299999999999998},"min":{"1min":0.45600000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.67100000000000004}]},{"osd":4,"last update":"Mon Dec 11 17:44:13 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.53500000000000003,"15min":0.53100000000000003},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.503},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.48699999999999999,"15min":0.49099999999999999},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.60399999999999998}]},{"osd":5,"last update":"Mon Dec 11 17:44:13 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.625,"15min":0.625},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.67200000000000004,"15min":0.66800000000000004},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.65000000000000002}]},{"osd":6,"last update":"Mon Dec 11 17:44:13 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.503,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.63100000000000001,"15min":0.63500000000000001},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.71599999999999997}]},{"osd":8,"last update":"Mon Dec 11 17:44:37 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.68300000000000005,"15min":0.69199999999999995},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.88200000000000001,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.70299999999999996,"15min":0.70899999999999996},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.92500000000000004,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.85999999999999999}]},{"osd":9,"last update":"Mon Dec 11 17:43:57 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.80600000000000005}]},{"osd":10,"last update":"Mon Dec 11 17:44:19 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.75900000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88700000000000001}]}]},{"osd":6,"up_from":45,"seq":193273528358,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":929792,"data_stored":460215,"data_compressed":11728,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 11 17:43:54 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.39500000000000002,"15min":0.38500000000000001},"min":{"1min":0.30499999999999999,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.27400000000000002},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.41299999999999998,"15min":0.40699999999999997},"min":{"1min":0.314,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.32600000000000001}]},{"osd":1,"last update":"Mon Dec 11 17:43:54 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.54500000000000004,"15min":0.53900000000000003},"min":{"1min":0.371,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.55700000000000005,"15min":0.54700000000000004},"min":{"1min":0.41299999999999998,"5min":0.375,"15min":0.375},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.65400000000000003}]},{"osd":2,"last update":"Mon Dec 11 17:43:54 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.56799999999999995,"15min":0.55400000000000005},"min":{"1min":0.44400000000000001,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.107,"5min":1.107,"15min":1.107},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.57699999999999996,"15min":0.55600000000000005},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.236,"5min":1.236,"15min":1.236},"last":0.42799999999999999}]},{"osd":3,"last update":"Mon Dec 11 17:43:54 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.49399999999999999,"15min":0.48199999999999998},"min":{"1min":0.34899999999999998,"5min":0.317,"15min":0.317},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.5,"15min":0.48699999999999999},"min":{"1min":0.318,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.503}]},{"osd":4,"last update":"Mon Dec 11 17:43:54 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.61099999999999999,"15min":0.60299999999999998},"min":{"1min":0.39900000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.63700000000000001,"15min":0.629},"min":{"1min":0.42799999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.62}]},{"osd":5,"last update":"Mon Dec 11 17:43:54 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.63800000000000001,"15min":0.628},"min":{"1min":0.45900000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.66200000000000003,"15min":0.65000000000000002},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.58599999999999997}]},{"osd":7,"last update":"Mon Dec 11 17:44:12 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.67300000000000004,"15min":0.66400000000000003},"min":{"1min":0.47899999999999998,"5min":0.442,"15min":0.442},"max":{"1min":1.3169999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.71199999999999997,"15min":0.70799999999999996},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.68100000000000005}]},{"osd":8,"last update":"Mon Dec 11 17:44:40 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.752,"15min":0.751},"min":{"1min":0.54900000000000004,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.002,"5min":1.157,"15min":1.157},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.69899999999999995,"15min":0.69699999999999995},"min":{"1min":0.495,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.97399999999999998,"5min":1.028,"15min":1.028},"last":0.47599999999999998}]},{"osd":9,"last update":"Mon Dec 11 17:43:56 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.371,"5min":1.371,"15min":1.371},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.45000000000000001}]},{"osd":10,"last update":"Mon Dec 11 17:44:19 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.425,"5min":1.425,"15min":1.425},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.288,"5min":1.288,"15min":1.288},"last":0.69999999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56200000000000006}]}]},{"osd":1,"up_from":13,"seq":55834574907,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":942080,"data_stored":464056,"data_compressed":11728,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 11 17:44:13 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.498,"15min":0.48099999999999998},"min":{"1min":0.34200000000000003,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.73799999999999999,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.52900000000000003,"15min":0.51200000000000001},"min":{"1min":0.40600000000000003,"5min":0.308,"15min":0.308},"max":{"1min":0.75600000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.80200000000000005}]},{"osd":2,"last update":"Mon Dec 11 17:44:30 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.54800000000000004,"15min":0.51800000000000002},"min":{"1min":0.34799999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":1.4690000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.94099999999999995},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.57099999999999995,"15min":0.54200000000000004},"min":{"1min":0.41699999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.4530000000000001,"5min":1.4530000000000001,"15min":1.4530000000000001},"last":0.66500000000000004}]},{"osd":3,"last update":"Mon Dec 11 17:43:53 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.51900000000000002,"15min":0.51500000000000001},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.94699999999999995,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.53000000000000003,"15min":0.51400000000000001},"min":{"1min":0.36199999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.81699999999999995}]},{"osd":4,"last update":"Mon Dec 11 17:44:13 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.42499999999999999,"15min":0.40200000000000002},"min":{"1min":0.28999999999999998,"5min":0.254,"15min":0.254},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.42799999999999999,"15min":0.39600000000000002},"min":{"1min":0.27700000000000002,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.53600000000000003}]},{"osd":5,"last update":"Mon Dec 11 17:44:36 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.59899999999999998,"15min":0.56299999999999994},"min":{"1min":0.41699999999999998,"5min":0.37,"15min":0.37},"max":{"1min":1.5109999999999999,"5min":1.5109999999999999,"15min":1.5109999999999999},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.60699999999999998,"15min":0.57599999999999996},"min":{"1min":0.53900000000000003,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.411,"5min":1.411,"15min":1.411},"last":0.85299999999999998}]},{"osd":6,"last update":"Mon Dec 11 17:43:53 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.59299999999999997,"15min":0.58699999999999997},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.629,"5min":0.60899999999999999,"15min":0.60499999999999998},"min":{"1min":0.38800000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.86699999999999999}]},{"osd":7,"last update":"Mon Dec 11 17:44:13 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.58499999999999996,"15min":0.57999999999999996},"min":{"1min":0.34499999999999997,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.996,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.58999999999999997,"15min":0.58899999999999997},"min":{"1min":0.435,"5min":0.309,"15min":0.309},"max":{"1min":0.94799999999999995,"5min":1.379,"15min":1.379},"last":0.61699999999999999}]},{"osd":8,"last update":"Mon Dec 11 17:44:36 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.71699999999999997,"15min":0.70699999999999996},"min":{"1min":0.48699999999999999,"5min":0.433,"15min":0.433},"max":{"1min":1.5349999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.755,"5min":0.69399999999999995,"15min":0.68400000000000005},"min":{"1min":0.48999999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.4259999999999999,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":0.78900000000000003}]},{"osd":9,"last update":"Mon Dec 11 17:44:00 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.97799999999999998}]},{"osd":10,"last update":"Mon Dec 11 17:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.69899999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91700000000000004}]}]},{"osd":0,"up_from":8,"seq":34359738431,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":933888,"data_stored":460305,"data_compressed":11728,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Dec 11 17:44:14 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.51000000000000001,"15min":0.502},"min":{"1min":0.34499999999999997,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.52300000000000002,"15min":0.48999999999999999},"min":{"1min":0.34200000000000003,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.86799999999999999}]},{"osd":2,"last update":"Mon Dec 11 17:44:38 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.56799999999999995,"15min":0.51500000000000001},"min":{"1min":0.38200000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":4.1029999999999998,"5min":4.1029999999999998,"15min":4.1029999999999998},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.55100000000000005,"15min":0.51100000000000001},"min":{"1min":0.33400000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.746,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.90200000000000002}]},{"osd":3,"last update":"Mon Dec 11 17:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.38800000000000001,"15min":0.38},"min":{"1min":0.30199999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.433,"15min":0.42299999999999999},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.38600000000000001}]},{"osd":4,"last update":"Mon Dec 11 17:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.53400000000000003,"15min":0.51800000000000002},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.81999999999999995,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.625,"5min":0.55400000000000005,"15min":0.54000000000000004},"min":{"1min":0.36499999999999999,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.626}]},{"osd":5,"last update":"Mon Dec 11 17:44:38 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.60599999999999998,"15min":0.58799999999999997},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":2.399,"5min":2.399,"15min":2.399},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.61299999999999999,"15min":0.59599999999999997},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.82699999999999996,"5min":0.995,"15min":0.995},"last":0.95199999999999996}]},{"osd":6,"last update":"Mon Dec 11 17:43:55 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.50900000000000001,"15min":0.499},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.59799999999999998,"15min":0.58599999999999997},"min":{"1min":0.34599999999999997,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.57499999999999996}]},{"osd":7,"last update":"Mon Dec 11 17:44:14 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.68000000000000005,"15min":0.67200000000000004},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.63,"15min":0.624},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.65000000000000002}]},{"osd":8,"last update":"Mon Dec 11 17:44:38 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.70299999999999996,"15min":0.68999999999999995},"min":{"1min":0.43099999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":3.3450000000000002,"5min":3.3450000000000002,"15min":3.3450000000000002},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.76100000000000001,"15min":0.76900000000000002},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.054,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.79700000000000004}]},{"osd":9,"last update":"Mon Dec 11 17:44:00 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.505},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.73199999999999998}]},{"osd":10,"last update":"Mon Dec 11 17:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.92700000000000005},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.76000000000000001}]},{"osd":11,"last update":"Mon Dec 11 17:44:42 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":2.456,"5min":2.456,"15min":2.456},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.107,"5min":1.107,"15min":1.107},"last":0.97899999999999998}]}]},{"osd":2,"up_from":18,"seq":77309411383,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":966656,"data_stored":465406,"data_compressed":11728,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 11 17:44:34 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.52900000000000003,"15min":0.502},"min":{"1min":0.32100000000000001,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.84899999999999998,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.51800000000000002,"15min":0.50800000000000001},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.79200000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.45300000000000001}]},{"osd":1,"last update":"Mon Dec 11 17:44:34 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.54100000000000004,"15min":0.51900000000000002},"min":{"1min":0.38900000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.56999999999999995,"15min":0.54100000000000004},"min":{"1min":0.35399999999999998,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.70099999999999996}]},{"osd":3,"last update":"Mon Dec 11 17:44:00 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.55600000000000005,"15min":0.53600000000000003},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.53400000000000003,"15min":0.52500000000000002},"min":{"1min":0.40600000000000003,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.67900000000000005}]},{"osd":4,"last update":"Mon Dec 11 17:44:13 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.54700000000000004,"15min":0.54000000000000004},"min":{"1min":0.38300000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.76200000000000001,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.58399999999999996,"15min":0.56899999999999995},"min":{"1min":0.41999999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.60299999999999998}]},{"osd":5,"last update":"Mon Dec 11 17:44:34 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.46800000000000003,"15min":0.44600000000000001},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.46000000000000002,"15min":0.441},"min":{"1min":0.28299999999999997,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.51800000000000002}]},{"osd":6,"last update":"Mon Dec 11 17:43:52 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.64100000000000001,"15min":0.63200000000000001},"min":{"1min":0.46100000000000002,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.60899999999999999,"15min":0.60499999999999998},"min":{"1min":0.48799999999999999,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.81799999999999995,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.56599999999999995}]},{"osd":7,"last update":"Mon Dec 11 17:44:17 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.60199999999999998,"15min":0.59799999999999998},"min":{"1min":0.46100000000000002,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.68100000000000005,"15min":0.67500000000000004},"min":{"1min":0.45600000000000002,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.749}]},{"osd":8,"last update":"Mon Dec 11 17:44:40 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.58199999999999996,"15min":0.57499999999999996},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.627},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.64600000000000002,"15min":0.64100000000000001},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.81799999999999995}]},{"osd":9,"last update":"Mon Dec 11 17:44:00 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.89500000000000002}]},{"osd":10,"last update":"Mon Dec 11 17:44:18 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.875}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78000000000000003}]}]},{"osd":3,"up_from":25,"seq":107374182450,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":942080,"data_stored":460168,"data_compressed":11728,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 11 17:43:53 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.376,"15min":0.35499999999999998},"min":{"1min":0.29399999999999998,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.433,"15min":0.42099999999999999},"min":{"1min":0.29599999999999999,"5min":0.251,"15min":0.251},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.51600000000000001}]},{"osd":1,"last update":"Mon Dec 11 17:43:53 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.57299999999999995,"15min":0.55700000000000005},"min":{"1min":0.42499999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.57799999999999996,"15min":0.56599999999999995},"min":{"1min":0.443,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.73199999999999998}]},{"osd":2,"last update":"Mon Dec 11 17:43:53 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.60999999999999999,"15min":0.60499999999999998},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.60299999999999998,"15min":0.60099999999999998},"min":{"1min":0.47799999999999998,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.92700000000000005}]},{"osd":4,"last update":"Mon Dec 11 17:44:19 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.58399999999999996,"15min":0.56499999999999995},"min":{"1min":0.435,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.99399999999999999},{"interface":"front","average":{"1min":0.627,"5min":0.55800000000000005,"15min":0.55200000000000005},"min":{"1min":0.41899999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.67400000000000004}]},{"osd":5,"last update":"Mon Dec 11 17:44:35 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.61299999999999999,"15min":0.58899999999999997},"min":{"1min":0.432,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":1.0069999999999999},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.64100000000000001,"15min":0.61799999999999999},"min":{"1min":0.41099999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.93100000000000005,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.63}]},{"osd":6,"last update":"Mon Dec 11 17:43:53 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.48299999999999998,"15min":0.46700000000000003},"min":{"1min":0.318,"5min":0.316,"15min":0.316},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.58899999999999997,"15min":0.55700000000000005},"min":{"1min":0.498,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.88100000000000001}]},{"osd":7,"last update":"Mon Dec 11 17:44:13 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.67300000000000004,"15min":0.65100000000000002},"min":{"1min":0.58799999999999997,"5min":0.432,"15min":0.432},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.94799999999999995},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.68899999999999995,"15min":0.67100000000000004},"min":{"1min":0.55900000000000005,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.96799999999999997}]},{"osd":8,"last update":"Mon Dec 11 17:43:38 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.66900000000000004}]},{"osd":9,"last update":"Mon Dec 11 17:43:57 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.751}]},{"osd":10,"last update":"Mon Dec 11 17:44:19 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.98299999999999998},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":1.024}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90600000000000003}]}]},{"osd":4,"up_from":32,"seq":137438953518,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":933888,"data_stored":460237,"data_compressed":11728,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 11 17:44:15 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.52800000000000002,"15min":0.51600000000000001},"min":{"1min":0.36899999999999999,"5min":0.31,"15min":0.31},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.51800000000000002,"15min":0.50700000000000001},"min":{"1min":0.35599999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.34399999999999997}]},{"osd":1,"last update":"Mon Dec 11 17:44:15 2023","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.39300000000000002,"15min":0.38500000000000001},"min":{"1min":0.28699999999999998,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.59399999999999997,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.39500000000000002,"15min":0.375},"min":{"1min":0.26000000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.434}]},{"osd":2,"last update":"Mon Dec 11 17:44:15 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.57499999999999996,"15min":0.55500000000000005},"min":{"1min":0.36099999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.58599999999999997,"15min":0.57999999999999996},"min":{"1min":0.41599999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.61299999999999999}]},{"osd":3,"last update":"Mon Dec 11 17:44:15 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.59799999999999998,"15min":0.58299999999999996},"min":{"1min":0.40000000000000002,"5min":0.372,"15min":0.372},"max":{"1min":0.82699999999999996,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.60499999999999998,"15min":0.58999999999999997},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.71499999999999997}]},{"osd":5,"last update":"Mon Dec 11 17:44:36 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.59499999999999997,"15min":0.58799999999999997},"min":{"1min":0.39500000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.57899999999999996,"15min":0.57399999999999995},"min":{"1min":0.38,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.82699999999999996,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.78500000000000003}]},{"osd":6,"last update":"Mon Dec 11 17:43:59 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.63100000000000001,"15min":0.61899999999999999},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.64400000000000002,"15min":0.63600000000000001},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.80100000000000005}]},{"osd":7,"last update":"Mon Dec 11 17:44:15 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.54500000000000004,"15min":0.54000000000000004},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.54400000000000004,"15min":0.53500000000000003},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.64700000000000002}]},{"osd":8,"last update":"Mon Dec 11 17:44:39 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.68400000000000005,"15min":0.67800000000000005},"min":{"1min":0.46200000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.66100000000000003,"15min":0.65500000000000003},"min":{"1min":0.44700000000000001,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.88300000000000001,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.83899999999999997}]},{"osd":9,"last update":"Mon Dec 11 17:43:59 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.85299999999999998}]},{"osd":10,"last update":"Mon Dec 11 17:44:17 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.76700000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75}]}]},{"osd":5,"up_from":37,"seq":158913789994,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1011712,"data_stored":1056544,"data_compressed":47493,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 11 17:44:31 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.501,"15min":0.497},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.66000000000000003,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.53000000000000003,"15min":0.52200000000000002},"min":{"1min":0.36599999999999999,"5min":0.314,"15min":0.314},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.70099999999999996}]},{"osd":1,"last update":"Mon Dec 11 17:44:31 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53200000000000003,"15min":0.52300000000000002},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.79600000000000004,"5min":0.872,"15min":0.872},"last":0.63},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.54100000000000004,"15min":0.53300000000000003},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.71799999999999997,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.65500000000000003}]},{"osd":2,"last update":"Mon Dec 11 17:44:31 2023","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.375,"15min":0.36399999999999999},"min":{"1min":0.28599999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.59899999999999998,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.377,"5min":0.40600000000000003,"15min":0.40400000000000003},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.57199999999999995,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.47899999999999998}]},{"osd":3,"last update":"Mon Dec 11 17:44:31 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.54700000000000004,"15min":0.53300000000000003},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.67000000000000004,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.56899999999999995,"15min":0.55800000000000005},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.78700000000000003,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.81399999999999995}]},{"osd":4,"last update":"Mon Dec 11 17:44:31 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.53600000000000003,"15min":0.52600000000000002},"min":{"1min":0.39200000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.71199999999999997,"5min":0.752,"15min":0.752},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58299999999999996,"15min":0.57199999999999995},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.88700000000000001,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.56399999999999995}]},{"osd":6,"last update":"Mon Dec 11 17:43:53 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.624,"15min":0.61599999999999999},"min":{"1min":0.49099999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.67400000000000004,"15min":0.66600000000000004},"min":{"1min":0.495,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.871}]},{"osd":7,"last update":"Mon Dec 11 17:44:14 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.63200000000000001,"15min":0.63},"min":{"1min":0.437,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61199999999999999,"15min":0.61099999999999999},"min":{"1min":0.44700000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.64300000000000002}]},{"osd":8,"last update":"Mon Dec 11 17:44:36 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.59199999999999997,"15min":0.60299999999999998},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.79500000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.628,"5min":0.64300000000000002,"15min":0.64500000000000002},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.80200000000000005,"5min":1.012,"15min":1.012},"last":0.79900000000000004}]},{"osd":9,"last update":"Mon Dec 11 17:43:59 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.246,"5min":1.246,"15min":1.246},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":0.89500000000000002}]},{"osd":10,"last update":"Mon Dec 11 17:44:17 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.82599999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34126,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34126,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35765,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35765,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35765,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35765,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":5191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-11T17:44:50.557 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-11T17:44:50.718 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:50.712+0000 7f172c573700 1 -- 172.21.15.59:0/4252257062 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1724101570 msgr2=0x7f17241019d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:50.718 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:50.712+0000 7f172c573700 1 --2- 172.21.15.59:0/4252257062 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1724101570 0x7f17241019d0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f17140099c0 tx=0x7f1714009cd0 comp rx=0 tx=0).stop 2023-12-11T17:44:50.718 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:50.713+0000 7f172c573700 1 -- 172.21.15.59:0/4252257062 shutdown_connections 2023-12-11T17:44:50.718 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:50.713+0000 7f172c573700 1 --2- 172.21.15.59:0/4252257062 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1724101f10 0x7f17241087c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:50.718 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:50.713+0000 7f172c573700 1 --2- 172.21.15.59:0/4252257062 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1724100380 0x7f17241007a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:50.718 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:50.713+0000 7f172c573700 1 --2- 172.21.15.59:0/4252257062 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1724101570 0x7f17241019d0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:50.718 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:50.713+0000 7f172c573700 1 -- 172.21.15.59:0/4252257062 >> 172.21.15.59:0/4252257062 conn(0x7f17240fb8f0 msgr2=0x7f17240fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:44:50.719 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:50.713+0000 7f172c573700 1 -- 172.21.15.59:0/4252257062 shutdown_connections 2023-12-11T17:44:50.719 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:50.713+0000 7f172c573700 1 -- 172.21.15.59:0/4252257062 wait complete. 2023-12-11T17:44:50.719 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:50.713+0000 7f172c573700 1 Processor -- start 2023-12-11T17:44:50.719 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:50.713+0000 7f172c573700 1 -- start start 2023-12-11T17:44:50.719 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:50.713+0000 7f172c573700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1724100380 0x7f17241042c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:50.719 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:50.713+0000 7f172c573700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1724101570 0x7f1724102950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:50.719 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:50.713+0000 7f172c573700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1724101f10 0x7f1724102e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:50.719 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:50.713+0000 7f172c573700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1724103630 con 0x7f1724100380 2023-12-11T17:44:50.719 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:50.713+0000 7f172c573700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17241037a0 con 0x7f1724101570 2023-12-11T17:44:50.719 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:50.713+0000 7f172c573700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17241a1470 con 0x7f1724101f10 2023-12-11T17:44:50.719 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:50.714+0000 7f172a30f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1724100380 0x7f17241042c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:50.719 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:50.714+0000 7f172a30f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1724100380 0x7f17241042c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33460/0 (socket says 172.21.15.59:33460) 2023-12-11T17:44:50.719 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:50.714+0000 7f172a30f700 1 -- 172.21.15.59:0/939935727 learned_addr learned my addr 172.21.15.59:0/939935727 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:44:50.719 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:50.714+0000 7f172ab10700 1 --2- 172.21.15.59:0/939935727 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1724101f10 0x7f1724102e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:50.720 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:50.714+0000 7f1729b0e700 1 --2- 172.21.15.59:0/939935727 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1724101570 0x7f1724102950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:50.720 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:50.714+0000 7f172a30f700 1 -- 172.21.15.59:0/939935727 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1724101f10 msgr2=0x7f1724102e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:50.720 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:50.714+0000 7f172a30f700 1 --2- 172.21.15.59:0/939935727 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1724101f10 0x7f1724102e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:50.720 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:50.714+0000 7f172a30f700 1 -- 172.21.15.59:0/939935727 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1724101570 msgr2=0x7f1724102950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:50.720 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:50.714+0000 7f172a30f700 1 --2- 172.21.15.59:0/939935727 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1724101570 0x7f1724102950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:50.720 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:50.714+0000 7f172a30f700 1 -- 172.21.15.59:0/939935727 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17140095f0 con 0x7f1724100380 2023-12-11T17:44:50.720 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:50.714+0000 7f1729b0e700 1 --2- 172.21.15.59:0/939935727 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1724101570 0x7f1724102950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-11T17:44:50.720 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:50.714+0000 7f172a30f700 1 --2- 172.21.15.59:0/939935727 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1724100380 0x7f17241042c0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f172000d980 tx=0x7f172000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:44:50.720 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:50.714+0000 7f171b7fe700 1 -- 172.21.15.59:0/939935727 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f172000de60 con 0x7f1724100380 2023-12-11T17:44:50.720 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:50.714+0000 7f172c573700 1 -- 172.21.15.59:0/939935727 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17241a1700 con 0x7f1724100380 2023-12-11T17:44:50.720 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:50.714+0000 7f171b7fe700 1 -- 172.21.15.59:0/939935727 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f17200044e0 con 0x7f1724100380 2023-12-11T17:44:50.720 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:50.714+0000 7f171b7fe700 1 -- 172.21.15.59:0/939935727 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1720003b40 con 0x7f1724100380 2023-12-11T17:44:50.721 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:50.714+0000 7f172c573700 1 -- 172.21.15.59:0/939935727 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17241a1c10 con 0x7f1724100380 2023-12-11T17:44:50.721 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:50.715+0000 7f172c573700 1 -- 172.21.15.59:0/939935727 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1724068230 con 0x7f1724100380 2023-12-11T17:44:50.721 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:50.717+0000 7f171b7fe700 1 -- 172.21.15.59:0/939935727 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f1720004870 con 0x7f1724100380 2023-12-11T17:44:50.721 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:50.717+0000 7f171b7fe700 1 --2- 172.21.15.59:0/939935727 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f17100a1480 0x7f17100a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:50.721 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:50.717+0000 7f171b7fe700 1 -- 172.21.15.59:0/939935727 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f17200cd0d0 con 0x7f1724100380 2023-12-11T17:44:50.722 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:50.717+0000 7f1729b0e700 1 --2- 172.21.15.59:0/939935727 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f17100a1480 0x7f17100a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:50.722 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:50.718+0000 7f1729b0e700 1 --2- 172.21.15.59:0/939935727 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f17100a1480 0x7f17100a3940 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1714009fd0 tx=0x7f1714009530 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:44:50.724 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:50.720+0000 7f171b7fe700 1 -- 172.21.15.59:0/939935727 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f17200964e0 con 0x7f1724100380 2023-12-11T17:44:50.791 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:50.787+0000 7f171b7fe700 1 -- 172.21.15.59:0/939935727 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f17200968b0 con 0x7f1724100380 2023-12-11T17:44:50.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:50 smithi059 ceph-mon[98847]: from='client.14871 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T17:44:50.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:50 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/1062268508' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T17:44:50.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:50 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/2579014155' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-11T17:44:50.929 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:50.925+0000 7f172c573700 1 -- 172.21.15.59:0/939935727 --> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f17240fdb40 con 0x7f17100a1480 2023-12-11T17:44:50.944 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:50.940+0000 7f171b7fe700 1 -- 172.21.15.59:0/939935727 <== mgr.14150 v2:172.21.15.59:6800/3627728353 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+422599 (secure 0 0 0) 0x7f17240fdb40 con 0x7f17100a1480 2023-12-11T17:44:50.949 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-12-11T17:44:50.952 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:50.944+0000 7f172c573700 1 -- 172.21.15.59:0/939935727 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f17100a1480 msgr2=0x7f17100a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:50.952 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:50.944+0000 7f172c573700 1 --2- 172.21.15.59:0/939935727 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f17100a1480 0x7f17100a3940 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1714009fd0 tx=0x7f1714009530 comp rx=0 tx=0).stop 2023-12-11T17:44:50.952 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:50.944+0000 7f172c573700 1 -- 172.21.15.59:0/939935727 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1724100380 msgr2=0x7f17241042c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:50.953 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:50.944+0000 7f172c573700 1 --2- 172.21.15.59:0/939935727 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1724100380 0x7f17241042c0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f172000d980 tx=0x7f172000dd40 comp rx=0 tx=0).stop 2023-12-11T17:44:50.953 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:50.944+0000 7f172c573700 1 -- 172.21.15.59:0/939935727 shutdown_connections 2023-12-11T17:44:50.953 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:50.944+0000 7f172c573700 1 --2- 172.21.15.59:0/939935727 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f17100a1480 0x7f17100a3940 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:50.953 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:50.944+0000 7f172c573700 1 --2- 172.21.15.59:0/939935727 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1724101f10 0x7f1724102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:50.953 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:50.944+0000 7f172c573700 1 --2- 172.21.15.59:0/939935727 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1724100380 0x7f17241042c0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:50.953 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:50.944+0000 7f172c573700 1 --2- 172.21.15.59:0/939935727 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1724101570 0x7f1724102950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:50.953 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:50.944+0000 7f172c573700 1 -- 172.21.15.59:0/939935727 >> 172.21.15.59:0/939935727 conn(0x7f17240fb8f0 msgr2=0x7f17240fd430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:44:50.953 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:50.944+0000 7f172c573700 1 -- 172.21.15.59:0/939935727 shutdown_connections 2023-12-11T17:44:50.953 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:50.944+0000 7f172c573700 1 -- 172.21.15.59:0/939935727 wait complete. 2023-12-11T17:44:50.953 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2023-12-11T17:44:50.960 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":274,"stamp":"2023-12-11T17:44:50.118430+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":597496,"num_objects":51,"num_object_clones":0,"num_object_copies":153,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":148,"num_read_kb":129,"num_write":293,"num_write_kb":2973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":32,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":236,"ondisk_log_size":236,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":335448,"kb_used_data":11244,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124606376,"statfs":{"total":1151940427776,"available":1151596929024,"internally_reserved":0,"allocated":11513856,"data_stored":7325246,"data_compressed":248031,"data_compressed_allocated":331776,"data_compressed_original":2162688,"omap_allocated":19142,"internal_metadata":331855162},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2489,"num_objects":27,"num_object_clones":0,"num_object_copies":81,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":63,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003835"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.675453+0000","last_change":"2023-12-11T17:44:20.624256+0000","last_active":"2023-12-11T17:44:24.675453+0000","last_peered":"2023-12-11T17:44:24.675453+0000","last_clean":"2023-12-11T17:44:24.675453+0000","last_became_active":"2023-12-11T17:44:20.623947+0000","last_became_peered":"2023-12-11T17:44:20.623947+0000","last_unstale":"2023-12-11T17:44:24.675453+0000","last_undegraded":"2023-12-11T17:44:24.675453+0000","last_fullsized":"2023-12-11T17:44:24.675453+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T18:12:43.669849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.683681+0000","last_change":"2023-12-11T17:44:20.620606+0000","last_active":"2023-12-11T17:44:24.683681+0000","last_peered":"2023-12-11T17:44:24.683681+0000","last_clean":"2023-12-11T17:44:24.683681+0000","last_became_active":"2023-12-11T17:44:20.620265+0000","last_became_peered":"2023-12-11T17:44:20.620265+0000","last_unstale":"2023-12-11T17:44:24.683681+0000","last_undegraded":"2023-12-11T17:44:24.683681+0000","last_fullsized":"2023-12-11T17:44:24.683681+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T19:53:35.349842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.680469+0000","last_change":"2023-12-11T17:44:20.620702+0000","last_active":"2023-12-11T17:44:24.680469+0000","last_peered":"2023-12-11T17:44:24.680469+0000","last_clean":"2023-12-11T17:44:24.680469+0000","last_became_active":"2023-12-11T17:44:20.620401+0000","last_became_peered":"2023-12-11T17:44:20.620401+0000","last_unstale":"2023-12-11T17:44:24.680469+0000","last_undegraded":"2023-12-11T17:44:24.680469+0000","last_fullsized":"2023-12-11T17:44:24.680469+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T21:14:21.156270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.687595+0000","last_change":"2023-12-11T17:44:20.614263+0000","last_active":"2023-12-11T17:44:24.687595+0000","last_peered":"2023-12-11T17:44:24.687595+0000","last_clean":"2023-12-11T17:44:24.687595+0000","last_became_active":"2023-12-11T17:44:20.613585+0000","last_became_peered":"2023-12-11T17:44:20.613585+0000","last_unstale":"2023-12-11T17:44:24.687595+0000","last_undegraded":"2023-12-11T17:44:24.687595+0000","last_fullsized":"2023-12-11T17:44:24.687595+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T23:03:26.514011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.680122+0000","last_change":"2023-12-11T17:44:20.615553+0000","last_active":"2023-12-11T17:44:24.680122+0000","last_peered":"2023-12-11T17:44:24.680122+0000","last_clean":"2023-12-11T17:44:24.680122+0000","last_became_active":"2023-12-11T17:44:20.615272+0000","last_became_peered":"2023-12-11T17:44:20.615272+0000","last_unstale":"2023-12-11T17:44:24.680122+0000","last_undegraded":"2023-12-11T17:44:24.680122+0000","last_fullsized":"2023-12-11T17:44:24.680122+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T20:37:02.178411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.695048+0000","last_change":"2023-12-11T17:44:20.612431+0000","last_active":"2023-12-11T17:44:24.695048+0000","last_peered":"2023-12-11T17:44:24.695048+0000","last_clean":"2023-12-11T17:44:24.695048+0000","last_became_active":"2023-12-11T17:44:20.611444+0000","last_became_peered":"2023-12-11T17:44:20.611444+0000","last_unstale":"2023-12-11T17:44:24.695048+0000","last_undegraded":"2023-12-11T17:44:24.695048+0000","last_fullsized":"2023-12-11T17:44:24.695048+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T22:45:12.892966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.686018+0000","last_change":"2023-12-11T17:44:20.623828+0000","last_active":"2023-12-11T17:44:24.686018+0000","last_peered":"2023-12-11T17:44:24.686018+0000","last_clean":"2023-12-11T17:44:24.686018+0000","last_became_active":"2023-12-11T17:44:20.623510+0000","last_became_peered":"2023-12-11T17:44:20.623510+0000","last_unstale":"2023-12-11T17:44:24.686018+0000","last_undegraded":"2023-12-11T17:44:24.686018+0000","last_fullsized":"2023-12-11T17:44:24.686018+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T22:04:32.298900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.680583+0000","last_change":"2023-12-11T17:44:20.620831+0000","last_active":"2023-12-11T17:44:24.680583+0000","last_peered":"2023-12-11T17:44:24.680583+0000","last_clean":"2023-12-11T17:44:24.680583+0000","last_became_active":"2023-12-11T17:44:20.620481+0000","last_became_peered":"2023-12-11T17:44:20.620481+0000","last_unstale":"2023-12-11T17:44:24.680583+0000","last_undegraded":"2023-12-11T17:44:24.680583+0000","last_fullsized":"2023-12-11T17:44:24.680583+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T22:14:28.838192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.679798+0000","last_change":"2023-12-11T17:44:20.619515+0000","last_active":"2023-12-11T17:44:24.679798+0000","last_peered":"2023-12-11T17:44:24.679798+0000","last_clean":"2023-12-11T17:44:24.679798+0000","last_became_active":"2023-12-11T17:44:20.619219+0000","last_became_peered":"2023-12-11T17:44:20.619219+0000","last_unstale":"2023-12-11T17:44:24.679798+0000","last_undegraded":"2023-12-11T17:44:24.679798+0000","last_fullsized":"2023-12-11T17:44:24.679798+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T23:08:55.112752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.695462+0000","last_change":"2023-12-11T17:44:20.612645+0000","last_active":"2023-12-11T17:44:24.695462+0000","last_peered":"2023-12-11T17:44:24.695462+0000","last_clean":"2023-12-11T17:44:24.695462+0000","last_became_active":"2023-12-11T17:44:20.611336+0000","last_became_peered":"2023-12-11T17:44:20.611336+0000","last_unstale":"2023-12-11T17:44:24.695462+0000","last_undegraded":"2023-12-11T17:44:24.695462+0000","last_fullsized":"2023-12-11T17:44:24.695462+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T02:43:07.447653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.688266+0000","last_change":"2023-12-11T17:44:20.621198+0000","last_active":"2023-12-11T17:44:24.688266+0000","last_peered":"2023-12-11T17:44:24.688266+0000","last_clean":"2023-12-11T17:44:24.688266+0000","last_became_active":"2023-12-11T17:44:20.620693+0000","last_became_peered":"2023-12-11T17:44:20.620693+0000","last_unstale":"2023-12-11T17:44:24.688266+0000","last_undegraded":"2023-12-11T17:44:24.688266+0000","last_fullsized":"2023-12-11T17:44:24.688266+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T22:10:23.396031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.687515+0000","last_change":"2023-12-11T17:44:20.614457+0000","last_active":"2023-12-11T17:44:24.687515+0000","last_peered":"2023-12-11T17:44:24.687515+0000","last_clean":"2023-12-11T17:44:24.687515+0000","last_became_active":"2023-12-11T17:44:20.613984+0000","last_became_peered":"2023-12-11T17:44:20.613984+0000","last_unstale":"2023-12-11T17:44:24.687515+0000","last_undegraded":"2023-12-11T17:44:24.687515+0000","last_fullsized":"2023-12-11T17:44:24.687515+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T00:50:01.101398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.675193+0000","last_change":"2023-12-11T17:44:20.619874+0000","last_active":"2023-12-11T17:44:24.675193+0000","last_peered":"2023-12-11T17:44:24.675193+0000","last_clean":"2023-12-11T17:44:24.675193+0000","last_became_active":"2023-12-11T17:44:20.618537+0000","last_became_peered":"2023-12-11T17:44:20.618537+0000","last_unstale":"2023-12-11T17:44:24.675193+0000","last_undegraded":"2023-12-11T17:44:24.675193+0000","last_fullsized":"2023-12-11T17:44:24.675193+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T18:42:43.587245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.681021+0000","last_change":"2023-12-11T17:44:20.616306+0000","last_active":"2023-12-11T17:44:24.681021+0000","last_peered":"2023-12-11T17:44:24.681021+0000","last_clean":"2023-12-11T17:44:24.681021+0000","last_became_active":"2023-12-11T17:44:20.615930+0000","last_became_peered":"2023-12-11T17:44:20.615930+0000","last_unstale":"2023-12-11T17:44:24.681021+0000","last_undegraded":"2023-12-11T17:44:24.681021+0000","last_fullsized":"2023-12-11T17:44:24.681021+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T05:40:24.495052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.680343+0000","last_change":"2023-12-11T17:44:20.624339+0000","last_active":"2023-12-11T17:44:24.680343+0000","last_peered":"2023-12-11T17:44:24.680343+0000","last_clean":"2023-12-11T17:44:24.680343+0000","last_became_active":"2023-12-11T17:44:20.624045+0000","last_became_peered":"2023-12-11T17:44:20.624045+0000","last_unstale":"2023-12-11T17:44:24.680343+0000","last_undegraded":"2023-12-11T17:44:24.680343+0000","last_fullsized":"2023-12-11T17:44:24.680343+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T01:32:30.792940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.679095+0000","last_change":"2023-12-11T17:44:20.620750+0000","last_active":"2023-12-11T17:44:24.679095+0000","last_peered":"2023-12-11T17:44:24.679095+0000","last_clean":"2023-12-11T17:44:24.679095+0000","last_became_active":"2023-12-11T17:44:20.620464+0000","last_became_peered":"2023-12-11T17:44:20.620464+0000","last_unstale":"2023-12-11T17:44:24.679095+0000","last_undegraded":"2023-12-11T17:44:24.679095+0000","last_fullsized":"2023-12-11T17:44:24.679095+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T05:43:33.680889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.675193+0000","last_change":"2023-12-11T17:44:20.624391+0000","last_active":"2023-12-11T17:44:24.675193+0000","last_peered":"2023-12-11T17:44:24.675193+0000","last_clean":"2023-12-11T17:44:24.675193+0000","last_became_active":"2023-12-11T17:44:20.624217+0000","last_became_peered":"2023-12-11T17:44:20.624217+0000","last_unstale":"2023-12-11T17:44:24.675193+0000","last_undegraded":"2023-12-11T17:44:24.675193+0000","last_fullsized":"2023-12-11T17:44:24.675193+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T01:06:41.660726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.696138+0000","last_change":"2023-12-11T17:44:20.612429+0000","last_active":"2023-12-11T17:44:24.696138+0000","last_peered":"2023-12-11T17:44:24.696138+0000","last_clean":"2023-12-11T17:44:24.696138+0000","last_became_active":"2023-12-11T17:44:20.612082+0000","last_became_peered":"2023-12-11T17:44:20.612082+0000","last_unstale":"2023-12-11T17:44:24.696138+0000","last_undegraded":"2023-12-11T17:44:24.696138+0000","last_fullsized":"2023-12-11T17:44:24.696138+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T02:34:42.364876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.688363+0000","last_change":"2023-12-11T17:44:20.614907+0000","last_active":"2023-12-11T17:44:24.688363+0000","last_peered":"2023-12-11T17:44:24.688363+0000","last_clean":"2023-12-11T17:44:24.688363+0000","last_became_active":"2023-12-11T17:44:20.614611+0000","last_became_peered":"2023-12-11T17:44:20.614611+0000","last_unstale":"2023-12-11T17:44:24.688363+0000","last_undegraded":"2023-12-11T17:44:24.688363+0000","last_fullsized":"2023-12-11T17:44:24.688363+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T05:29:50.095253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T17:44:25.781615+0000","last_change":"2023-12-11T17:44:24.690974+0000","last_active":"2023-12-11T17:44:25.781615+0000","last_peered":"2023-12-11T17:44:25.781615+0000","last_clean":"2023-12-11T17:44:25.781615+0000","last_became_active":"2023-12-11T17:44:24.690579+0000","last_became_peered":"2023-12-11T17:44:24.690579+0000","last_unstale":"2023-12-11T17:44:25.781615+0000","last_undegraded":"2023-12-11T17:44:25.781615+0000","last_fullsized":"2023-12-11T17:44:25.781615+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_clean_scrub_stamp":"2023-12-11T17:44:23.663161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:31:59.830486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'9","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T17:44:46.775802+0000","last_change":"2023-12-11T17:44:19.659110+0000","last_active":"2023-12-11T17:44:46.775802+0000","last_peered":"2023-12-11T17:44:46.775802+0000","last_clean":"2023-12-11T17:44:46.775802+0000","last_became_active":"2023-12-11T17:44:19.658780+0000","last_became_peered":"2023-12-11T17:44:19.658780+0000","last_unstale":"2023-12-11T17:44:46.775802+0000","last_undegraded":"2023-12-11T17:44:46.775802+0000","last_fullsized":"2023-12-11T17:44:46.775802+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_clean_scrub_stamp":"2023-12-11T17:44:18.636443+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T00:08:58.425711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.680643+0000","last_change":"2023-12-11T17:44:20.621928+0000","last_active":"2023-12-11T17:44:24.680643+0000","last_peered":"2023-12-11T17:44:24.680643+0000","last_clean":"2023-12-11T17:44:24.680643+0000","last_became_active":"2023-12-11T17:44:20.621575+0000","last_became_peered":"2023-12-11T17:44:20.621575+0000","last_unstale":"2023-12-11T17:44:24.680643+0000","last_undegraded":"2023-12-11T17:44:24.680643+0000","last_fullsized":"2023-12-11T17:44:24.680643+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T17:57:44.860200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T17:44:25.676954+0000","last_change":"2023-12-11T17:44:24.704170+0000","last_active":"2023-12-11T17:44:25.676954+0000","last_peered":"2023-12-11T17:44:25.676954+0000","last_clean":"2023-12-11T17:44:25.676954+0000","last_became_active":"2023-12-11T17:44:24.703800+0000","last_became_peered":"2023-12-11T17:44:24.703800+0000","last_unstale":"2023-12-11T17:44:25.676954+0000","last_undegraded":"2023-12-11T17:44:25.676954+0000","last_fullsized":"2023-12-11T17:44:25.676954+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_clean_scrub_stamp":"2023-12-11T17:44:23.663161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T01:26:29.578910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T17:44:25.677720+0000","last_change":"2023-12-11T17:44:19.659660+0000","last_active":"2023-12-11T17:44:25.677720+0000","last_peered":"2023-12-11T17:44:25.677720+0000","last_clean":"2023-12-11T17:44:25.677720+0000","last_became_active":"2023-12-11T17:44:19.659283+0000","last_became_peered":"2023-12-11T17:44:19.659283+0000","last_unstale":"2023-12-11T17:44:25.677720+0000","last_undegraded":"2023-12-11T17:44:25.677720+0000","last_fullsized":"2023-12-11T17:44:25.677720+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_clean_scrub_stamp":"2023-12-11T17:44:18.636443+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T22:29:43.427794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.682875+0000","last_change":"2023-12-11T17:44:20.624962+0000","last_active":"2023-12-11T17:44:24.682875+0000","last_peered":"2023-12-11T17:44:24.682875+0000","last_clean":"2023-12-11T17:44:24.682875+0000","last_became_active":"2023-12-11T17:44:20.624640+0000","last_became_peered":"2023-12-11T17:44:20.624640+0000","last_unstale":"2023-12-11T17:44:24.682875+0000","last_undegraded":"2023-12-11T17:44:24.682875+0000","last_fullsized":"2023-12-11T17:44:24.682875+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T23:37:43.252062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T17:44:25.677149+0000","last_change":"2023-12-11T17:44:24.698866+0000","last_active":"2023-12-11T17:44:25.677149+0000","last_peered":"2023-12-11T17:44:25.677149+0000","last_clean":"2023-12-11T17:44:25.677149+0000","last_became_active":"2023-12-11T17:44:24.698481+0000","last_became_peered":"2023-12-11T17:44:24.698481+0000","last_unstale":"2023-12-11T17:44:25.677149+0000","last_undegraded":"2023-12-11T17:44:25.677149+0000","last_fullsized":"2023-12-11T17:44:25.677149+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_clean_scrub_stamp":"2023-12-11T17:44:23.663161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T20:50:19.571616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T17:44:46.774408+0000","last_change":"2023-12-11T17:44:19.662637+0000","last_active":"2023-12-11T17:44:46.774408+0000","last_peered":"2023-12-11T17:44:46.774408+0000","last_clean":"2023-12-11T17:44:46.774408+0000","last_became_active":"2023-12-11T17:44:19.662355+0000","last_became_peered":"2023-12-11T17:44:19.662355+0000","last_unstale":"2023-12-11T17:44:46.774408+0000","last_undegraded":"2023-12-11T17:44:46.774408+0000","last_fullsized":"2023-12-11T17:44:46.774408+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_clean_scrub_stamp":"2023-12-11T17:44:18.636443+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T17:56:37.436037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.679261+0000","last_change":"2023-12-11T17:44:20.615119+0000","last_active":"2023-12-11T17:44:24.679261+0000","last_peered":"2023-12-11T17:44:24.679261+0000","last_clean":"2023-12-11T17:44:24.679261+0000","last_became_active":"2023-12-11T17:44:20.614775+0000","last_became_peered":"2023-12-11T17:44:20.614775+0000","last_unstale":"2023-12-11T17:44:24.679261+0000","last_undegraded":"2023-12-11T17:44:24.679261+0000","last_fullsized":"2023-12-11T17:44:24.679261+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T20:45:47.162890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T17:44:25.677063+0000","last_change":"2023-12-11T17:44:24.707718+0000","last_active":"2023-12-11T17:44:25.677063+0000","last_peered":"2023-12-11T17:44:25.677063+0000","last_clean":"2023-12-11T17:44:25.677063+0000","last_became_active":"2023-12-11T17:44:24.707404+0000","last_became_peered":"2023-12-11T17:44:24.707404+0000","last_unstale":"2023-12-11T17:44:25.677063+0000","last_undegraded":"2023-12-11T17:44:25.677063+0000","last_fullsized":"2023-12-11T17:44:25.677063+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_clean_scrub_stamp":"2023-12-11T17:44:23.663161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T22:53:53.918649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"81'2","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.682642+0000","last_change":"2023-12-11T17:44:19.665282+0000","last_active":"2023-12-11T17:44:24.682642+0000","last_peered":"2023-12-11T17:44:24.682642+0000","last_clean":"2023-12-11T17:44:24.682642+0000","last_became_active":"2023-12-11T17:44:19.664972+0000","last_became_peered":"2023-12-11T17:44:19.664972+0000","last_unstale":"2023-12-11T17:44:24.682642+0000","last_undegraded":"2023-12-11T17:44:24.682642+0000","last_fullsized":"2023-12-11T17:44:24.682642+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_clean_scrub_stamp":"2023-12-11T17:44:18.636443+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T01:59:50.809386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.684473+0000","last_change":"2023-12-11T17:44:20.615894+0000","last_active":"2023-12-11T17:44:24.684473+0000","last_peered":"2023-12-11T17:44:24.684473+0000","last_clean":"2023-12-11T17:44:24.684473+0000","last_became_active":"2023-12-11T17:44:20.615582+0000","last_became_peered":"2023-12-11T17:44:20.615582+0000","last_unstale":"2023-12-11T17:44:24.684473+0000","last_undegraded":"2023-12-11T17:44:24.684473+0000","last_fullsized":"2023-12-11T17:44:24.684473+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T19:04:50.725088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T17:44:25.781958+0000","last_change":"2023-12-11T17:44:24.691459+0000","last_active":"2023-12-11T17:44:25.781958+0000","last_peered":"2023-12-11T17:44:25.781958+0000","last_clean":"2023-12-11T17:44:25.781958+0000","last_became_active":"2023-12-11T17:44:24.690301+0000","last_became_peered":"2023-12-11T17:44:24.690301+0000","last_unstale":"2023-12-11T17:44:25.781958+0000","last_undegraded":"2023-12-11T17:44:25.781958+0000","last_fullsized":"2023-12-11T17:44:25.781958+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_clean_scrub_stamp":"2023-12-11T17:44:23.663161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T17:58:04.685785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"86'6","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T17:44:48.782361+0000","last_change":"2023-12-11T17:44:19.661360+0000","last_active":"2023-12-11T17:44:48.782361+0000","last_peered":"2023-12-11T17:44:48.782361+0000","last_clean":"2023-12-11T17:44:48.782361+0000","last_became_active":"2023-12-11T17:44:19.661056+0000","last_became_peered":"2023-12-11T17:44:19.661056+0000","last_unstale":"2023-12-11T17:44:48.782361+0000","last_undegraded":"2023-12-11T17:44:48.782361+0000","last_fullsized":"2023-12-11T17:44:48.782361+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_clean_scrub_stamp":"2023-12-11T17:44:18.636443+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:21:19.212002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1011,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.680386+0000","last_change":"2023-12-11T17:44:20.608280+0000","last_active":"2023-12-11T17:44:24.680386+0000","last_peered":"2023-12-11T17:44:24.680386+0000","last_clean":"2023-12-11T17:44:24.680386+0000","last_became_active":"2023-12-11T17:44:20.607980+0000","last_became_peered":"2023-12-11T17:44:20.607980+0000","last_unstale":"2023-12-11T17:44:24.680386+0000","last_undegraded":"2023-12-11T17:44:24.680386+0000","last_fullsized":"2023-12-11T17:44:24.680386+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T04:45:56.108916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T17:44:25.781523+0000","last_change":"2023-12-11T17:44:24.703629+0000","last_active":"2023-12-11T17:44:25.781523+0000","last_peered":"2023-12-11T17:44:25.781523+0000","last_clean":"2023-12-11T17:44:25.781523+0000","last_became_active":"2023-12-11T17:44:24.703219+0000","last_became_peered":"2023-12-11T17:44:24.703219+0000","last_unstale":"2023-12-11T17:44:25.781523+0000","last_undegraded":"2023-12-11T17:44:25.781523+0000","last_fullsized":"2023-12-11T17:44:25.781523+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_clean_scrub_stamp":"2023-12-11T17:44:23.663161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T19:57:44.998039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T17:44:46.777497+0000","last_change":"2023-12-11T17:44:19.659311+0000","last_active":"2023-12-11T17:44:46.777497+0000","last_peered":"2023-12-11T17:44:46.777497+0000","last_clean":"2023-12-11T17:44:46.777497+0000","last_became_active":"2023-12-11T17:44:19.658994+0000","last_became_peered":"2023-12-11T17:44:19.658994+0000","last_unstale":"2023-12-11T17:44:46.777497+0000","last_undegraded":"2023-12-11T17:44:46.777497+0000","last_fullsized":"2023-12-11T17:44:46.777497+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_clean_scrub_stamp":"2023-12-11T17:44:18.636443+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T17:46:35.701179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.684251+0000","last_change":"2023-12-11T17:44:20.615684+0000","last_active":"2023-12-11T17:44:24.684251+0000","last_peered":"2023-12-11T17:44:24.684251+0000","last_clean":"2023-12-11T17:44:24.684251+0000","last_became_active":"2023-12-11T17:44:20.615373+0000","last_became_peered":"2023-12-11T17:44:20.615373+0000","last_unstale":"2023-12-11T17:44:24.684251+0000","last_undegraded":"2023-12-11T17:44:24.684251+0000","last_fullsized":"2023-12-11T17:44:24.684251+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T19:45:59.980253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T17:44:25.782048+0000","last_change":"2023-12-11T17:44:24.699085+0000","last_active":"2023-12-11T17:44:25.782048+0000","last_peered":"2023-12-11T17:44:25.782048+0000","last_clean":"2023-12-11T17:44:25.782048+0000","last_became_active":"2023-12-11T17:44:24.698717+0000","last_became_peered":"2023-12-11T17:44:24.698717+0000","last_unstale":"2023-12-11T17:44:25.782048+0000","last_undegraded":"2023-12-11T17:44:25.782048+0000","last_fullsized":"2023-12-11T17:44:25.782048+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_clean_scrub_stamp":"2023-12-11T17:44:23.663161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T05:00:52.341938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.675120+0000","last_change":"2023-12-11T17:44:19.660392+0000","last_active":"2023-12-11T17:44:24.675120+0000","last_peered":"2023-12-11T17:44:24.675120+0000","last_clean":"2023-12-11T17:44:24.675120+0000","last_became_active":"2023-12-11T17:44:19.660099+0000","last_became_peered":"2023-12-11T17:44:19.660099+0000","last_unstale":"2023-12-11T17:44:24.675120+0000","last_undegraded":"2023-12-11T17:44:24.675120+0000","last_fullsized":"2023-12-11T17:44:24.675120+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_clean_scrub_stamp":"2023-12-11T17:44:18.636443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T00:38:37.279230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.680701+0000","last_change":"2023-12-11T17:44:20.615629+0000","last_active":"2023-12-11T17:44:24.680701+0000","last_peered":"2023-12-11T17:44:24.680701+0000","last_clean":"2023-12-11T17:44:24.680701+0000","last_became_active":"2023-12-11T17:44:20.615193+0000","last_became_peered":"2023-12-11T17:44:20.615193+0000","last_unstale":"2023-12-11T17:44:24.680701+0000","last_undegraded":"2023-12-11T17:44:24.680701+0000","last_fullsized":"2023-12-11T17:44:24.680701+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:09:56.237298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T17:44:25.781425+0000","last_change":"2023-12-11T17:44:24.705845+0000","last_active":"2023-12-11T17:44:25.781425+0000","last_peered":"2023-12-11T17:44:25.781425+0000","last_clean":"2023-12-11T17:44:25.781425+0000","last_became_active":"2023-12-11T17:44:24.705434+0000","last_became_peered":"2023-12-11T17:44:24.705434+0000","last_unstale":"2023-12-11T17:44:25.781425+0000","last_undegraded":"2023-12-11T17:44:25.781425+0000","last_fullsized":"2023-12-11T17:44:25.781425+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_clean_scrub_stamp":"2023-12-11T17:44:23.663161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T02:37:25.663753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T17:44:25.782676+0000","last_change":"2023-12-11T17:44:19.661317+0000","last_active":"2023-12-11T17:44:25.782676+0000","last_peered":"2023-12-11T17:44:25.782676+0000","last_clean":"2023-12-11T17:44:25.782676+0000","last_became_active":"2023-12-11T17:44:19.660993+0000","last_became_peered":"2023-12-11T17:44:19.660993+0000","last_unstale":"2023-12-11T17:44:25.782676+0000","last_undegraded":"2023-12-11T17:44:25.782676+0000","last_fullsized":"2023-12-11T17:44:25.782676+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_clean_scrub_stamp":"2023-12-11T17:44:18.636443+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T23:00:07.345964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.695205+0000","last_change":"2023-12-11T17:44:20.623681+0000","last_active":"2023-12-11T17:44:24.695205+0000","last_peered":"2023-12-11T17:44:24.695205+0000","last_clean":"2023-12-11T17:44:24.695205+0000","last_became_active":"2023-12-11T17:44:20.623288+0000","last_became_peered":"2023-12-11T17:44:20.623288+0000","last_unstale":"2023-12-11T17:44:24.695205+0000","last_undegraded":"2023-12-11T17:44:24.695205+0000","last_fullsized":"2023-12-11T17:44:24.695205+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T23:47:55.348480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T17:44:25.677106+0000","last_change":"2023-12-11T17:44:24.687613+0000","last_active":"2023-12-11T17:44:25.677106+0000","last_peered":"2023-12-11T17:44:25.677106+0000","last_clean":"2023-12-11T17:44:25.677106+0000","last_became_active":"2023-12-11T17:44:24.687240+0000","last_became_peered":"2023-12-11T17:44:24.687240+0000","last_unstale":"2023-12-11T17:44:25.677106+0000","last_undegraded":"2023-12-11T17:44:25.677106+0000","last_fullsized":"2023-12-11T17:44:25.677106+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_clean_scrub_stamp":"2023-12-11T17:44:23.663161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T20:54:25.409631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"86'3","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T17:44:45.765254+0000","last_change":"2023-12-11T17:44:19.653571+0000","last_active":"2023-12-11T17:44:45.765254+0000","last_peered":"2023-12-11T17:44:45.765254+0000","last_clean":"2023-12-11T17:44:45.765254+0000","last_became_active":"2023-12-11T17:44:19.653035+0000","last_became_peered":"2023-12-11T17:44:19.653035+0000","last_unstale":"2023-12-11T17:44:45.765254+0000","last_undegraded":"2023-12-11T17:44:45.765254+0000","last_fullsized":"2023-12-11T17:44:45.765254+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_clean_scrub_stamp":"2023-12-11T17:44:18.636443+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T02:29:38.310132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.679683+0000","last_change":"2023-12-11T17:44:20.623777+0000","last_active":"2023-12-11T17:44:24.679683+0000","last_peered":"2023-12-11T17:44:24.679683+0000","last_clean":"2023-12-11T17:44:24.679683+0000","last_became_active":"2023-12-11T17:44:20.623449+0000","last_became_peered":"2023-12-11T17:44:20.623449+0000","last_unstale":"2023-12-11T17:44:24.679683+0000","last_undegraded":"2023-12-11T17:44:24.679683+0000","last_fullsized":"2023-12-11T17:44:24.679683+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T05:23:03.493677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T17:44:25.782574+0000","last_change":"2023-12-11T17:44:24.693893+0000","last_active":"2023-12-11T17:44:25.782574+0000","last_peered":"2023-12-11T17:44:25.782574+0000","last_clean":"2023-12-11T17:44:25.782574+0000","last_became_active":"2023-12-11T17:44:24.693441+0000","last_became_peered":"2023-12-11T17:44:24.693441+0000","last_unstale":"2023-12-11T17:44:25.782574+0000","last_undegraded":"2023-12-11T17:44:25.782574+0000","last_fullsized":"2023-12-11T17:44:25.782574+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_clean_scrub_stamp":"2023-12-11T17:44:23.663161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T23:25:06.457176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T17:44:45.763408+0000","last_change":"2023-12-11T17:44:19.662988+0000","last_active":"2023-12-11T17:44:45.763408+0000","last_peered":"2023-12-11T17:44:45.763408+0000","last_clean":"2023-12-11T17:44:45.763408+0000","last_became_active":"2023-12-11T17:44:19.662660+0000","last_became_peered":"2023-12-11T17:44:19.662660+0000","last_unstale":"2023-12-11T17:44:45.763408+0000","last_undegraded":"2023-12-11T17:44:45.763408+0000","last_fullsized":"2023-12-11T17:44:45.763408+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_clean_scrub_stamp":"2023-12-11T17:44:18.636443+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T19:08:50.703161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.682688+0000","last_change":"2023-12-11T17:44:20.622993+0000","last_active":"2023-12-11T17:44:24.682688+0000","last_peered":"2023-12-11T17:44:24.682688+0000","last_clean":"2023-12-11T17:44:24.682688+0000","last_became_active":"2023-12-11T17:44:20.621050+0000","last_became_peered":"2023-12-11T17:44:20.621050+0000","last_unstale":"2023-12-11T17:44:24.682688+0000","last_undegraded":"2023-12-11T17:44:24.682688+0000","last_fullsized":"2023-12-11T17:44:24.682688+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T22:28:03.230596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T17:44:25.677339+0000","last_change":"2023-12-11T17:44:24.698715+0000","last_active":"2023-12-11T17:44:25.677339+0000","last_peered":"2023-12-11T17:44:25.677339+0000","last_clean":"2023-12-11T17:44:25.677339+0000","last_became_active":"2023-12-11T17:44:24.698359+0000","last_became_peered":"2023-12-11T17:44:24.698359+0000","last_unstale":"2023-12-11T17:44:25.677339+0000","last_undegraded":"2023-12-11T17:44:25.677339+0000","last_fullsized":"2023-12-11T17:44:25.677339+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_clean_scrub_stamp":"2023-12-11T17:44:23.663161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T01:07:23.870922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T17:44:45.763733+0000","last_change":"2023-12-11T17:44:19.655914+0000","last_active":"2023-12-11T17:44:45.763733+0000","last_peered":"2023-12-11T17:44:45.763733+0000","last_clean":"2023-12-11T17:44:45.763733+0000","last_became_active":"2023-12-11T17:44:19.655593+0000","last_became_peered":"2023-12-11T17:44:19.655593+0000","last_unstale":"2023-12-11T17:44:45.763733+0000","last_undegraded":"2023-12-11T17:44:45.763733+0000","last_fullsized":"2023-12-11T17:44:45.763733+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_clean_scrub_stamp":"2023-12-11T17:44:18.636443+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T01:54:50.246100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.679438+0000","last_change":"2023-12-11T17:44:20.618469+0000","last_active":"2023-12-11T17:44:24.679438+0000","last_peered":"2023-12-11T17:44:24.679438+0000","last_clean":"2023-12-11T17:44:24.679438+0000","last_became_active":"2023-12-11T17:44:20.618186+0000","last_became_peered":"2023-12-11T17:44:20.618186+0000","last_unstale":"2023-12-11T17:44:24.679438+0000","last_undegraded":"2023-12-11T17:44:24.679438+0000","last_fullsized":"2023-12-11T17:44:24.679438+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T20:53:22.405100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T17:44:25.781421+0000","last_change":"2023-12-11T17:44:24.697048+0000","last_active":"2023-12-11T17:44:25.781421+0000","last_peered":"2023-12-11T17:44:25.781421+0000","last_clean":"2023-12-11T17:44:25.781421+0000","last_became_active":"2023-12-11T17:44:24.696660+0000","last_became_peered":"2023-12-11T17:44:24.696660+0000","last_unstale":"2023-12-11T17:44:25.781421+0000","last_undegraded":"2023-12-11T17:44:25.781421+0000","last_fullsized":"2023-12-11T17:44:25.781421+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_clean_scrub_stamp":"2023-12-11T17:44:23.663161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T21:50:51.018386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"86'12","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T17:44:45.764204+0000","last_change":"2023-12-11T17:44:19.653528+0000","last_active":"2023-12-11T17:44:45.764204+0000","last_peered":"2023-12-11T17:44:45.764204+0000","last_clean":"2023-12-11T17:44:45.764204+0000","last_became_active":"2023-12-11T17:44:19.653215+0000","last_became_peered":"2023-12-11T17:44:19.653215+0000","last_unstale":"2023-12-11T17:44:45.764204+0000","last_undegraded":"2023-12-11T17:44:45.764204+0000","last_fullsized":"2023-12-11T17:44:45.764204+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_clean_scrub_stamp":"2023-12-11T17:44:18.636443+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T21:19:12.024008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3841,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.680635+0000","last_change":"2023-12-11T17:44:20.614085+0000","last_active":"2023-12-11T17:44:24.680635+0000","last_peered":"2023-12-11T17:44:24.680635+0000","last_clean":"2023-12-11T17:44:24.680635+0000","last_became_active":"2023-12-11T17:44:20.613768+0000","last_became_peered":"2023-12-11T17:44:20.613768+0000","last_unstale":"2023-12-11T17:44:24.680635+0000","last_undegraded":"2023-12-11T17:44:24.680635+0000","last_fullsized":"2023-12-11T17:44:24.680635+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T23:42:52.133207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T17:44:25.782452+0000","last_change":"2023-12-11T17:44:24.702039+0000","last_active":"2023-12-11T17:44:25.782452+0000","last_peered":"2023-12-11T17:44:25.782452+0000","last_clean":"2023-12-11T17:44:25.782452+0000","last_became_active":"2023-12-11T17:44:24.701539+0000","last_became_peered":"2023-12-11T17:44:24.701539+0000","last_unstale":"2023-12-11T17:44:25.782452+0000","last_undegraded":"2023-12-11T17:44:25.782452+0000","last_fullsized":"2023-12-11T17:44:25.782452+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_clean_scrub_stamp":"2023-12-11T17:44:23.663161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T02:23:55.393595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T17:44:46.773873+0000","last_change":"2023-12-11T17:44:19.662118+0000","last_active":"2023-12-11T17:44:46.773873+0000","last_peered":"2023-12-11T17:44:46.773873+0000","last_clean":"2023-12-11T17:44:46.773873+0000","last_became_active":"2023-12-11T17:44:19.661796+0000","last_became_peered":"2023-12-11T17:44:19.661796+0000","last_unstale":"2023-12-11T17:44:46.773873+0000","last_undegraded":"2023-12-11T17:44:46.773873+0000","last_fullsized":"2023-12-11T17:44:46.773873+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_clean_scrub_stamp":"2023-12-11T17:44:18.636443+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T21:08:02.842544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.675150+0000","last_change":"2023-12-11T17:44:20.615403+0000","last_active":"2023-12-11T17:44:24.675150+0000","last_peered":"2023-12-11T17:44:24.675150+0000","last_clean":"2023-12-11T17:44:24.675150+0000","last_became_active":"2023-12-11T17:44:20.615106+0000","last_became_peered":"2023-12-11T17:44:20.615106+0000","last_unstale":"2023-12-11T17:44:24.675150+0000","last_undegraded":"2023-12-11T17:44:24.675150+0000","last_fullsized":"2023-12-11T17:44:24.675150+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T02:10:24.047720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T17:44:25.781715+0000","last_change":"2023-12-11T17:44:24.699737+0000","last_active":"2023-12-11T17:44:25.781715+0000","last_peered":"2023-12-11T17:44:25.781715+0000","last_clean":"2023-12-11T17:44:25.781715+0000","last_became_active":"2023-12-11T17:44:24.699345+0000","last_became_peered":"2023-12-11T17:44:24.699345+0000","last_unstale":"2023-12-11T17:44:25.781715+0000","last_undegraded":"2023-12-11T17:44:25.781715+0000","last_fullsized":"2023-12-11T17:44:25.781715+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_clean_scrub_stamp":"2023-12-11T17:44:23.663161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T01:30:25.471636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T17:44:45.764021+0000","last_change":"2023-12-11T17:44:19.661325+0000","last_active":"2023-12-11T17:44:45.764021+0000","last_peered":"2023-12-11T17:44:45.764021+0000","last_clean":"2023-12-11T17:44:45.764021+0000","last_became_active":"2023-12-11T17:44:19.661012+0000","last_became_peered":"2023-12-11T17:44:19.661012+0000","last_unstale":"2023-12-11T17:44:45.764021+0000","last_undegraded":"2023-12-11T17:44:45.764021+0000","last_fullsized":"2023-12-11T17:44:45.764021+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_clean_scrub_stamp":"2023-12-11T17:44:18.636443+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T19:33:08.258019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.686029+0000","last_change":"2023-12-11T17:44:20.614795+0000","last_active":"2023-12-11T17:44:24.686029+0000","last_peered":"2023-12-11T17:44:24.686029+0000","last_clean":"2023-12-11T17:44:24.686029+0000","last_became_active":"2023-12-11T17:44:20.613826+0000","last_became_peered":"2023-12-11T17:44:20.613826+0000","last_unstale":"2023-12-11T17:44:24.686029+0000","last_undegraded":"2023-12-11T17:44:24.686029+0000","last_fullsized":"2023-12-11T17:44:24.686029+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T21:32:19.029040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T17:44:25.781716+0000","last_change":"2023-12-11T17:44:24.701569+0000","last_active":"2023-12-11T17:44:25.781716+0000","last_peered":"2023-12-11T17:44:25.781716+0000","last_clean":"2023-12-11T17:44:25.781716+0000","last_became_active":"2023-12-11T17:44:24.701173+0000","last_became_peered":"2023-12-11T17:44:24.701173+0000","last_unstale":"2023-12-11T17:44:25.781716+0000","last_undegraded":"2023-12-11T17:44:25.781716+0000","last_fullsized":"2023-12-11T17:44:25.781716+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_clean_scrub_stamp":"2023-12-11T17:44:23.663161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T23:02:45.843827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.684158+0000","last_change":"2023-12-11T17:44:19.659643+0000","last_active":"2023-12-11T17:44:24.684158+0000","last_peered":"2023-12-11T17:44:24.684158+0000","last_clean":"2023-12-11T17:44:24.684158+0000","last_became_active":"2023-12-11T17:44:19.659320+0000","last_became_peered":"2023-12-11T17:44:19.659320+0000","last_unstale":"2023-12-11T17:44:24.684158+0000","last_undegraded":"2023-12-11T17:44:24.684158+0000","last_fullsized":"2023-12-11T17:44:24.684158+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_clean_scrub_stamp":"2023-12-11T17:44:18.636443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T20:02:03.633439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.685932+0000","last_change":"2023-12-11T17:44:20.619445+0000","last_active":"2023-12-11T17:44:24.685932+0000","last_peered":"2023-12-11T17:44:24.685932+0000","last_clean":"2023-12-11T17:44:24.685932+0000","last_became_active":"2023-12-11T17:44:20.619094+0000","last_became_peered":"2023-12-11T17:44:20.619094+0000","last_unstale":"2023-12-11T17:44:24.685932+0000","last_undegraded":"2023-12-11T17:44:24.685932+0000","last_fullsized":"2023-12-11T17:44:24.685932+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T04:03:21.917498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T17:44:25.781832+0000","last_change":"2023-12-11T17:44:24.701821+0000","last_active":"2023-12-11T17:44:25.781832+0000","last_peered":"2023-12-11T17:44:25.781832+0000","last_clean":"2023-12-11T17:44:25.781832+0000","last_became_active":"2023-12-11T17:44:24.700590+0000","last_became_peered":"2023-12-11T17:44:24.700590+0000","last_unstale":"2023-12-11T17:44:25.781832+0000","last_undegraded":"2023-12-11T17:44:25.781832+0000","last_fullsized":"2023-12-11T17:44:25.781832+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_clean_scrub_stamp":"2023-12-11T17:44:23.663161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T19:58:28.422648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":33,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T17:44:46.775889+0000","last_change":"2023-12-11T17:44:19.662540+0000","last_active":"2023-12-11T17:44:46.775889+0000","last_peered":"2023-12-11T17:44:46.775889+0000","last_clean":"2023-12-11T17:44:46.775889+0000","last_became_active":"2023-12-11T17:44:19.662231+0000","last_became_peered":"2023-12-11T17:44:19.662231+0000","last_unstale":"2023-12-11T17:44:46.775889+0000","last_undegraded":"2023-12-11T17:44:46.775889+0000","last_fullsized":"2023-12-11T17:44:46.775889+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_clean_scrub_stamp":"2023-12-11T17:44:18.636443+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T21:56:45.241138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.683927+0000","last_change":"2023-12-11T17:44:20.620457+0000","last_active":"2023-12-11T17:44:24.683927+0000","last_peered":"2023-12-11T17:44:24.683927+0000","last_clean":"2023-12-11T17:44:24.683927+0000","last_became_active":"2023-12-11T17:44:20.619949+0000","last_became_peered":"2023-12-11T17:44:20.619949+0000","last_unstale":"2023-12-11T17:44:24.683927+0000","last_undegraded":"2023-12-11T17:44:24.683927+0000","last_fullsized":"2023-12-11T17:44:24.683927+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T22:59:14.328817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T17:44:25.782384+0000","last_change":"2023-12-11T17:44:24.691573+0000","last_active":"2023-12-11T17:44:25.782384+0000","last_peered":"2023-12-11T17:44:25.782384+0000","last_clean":"2023-12-11T17:44:25.782384+0000","last_became_active":"2023-12-11T17:44:24.691182+0000","last_became_peered":"2023-12-11T17:44:24.691182+0000","last_unstale":"2023-12-11T17:44:25.782384+0000","last_undegraded":"2023-12-11T17:44:25.782384+0000","last_fullsized":"2023-12-11T17:44:25.782384+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_clean_scrub_stamp":"2023-12-11T17:44:23.663161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T02:18:36.582108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"81'1","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T17:44:25.677343+0000","last_change":"2023-12-11T17:44:19.653470+0000","last_active":"2023-12-11T17:44:25.677343+0000","last_peered":"2023-12-11T17:44:25.677343+0000","last_clean":"2023-12-11T17:44:25.677343+0000","last_became_active":"2023-12-11T17:44:19.653148+0000","last_became_peered":"2023-12-11T17:44:19.653148+0000","last_unstale":"2023-12-11T17:44:25.677343+0000","last_undegraded":"2023-12-11T17:44:25.677343+0000","last_fullsized":"2023-12-11T17:44:25.677343+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_clean_scrub_stamp":"2023-12-11T17:44:18.636443+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T05:26:12.017814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.674922+0000","last_change":"2023-12-11T17:44:20.617019+0000","last_active":"2023-12-11T17:44:24.674922+0000","last_peered":"2023-12-11T17:44:24.674922+0000","last_clean":"2023-12-11T17:44:24.674922+0000","last_became_active":"2023-12-11T17:44:20.616770+0000","last_became_peered":"2023-12-11T17:44:20.616770+0000","last_unstale":"2023-12-11T17:44:24.674922+0000","last_undegraded":"2023-12-11T17:44:24.674922+0000","last_fullsized":"2023-12-11T17:44:24.674922+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T18:32:15.050179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T17:44:25.676860+0000","last_change":"2023-12-11T17:44:24.694843+0000","last_active":"2023-12-11T17:44:25.676860+0000","last_peered":"2023-12-11T17:44:25.676860+0000","last_clean":"2023-12-11T17:44:25.676860+0000","last_became_active":"2023-12-11T17:44:24.694423+0000","last_became_peered":"2023-12-11T17:44:24.694423+0000","last_unstale":"2023-12-11T17:44:25.676860+0000","last_undegraded":"2023-12-11T17:44:25.676860+0000","last_fullsized":"2023-12-11T17:44:25.676860+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_clean_scrub_stamp":"2023-12-11T17:44:23.663161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T18:42:35.281782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.695692+0000","last_change":"2023-12-11T17:44:19.655676+0000","last_active":"2023-12-11T17:44:24.695692+0000","last_peered":"2023-12-11T17:44:24.695692+0000","last_clean":"2023-12-11T17:44:24.695692+0000","last_became_active":"2023-12-11T17:44:19.655331+0000","last_became_peered":"2023-12-11T17:44:19.655331+0000","last_unstale":"2023-12-11T17:44:24.695692+0000","last_undegraded":"2023-12-11T17:44:24.695692+0000","last_fullsized":"2023-12-11T17:44:24.695692+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_clean_scrub_stamp":"2023-12-11T17:44:18.636443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T00:01:27.963540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.695557+0000","last_change":"2023-12-11T17:44:20.617396+0000","last_active":"2023-12-11T17:44:24.695557+0000","last_peered":"2023-12-11T17:44:24.695557+0000","last_clean":"2023-12-11T17:44:24.695557+0000","last_became_active":"2023-12-11T17:44:20.617086+0000","last_became_peered":"2023-12-11T17:44:20.617086+0000","last_unstale":"2023-12-11T17:44:24.695557+0000","last_undegraded":"2023-12-11T17:44:24.695557+0000","last_fullsized":"2023-12-11T17:44:24.695557+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T02:31:55.763125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T17:44:25.781340+0000","last_change":"2023-12-11T17:44:24.703283+0000","last_active":"2023-12-11T17:44:25.781340+0000","last_peered":"2023-12-11T17:44:25.781340+0000","last_clean":"2023-12-11T17:44:25.781340+0000","last_became_active":"2023-12-11T17:44:24.701703+0000","last_became_peered":"2023-12-11T17:44:24.701703+0000","last_unstale":"2023-12-11T17:44:25.781340+0000","last_undegraded":"2023-12-11T17:44:25.781340+0000","last_fullsized":"2023-12-11T17:44:25.781340+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_clean_scrub_stamp":"2023-12-11T17:44:23.663161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T18:52:50.570036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T17:44:25.781972+0000","last_change":"2023-12-11T17:44:19.662851+0000","last_active":"2023-12-11T17:44:25.781972+0000","last_peered":"2023-12-11T17:44:25.781972+0000","last_clean":"2023-12-11T17:44:25.781972+0000","last_became_active":"2023-12-11T17:44:19.662548+0000","last_became_peered":"2023-12-11T17:44:19.662548+0000","last_unstale":"2023-12-11T17:44:25.781972+0000","last_undegraded":"2023-12-11T17:44:25.781972+0000","last_fullsized":"2023-12-11T17:44:25.781972+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_clean_scrub_stamp":"2023-12-11T17:44:18.636443+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T02:34:14.914687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.695033+0000","last_change":"2023-12-11T17:44:20.612750+0000","last_active":"2023-12-11T17:44:24.695033+0000","last_peered":"2023-12-11T17:44:24.695033+0000","last_clean":"2023-12-11T17:44:24.695033+0000","last_became_active":"2023-12-11T17:44:20.612479+0000","last_became_peered":"2023-12-11T17:44:20.612479+0000","last_unstale":"2023-12-11T17:44:24.695033+0000","last_undegraded":"2023-12-11T17:44:24.695033+0000","last_fullsized":"2023-12-11T17:44:24.695033+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:45:03.217844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T17:44:25.782005+0000","last_change":"2023-12-11T17:44:24.703050+0000","last_active":"2023-12-11T17:44:25.782005+0000","last_peered":"2023-12-11T17:44:25.782005+0000","last_clean":"2023-12-11T17:44:25.782005+0000","last_became_active":"2023-12-11T17:44:24.701408+0000","last_became_peered":"2023-12-11T17:44:24.701408+0000","last_unstale":"2023-12-11T17:44:25.782005+0000","last_undegraded":"2023-12-11T17:44:25.782005+0000","last_fullsized":"2023-12-11T17:44:25.782005+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_clean_scrub_stamp":"2023-12-11T17:44:23.663161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T01:02:44.726831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.680820+0000","last_change":"2023-12-11T17:44:20.616004+0000","last_active":"2023-12-11T17:44:24.680820+0000","last_peered":"2023-12-11T17:44:24.680820+0000","last_clean":"2023-12-11T17:44:24.680820+0000","last_became_active":"2023-12-11T17:44:20.614985+0000","last_became_peered":"2023-12-11T17:44:20.614985+0000","last_unstale":"2023-12-11T17:44:24.680820+0000","last_undegraded":"2023-12-11T17:44:24.680820+0000","last_fullsized":"2023-12-11T17:44:24.680820+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T05:31:13.414818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'1","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T17:44:25.782150+0000","last_change":"2023-12-11T17:44:19.663092+0000","last_active":"2023-12-11T17:44:25.782150+0000","last_peered":"2023-12-11T17:44:25.782150+0000","last_clean":"2023-12-11T17:44:25.782150+0000","last_became_active":"2023-12-11T17:44:19.662728+0000","last_became_peered":"2023-12-11T17:44:19.662728+0000","last_unstale":"2023-12-11T17:44:25.782150+0000","last_undegraded":"2023-12-11T17:44:25.782150+0000","last_fullsized":"2023-12-11T17:44:25.782150+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_clean_scrub_stamp":"2023-12-11T17:44:18.636443+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T20:05:23.952470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T17:44:25.782684+0000","last_change":"2023-12-11T17:44:24.694194+0000","last_active":"2023-12-11T17:44:25.782684+0000","last_peered":"2023-12-11T17:44:25.782684+0000","last_clean":"2023-12-11T17:44:25.782684+0000","last_became_active":"2023-12-11T17:44:24.693860+0000","last_became_peered":"2023-12-11T17:44:24.693860+0000","last_unstale":"2023-12-11T17:44:25.782684+0000","last_undegraded":"2023-12-11T17:44:25.782684+0000","last_fullsized":"2023-12-11T17:44:25.782684+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_clean_scrub_stamp":"2023-12-11T17:44:23.663161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T01:16:04.877531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.679992+0000","last_change":"2023-12-11T17:44:20.615445+0000","last_active":"2023-12-11T17:44:24.679992+0000","last_peered":"2023-12-11T17:44:24.679992+0000","last_clean":"2023-12-11T17:44:24.679992+0000","last_became_active":"2023-12-11T17:44:20.614973+0000","last_became_peered":"2023-12-11T17:44:20.614973+0000","last_unstale":"2023-12-11T17:44:24.679992+0000","last_undegraded":"2023-12-11T17:44:24.679992+0000","last_fullsized":"2023-12-11T17:44:24.679992+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T21:51:31.923310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.687462+0000","last_change":"2023-12-11T17:44:19.651980+0000","last_active":"2023-12-11T17:44:24.687462+0000","last_peered":"2023-12-11T17:44:24.687462+0000","last_clean":"2023-12-11T17:44:24.687462+0000","last_became_active":"2023-12-11T17:44:19.651659+0000","last_became_peered":"2023-12-11T17:44:19.651659+0000","last_unstale":"2023-12-11T17:44:24.687462+0000","last_undegraded":"2023-12-11T17:44:24.687462+0000","last_fullsized":"2023-12-11T17:44:24.687462+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_clean_scrub_stamp":"2023-12-11T17:44:18.636443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T19:29:21.453758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":320,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.685586+0000","last_change":"2023-12-11T17:43:21.021919+0000","last_active":"2023-12-11T17:44:24.685586+0000","last_peered":"2023-12-11T17:44:24.685586+0000","last_clean":"2023-12-11T17:44:24.685586+0000","last_became_active":"2023-12-11T17:43:21.021489+0000","last_became_peered":"2023-12-11T17:43:21.021489+0000","last_unstale":"2023-12-11T17:44:24.685586+0000","last_undegraded":"2023-12-11T17:44:24.685586+0000","last_fullsized":"2023-12-11T17:44:24.685586+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:40:27.026768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:40:27.026768+0000","last_clean_scrub_stamp":"2023-12-11T17:40:27.026768+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T22:30:14.730794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T17:44:25.781461+0000","last_change":"2023-12-11T17:44:24.701996+0000","last_active":"2023-12-11T17:44:25.781461+0000","last_peered":"2023-12-11T17:44:25.781461+0000","last_clean":"2023-12-11T17:44:25.781461+0000","last_became_active":"2023-12-11T17:44:24.701574+0000","last_became_peered":"2023-12-11T17:44:24.701574+0000","last_unstale":"2023-12-11T17:44:25.781461+0000","last_undegraded":"2023-12-11T17:44:25.781461+0000","last_fullsized":"2023-12-11T17:44:25.781461+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_clean_scrub_stamp":"2023-12-11T17:44:23.663161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T19:49:01.458432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.674905+0000","last_change":"2023-12-11T17:44:20.614765+0000","last_active":"2023-12-11T17:44:24.674905+0000","last_peered":"2023-12-11T17:44:24.674905+0000","last_clean":"2023-12-11T17:44:24.674905+0000","last_became_active":"2023-12-11T17:44:20.614462+0000","last_became_peered":"2023-12-11T17:44:20.614462+0000","last_unstale":"2023-12-11T17:44:24.674905+0000","last_undegraded":"2023-12-11T17:44:24.674905+0000","last_fullsized":"2023-12-11T17:44:24.674905+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:08:20.784124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T17:44:46.774838+0000","last_change":"2023-12-11T17:44:19.661528+0000","last_active":"2023-12-11T17:44:46.774838+0000","last_peered":"2023-12-11T17:44:46.774838+0000","last_clean":"2023-12-11T17:44:46.774838+0000","last_became_active":"2023-12-11T17:44:19.661208+0000","last_became_peered":"2023-12-11T17:44:19.661208+0000","last_unstale":"2023-12-11T17:44:46.774838+0000","last_undegraded":"2023-12-11T17:44:46.774838+0000","last_fullsized":"2023-12-11T17:44:46.774838+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_clean_scrub_stamp":"2023-12-11T17:44:18.636443+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T18:31:23.295193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T17:44:25.781318+0000","last_change":"2023-12-11T17:44:24.688218+0000","last_active":"2023-12-11T17:44:25.781318+0000","last_peered":"2023-12-11T17:44:25.781318+0000","last_clean":"2023-12-11T17:44:25.781318+0000","last_became_active":"2023-12-11T17:44:24.687809+0000","last_became_peered":"2023-12-11T17:44:24.687809+0000","last_unstale":"2023-12-11T17:44:25.781318+0000","last_undegraded":"2023-12-11T17:44:25.781318+0000","last_fullsized":"2023-12-11T17:44:25.781318+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_clean_scrub_stamp":"2023-12-11T17:44:23.663161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T02:12:50.975614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T17:44:45.762595+0000","last_change":"2023-12-11T17:44:19.657797+0000","last_active":"2023-12-11T17:44:45.762595+0000","last_peered":"2023-12-11T17:44:45.762595+0000","last_clean":"2023-12-11T17:44:45.762595+0000","last_became_active":"2023-12-11T17:44:19.657483+0000","last_became_peered":"2023-12-11T17:44:19.657483+0000","last_unstale":"2023-12-11T17:44:45.762595+0000","last_undegraded":"2023-12-11T17:44:45.762595+0000","last_fullsized":"2023-12-11T17:44:45.762595+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_clean_scrub_stamp":"2023-12-11T17:44:18.636443+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T22:43:35.785943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.676427+0000","last_change":"2023-12-11T17:44:20.610002+0000","last_active":"2023-12-11T17:44:24.676427+0000","last_peered":"2023-12-11T17:44:24.676427+0000","last_clean":"2023-12-11T17:44:24.676427+0000","last_became_active":"2023-12-11T17:44:20.609579+0000","last_became_peered":"2023-12-11T17:44:20.609579+0000","last_unstale":"2023-12-11T17:44:24.676427+0000","last_undegraded":"2023-12-11T17:44:24.676427+0000","last_fullsized":"2023-12-11T17:44:24.676427+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:18:25.670403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T17:44:25.676923+0000","last_change":"2023-12-11T17:44:24.694707+0000","last_active":"2023-12-11T17:44:25.676923+0000","last_peered":"2023-12-11T17:44:25.676923+0000","last_clean":"2023-12-11T17:44:25.676923+0000","last_became_active":"2023-12-11T17:44:24.694275+0000","last_became_peered":"2023-12-11T17:44:24.694275+0000","last_unstale":"2023-12-11T17:44:25.676923+0000","last_undegraded":"2023-12-11T17:44:25.676923+0000","last_fullsized":"2023-12-11T17:44:25.676923+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_clean_scrub_stamp":"2023-12-11T17:44:23.663161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T18:28:12.922991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"86'3","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T17:44:46.775636+0000","last_change":"2023-12-11T17:44:19.658825+0000","last_active":"2023-12-11T17:44:46.775636+0000","last_peered":"2023-12-11T17:44:46.775636+0000","last_clean":"2023-12-11T17:44:46.775636+0000","last_became_active":"2023-12-11T17:44:19.658425+0000","last_became_peered":"2023-12-11T17:44:19.658425+0000","last_unstale":"2023-12-11T17:44:46.775636+0000","last_undegraded":"2023-12-11T17:44:46.775636+0000","last_fullsized":"2023-12-11T17:44:46.775636+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_clean_scrub_stamp":"2023-12-11T17:44:18.636443+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T04:06:15.289454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.680643+0000","last_change":"2023-12-11T17:44:20.617557+0000","last_active":"2023-12-11T17:44:24.680643+0000","last_peered":"2023-12-11T17:44:24.680643+0000","last_clean":"2023-12-11T17:44:24.680643+0000","last_became_active":"2023-12-11T17:44:20.616879+0000","last_became_peered":"2023-12-11T17:44:20.616879+0000","last_unstale":"2023-12-11T17:44:24.680643+0000","last_undegraded":"2023-12-11T17:44:24.680643+0000","last_fullsized":"2023-12-11T17:44:24.680643+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T19:34:04.954862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T17:44:25.781207+0000","last_change":"2023-12-11T17:44:24.691471+0000","last_active":"2023-12-11T17:44:25.781207+0000","last_peered":"2023-12-11T17:44:25.781207+0000","last_clean":"2023-12-11T17:44:25.781207+0000","last_became_active":"2023-12-11T17:44:24.690390+0000","last_became_peered":"2023-12-11T17:44:24.690390+0000","last_unstale":"2023-12-11T17:44:25.781207+0000","last_undegraded":"2023-12-11T17:44:25.781207+0000","last_fullsized":"2023-12-11T17:44:25.781207+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_clean_scrub_stamp":"2023-12-11T17:44:23.663161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T00:26:20.141717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"81'2","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.685191+0000","last_change":"2023-12-11T17:44:19.661004+0000","last_active":"2023-12-11T17:44:24.685191+0000","last_peered":"2023-12-11T17:44:24.685191+0000","last_clean":"2023-12-11T17:44:24.685191+0000","last_became_active":"2023-12-11T17:44:19.660691+0000","last_became_peered":"2023-12-11T17:44:19.660691+0000","last_unstale":"2023-12-11T17:44:24.685191+0000","last_undegraded":"2023-12-11T17:44:24.685191+0000","last_fullsized":"2023-12-11T17:44:24.685191+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_clean_scrub_stamp":"2023-12-11T17:44:18.636443+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T17:54:53.956473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.683776+0000","last_change":"2023-12-11T17:44:20.621437+0000","last_active":"2023-12-11T17:44:24.683776+0000","last_peered":"2023-12-11T17:44:24.683776+0000","last_clean":"2023-12-11T17:44:24.683776+0000","last_became_active":"2023-12-11T17:44:20.621113+0000","last_became_peered":"2023-12-11T17:44:20.621113+0000","last_unstale":"2023-12-11T17:44:24.683776+0000","last_undegraded":"2023-12-11T17:44:24.683776+0000","last_fullsized":"2023-12-11T17:44:24.683776+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T21:05:50.097605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T17:44:25.677532+0000","last_change":"2023-12-11T17:44:24.697432+0000","last_active":"2023-12-11T17:44:25.677532+0000","last_peered":"2023-12-11T17:44:25.677532+0000","last_clean":"2023-12-11T17:44:25.677532+0000","last_became_active":"2023-12-11T17:44:24.696924+0000","last_became_peered":"2023-12-11T17:44:24.696924+0000","last_unstale":"2023-12-11T17:44:25.677532+0000","last_undegraded":"2023-12-11T17:44:25.677532+0000","last_fullsized":"2023-12-11T17:44:25.677532+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_clean_scrub_stamp":"2023-12-11T17:44:23.663161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T21:00:01.967894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T17:44:45.763470+0000","last_change":"2023-12-11T17:44:19.665080+0000","last_active":"2023-12-11T17:44:45.763470+0000","last_peered":"2023-12-11T17:44:45.763470+0000","last_clean":"2023-12-11T17:44:45.763470+0000","last_became_active":"2023-12-11T17:44:19.664862+0000","last_became_peered":"2023-12-11T17:44:19.664862+0000","last_unstale":"2023-12-11T17:44:45.763470+0000","last_undegraded":"2023-12-11T17:44:45.763470+0000","last_fullsized":"2023-12-11T17:44:45.763470+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_clean_scrub_stamp":"2023-12-11T17:44:18.636443+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T20:22:05.909035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.679991+0000","last_change":"2023-12-11T17:44:20.614140+0000","last_active":"2023-12-11T17:44:24.679991+0000","last_peered":"2023-12-11T17:44:24.679991+0000","last_clean":"2023-12-11T17:44:24.679991+0000","last_became_active":"2023-12-11T17:44:20.613808+0000","last_became_peered":"2023-12-11T17:44:20.613808+0000","last_unstale":"2023-12-11T17:44:24.679991+0000","last_undegraded":"2023-12-11T17:44:24.679991+0000","last_fullsized":"2023-12-11T17:44:24.679991+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T20:55:57.572836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T17:44:25.781960+0000","last_change":"2023-12-11T17:44:24.697388+0000","last_active":"2023-12-11T17:44:25.781960+0000","last_peered":"2023-12-11T17:44:25.781960+0000","last_clean":"2023-12-11T17:44:25.781960+0000","last_became_active":"2023-12-11T17:44:24.696964+0000","last_became_peered":"2023-12-11T17:44:24.696964+0000","last_unstale":"2023-12-11T17:44:25.781960+0000","last_undegraded":"2023-12-11T17:44:25.781960+0000","last_fullsized":"2023-12-11T17:44:25.781960+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_clean_scrub_stamp":"2023-12-11T17:44:23.663161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T18:08:04.162075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T17:44:45.764439+0000","last_change":"2023-12-11T17:44:19.660327+0000","last_active":"2023-12-11T17:44:45.764439+0000","last_peered":"2023-12-11T17:44:45.764439+0000","last_clean":"2023-12-11T17:44:45.764439+0000","last_became_active":"2023-12-11T17:44:19.659999+0000","last_became_peered":"2023-12-11T17:44:19.659999+0000","last_unstale":"2023-12-11T17:44:45.764439+0000","last_undegraded":"2023-12-11T17:44:45.764439+0000","last_fullsized":"2023-12-11T17:44:45.764439+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_clean_scrub_stamp":"2023-12-11T17:44:18.636443+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T23:51:24.580854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.674733+0000","last_change":"2023-12-11T17:44:20.624126+0000","last_active":"2023-12-11T17:44:24.674733+0000","last_peered":"2023-12-11T17:44:24.674733+0000","last_clean":"2023-12-11T17:44:24.674733+0000","last_became_active":"2023-12-11T17:44:20.623836+0000","last_became_peered":"2023-12-11T17:44:20.623836+0000","last_unstale":"2023-12-11T17:44:24.674733+0000","last_undegraded":"2023-12-11T17:44:24.674733+0000","last_fullsized":"2023-12-11T17:44:24.674733+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T21:43:39.123299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T17:44:25.677271+0000","last_change":"2023-12-11T17:44:24.698994+0000","last_active":"2023-12-11T17:44:25.677271+0000","last_peered":"2023-12-11T17:44:25.677271+0000","last_clean":"2023-12-11T17:44:25.677271+0000","last_became_active":"2023-12-11T17:44:24.698583+0000","last_became_peered":"2023-12-11T17:44:24.698583+0000","last_unstale":"2023-12-11T17:44:25.677271+0000","last_undegraded":"2023-12-11T17:44:25.677271+0000","last_fullsized":"2023-12-11T17:44:25.677271+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_clean_scrub_stamp":"2023-12-11T17:44:23.663161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T00:03:20.178331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.686821+0000","last_change":"2023-12-11T17:44:19.662505+0000","last_active":"2023-12-11T17:44:24.686821+0000","last_peered":"2023-12-11T17:44:24.686821+0000","last_clean":"2023-12-11T17:44:24.686821+0000","last_became_active":"2023-12-11T17:44:19.661478+0000","last_became_peered":"2023-12-11T17:44:19.661478+0000","last_unstale":"2023-12-11T17:44:24.686821+0000","last_undegraded":"2023-12-11T17:44:24.686821+0000","last_fullsized":"2023-12-11T17:44:24.686821+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_clean_scrub_stamp":"2023-12-11T17:44:18.636443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T23:28:09.345693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.680169+0000","last_change":"2023-12-11T17:44:20.615026+0000","last_active":"2023-12-11T17:44:24.680169+0000","last_peered":"2023-12-11T17:44:24.680169+0000","last_clean":"2023-12-11T17:44:24.680169+0000","last_became_active":"2023-12-11T17:44:20.614669+0000","last_became_peered":"2023-12-11T17:44:20.614669+0000","last_unstale":"2023-12-11T17:44:24.680169+0000","last_undegraded":"2023-12-11T17:44:24.680169+0000","last_fullsized":"2023-12-11T17:44:24.680169+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T20:52:20.455326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T17:44:25.677459+0000","last_change":"2023-12-11T17:44:24.694681+0000","last_active":"2023-12-11T17:44:25.677459+0000","last_peered":"2023-12-11T17:44:25.677459+0000","last_clean":"2023-12-11T17:44:25.677459+0000","last_became_active":"2023-12-11T17:44:24.694172+0000","last_became_peered":"2023-12-11T17:44:24.694172+0000","last_unstale":"2023-12-11T17:44:25.677459+0000","last_undegraded":"2023-12-11T17:44:25.677459+0000","last_fullsized":"2023-12-11T17:44:25.677459+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_clean_scrub_stamp":"2023-12-11T17:44:23.663161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T20:54:46.367735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T17:44:45.763580+0000","last_change":"2023-12-11T17:44:19.664254+0000","last_active":"2023-12-11T17:44:45.763580+0000","last_peered":"2023-12-11T17:44:45.763580+0000","last_clean":"2023-12-11T17:44:45.763580+0000","last_became_active":"2023-12-11T17:44:19.663814+0000","last_became_peered":"2023-12-11T17:44:19.663814+0000","last_unstale":"2023-12-11T17:44:45.763580+0000","last_undegraded":"2023-12-11T17:44:45.763580+0000","last_fullsized":"2023-12-11T17:44:45.763580+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_clean_scrub_stamp":"2023-12-11T17:44:18.636443+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T20:11:23.717547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.686362+0000","last_change":"2023-12-11T17:44:20.621879+0000","last_active":"2023-12-11T17:44:24.686362+0000","last_peered":"2023-12-11T17:44:24.686362+0000","last_clean":"2023-12-11T17:44:24.686362+0000","last_became_active":"2023-12-11T17:44:20.621430+0000","last_became_peered":"2023-12-11T17:44:20.621430+0000","last_unstale":"2023-12-11T17:44:24.686362+0000","last_undegraded":"2023-12-11T17:44:24.686362+0000","last_fullsized":"2023-12-11T17:44:24.686362+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T18:08:46.359629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T17:44:25.782485+0000","last_change":"2023-12-11T17:44:24.700599+0000","last_active":"2023-12-11T17:44:25.782485+0000","last_peered":"2023-12-11T17:44:25.782485+0000","last_clean":"2023-12-11T17:44:25.782485+0000","last_became_active":"2023-12-11T17:44:24.700121+0000","last_became_peered":"2023-12-11T17:44:24.700121+0000","last_unstale":"2023-12-11T17:44:25.782485+0000","last_undegraded":"2023-12-11T17:44:25.782485+0000","last_fullsized":"2023-12-11T17:44:25.782485+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_clean_scrub_stamp":"2023-12-11T17:44:23.663161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T01:08:18.000642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T17:44:46.775265+0000","last_change":"2023-12-11T17:44:19.662414+0000","last_active":"2023-12-11T17:44:46.775265+0000","last_peered":"2023-12-11T17:44:46.775265+0000","last_clean":"2023-12-11T17:44:46.775265+0000","last_became_active":"2023-12-11T17:44:19.661753+0000","last_became_peered":"2023-12-11T17:44:19.661753+0000","last_unstale":"2023-12-11T17:44:46.775265+0000","last_undegraded":"2023-12-11T17:44:46.775265+0000","last_fullsized":"2023-12-11T17:44:46.775265+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_clean_scrub_stamp":"2023-12-11T17:44:18.636443+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T00:49:11.713347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.688473+0000","last_change":"2023-12-11T17:44:20.620973+0000","last_active":"2023-12-11T17:44:24.688473+0000","last_peered":"2023-12-11T17:44:24.688473+0000","last_clean":"2023-12-11T17:44:24.688473+0000","last_became_active":"2023-12-11T17:44:20.620284+0000","last_became_peered":"2023-12-11T17:44:20.620284+0000","last_unstale":"2023-12-11T17:44:24.688473+0000","last_undegraded":"2023-12-11T17:44:24.688473+0000","last_fullsized":"2023-12-11T17:44:24.688473+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:00:52.529062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T17:44:25.781009+0000","last_change":"2023-12-11T17:44:24.696888+0000","last_active":"2023-12-11T17:44:25.781009+0000","last_peered":"2023-12-11T17:44:25.781009+0000","last_clean":"2023-12-11T17:44:25.781009+0000","last_became_active":"2023-12-11T17:44:24.696475+0000","last_became_peered":"2023-12-11T17:44:24.696475+0000","last_unstale":"2023-12-11T17:44:25.781009+0000","last_undegraded":"2023-12-11T17:44:25.781009+0000","last_fullsized":"2023-12-11T17:44:25.781009+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_clean_scrub_stamp":"2023-12-11T17:44:23.663161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T21:42:17.287557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"81'1","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T17:44:25.782089+0000","last_change":"2023-12-11T17:44:19.659094+0000","last_active":"2023-12-11T17:44:25.782089+0000","last_peered":"2023-12-11T17:44:25.782089+0000","last_clean":"2023-12-11T17:44:25.782089+0000","last_became_active":"2023-12-11T17:44:19.658794+0000","last_became_peered":"2023-12-11T17:44:19.658794+0000","last_unstale":"2023-12-11T17:44:25.782089+0000","last_undegraded":"2023-12-11T17:44:25.782089+0000","last_fullsized":"2023-12-11T17:44:25.782089+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_clean_scrub_stamp":"2023-12-11T17:44:18.636443+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T01:39:41.151915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.687828+0000","last_change":"2023-12-11T17:44:20.623342+0000","last_active":"2023-12-11T17:44:24.687828+0000","last_peered":"2023-12-11T17:44:24.687828+0000","last_clean":"2023-12-11T17:44:24.687828+0000","last_became_active":"2023-12-11T17:44:20.622957+0000","last_became_peered":"2023-12-11T17:44:20.622957+0000","last_unstale":"2023-12-11T17:44:24.687828+0000","last_undegraded":"2023-12-11T17:44:24.687828+0000","last_fullsized":"2023-12-11T17:44:24.687828+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:25:00.910645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-11T17:44:25.782300+0000","last_change":"2023-12-11T17:44:24.690746+0000","last_active":"2023-12-11T17:44:25.782300+0000","last_peered":"2023-12-11T17:44:25.782300+0000","last_clean":"2023-12-11T17:44:25.782300+0000","last_became_active":"2023-12-11T17:44:24.690290+0000","last_became_peered":"2023-12-11T17:44:24.690290+0000","last_unstale":"2023-12-11T17:44:25.782300+0000","last_undegraded":"2023-12-11T17:44:25.782300+0000","last_fullsized":"2023-12-11T17:44:25.782300+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_clean_scrub_stamp":"2023-12-11T17:44:23.663161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T01:26:14.409832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"81'2","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.676412+0000","last_change":"2023-12-11T17:44:19.655775+0000","last_active":"2023-12-11T17:44:24.676412+0000","last_peered":"2023-12-11T17:44:24.676412+0000","last_clean":"2023-12-11T17:44:24.676412+0000","last_became_active":"2023-12-11T17:44:19.654360+0000","last_became_peered":"2023-12-11T17:44:19.654360+0000","last_unstale":"2023-12-11T17:44:24.676412+0000","last_undegraded":"2023-12-11T17:44:24.676412+0000","last_fullsized":"2023-12-11T17:44:24.676412+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_clean_scrub_stamp":"2023-12-11T17:44:18.636443+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T00:07:08.678389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.688216+0000","last_change":"2023-12-11T17:44:20.614898+0000","last_active":"2023-12-11T17:44:24.688216+0000","last_peered":"2023-12-11T17:44:24.688216+0000","last_clean":"2023-12-11T17:44:24.688216+0000","last_became_active":"2023-12-11T17:44:20.613972+0000","last_became_peered":"2023-12-11T17:44:20.613972+0000","last_unstale":"2023-12-11T17:44:24.688216+0000","last_undegraded":"2023-12-11T17:44:24.688216+0000","last_fullsized":"2023-12-11T17:44:24.688216+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T04:28:51.576955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.674777+0000","last_change":"2023-12-11T17:44:20.617502+0000","last_active":"2023-12-11T17:44:24.674777+0000","last_peered":"2023-12-11T17:44:24.674777+0000","last_clean":"2023-12-11T17:44:24.674777+0000","last_became_active":"2023-12-11T17:44:20.617252+0000","last_became_peered":"2023-12-11T17:44:20.617252+0000","last_unstale":"2023-12-11T17:44:24.674777+0000","last_undegraded":"2023-12-11T17:44:24.674777+0000","last_fullsized":"2023-12-11T17:44:24.674777+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T19:23:37.133765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.686677+0000","last_change":"2023-12-11T17:44:20.619325+0000","last_active":"2023-12-11T17:44:24.686677+0000","last_peered":"2023-12-11T17:44:24.686677+0000","last_clean":"2023-12-11T17:44:24.686677+0000","last_became_active":"2023-12-11T17:44:20.619016+0000","last_became_peered":"2023-12-11T17:44:20.619016+0000","last_unstale":"2023-12-11T17:44:24.686677+0000","last_undegraded":"2023-12-11T17:44:24.686677+0000","last_fullsized":"2023-12-11T17:44:24.686677+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T20:21:24.877379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.686636+0000","last_change":"2023-12-11T17:44:20.624432+0000","last_active":"2023-12-11T17:44:24.686636+0000","last_peered":"2023-12-11T17:44:24.686636+0000","last_clean":"2023-12-11T17:44:24.686636+0000","last_became_active":"2023-12-11T17:44:20.624148+0000","last_became_peered":"2023-12-11T17:44:20.624148+0000","last_unstale":"2023-12-11T17:44:24.686636+0000","last_undegraded":"2023-12-11T17:44:24.686636+0000","last_fullsized":"2023-12-11T17:44:24.686636+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T19:15:46.054968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.681296+0000","last_change":"2023-12-11T17:44:20.624134+0000","last_active":"2023-12-11T17:44:24.681296+0000","last_peered":"2023-12-11T17:44:24.681296+0000","last_clean":"2023-12-11T17:44:24.681296+0000","last_became_active":"2023-12-11T17:44:20.623858+0000","last_became_peered":"2023-12-11T17:44:20.623858+0000","last_unstale":"2023-12-11T17:44:24.681296+0000","last_undegraded":"2023-12-11T17:44:24.681296+0000","last_fullsized":"2023-12-11T17:44:24.681296+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:24:48.946619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.677112+0000","last_change":"2023-12-11T17:44:20.620407+0000","last_active":"2023-12-11T17:44:24.677112+0000","last_peered":"2023-12-11T17:44:24.677112+0000","last_clean":"2023-12-11T17:44:24.677112+0000","last_became_active":"2023-12-11T17:44:20.620106+0000","last_became_peered":"2023-12-11T17:44:20.620106+0000","last_unstale":"2023-12-11T17:44:24.677112+0000","last_undegraded":"2023-12-11T17:44:24.677112+0000","last_fullsized":"2023-12-11T17:44:24.677112+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T22:18:36.780037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.675040+0000","last_change":"2023-12-11T17:44:20.618426+0000","last_active":"2023-12-11T17:44:24.675040+0000","last_peered":"2023-12-11T17:44:24.675040+0000","last_clean":"2023-12-11T17:44:24.675040+0000","last_became_active":"2023-12-11T17:44:20.618136+0000","last_became_peered":"2023-12-11T17:44:20.618136+0000","last_unstale":"2023-12-11T17:44:24.675040+0000","last_undegraded":"2023-12-11T17:44:24.675040+0000","last_fullsized":"2023-12-11T17:44:24.675040+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T05:02:04.096866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.687279+0000","last_change":"2023-12-11T17:44:20.619609+0000","last_active":"2023-12-11T17:44:24.687279+0000","last_peered":"2023-12-11T17:44:24.687279+0000","last_clean":"2023-12-11T17:44:24.687279+0000","last_became_active":"2023-12-11T17:44:20.619345+0000","last_became_peered":"2023-12-11T17:44:20.619345+0000","last_unstale":"2023-12-11T17:44:24.687279+0000","last_undegraded":"2023-12-11T17:44:24.687279+0000","last_fullsized":"2023-12-11T17:44:24.687279+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T19:56:00.839819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.680739+0000","last_change":"2023-12-11T17:44:20.614115+0000","last_active":"2023-12-11T17:44:24.680739+0000","last_peered":"2023-12-11T17:44:24.680739+0000","last_clean":"2023-12-11T17:44:24.680739+0000","last_became_active":"2023-12-11T17:44:20.613800+0000","last_became_peered":"2023-12-11T17:44:20.613800+0000","last_unstale":"2023-12-11T17:44:24.680739+0000","last_undegraded":"2023-12-11T17:44:24.680739+0000","last_fullsized":"2023-12-11T17:44:24.680739+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T20:33:37.170611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.680726+0000","last_change":"2023-12-11T17:44:20.621737+0000","last_active":"2023-12-11T17:44:24.680726+0000","last_peered":"2023-12-11T17:44:24.680726+0000","last_clean":"2023-12-11T17:44:24.680726+0000","last_became_active":"2023-12-11T17:44:20.621320+0000","last_became_peered":"2023-12-11T17:44:20.621320+0000","last_unstale":"2023-12-11T17:44:24.680726+0000","last_undegraded":"2023-12-11T17:44:24.680726+0000","last_fullsized":"2023-12-11T17:44:24.680726+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T22:37:02.316528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.680302+0000","last_change":"2023-12-11T17:44:20.617195+0000","last_active":"2023-12-11T17:44:24.680302+0000","last_peered":"2023-12-11T17:44:24.680302+0000","last_clean":"2023-12-11T17:44:24.680302+0000","last_became_active":"2023-12-11T17:44:20.616529+0000","last_became_peered":"2023-12-11T17:44:20.616529+0000","last_unstale":"2023-12-11T17:44:24.680302+0000","last_undegraded":"2023-12-11T17:44:24.680302+0000","last_fullsized":"2023-12-11T17:44:24.680302+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T23:53:50.558483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.679469+0000","last_change":"2023-12-11T17:44:20.618290+0000","last_active":"2023-12-11T17:44:24.679469+0000","last_peered":"2023-12-11T17:44:24.679469+0000","last_clean":"2023-12-11T17:44:24.679469+0000","last_became_active":"2023-12-11T17:44:20.617982+0000","last_became_peered":"2023-12-11T17:44:20.617982+0000","last_unstale":"2023-12-11T17:44:24.679469+0000","last_undegraded":"2023-12-11T17:44:24.679469+0000","last_fullsized":"2023-12-11T17:44:24.679469+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T21:58:14.444462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.675268+0000","last_change":"2023-12-11T17:44:20.619958+0000","last_active":"2023-12-11T17:44:24.675268+0000","last_peered":"2023-12-11T17:44:24.675268+0000","last_clean":"2023-12-11T17:44:24.675268+0000","last_became_active":"2023-12-11T17:44:20.619504+0000","last_became_peered":"2023-12-11T17:44:20.619504+0000","last_unstale":"2023-12-11T17:44:24.675268+0000","last_undegraded":"2023-12-11T17:44:24.675268+0000","last_fullsized":"2023-12-11T17:44:24.675268+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T19:36:41.193000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-11T17:44:24.675091+0000","last_change":"2023-12-11T17:44:20.612953+0000","last_active":"2023-12-11T17:44:24.675091+0000","last_peered":"2023-12-11T17:44:24.675091+0000","last_clean":"2023-12-11T17:44:24.675091+0000","last_became_active":"2023-12-11T17:44:20.612618+0000","last_became_peered":"2023-12-11T17:44:20.612618+0000","last_unstale":"2023-12-11T17:44:24.675091+0000","last_undegraded":"2023-12-11T17:44:24.675091+0000","last_fullsized":"2023-12-11T17:44:24.675091+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T04:25:39.528227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":12288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":3542208,"data_compressed":211312,"data_compressed_allocated":270336,"data_compressed_original":3506176,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":7128,"num_objects":49,"num_object_clones":0,"num_object_copies":147,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":112,"num_write_kb":35,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":32,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":208896,"data_stored":20642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":99,"ondisk_log_size":99,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":77,"seq":330712481809,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":933888,"data_stored":464187,"data_compressed":11728,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 11 17:44:38 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.72599999999999998}]},{"osd":1,"last update":"Mon Dec 11 17:44:38 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.55000000000000004}]},{"osd":2,"last update":"Mon Dec 11 17:44:38 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.50900000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48599999999999999}]},{"osd":4,"last update":"Mon Dec 11 17:44:38 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.49199999999999999}]},{"osd":5,"last update":"Mon Dec 11 17:44:38 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.745}]},{"osd":6,"last update":"Mon Dec 11 17:44:38 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.448,"5min":1.448,"15min":1.448},"last":0.77200000000000002}]},{"osd":7,"last update":"Mon Dec 11 17:44:38 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.385,"5min":1.385,"15min":1.385},"last":0.61399999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80600000000000005}]},{"osd":9,"last update":"Mon Dec 11 17:44:38 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.5089999999999999,"5min":1.5089999999999999,"15min":1.5089999999999999},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.86899999999999999}]},{"osd":10,"last update":"Mon Dec 11 17:44:38 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.4870000000000001,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.5309999999999999,"5min":1.5309999999999999,"15min":1.5309999999999999},"last":0.85299999999999998}]}]},{"osd":10,"up_from":70,"seq":300647710739,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1019904,"data_stored":1055671,"data_compressed":47493,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36499999999999999}]},{"osd":1,"last update":"Mon Dec 11 17:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.59299999999999997}]},{"osd":2,"last update":"Mon Dec 11 17:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.63200000000000001}]},{"osd":3,"last update":"Mon Dec 11 17:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.49299999999999999}]},{"osd":4,"last update":"Mon Dec 11 17:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.46200000000000002}]},{"osd":5,"last update":"Mon Dec 11 17:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.68999999999999995}]},{"osd":6,"last update":"Mon Dec 11 17:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.70399999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56799999999999995}]},{"osd":8,"last update":"Mon Dec 11 17:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.327,"5min":1.327,"15min":1.327},"last":0.67200000000000004}]},{"osd":9,"last update":"Mon Dec 11 17:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.3979999999999999,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.71999999999999997}]},{"osd":11,"last update":"Mon Dec 11 17:44:40 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.745},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.75600000000000001}]}]},{"osd":9,"up_from":65,"seq":279172874265,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":461033,"data_compressed":11728,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 11 17:43:57 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.45300000000000001}]},{"osd":1,"last update":"Mon Dec 11 17:43:57 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.54800000000000004}]},{"osd":2,"last update":"Mon Dec 11 17:43:57 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.5},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.42599999999999999}]},{"osd":3,"last update":"Mon Dec 11 17:43:57 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.40799999999999997}]},{"osd":4,"last update":"Mon Dec 11 17:43:57 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.157,"5min":1.157,"15min":1.157},"last":0.68700000000000006}]},{"osd":5,"last update":"Mon Dec 11 17:43:57 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.748},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.442}]},{"osd":6,"last update":"Mon Dec 11 17:43:57 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.5289999999999999,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.44800000000000001}]},{"osd":7,"last update":"Mon Dec 11 17:43:57 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.224,"5min":1.224,"15min":1.224},"last":0.60999999999999999}]},{"osd":8,"last update":"Mon Dec 11 17:43:57 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.77600000000000002}]},{"osd":10,"last update":"Mon Dec 11 17:44:18 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.622},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.252,"5min":1.252,"15min":1.252},"last":0.79600000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81699999999999995}]}]},{"osd":8,"up_from":57,"seq":244813135902,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":954368,"data_stored":465049,"data_compressed":11728,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 11 17:44:37 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.54300000000000004,"15min":0.53600000000000003},"min":{"1min":0.36499999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.437},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.56200000000000006,"15min":0.56299999999999994},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.78000000000000003,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.46200000000000002}]},{"osd":1,"last update":"Mon Dec 11 17:44:37 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.57799999999999996,"15min":0.57599999999999996},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.94999999999999996,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.62,"5min":0.59699999999999998,"15min":0.59299999999999997},"min":{"1min":0.42799999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.69699999999999995}]},{"osd":2,"last update":"Mon Dec 11 17:44:37 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.437,"15min":0.42999999999999999},"min":{"1min":0.32000000000000001,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.503},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.47899999999999998,"15min":0.48199999999999998},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.68899999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.53300000000000003}]},{"osd":3,"last update":"Mon Dec 11 17:44:37 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.59599999999999997,"15min":0.59999999999999998},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.73099999999999998,"5min":1.077,"15min":1.077},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.57899999999999996,"15min":0.57599999999999996},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.80200000000000005,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.56899999999999995}]},{"osd":4,"last update":"Mon Dec 11 17:44:37 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.628,"15min":0.625},"min":{"1min":0.47299999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.87,"5min":1.141,"15min":1.141},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.621,"15min":0.61099999999999999},"min":{"1min":0.53000000000000003,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.0069999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.85199999999999998}]},{"osd":5,"last update":"Mon Dec 11 17:44:37 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.56299999999999994,"15min":0.56499999999999995},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.72799999999999998,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.59199999999999997,"15min":0.59799999999999998},"min":{"1min":0.35599999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.76300000000000001,"5min":1.482,"15min":1.482},"last":0.622}]},{"osd":6,"last update":"Mon Dec 11 17:44:37 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.64300000000000002,"15min":0.63400000000000001},"min":{"1min":0.45000000000000001,"5min":0.37,"15min":0.37},"max":{"1min":0.84599999999999997,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.751},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.68500000000000005,"15min":0.68300000000000005},"min":{"1min":0.53600000000000003,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.87,"5min":1.008,"15min":1.008},"last":0.77100000000000002}]},{"osd":7,"last update":"Mon Dec 11 17:44:37 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.68000000000000005,"15min":0.67300000000000004},"min":{"1min":0.48699999999999999,"5min":0.432,"15min":0.432},"max":{"1min":0.93300000000000005,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.73099999999999998,"15min":0.72499999999999998},"min":{"1min":0.55400000000000005,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.546,"5min":1.546,"15min":1.546},"last":0.81699999999999995}]},{"osd":9,"last update":"Mon Dec 11 17:43:59 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.80500000000000005}]},{"osd":10,"last update":"Mon Dec 11 17:44:17 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.88}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79200000000000004}]}]},{"osd":7,"up_from":50,"seq":214748364833,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":995328,"data_stored":1052375,"data_compressed":47493,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 11 17:44:13 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.54700000000000004,"15min":0.54300000000000004},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.44},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.55200000000000005,"15min":0.54800000000000004},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.42399999999999999}]},{"osd":1,"last update":"Mon Dec 11 17:44:13 2023","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.40999999999999998,"15min":0.40999999999999998},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.56799999999999995,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.441},{"interface":"front","average":{"1min":0.436,"5min":0.44500000000000001,"15min":0.44700000000000001},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.60399999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.55500000000000005}]},{"osd":2,"last update":"Mon Dec 11 17:44:13 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.41299999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.627,"5min":0.59899999999999998,"15min":0.59399999999999997},"min":{"1min":0.46200000000000002,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.69299999999999995}]},{"osd":3,"last update":"Mon Dec 11 17:44:13 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.56999999999999995,"15min":0.56399999999999995},"min":{"1min":0.38700000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.60699999999999998,"15min":0.60299999999999998},"min":{"1min":0.45600000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.67100000000000004}]},{"osd":4,"last update":"Mon Dec 11 17:44:13 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.53500000000000003,"15min":0.53100000000000003},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.503},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.48699999999999999,"15min":0.49099999999999999},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.60399999999999998}]},{"osd":5,"last update":"Mon Dec 11 17:44:13 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.625,"15min":0.625},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.67200000000000004,"15min":0.66800000000000004},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.65000000000000002}]},{"osd":6,"last update":"Mon Dec 11 17:44:13 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.503,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.63100000000000001,"15min":0.63500000000000001},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.71599999999999997}]},{"osd":8,"last update":"Mon Dec 11 17:44:37 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.68300000000000005,"15min":0.69199999999999995},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.88200000000000001,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.70299999999999996,"15min":0.70899999999999996},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.92500000000000004,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.85999999999999999}]},{"osd":9,"last update":"Mon Dec 11 17:43:57 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.80600000000000005}]},{"osd":10,"last update":"Mon Dec 11 17:44:19 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.75900000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88700000000000001}]}]},{"osd":6,"up_from":45,"seq":193273528358,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":929792,"data_stored":460215,"data_compressed":11728,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 11 17:43:54 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.39500000000000002,"15min":0.38500000000000001},"min":{"1min":0.30499999999999999,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.27400000000000002},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.41299999999999998,"15min":0.40699999999999997},"min":{"1min":0.314,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.32600000000000001}]},{"osd":1,"last update":"Mon Dec 11 17:43:54 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.54500000000000004,"15min":0.53900000000000003},"min":{"1min":0.371,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.55700000000000005,"15min":0.54700000000000004},"min":{"1min":0.41299999999999998,"5min":0.375,"15min":0.375},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.65400000000000003}]},{"osd":2,"last update":"Mon Dec 11 17:43:54 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.56799999999999995,"15min":0.55400000000000005},"min":{"1min":0.44400000000000001,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.107,"5min":1.107,"15min":1.107},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.57699999999999996,"15min":0.55600000000000005},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.236,"5min":1.236,"15min":1.236},"last":0.42799999999999999}]},{"osd":3,"last update":"Mon Dec 11 17:43:54 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.49399999999999999,"15min":0.48199999999999998},"min":{"1min":0.34899999999999998,"5min":0.317,"15min":0.317},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.5,"15min":0.48699999999999999},"min":{"1min":0.318,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.503}]},{"osd":4,"last update":"Mon Dec 11 17:43:54 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.61099999999999999,"15min":0.60299999999999998},"min":{"1min":0.39900000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.63700000000000001,"15min":0.629},"min":{"1min":0.42799999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.62}]},{"osd":5,"last update":"Mon Dec 11 17:43:54 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.63800000000000001,"15min":0.628},"min":{"1min":0.45900000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.66200000000000003,"15min":0.65000000000000002},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.58599999999999997}]},{"osd":7,"last update":"Mon Dec 11 17:44:12 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.67300000000000004,"15min":0.66400000000000003},"min":{"1min":0.47899999999999998,"5min":0.442,"15min":0.442},"max":{"1min":1.3169999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.71199999999999997,"15min":0.70799999999999996},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.68100000000000005}]},{"osd":8,"last update":"Mon Dec 11 17:44:40 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.752,"15min":0.751},"min":{"1min":0.54900000000000004,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.002,"5min":1.157,"15min":1.157},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.69899999999999995,"15min":0.69699999999999995},"min":{"1min":0.495,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.97399999999999998,"5min":1.028,"15min":1.028},"last":0.47599999999999998}]},{"osd":9,"last update":"Mon Dec 11 17:43:56 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.371,"5min":1.371,"15min":1.371},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.45000000000000001}]},{"osd":10,"last update":"Mon Dec 11 17:44:19 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.425,"5min":1.425,"15min":1.425},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.288,"5min":1.288,"15min":1.288},"last":0.69999999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56200000000000006}]}]},{"osd":1,"up_from":13,"seq":55834574907,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":942080,"data_stored":464056,"data_compressed":11728,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 11 17:44:13 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.498,"15min":0.48099999999999998},"min":{"1min":0.34200000000000003,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.73799999999999999,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.52900000000000003,"15min":0.51200000000000001},"min":{"1min":0.40600000000000003,"5min":0.308,"15min":0.308},"max":{"1min":0.75600000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.80200000000000005}]},{"osd":2,"last update":"Mon Dec 11 17:44:30 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.54800000000000004,"15min":0.51800000000000002},"min":{"1min":0.34799999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":1.4690000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.94099999999999995},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.57099999999999995,"15min":0.54200000000000004},"min":{"1min":0.41699999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.4530000000000001,"5min":1.4530000000000001,"15min":1.4530000000000001},"last":0.66500000000000004}]},{"osd":3,"last update":"Mon Dec 11 17:43:53 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.51900000000000002,"15min":0.51500000000000001},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.94699999999999995,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.53000000000000003,"15min":0.51400000000000001},"min":{"1min":0.36199999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.81699999999999995}]},{"osd":4,"last update":"Mon Dec 11 17:44:13 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.42499999999999999,"15min":0.40200000000000002},"min":{"1min":0.28999999999999998,"5min":0.254,"15min":0.254},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.42799999999999999,"15min":0.39600000000000002},"min":{"1min":0.27700000000000002,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.53600000000000003}]},{"osd":5,"last update":"Mon Dec 11 17:44:36 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.59899999999999998,"15min":0.56299999999999994},"min":{"1min":0.41699999999999998,"5min":0.37,"15min":0.37},"max":{"1min":1.5109999999999999,"5min":1.5109999999999999,"15min":1.5109999999999999},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.60699999999999998,"15min":0.57599999999999996},"min":{"1min":0.53900000000000003,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.411,"5min":1.411,"15min":1.411},"last":0.85299999999999998}]},{"osd":6,"last update":"Mon Dec 11 17:43:53 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.59299999999999997,"15min":0.58699999999999997},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.629,"5min":0.60899999999999999,"15min":0.60499999999999998},"min":{"1min":0.38800000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.86699999999999999}]},{"osd":7,"last update":"Mon Dec 11 17:44:13 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.58499999999999996,"15min":0.57999999999999996},"min":{"1min":0.34499999999999997,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.996,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.58999999999999997,"15min":0.58899999999999997},"min":{"1min":0.435,"5min":0.309,"15min":0.309},"max":{"1min":0.94799999999999995,"5min":1.379,"15min":1.379},"last":0.61699999999999999}]},{"osd":8,"last update":"Mon Dec 11 17:44:36 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.71699999999999997,"15min":0.70699999999999996},"min":{"1min":0.48699999999999999,"5min":0.433,"15min":0.433},"max":{"1min":1.5349999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.755,"5min":0.69399999999999995,"15min":0.68400000000000005},"min":{"1min":0.48999999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.4259999999999999,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":0.78900000000000003}]},{"osd":9,"last update":"Mon Dec 11 17:44:00 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.97799999999999998}]},{"osd":10,"last update":"Mon Dec 11 17:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.69899999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91700000000000004}]}]},{"osd":0,"up_from":8,"seq":34359738431,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":933888,"data_stored":460305,"data_compressed":11728,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Dec 11 17:44:14 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.51000000000000001,"15min":0.502},"min":{"1min":0.34499999999999997,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.52300000000000002,"15min":0.48999999999999999},"min":{"1min":0.34200000000000003,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.86799999999999999}]},{"osd":2,"last update":"Mon Dec 11 17:44:38 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.56799999999999995,"15min":0.51500000000000001},"min":{"1min":0.38200000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":4.1029999999999998,"5min":4.1029999999999998,"15min":4.1029999999999998},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.55100000000000005,"15min":0.51100000000000001},"min":{"1min":0.33400000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.746,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.90200000000000002}]},{"osd":3,"last update":"Mon Dec 11 17:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.38800000000000001,"15min":0.38},"min":{"1min":0.30199999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.433,"15min":0.42299999999999999},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.38600000000000001}]},{"osd":4,"last update":"Mon Dec 11 17:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.53400000000000003,"15min":0.51800000000000002},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.81999999999999995,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.625,"5min":0.55400000000000005,"15min":0.54000000000000004},"min":{"1min":0.36499999999999999,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.626}]},{"osd":5,"last update":"Mon Dec 11 17:44:38 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.60599999999999998,"15min":0.58799999999999997},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":2.399,"5min":2.399,"15min":2.399},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.61299999999999999,"15min":0.59599999999999997},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.82699999999999996,"5min":0.995,"15min":0.995},"last":0.95199999999999996}]},{"osd":6,"last update":"Mon Dec 11 17:43:55 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.50900000000000001,"15min":0.499},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.59799999999999998,"15min":0.58599999999999997},"min":{"1min":0.34599999999999997,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.57499999999999996}]},{"osd":7,"last update":"Mon Dec 11 17:44:14 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.68000000000000005,"15min":0.67200000000000004},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.63,"15min":0.624},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.65000000000000002}]},{"osd":8,"last update":"Mon Dec 11 17:44:38 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.70299999999999996,"15min":0.68999999999999995},"min":{"1min":0.43099999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":3.3450000000000002,"5min":3.3450000000000002,"15min":3.3450000000000002},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.76100000000000001,"15min":0.76900000000000002},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.054,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.79700000000000004}]},{"osd":9,"last update":"Mon Dec 11 17:44:00 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.505},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.73199999999999998}]},{"osd":10,"last update":"Mon Dec 11 17:44:20 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.92700000000000005},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.76000000000000001}]},{"osd":11,"last update":"Mon Dec 11 17:44:42 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":2.456,"5min":2.456,"15min":2.456},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.107,"5min":1.107,"15min":1.107},"last":0.97899999999999998}]}]},{"osd":2,"up_from":18,"seq":77309411383,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":966656,"data_stored":465406,"data_compressed":11728,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 11 17:44:34 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.52900000000000003,"15min":0.502},"min":{"1min":0.32100000000000001,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.84899999999999998,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.51800000000000002,"15min":0.50800000000000001},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.79200000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.45300000000000001}]},{"osd":1,"last update":"Mon Dec 11 17:44:34 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.54100000000000004,"15min":0.51900000000000002},"min":{"1min":0.38900000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.56999999999999995,"15min":0.54100000000000004},"min":{"1min":0.35399999999999998,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.70099999999999996}]},{"osd":3,"last update":"Mon Dec 11 17:44:00 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.55600000000000005,"15min":0.53600000000000003},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.53400000000000003,"15min":0.52500000000000002},"min":{"1min":0.40600000000000003,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.67900000000000005}]},{"osd":4,"last update":"Mon Dec 11 17:44:13 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.54700000000000004,"15min":0.54000000000000004},"min":{"1min":0.38300000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.76200000000000001,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.58399999999999996,"15min":0.56899999999999995},"min":{"1min":0.41999999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.60299999999999998}]},{"osd":5,"last update":"Mon Dec 11 17:44:34 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.46800000000000003,"15min":0.44600000000000001},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.46000000000000002,"15min":0.441},"min":{"1min":0.28299999999999997,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.51800000000000002}]},{"osd":6,"last update":"Mon Dec 11 17:43:52 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.64100000000000001,"15min":0.63200000000000001},"min":{"1min":0.46100000000000002,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.60899999999999999,"15min":0.60499999999999998},"min":{"1min":0.48799999999999999,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.81799999999999995,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.56599999999999995}]},{"osd":7,"last update":"Mon Dec 11 17:44:17 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.60199999999999998,"15min":0.59799999999999998},"min":{"1min":0.46100000000000002,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.68100000000000005,"15min":0.67500000000000004},"min":{"1min":0.45600000000000002,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.749}]},{"osd":8,"last update":"Mon Dec 11 17:44:40 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.58199999999999996,"15min":0.57499999999999996},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.627},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.64600000000000002,"15min":0.64100000000000001},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.81799999999999995}]},{"osd":9,"last update":"Mon Dec 11 17:44:00 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.89500000000000002}]},{"osd":10,"last update":"Mon Dec 11 17:44:18 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.875}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78000000000000003}]}]},{"osd":3,"up_from":25,"seq":107374182450,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":942080,"data_stored":460168,"data_compressed":11728,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 11 17:43:53 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.376,"15min":0.35499999999999998},"min":{"1min":0.29399999999999998,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.433,"15min":0.42099999999999999},"min":{"1min":0.29599999999999999,"5min":0.251,"15min":0.251},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.51600000000000001}]},{"osd":1,"last update":"Mon Dec 11 17:43:53 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.57299999999999995,"15min":0.55700000000000005},"min":{"1min":0.42499999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.57799999999999996,"15min":0.56599999999999995},"min":{"1min":0.443,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.73199999999999998}]},{"osd":2,"last update":"Mon Dec 11 17:43:53 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.60999999999999999,"15min":0.60499999999999998},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.60299999999999998,"15min":0.60099999999999998},"min":{"1min":0.47799999999999998,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.92700000000000005}]},{"osd":4,"last update":"Mon Dec 11 17:44:19 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.58399999999999996,"15min":0.56499999999999995},"min":{"1min":0.435,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.99399999999999999},{"interface":"front","average":{"1min":0.627,"5min":0.55800000000000005,"15min":0.55200000000000005},"min":{"1min":0.41899999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.67400000000000004}]},{"osd":5,"last update":"Mon Dec 11 17:44:35 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.61299999999999999,"15min":0.58899999999999997},"min":{"1min":0.432,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":1.0069999999999999},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.64100000000000001,"15min":0.61799999999999999},"min":{"1min":0.41099999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.93100000000000005,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.63}]},{"osd":6,"last update":"Mon Dec 11 17:43:53 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.48299999999999998,"15min":0.46700000000000003},"min":{"1min":0.318,"5min":0.316,"15min":0.316},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.58899999999999997,"15min":0.55700000000000005},"min":{"1min":0.498,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.88100000000000001}]},{"osd":7,"last update":"Mon Dec 11 17:44:13 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.67300000000000004,"15min":0.65100000000000002},"min":{"1min":0.58799999999999997,"5min":0.432,"15min":0.432},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.94799999999999995},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.68899999999999995,"15min":0.67100000000000004},"min":{"1min":0.55900000000000005,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.96799999999999997}]},{"osd":8,"last update":"Mon Dec 11 17:43:38 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.66900000000000004}]},{"osd":9,"last update":"Mon Dec 11 17:43:57 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.751}]},{"osd":10,"last update":"Mon Dec 11 17:44:19 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.98299999999999998},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":1.024}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90600000000000003}]}]},{"osd":4,"up_from":32,"seq":137438953518,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":933888,"data_stored":460237,"data_compressed":11728,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 11 17:44:15 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.52800000000000002,"15min":0.51600000000000001},"min":{"1min":0.36899999999999999,"5min":0.31,"15min":0.31},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.51800000000000002,"15min":0.50700000000000001},"min":{"1min":0.35599999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.34399999999999997}]},{"osd":1,"last update":"Mon Dec 11 17:44:15 2023","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.39300000000000002,"15min":0.38500000000000001},"min":{"1min":0.28699999999999998,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.59399999999999997,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.39500000000000002,"15min":0.375},"min":{"1min":0.26000000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.434}]},{"osd":2,"last update":"Mon Dec 11 17:44:15 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.57499999999999996,"15min":0.55500000000000005},"min":{"1min":0.36099999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.58599999999999997,"15min":0.57999999999999996},"min":{"1min":0.41599999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.61299999999999999}]},{"osd":3,"last update":"Mon Dec 11 17:44:15 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.59799999999999998,"15min":0.58299999999999996},"min":{"1min":0.40000000000000002,"5min":0.372,"15min":0.372},"max":{"1min":0.82699999999999996,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.60499999999999998,"15min":0.58999999999999997},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.71499999999999997}]},{"osd":5,"last update":"Mon Dec 11 17:44:36 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.59499999999999997,"15min":0.58799999999999997},"min":{"1min":0.39500000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.57899999999999996,"15min":0.57399999999999995},"min":{"1min":0.38,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.82699999999999996,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.78500000000000003}]},{"osd":6,"last update":"Mon Dec 11 17:43:59 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.63100000000000001,"15min":0.61899999999999999},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.64400000000000002,"15min":0.63600000000000001},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.80100000000000005}]},{"osd":7,"last update":"Mon Dec 11 17:44:15 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.54500000000000004,"15min":0.54000000000000004},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.54400000000000004,"15min":0.53500000000000003},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.64700000000000002}]},{"osd":8,"last update":"Mon Dec 11 17:44:39 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.68400000000000005,"15min":0.67800000000000005},"min":{"1min":0.46200000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.66100000000000003,"15min":0.65500000000000003},"min":{"1min":0.44700000000000001,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.88300000000000001,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.83899999999999997}]},{"osd":9,"last update":"Mon Dec 11 17:43:59 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.85299999999999998}]},{"osd":10,"last update":"Mon Dec 11 17:44:17 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.76700000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75}]}]},{"osd":5,"up_from":37,"seq":158913789994,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1011712,"data_stored":1056544,"data_compressed":47493,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 11 17:44:31 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.501,"15min":0.497},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.66000000000000003,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.53000000000000003,"15min":0.52200000000000002},"min":{"1min":0.36599999999999999,"5min":0.314,"15min":0.314},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.70099999999999996}]},{"osd":1,"last update":"Mon Dec 11 17:44:31 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53200000000000003,"15min":0.52300000000000002},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.79600000000000004,"5min":0.872,"15min":0.872},"last":0.63},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.54100000000000004,"15min":0.53300000000000003},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.71799999999999997,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.65500000000000003}]},{"osd":2,"last update":"Mon Dec 11 17:44:31 2023","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.375,"15min":0.36399999999999999},"min":{"1min":0.28599999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.59899999999999998,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.377,"5min":0.40600000000000003,"15min":0.40400000000000003},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.57199999999999995,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.47899999999999998}]},{"osd":3,"last update":"Mon Dec 11 17:44:31 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.54700000000000004,"15min":0.53300000000000003},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.67000000000000004,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.56899999999999995,"15min":0.55800000000000005},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.78700000000000003,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.81399999999999995}]},{"osd":4,"last update":"Mon Dec 11 17:44:31 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.53600000000000003,"15min":0.52600000000000002},"min":{"1min":0.39200000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.71199999999999997,"5min":0.752,"15min":0.752},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58299999999999996,"15min":0.57199999999999995},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.88700000000000001,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.56399999999999995}]},{"osd":6,"last update":"Mon Dec 11 17:43:53 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.624,"15min":0.61599999999999999},"min":{"1min":0.49099999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.67400000000000004,"15min":0.66600000000000004},"min":{"1min":0.495,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.871}]},{"osd":7,"last update":"Mon Dec 11 17:44:14 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.63200000000000001,"15min":0.63},"min":{"1min":0.437,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61199999999999999,"15min":0.61099999999999999},"min":{"1min":0.44700000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.64300000000000002}]},{"osd":8,"last update":"Mon Dec 11 17:44:36 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.59199999999999997,"15min":0.60299999999999998},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.79500000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.628,"5min":0.64300000000000002,"15min":0.64500000000000002},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.80200000000000005,"5min":1.012,"15min":1.012},"last":0.79900000000000004}]},{"osd":9,"last update":"Mon Dec 11 17:43:59 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.246,"5min":1.246,"15min":1.246},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":0.89500000000000002}]},{"osd":10,"last update":"Mon Dec 11 17:44:17 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.82599999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34126,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34126,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35765,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35765,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35765,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35765,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":5191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-11T17:44:50.967 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-12-11T17:44:50.967 INFO:tasks.cephadm:Setting up client nodes... 2023-12-11T17:44:50.968 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d0a07eaa-984b-11ee-95a3-87774f69a715 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2023-12-11T17:44:51.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:50 smithi115 ceph-mon[102748]: from='client.14871 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T17:44:51.034 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:50 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/1062268508' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T17:44:51.034 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:50 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/2579014155' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-11T17:44:51.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:50 smithi181 ceph-mon[103627]: from='client.14871 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T17:44:51.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:50 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/1062268508' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T17:44:51.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:50 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/2579014155' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-11T17:44:51.533 INFO:journalctl@ceph.mds.b.smithi115.stdout:Dec 11 17:44:51 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mds-b[139691]: 2023-12-11T17:44:51.128+0000 7fd52fbae700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2023-12-11T17:44:51.533 INFO:journalctl@ceph.mds.b.smithi115.stdout:Dec 11 17:44:51 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mds-b[139691]: 2023-12-11T17:44:51.128+0000 7fd52fbae700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2023-12-11T17:44:51.604 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/mon.a/config 2023-12-11T17:44:52.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:51 smithi181 ceph-mon[103627]: pgmap v274: 129 pgs: 129 active+clean; 583 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1.3 KiB/s wr, 6 op/s 2023-12-11T17:44:52.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:51 smithi181 ceph-mon[103627]: from='client.14889 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-11T17:44:52.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:51 smithi181 ceph-mon[103627]: mgrmap e17: x(active, since 6m), standbys: z, y 2023-12-11T17:44:52.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:51 smithi181 ceph-mon[103627]: from='client.14895 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-11T17:44:52.149 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:51 smithi059 ceph-mon[98847]: pgmap v274: 129 pgs: 129 active+clean; 583 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1.3 KiB/s wr, 6 op/s 2023-12-11T17:44:52.149 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:51 smithi059 ceph-mon[98847]: from='client.14889 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-11T17:44:52.149 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:51 smithi059 ceph-mon[98847]: mgrmap e17: x(active, since 6m), standbys: z, y 2023-12-11T17:44:52.150 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:51 smithi059 ceph-mon[98847]: from='client.14895 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-11T17:44:52.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:51 smithi115 ceph-mon[102748]: pgmap v274: 129 pgs: 129 active+clean; 583 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1.3 KiB/s wr, 6 op/s 2023-12-11T17:44:52.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:51 smithi115 ceph-mon[102748]: from='client.14889 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-11T17:44:52.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:51 smithi115 ceph-mon[102748]: mgrmap e17: x(active, since 6m), standbys: z, y 2023-12-11T17:44:52.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:51 smithi115 ceph-mon[102748]: from='client.14895 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-11T17:44:52.977 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:52.972+0000 7f540ed6d700 1 -- 172.21.15.59:0/912976563 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f54081004e0 msgr2=0x7f5408100900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:52.977 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:52.972+0000 7f540ed6d700 1 --2- 172.21.15.59:0/912976563 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f54081004e0 0x7f5408100900 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f53fc0099c0 tx=0x7f53fc009cd0 comp rx=0 tx=0).stop 2023-12-11T17:44:52.977 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:52.972+0000 7f540ed6d700 1 -- 172.21.15.59:0/912976563 shutdown_connections 2023-12-11T17:44:52.977 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:52.972+0000 7f540ed6d700 1 --2- 172.21.15.59:0/912976563 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5408102080 0x7f5408108930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:52.977 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:52.972+0000 7f540ed6d700 1 --2- 172.21.15.59:0/912976563 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f54081004e0 0x7f5408100900 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:52.977 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:52.972+0000 7f540ed6d700 1 --2- 172.21.15.59:0/912976563 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f54081016e0 0x7f5408101b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:52.977 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:52.972+0000 7f540ed6d700 1 -- 172.21.15.59:0/912976563 >> 172.21.15.59:0/912976563 conn(0x7f54080fbaa0 msgr2=0x7f54080fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:44:52.977 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:52.972+0000 7f540ed6d700 1 -- 172.21.15.59:0/912976563 shutdown_connections 2023-12-11T17:44:52.978 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:52.972+0000 7f540ed6d700 1 -- 172.21.15.59:0/912976563 wait complete. 2023-12-11T17:44:52.978 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:52.973+0000 7f540ed6d700 1 Processor -- start 2023-12-11T17:44:52.978 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:52.973+0000 7f540ed6d700 1 -- start start 2023-12-11T17:44:52.978 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:52.973+0000 7f540ed6d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f54081004e0 0x7f5408197510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:52.978 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:52.973+0000 7f540ed6d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f54081016e0 0x7f5408197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:52.978 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:52.973+0000 7f540ed6d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5408102080 0x7f540819ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:52.978 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:52.973+0000 7f540ed6d700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f540819f050 con 0x7f54081004e0 2023-12-11T17:44:52.978 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:52.973+0000 7f540ed6d700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f540819f190 con 0x7f54081016e0 2023-12-11T17:44:52.978 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:52.973+0000 7f540ed6d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f540819f320 con 0x7f5408102080 2023-12-11T17:44:52.978 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:52.973+0000 7f540cb09700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f54081004e0 0x7f5408197510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:52.978 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:52.973+0000 7f540cb09700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f54081004e0 0x7f5408197510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33474/0 (socket says 172.21.15.59:33474) 2023-12-11T17:44:52.978 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:52.973+0000 7f540cb09700 1 -- 172.21.15.59:0/1140138599 learned_addr learned my addr 172.21.15.59:0/1140138599 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:44:52.978 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:52.973+0000 7f540d30a700 1 --2- 172.21.15.59:0/1140138599 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5408102080 0x7f540819ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:52.979 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:52.973+0000 7f5407fff700 1 --2- 172.21.15.59:0/1140138599 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f54081016e0 0x7f5408197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:52.979 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:52.973+0000 7f540cb09700 1 -- 172.21.15.59:0/1140138599 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5408102080 msgr2=0x7f540819ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:52.979 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:52.973+0000 7f540cb09700 1 --2- 172.21.15.59:0/1140138599 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5408102080 0x7f540819ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:52.979 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:52.973+0000 7f540cb09700 1 -- 172.21.15.59:0/1140138599 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f54081016e0 msgr2=0x7f5408197a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:52.979 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:52.973+0000 7f540cb09700 1 --2- 172.21.15.59:0/1140138599 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f54081016e0 0x7f5408197a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:52.979 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:52.973+0000 7f540cb09700 1 -- 172.21.15.59:0/1140138599 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53fc0095f0 con 0x7f54081004e0 2023-12-11T17:44:52.979 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:52.974+0000 7f540cb09700 1 --2- 172.21.15.59:0/1140138599 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f54081004e0 0x7f5408197510 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f53fc0036f0 tx=0x7f53fc0047a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:44:52.979 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:52.974+0000 7f5405ffb700 1 -- 172.21.15.59:0/1140138599 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53fc01d070 con 0x7f54081004e0 2023-12-11T17:44:52.979 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:52.974+0000 7f5405ffb700 1 -- 172.21.15.59:0/1140138599 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f53fc0041f0 con 0x7f54081004e0 2023-12-11T17:44:52.979 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:52.974+0000 7f540ed6d700 1 -- 172.21.15.59:0/1140138599 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f540810c8f0 con 0x7f54081004e0 2023-12-11T17:44:52.979 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:52.974+0000 7f5405ffb700 1 -- 172.21.15.59:0/1140138599 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53fc00f460 con 0x7f54081004e0 2023-12-11T17:44:52.980 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:52.974+0000 7f540ed6d700 1 -- 172.21.15.59:0/1140138599 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f540810cde0 con 0x7f54081004e0 2023-12-11T17:44:52.980 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:52.975+0000 7f540ed6d700 1 -- 172.21.15.59:0/1140138599 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f540810c620 con 0x7f54081004e0 2023-12-11T17:44:52.980 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:52.976+0000 7f5405ffb700 1 -- 172.21.15.59:0/1140138599 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f53fc022650 con 0x7f54081004e0 2023-12-11T17:44:52.980 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:52.976+0000 7f5405ffb700 1 --2- 172.21.15.59:0/1140138599 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f53f80a1590 0x7f53f80a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:52.980 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:52.977+0000 7f5405ffb700 1 -- 172.21.15.59:0/1140138599 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f53fc0ce480 con 0x7f54081004e0 2023-12-11T17:44:52.980 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:52.977+0000 7f5407fff700 1 --2- 172.21.15.59:0/1140138599 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f53f80a1590 0x7f53f80a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:52.981 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:52.977+0000 7f5407fff700 1 --2- 172.21.15.59:0/1140138599 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f53f80a1590 0x7f53f80a3a50 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f53f4005fd0 tx=0x7f53f400a560 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:44:52.984 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:52.981+0000 7f5405ffb700 1 -- 172.21.15.59:0/1140138599 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f53fc097940 con 0x7f54081004e0 2023-12-11T17:44:53.264 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:53.260+0000 7f540ed6d700 1 -- 172.21.15.59:0/1140138599 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f540810bf20 con 0x7f54081004e0 2023-12-11T17:44:53.266 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:53.262+0000 7f5405ffb700 1 -- 172.21.15.59:0/1140138599 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v31) v1 ==== 170+0+59 (secure 0 0 0) 0x7f53fc09af60 con 0x7f54081004e0 2023-12-11T17:44:53.266 INFO:teuthology.orchestra.run.smithi059.stdout:[client.0] 2023-12-11T17:44:53.266 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQCVSndlk3qrDxAAnK62ze3LhcOl3usWDDwAJw== 2023-12-11T17:44:53.269 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:53.265+0000 7f540ed6d700 1 -- 172.21.15.59:0/1140138599 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f53f80a1590 msgr2=0x7f53f80a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:53.269 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:53.265+0000 7f540ed6d700 1 --2- 172.21.15.59:0/1140138599 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f53f80a1590 0x7f53f80a3a50 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f53f4005fd0 tx=0x7f53f400a560 comp rx=0 tx=0).stop 2023-12-11T17:44:53.269 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:53.265+0000 7f540ed6d700 1 -- 172.21.15.59:0/1140138599 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f54081004e0 msgr2=0x7f5408197510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:53.269 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:53.265+0000 7f540ed6d700 1 --2- 172.21.15.59:0/1140138599 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f54081004e0 0x7f5408197510 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f53fc0036f0 tx=0x7f53fc0047a0 comp rx=0 tx=0).stop 2023-12-11T17:44:53.269 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:53.265+0000 7f540ed6d700 1 -- 172.21.15.59:0/1140138599 shutdown_connections 2023-12-11T17:44:53.269 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:53.265+0000 7f540ed6d700 1 --2- 172.21.15.59:0/1140138599 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f53f80a1590 0x7f53f80a3a50 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:53.269 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:53.265+0000 7f540ed6d700 1 --2- 172.21.15.59:0/1140138599 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5408102080 0x7f540819ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:53.269 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:53.265+0000 7f540ed6d700 1 --2- 172.21.15.59:0/1140138599 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f54081004e0 0x7f5408197510 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:53.269 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:53.265+0000 7f540ed6d700 1 --2- 172.21.15.59:0/1140138599 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f54081016e0 0x7f5408197a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:53.269 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:53.265+0000 7f540ed6d700 1 -- 172.21.15.59:0/1140138599 >> 172.21.15.59:0/1140138599 conn(0x7f54080fbaa0 msgr2=0x7f54080fc140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:44:53.269 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:53.265+0000 7f540ed6d700 1 -- 172.21.15.59:0/1140138599 shutdown_connections 2023-12-11T17:44:53.269 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:53.265+0000 7f540ed6d700 1 -- 172.21.15.59:0/1140138599 wait complete. 2023-12-11T17:44:53.678 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2023-12-11T17:44:53.678 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2023-12-11T17:44:53.678 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-12-11T17:44:53.719 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-12-11T17:44:53.719 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2023-12-11T17:44:53.720 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell --fsid d0a07eaa-984b-11ee-95a3-87774f69a715 -- ceph mgr dump --format=json 2023-12-11T17:44:53.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:53 smithi059 ceph-mon[98847]: pgmap v275: 129 pgs: 129 active+clean; 585 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 2.3 KiB/s wr, 9 op/s 2023-12-11T17:44:53.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:53 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/1140138599' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-12-11T17:44:53.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:53 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/1140138599' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-12-11T17:44:54.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:53 smithi181 ceph-mon[103627]: pgmap v275: 129 pgs: 129 active+clean; 585 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 2.3 KiB/s wr, 9 op/s 2023-12-11T17:44:54.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:53 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/1140138599' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-12-11T17:44:54.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:53 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/1140138599' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-12-11T17:44:54.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:53 smithi115 ceph-mon[102748]: pgmap v275: 129 pgs: 129 active+clean; 585 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 2.3 KiB/s wr, 9 op/s 2023-12-11T17:44:54.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:53 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/1140138599' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-12-11T17:44:54.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:53 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/1140138599' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-12-11T17:44:54.361 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/mon.a/config 2023-12-11T17:44:55.694 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:55.688+0000 7fbb8fafe700 1 -- 172.21.15.59:0/306279937 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb88101680 msgr2=0x7fbb88101b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:55.695 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:55.688+0000 7fbb8fafe700 1 --2- 172.21.15.59:0/306279937 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb88101680 0x7fbb88101b00 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fbb7c009990 tx=0x7fbb7c009ca0 comp rx=0 tx=0).stop 2023-12-11T17:44:55.695 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:55.688+0000 7fbb8fafe700 1 -- 172.21.15.59:0/306279937 shutdown_connections 2023-12-11T17:44:55.695 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:55.688+0000 7fbb8fafe700 1 --2- 172.21.15.59:0/306279937 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb88102040 0x7fbb8810e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:55.695 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:55.688+0000 7fbb8fafe700 1 --2- 172.21.15.59:0/306279937 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbb881065a0 0x7fbb88106980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:55.695 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:55.688+0000 7fbb8fafe700 1 --2- 172.21.15.59:0/306279937 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb88101680 0x7fbb88101b00 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:55.695 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:55.688+0000 7fbb8fafe700 1 -- 172.21.15.59:0/306279937 >> 172.21.15.59:0/306279937 conn(0x7fbb88078730 msgr2=0x7fbb88078b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:44:55.695 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:55.689+0000 7fbb8fafe700 1 -- 172.21.15.59:0/306279937 shutdown_connections 2023-12-11T17:44:55.695 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:55.689+0000 7fbb8fafe700 1 -- 172.21.15.59:0/306279937 wait complete. 2023-12-11T17:44:55.695 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:55.689+0000 7fbb8fafe700 1 Processor -- start 2023-12-11T17:44:55.695 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:55.689+0000 7fbb8fafe700 1 -- start start 2023-12-11T17:44:55.695 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:55.690+0000 7fbb8fafe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb88101680 0x7fbb8819d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:55.695 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:55.690+0000 7fbb8fafe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb88102040 0x7fbb8819dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:55.695 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:55.690+0000 7fbb8fafe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbb881065a0 0x7fbb88197800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:55.696 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:55.690+0000 7fbb8fafe700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb88197e70 con 0x7fbb88102040 2023-12-11T17:44:55.696 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:55.690+0000 7fbb8fafe700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb88197fe0 con 0x7fbb881065a0 2023-12-11T17:44:55.696 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:55.690+0000 7fbb8fafe700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb881981a0 con 0x7fbb88101680 2023-12-11T17:44:55.696 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:55.690+0000 7fbb8d099700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb88102040 0x7fbb8819dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:55.696 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:55.690+0000 7fbb8d099700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb88102040 0x7fbb8819dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33488/0 (socket says 172.21.15.59:33488) 2023-12-11T17:44:55.696 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:55.690+0000 7fbb8d099700 1 -- 172.21.15.59:0/4177129610 learned_addr learned my addr 172.21.15.59:0/4177129610 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:44:55.696 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:55.690+0000 7fbb8d89a700 1 --2- 172.21.15.59:0/4177129610 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb88101680 0x7fbb8819d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:55.696 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:55.690+0000 7fbb8d099700 1 -- 172.21.15.59:0/4177129610 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb88101680 msgr2=0x7fbb8819d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:55.696 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:55.690+0000 7fbb8d099700 1 --2- 172.21.15.59:0/4177129610 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb88101680 0x7fbb8819d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:55.696 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:55.690+0000 7fbb8d099700 1 -- 172.21.15.59:0/4177129610 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbb881065a0 msgr2=0x7fbb88197800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:55.696 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:55.690+0000 7fbb8d099700 1 --2- 172.21.15.59:0/4177129610 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbb881065a0 0x7fbb88197800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:55.696 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:55.690+0000 7fbb8d099700 1 -- 172.21.15.59:0/4177129610 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb7c0095f0 con 0x7fbb88102040 2023-12-11T17:44:55.696 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:55.690+0000 7fbb8d099700 1 --2- 172.21.15.59:0/4177129610 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb88102040 0x7fbb8819dc00 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fbb7c0098e0 tx=0x7fbb7c00f720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:44:55.696 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:55.690+0000 7fbb7affd700 1 -- 172.21.15.59:0/4177129610 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb7c01d070 con 0x7fbb88102040 2023-12-11T17:44:55.696 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:55.690+0000 7fbb7affd700 1 -- 172.21.15.59:0/4177129610 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbb7c00fb90 con 0x7fbb88102040 2023-12-11T17:44:55.697 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:55.690+0000 7fbb8fafe700 1 -- 172.21.15.59:0/4177129610 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb88198420 con 0x7fbb88102040 2023-12-11T17:44:55.697 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:55.690+0000 7fbb7affd700 1 -- 172.21.15.59:0/4177129610 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb7c017650 con 0x7fbb88102040 2023-12-11T17:44:55.697 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:55.690+0000 7fbb8fafe700 1 -- 172.21.15.59:0/4177129610 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb881a46b0 con 0x7fbb88102040 2023-12-11T17:44:55.697 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:55.691+0000 7fbb8fafe700 1 -- 172.21.15.59:0/4177129610 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb88109bb0 con 0x7fbb88102040 2023-12-11T17:44:55.698 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:55.692+0000 7fbb7affd700 1 -- 172.21.15.59:0/4177129610 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7fbb7c00fd00 con 0x7fbb88102040 2023-12-11T17:44:55.698 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:55.693+0000 7fbb7affd700 1 --2- 172.21.15.59:0/4177129610 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fbb740a1510 0x7fbb740a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:55.698 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:55.693+0000 7fbb7affd700 1 -- 172.21.15.59:0/4177129610 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fbb7c0ce1d0 con 0x7fbb88102040 2023-12-11T17:44:55.698 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:55.693+0000 7fbb8d89a700 1 --2- 172.21.15.59:0/4177129610 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fbb740a1510 0x7fbb740a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:55.698 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:55.693+0000 7fbb8d89a700 1 --2- 172.21.15.59:0/4177129610 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fbb740a1510 0x7fbb740a39d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fbb84005950 tx=0x7fbb840058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:44:55.701 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:55.698+0000 7fbb7affd700 1 -- 172.21.15.59:0/4177129610 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fbb7c097570 con 0x7fbb88102040 2023-12-11T17:44:55.958 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:55.954+0000 7fbb8fafe700 1 -- 172.21.15.59:0/4177129610 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fbb88102ca0 con 0x7fbb88102040 2023-12-11T17:44:55.963 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:55.959+0000 7fbb7affd700 1 -- 172.21.15.59:0/4177129610 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) v1 ==== 74+0+260410 (secure 0 0 0) 0x7fbb7c09ab90 con 0x7fbb88102040 2023-12-11T17:44:55.966 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-12-11T17:44:55.967 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:55.962+0000 7fbb8fafe700 1 -- 172.21.15.59:0/4177129610 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fbb740a1510 msgr2=0x7fbb740a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:55.967 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:55.962+0000 7fbb8fafe700 1 --2- 172.21.15.59:0/4177129610 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fbb740a1510 0x7fbb740a39d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fbb84005950 tx=0x7fbb840058e0 comp rx=0 tx=0).stop 2023-12-11T17:44:55.968 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:55.962+0000 7fbb8fafe700 1 -- 172.21.15.59:0/4177129610 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb88102040 msgr2=0x7fbb8819dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:55.968 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:55.962+0000 7fbb8fafe700 1 --2- 172.21.15.59:0/4177129610 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb88102040 0x7fbb8819dc00 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fbb7c0098e0 tx=0x7fbb7c00f720 comp rx=0 tx=0).stop 2023-12-11T17:44:55.968 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:55.962+0000 7fbb8fafe700 1 -- 172.21.15.59:0/4177129610 shutdown_connections 2023-12-11T17:44:55.968 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:55.962+0000 7fbb8fafe700 1 --2- 172.21.15.59:0/4177129610 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fbb740a1510 0x7fbb740a39d0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:55.968 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:55.962+0000 7fbb8fafe700 1 --2- 172.21.15.59:0/4177129610 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbb881065a0 0x7fbb88197800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:55.968 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:55.962+0000 7fbb8fafe700 1 --2- 172.21.15.59:0/4177129610 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb88101680 0x7fbb8819d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:55.968 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:55.962+0000 7fbb8fafe700 1 --2- 172.21.15.59:0/4177129610 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb88102040 0x7fbb8819dc00 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:55.968 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:55.962+0000 7fbb8fafe700 1 -- 172.21.15.59:0/4177129610 >> 172.21.15.59:0/4177129610 conn(0x7fbb88078730 msgr2=0x7fbb880ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:44:55.968 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:55.962+0000 7fbb8fafe700 1 -- 172.21.15.59:0/4177129610 shutdown_connections 2023-12-11T17:44:55.968 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:55.962+0000 7fbb8fafe700 1 -- 172.21.15.59:0/4177129610 wait complete. 2023-12-11T17:44:56.115 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:55 smithi059 ceph-mon[98847]: pgmap v276: 129 pgs: 129 active+clean; 623 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 5.9 KiB/s wr, 12 op/s 2023-12-11T17:44:56.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:55 smithi181 ceph-mon[103627]: pgmap v276: 129 pgs: 129 active+clean; 623 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 5.9 KiB/s wr, 12 op/s 2023-12-11T17:44:56.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:55 smithi115 ceph-mon[102748]: pgmap v276: 129 pgs: 129 active+clean; 623 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 5.9 KiB/s wr, 12 op/s 2023-12-11T17:44:56.335 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":17,"active_gid":14150,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3627728353},{"type":"v1","addr":"172.21.15.59:6801","nonce":3627728353}]},"active_addr":"172.21.15.59:6801/3627728353","active_change":"2023-12-11T17:38:14.038067+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24113,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.59:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.59:0","nonce":4159306605}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.59:0","nonce":2112420116}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.59:0","nonce":2683787475}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.59:0","nonce":644616907}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.59:0","nonce":1373005911}]}]} 2023-12-11T17:44:56.340 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2023-12-11T17:44:56.340 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2023-12-11T17:44:56.340 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell --fsid d0a07eaa-984b-11ee-95a3-87774f69a715 -- ceph osd dump --format=json 2023-12-11T17:44:56.942 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/mon.a/config 2023-12-11T17:44:57.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:56 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/4177129610' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-12-11T17:44:57.155 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:56 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/4177129610' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-12-11T17:44:57.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:56 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/4177129610' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-12-11T17:44:57.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:57 smithi059 ceph-mon[98847]: pgmap v277: 129 pgs: 129 active+clean; 623 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 5.9 KiB/s wr, 12 op/s 2023-12-11T17:44:58.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:57 smithi181 ceph-mon[103627]: pgmap v277: 129 pgs: 129 active+clean; 623 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 5.9 KiB/s wr, 12 op/s 2023-12-11T17:44:58.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:57 smithi115 ceph-mon[102748]: pgmap v277: 129 pgs: 129 active+clean; 623 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 5.9 KiB/s wr, 12 op/s 2023-12-11T17:44:58.513 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:58.510+0000 7f4f59a15700 1 -- 172.21.15.59:0/2550026656 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4f541004c0 msgr2=0x7f4f54100940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:58.513 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:58.510+0000 7f4f59a15700 1 --2- 172.21.15.59:0/2550026656 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4f541004c0 0x7f4f54100940 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f4f480099c0 tx=0x7f4f48009cd0 comp rx=0 tx=0).stop 2023-12-11T17:44:58.514 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:58.510+0000 7f4f59a15700 1 -- 172.21.15.59:0/2550026656 shutdown_connections 2023-12-11T17:44:58.514 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:58.510+0000 7f4f59a15700 1 --2- 172.21.15.59:0/2550026656 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f54100e80 0x7f4f5410d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:58.514 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:58.510+0000 7f4f59a15700 1 --2- 172.21.15.59:0/2550026656 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f54106570 0x7f4f54106950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:58.514 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:58.510+0000 7f4f59a15700 1 --2- 172.21.15.59:0/2550026656 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4f541004c0 0x7f4f54100940 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:58.514 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:58.510+0000 7f4f59a15700 1 -- 172.21.15.59:0/2550026656 >> 172.21.15.59:0/2550026656 conn(0x7f4f540774d0 msgr2=0x7f4f540778e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:44:58.514 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:58.510+0000 7f4f59a15700 1 -- 172.21.15.59:0/2550026656 shutdown_connections 2023-12-11T17:44:58.514 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:58.511+0000 7f4f59a15700 1 -- 172.21.15.59:0/2550026656 wait complete. 2023-12-11T17:44:58.514 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:58.511+0000 7f4f59a15700 1 Processor -- start 2023-12-11T17:44:58.516 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:58.511+0000 7f4f59a15700 1 -- start start 2023-12-11T17:44:58.516 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:58.512+0000 7f4f59a15700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4f541004c0 0x7f4f54198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:58.516 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:58.512+0000 7f4f59a15700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f54100e80 0x7f4f54198e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:58.516 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:58.512+0000 7f4f59a15700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f54106570 0x7f4f541a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:58.516 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:58.512+0000 7f4f59a15700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f541996f0 con 0x7f4f541004c0 2023-12-11T17:44:58.516 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:58.512+0000 7f4f59a15700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f541a3f00 con 0x7f4f54106570 2023-12-11T17:44:58.516 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:58.512+0000 7f4f59a15700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f541a4040 con 0x7f4f54100e80 2023-12-11T17:44:58.516 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:58.512+0000 7f4f52ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4f541004c0 0x7f4f54198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:58.516 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:58.512+0000 7f4f52ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4f541004c0 0x7f4f54198940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33500/0 (socket says 172.21.15.59:33500) 2023-12-11T17:44:58.516 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:58.512+0000 7f4f52ffd700 1 -- 172.21.15.59:0/932713681 learned_addr learned my addr 172.21.15.59:0/932713681 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:44:58.516 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:58.512+0000 7f4f527fc700 1 --2- 172.21.15.59:0/932713681 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f54100e80 0x7f4f54198e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:58.516 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:58.512+0000 7f4f537fe700 1 --2- 172.21.15.59:0/932713681 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f54106570 0x7f4f541a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:58.516 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:58.512+0000 7f4f52ffd700 1 -- 172.21.15.59:0/932713681 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f54100e80 msgr2=0x7f4f54198e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:58.517 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:58.512+0000 7f4f52ffd700 1 --2- 172.21.15.59:0/932713681 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f54100e80 0x7f4f54198e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:58.517 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:58.512+0000 7f4f52ffd700 1 -- 172.21.15.59:0/932713681 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f54106570 msgr2=0x7f4f541a39c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:58.517 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:58.512+0000 7f4f52ffd700 1 --2- 172.21.15.59:0/932713681 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f54106570 0x7f4f541a39c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:58.517 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:58.512+0000 7f4f52ffd700 1 -- 172.21.15.59:0/932713681 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f480095f0 con 0x7f4f541004c0 2023-12-11T17:44:58.517 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:58.512+0000 7f4f52ffd700 1 --2- 172.21.15.59:0/932713681 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4f541004c0 0x7f4f54198940 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f4f4400b530 tx=0x7f4f4400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:44:58.517 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:58.512+0000 7f4f58a13700 1 -- 172.21.15.59:0/932713681 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f44011830 con 0x7f4f541004c0 2023-12-11T17:44:58.517 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:58.513+0000 7f4f59a15700 1 -- 172.21.15.59:0/932713681 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f541a42d0 con 0x7f4f541004c0 2023-12-11T17:44:58.517 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:58.513+0000 7f4f58a13700 1 -- 172.21.15.59:0/932713681 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4f44011e50 con 0x7f4f541004c0 2023-12-11T17:44:58.517 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:58.513+0000 7f4f59a15700 1 -- 172.21.15.59:0/932713681 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f541a47f0 con 0x7f4f541004c0 2023-12-11T17:44:58.517 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:58.513+0000 7f4f58a13700 1 -- 172.21.15.59:0/932713681 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f4401a400 con 0x7f4f541004c0 2023-12-11T17:44:58.517 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:58.513+0000 7f4f59a15700 1 -- 172.21.15.59:0/932713681 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f54108a00 con 0x7f4f541004c0 2023-12-11T17:44:58.518 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:58.515+0000 7f4f58a13700 1 -- 172.21.15.59:0/932713681 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f4f44010380 con 0x7f4f541004c0 2023-12-11T17:44:58.518 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:58.515+0000 7f4f58a13700 1 --2- 172.21.15.59:0/932713681 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f4f3c0a1510 0x7f4f3c0a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:44:58.518 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:58.516+0000 7f4f527fc700 1 --2- 172.21.15.59:0/932713681 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f4f3c0a1510 0x7f4f3c0a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:44:58.518 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:58.516+0000 7f4f58a13700 1 -- 172.21.15.59:0/932713681 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f4f440cd010 con 0x7f4f541004c0 2023-12-11T17:44:58.518 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:58.516+0000 7f4f527fc700 1 --2- 172.21.15.59:0/932713681 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f4f3c0a1510 0x7f4f3c0a39d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f4f48009990 tx=0x7f4f48005040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:44:58.521 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:58.519+0000 7f4f58a13700 1 -- 172.21.15.59:0/932713681 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f4f44096430 con 0x7f4f541004c0 2023-12-11T17:44:58.721 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:58.718+0000 7f4f59a15700 1 -- 172.21.15.59:0/932713681 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4f54101be0 con 0x7f4f541004c0 2023-12-11T17:44:58.722 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:58.720+0000 7f4f58a13700 1 -- 172.21.15.59:0/932713681 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+23960 (secure 0 0 0) 0x7f4f44099a50 con 0x7f4f541004c0 2023-12-11T17:44:58.723 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-12-11T17:44:58.723 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":86,"fsid":"d0a07eaa-984b-11ee-95a3-87774f69a715","created":"2023-12-11T17:37:14.526539+0000","modified":"2023-12-11T17:44:25.096094+0000","last_up_change":"2023-12-11T17:43:38.535503+0000","last_in_change":"2023-12-11T17:43:21.014888+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-11T17:40:26.596029+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-11T17:44:18.586290+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-11T17:44:19.432952+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-11T17:44:23.097308+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3c0b3218-4b2b-40f8-8bef-ed8cd5633607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":1040759398},{"type":"v1","addr":"172.21.15.59:6803","nonce":1040759398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":1040759398},{"type":"v1","addr":"172.21.15.59:6805","nonce":1040759398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":1040759398},{"type":"v1","addr":"172.21.15.59:6809","nonce":1040759398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":1040759398},{"type":"v1","addr":"172.21.15.59:6807","nonce":1040759398}]},"public_addr":"172.21.15.59:6803/1040759398","cluster_addr":"172.21.15.59:6805/1040759398","heartbeat_back_addr":"172.21.15.59:6809/1040759398","heartbeat_front_addr":"172.21.15.59:6807/1040759398","state":["exists","up"]},{"osd":1,"uuid":"2ead7578-765b-4f5a-a72c-863731ed69e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":2776354245},{"type":"v1","addr":"172.21.15.115:6801","nonce":2776354245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":2776354245},{"type":"v1","addr":"172.21.15.115:6803","nonce":2776354245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":2776354245},{"type":"v1","addr":"172.21.15.115:6807","nonce":2776354245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":2776354245},{"type":"v1","addr":"172.21.15.115:6805","nonce":2776354245}]},"public_addr":"172.21.15.115:6801/2776354245","cluster_addr":"172.21.15.115:6803/2776354245","heartbeat_back_addr":"172.21.15.115:6807/2776354245","heartbeat_front_addr":"172.21.15.115:6805/2776354245","state":["exists","up"]},{"osd":2,"uuid":"8cb1d7ec-53cc-4672-b0b6-70d73797c44f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4287675481},{"type":"v1","addr":"172.21.15.181:6801","nonce":4287675481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4287675481},{"type":"v1","addr":"172.21.15.181:6803","nonce":4287675481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4287675481},{"type":"v1","addr":"172.21.15.181:6807","nonce":4287675481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4287675481},{"type":"v1","addr":"172.21.15.181:6805","nonce":4287675481}]},"public_addr":"172.21.15.181:6801/4287675481","cluster_addr":"172.21.15.181:6803/4287675481","heartbeat_back_addr":"172.21.15.181:6807/4287675481","heartbeat_front_addr":"172.21.15.181:6805/4287675481","state":["exists","up"]},{"osd":3,"uuid":"5d64d9af-831b-4d0a-8ace-a452169351d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":219909995},{"type":"v1","addr":"172.21.15.59:6811","nonce":219909995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":219909995},{"type":"v1","addr":"172.21.15.59:6813","nonce":219909995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":219909995},{"type":"v1","addr":"172.21.15.59:6817","nonce":219909995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":219909995},{"type":"v1","addr":"172.21.15.59:6815","nonce":219909995}]},"public_addr":"172.21.15.59:6811/219909995","cluster_addr":"172.21.15.59:6813/219909995","heartbeat_back_addr":"172.21.15.59:6817/219909995","heartbeat_front_addr":"172.21.15.59:6815/219909995","state":["exists","up"]},{"osd":4,"uuid":"ef8fad28-1db5-4cba-b5f2-10e54e8e1549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":1021473826},{"type":"v1","addr":"172.21.15.115:6809","nonce":1021473826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":1021473826},{"type":"v1","addr":"172.21.15.115:6811","nonce":1021473826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":1021473826},{"type":"v1","addr":"172.21.15.115:6815","nonce":1021473826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":1021473826},{"type":"v1","addr":"172.21.15.115:6813","nonce":1021473826}]},"public_addr":"172.21.15.115:6809/1021473826","cluster_addr":"172.21.15.115:6811/1021473826","heartbeat_back_addr":"172.21.15.115:6815/1021473826","heartbeat_front_addr":"172.21.15.115:6813/1021473826","state":["exists","up"]},{"osd":5,"uuid":"4233a512-9e8f-4db8-aed2-6c3af706e589","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":2405172521},{"type":"v1","addr":"172.21.15.181:6809","nonce":2405172521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":2405172521},{"type":"v1","addr":"172.21.15.181:6811","nonce":2405172521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2405172521},{"type":"v1","addr":"172.21.15.181:6815","nonce":2405172521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":2405172521},{"type":"v1","addr":"172.21.15.181:6813","nonce":2405172521}]},"public_addr":"172.21.15.181:6809/2405172521","cluster_addr":"172.21.15.181:6811/2405172521","heartbeat_back_addr":"172.21.15.181:6815/2405172521","heartbeat_front_addr":"172.21.15.181:6813/2405172521","state":["exists","up"]},{"osd":6,"uuid":"8faab332-016e-4c77-8370-8bef95fc6f34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":815374853},{"type":"v1","addr":"172.21.15.59:6819","nonce":815374853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":815374853},{"type":"v1","addr":"172.21.15.59:6821","nonce":815374853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":815374853},{"type":"v1","addr":"172.21.15.59:6825","nonce":815374853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":815374853},{"type":"v1","addr":"172.21.15.59:6823","nonce":815374853}]},"public_addr":"172.21.15.59:6819/815374853","cluster_addr":"172.21.15.59:6821/815374853","heartbeat_back_addr":"172.21.15.59:6825/815374853","heartbeat_front_addr":"172.21.15.59:6823/815374853","state":["exists","up"]},{"osd":7,"uuid":"f20f470a-0d40-4ed3-8e0e-fd58675cf143","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":1314971870},{"type":"v1","addr":"172.21.15.115:6817","nonce":1314971870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":1314971870},{"type":"v1","addr":"172.21.15.115:6819","nonce":1314971870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":1314971870},{"type":"v1","addr":"172.21.15.115:6823","nonce":1314971870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":1314971870},{"type":"v1","addr":"172.21.15.115:6821","nonce":1314971870}]},"public_addr":"172.21.15.115:6817/1314971870","cluster_addr":"172.21.15.115:6819/1314971870","heartbeat_back_addr":"172.21.15.115:6823/1314971870","heartbeat_front_addr":"172.21.15.115:6821/1314971870","state":["exists","up"]},{"osd":8,"uuid":"eb657d32-1a95-4fd7-a293-d0a970837fc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":3954509873},{"type":"v1","addr":"172.21.15.181:6817","nonce":3954509873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":3954509873},{"type":"v1","addr":"172.21.15.181:6819","nonce":3954509873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":3954509873},{"type":"v1","addr":"172.21.15.181:6823","nonce":3954509873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":3954509873},{"type":"v1","addr":"172.21.15.181:6821","nonce":3954509873}]},"public_addr":"172.21.15.181:6817/3954509873","cluster_addr":"172.21.15.181:6819/3954509873","heartbeat_back_addr":"172.21.15.181:6823/3954509873","heartbeat_front_addr":"172.21.15.181:6821/3954509873","state":["exists","up"]},{"osd":9,"uuid":"8f7717d4-b6f2-4ec1-9b58-62feb0c9a146","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":1163095677},{"type":"v1","addr":"172.21.15.59:6827","nonce":1163095677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":1163095677},{"type":"v1","addr":"172.21.15.59:6829","nonce":1163095677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":1163095677},{"type":"v1","addr":"172.21.15.59:6833","nonce":1163095677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":1163095677},{"type":"v1","addr":"172.21.15.59:6831","nonce":1163095677}]},"public_addr":"172.21.15.59:6827/1163095677","cluster_addr":"172.21.15.59:6829/1163095677","heartbeat_back_addr":"172.21.15.59:6833/1163095677","heartbeat_front_addr":"172.21.15.59:6831/1163095677","state":["exists","up"]},{"osd":10,"uuid":"b75c179a-875d-43bc-b4c1-e8eb80fb64fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":2199610269},{"type":"v1","addr":"172.21.15.115:6825","nonce":2199610269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":2199610269},{"type":"v1","addr":"172.21.15.115:6827","nonce":2199610269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":2199610269},{"type":"v1","addr":"172.21.15.115:6831","nonce":2199610269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":2199610269},{"type":"v1","addr":"172.21.15.115:6829","nonce":2199610269}]},"public_addr":"172.21.15.115:6825/2199610269","cluster_addr":"172.21.15.115:6827/2199610269","heartbeat_back_addr":"172.21.15.115:6831/2199610269","heartbeat_front_addr":"172.21.15.115:6829/2199610269","state":["exists","up"]},{"osd":11,"uuid":"5e141dfe-8a00-4bbc-8204-2dc9be24e18d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":226506989},{"type":"v1","addr":"172.21.15.181:6825","nonce":226506989}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":226506989},{"type":"v1","addr":"172.21.15.181:6827","nonce":226506989}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":226506989},{"type":"v1","addr":"172.21.15.181:6831","nonce":226506989}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":226506989},{"type":"v1","addr":"172.21.15.181:6829","nonce":226506989}]},"public_addr":"172.21.15.181:6825/226506989","cluster_addr":"172.21.15.181:6827/226506989","heartbeat_back_addr":"172.21.15.181:6831/226506989","heartbeat_front_addr":"172.21.15.181:6829/226506989","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:39:42.687084+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:40:03.903448+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:40:24.899908+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:40:46.600251+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:41:07.690037+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:41:28.488855+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:41:49.459274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:42:10.863881+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:42:32.248391+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:42:54.112265+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:43:15.634872+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:43:36.922534+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.59:6800/1259916482":"2023-12-12T17:38:14.037922+0000","172.21.15.59:0/17457774":"2023-12-12T17:38:14.037922+0000","172.21.15.59:6801/1672530462":"2023-12-12T17:37:44.773925+0000","172.21.15.59:0/1053944547":"2023-12-12T17:38:14.037922+0000","172.21.15.59:6801/1259916482":"2023-12-12T17:38:14.037922+0000","172.21.15.59:6800/1672530462":"2023-12-12T17:37:44.773925+0000","172.21.15.59:0/3570767650":"2023-12-12T17:37:44.773925+0000","172.21.15.59:0/3853462899":"2023-12-12T17:37:44.773925+0000","172.21.15.59:0/2859415380":"2023-12-12T17:38:14.037922+0000","172.21.15.59:0/2192742819":"2023-12-12T17:37:44.773925+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-11T17:44:58.725 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:58.722+0000 7f4f59a15700 1 -- 172.21.15.59:0/932713681 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f4f3c0a1510 msgr2=0x7f4f3c0a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:58.725 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:58.722+0000 7f4f59a15700 1 --2- 172.21.15.59:0/932713681 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f4f3c0a1510 0x7f4f3c0a39d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f4f48009990 tx=0x7f4f48005040 comp rx=0 tx=0).stop 2023-12-11T17:44:58.725 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:58.722+0000 7f4f59a15700 1 -- 172.21.15.59:0/932713681 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4f541004c0 msgr2=0x7f4f54198940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:44:58.725 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:58.722+0000 7f4f59a15700 1 --2- 172.21.15.59:0/932713681 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4f541004c0 0x7f4f54198940 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f4f4400b530 tx=0x7f4f4400b840 comp rx=0 tx=0).stop 2023-12-11T17:44:58.725 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:58.722+0000 7f4f59a15700 1 -- 172.21.15.59:0/932713681 shutdown_connections 2023-12-11T17:44:58.725 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:58.722+0000 7f4f59a15700 1 --2- 172.21.15.59:0/932713681 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f4f3c0a1510 0x7f4f3c0a39d0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:58.725 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:58.722+0000 7f4f59a15700 1 --2- 172.21.15.59:0/932713681 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f54106570 0x7f4f541a39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:58.725 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:58.722+0000 7f4f59a15700 1 --2- 172.21.15.59:0/932713681 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4f541004c0 0x7f4f54198940 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:58.725 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:58.722+0000 7f4f59a15700 1 --2- 172.21.15.59:0/932713681 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f54100e80 0x7f4f54198e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:44:58.725 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:58.722+0000 7f4f59a15700 1 -- 172.21.15.59:0/932713681 >> 172.21.15.59:0/932713681 conn(0x7f4f540774d0 msgr2=0x7f4f540ff280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:44:58.726 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:58.722+0000 7f4f59a15700 1 -- 172.21.15.59:0/932713681 shutdown_connections 2023-12-11T17:44:58.726 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:44:58.722+0000 7f4f59a15700 1 -- 172.21.15.59:0/932713681 wait complete. 2023-12-11T17:44:59.097 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:58 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/932713681' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-11T17:44:59.123 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2023-12-11T17:44:59.124 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell --fsid d0a07eaa-984b-11ee-95a3-87774f69a715 -- ceph osd dump --format=json 2023-12-11T17:44:59.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:58 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/932713681' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-11T17:44:59.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:58 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/932713681' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-11T17:44:59.729 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/mon.a/config 2023-12-11T17:45:00.119 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:44:59 smithi059 ceph-mon[98847]: pgmap v278: 129 pgs: 129 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 7.5 KiB/s wr, 15 op/s 2023-12-11T17:45:00.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:44:59 smithi181 ceph-mon[103627]: pgmap v278: 129 pgs: 129 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 7.5 KiB/s wr, 15 op/s 2023-12-11T17:45:00.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:44:59 smithi115 ceph-mon[102748]: pgmap v278: 129 pgs: 129 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 7.5 KiB/s wr, 15 op/s 2023-12-11T17:45:00.960 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:00.955+0000 7f1d2088d700 1 -- 172.21.15.59:0/2646698695 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1d1c1066b0 msgr2=0x7f1d1c106a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:00.960 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:00.955+0000 7f1d2088d700 1 --2- 172.21.15.59:0/2646698695 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1d1c1066b0 0x7f1d1c106a90 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f1d0c009990 tx=0x7f1d0c009ca0 comp rx=0 tx=0).stop 2023-12-11T17:45:00.960 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:00.955+0000 7f1d2088d700 1 -- 172.21.15.59:0/2646698695 shutdown_connections 2023-12-11T17:45:00.960 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:00.955+0000 7f1d2088d700 1 --2- 172.21.15.59:0/2646698695 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1d1c100910 0x7f1d1c10e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:00.960 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:00.955+0000 7f1d2088d700 1 --2- 172.21.15.59:0/2646698695 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1d1c1066b0 0x7f1d1c106a90 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:00.960 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:00.955+0000 7f1d2088d700 1 --2- 172.21.15.59:0/2646698695 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d1c0fff70 0x7f1d1c1003d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:00.960 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:00.955+0000 7f1d2088d700 1 -- 172.21.15.59:0/2646698695 >> 172.21.15.59:0/2646698695 conn(0x7f1d1c0757f0 msgr2=0x7f1d1c075c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:45:00.960 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:00.955+0000 7f1d2088d700 1 -- 172.21.15.59:0/2646698695 shutdown_connections 2023-12-11T17:45:00.961 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:00.955+0000 7f1d2088d700 1 -- 172.21.15.59:0/2646698695 wait complete. 2023-12-11T17:45:00.961 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:00.956+0000 7f1d2088d700 1 Processor -- start 2023-12-11T17:45:00.961 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:00.956+0000 7f1d2088d700 1 -- start start 2023-12-11T17:45:00.961 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:00.956+0000 7f1d2088d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d1c0fff70 0x7f1d1c197a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:00.961 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:00.956+0000 7f1d2088d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1d1c100910 0x7f1d1c197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:00.961 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:00.956+0000 7f1d2088d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1d1c1066b0 0x7f1d1c19c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:00.961 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:00.956+0000 7f1d2088d700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d1c19c930 con 0x7f1d1c1066b0 2023-12-11T17:45:00.961 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:00.956+0000 7f1d2088d700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d1c19caa0 con 0x7f1d1c100910 2023-12-11T17:45:00.961 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:00.956+0000 7f1d2088d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d1c19cc60 con 0x7f1d1c0fff70 2023-12-11T17:45:00.961 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:00.956+0000 7f1d1ad9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1d1c1066b0 0x7f1d1c19c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:00.961 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:00.957+0000 7f1d1ad9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1d1c1066b0 0x7f1d1c19c360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33518/0 (socket says 172.21.15.59:33518) 2023-12-11T17:45:00.961 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:00.957+0000 7f1d1ad9d700 1 -- 172.21.15.59:0/1931471969 learned_addr learned my addr 172.21.15.59:0/1931471969 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:45:00.961 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:00.957+0000 7f1d1a59c700 1 --2- 172.21.15.59:0/1931471969 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d1c0fff70 0x7f1d1c197a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:00.961 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:00.957+0000 7f1d19d9b700 1 --2- 172.21.15.59:0/1931471969 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1d1c100910 0x7f1d1c197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:00.961 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:00.957+0000 7f1d1ad9d700 1 -- 172.21.15.59:0/1931471969 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d1c0fff70 msgr2=0x7f1d1c197a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:00.962 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:00.957+0000 7f1d1ad9d700 1 --2- 172.21.15.59:0/1931471969 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d1c0fff70 0x7f1d1c197a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:00.962 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:00.957+0000 7f1d1ad9d700 1 -- 172.21.15.59:0/1931471969 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1d1c100910 msgr2=0x7f1d1c197f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:00.962 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:00.957+0000 7f1d1ad9d700 1 --2- 172.21.15.59:0/1931471969 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1d1c100910 0x7f1d1c197f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:00.962 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:00.957+0000 7f1d1ad9d700 1 -- 172.21.15.59:0/1931471969 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d0c0095f0 con 0x7f1d1c1066b0 2023-12-11T17:45:00.962 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:00.957+0000 7f1d1ad9d700 1 --2- 172.21.15.59:0/1931471969 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1d1c1066b0 0x7f1d1c19c360 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f1d1400ada0 tx=0x7f1d1400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:00.962 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:00.957+0000 7f1d0b7fe700 1 -- 172.21.15.59:0/1931471969 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d1400f070 con 0x7f1d1c1066b0 2023-12-11T17:45:00.962 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:00.957+0000 7f1d0b7fe700 1 -- 172.21.15.59:0/1931471969 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1d14007b20 con 0x7f1d1c1066b0 2023-12-11T17:45:00.962 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:00.957+0000 7f1d2088d700 1 -- 172.21.15.59:0/1931471969 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d1c19cf40 con 0x7f1d1c1066b0 2023-12-11T17:45:00.962 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:00.957+0000 7f1d0b7fe700 1 -- 172.21.15.59:0/1931471969 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d14004290 con 0x7f1d1c1066b0 2023-12-11T17:45:00.962 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:00.957+0000 7f1d2088d700 1 -- 172.21.15.59:0/1931471969 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d1c1a47e0 con 0x7f1d1c1066b0 2023-12-11T17:45:00.963 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:00.958+0000 7f1d2088d700 1 -- 172.21.15.59:0/1931471969 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d1c109b10 con 0x7f1d1c1066b0 2023-12-11T17:45:00.963 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:00.959+0000 7f1d0b7fe700 1 -- 172.21.15.59:0/1931471969 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f1d1400c980 con 0x7f1d1c1066b0 2023-12-11T17:45:00.963 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:00.960+0000 7f1d0b7fe700 1 --2- 172.21.15.59:0/1931471969 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f1d040a1590 0x7f1d040a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:00.963 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:00.960+0000 7f1d0b7fe700 1 -- 172.21.15.59:0/1931471969 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f1d140cc440 con 0x7f1d1c1066b0 2023-12-11T17:45:00.964 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:00.960+0000 7f1d1a59c700 1 --2- 172.21.15.59:0/1931471969 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f1d040a1590 0x7f1d040a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:00.964 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:00.960+0000 7f1d1a59c700 1 --2- 172.21.15.59:0/1931471969 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f1d040a1590 0x7f1d040a3a50 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f1d0c00b5c0 tx=0x7f1d0c005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:00.966 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:00.964+0000 7f1d0b7fe700 1 -- 172.21.15.59:0/1931471969 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f1d140956f0 con 0x7f1d1c1066b0 2023-12-11T17:45:01.170 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:01.168+0000 7f1d2088d700 1 -- 172.21.15.59:0/1931471969 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1d1c04f290 con 0x7f1d1c1066b0 2023-12-11T17:45:01.172 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:01.170+0000 7f1d0b7fe700 1 -- 172.21.15.59:0/1931471969 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+23960 (secure 0 0 0) 0x7f1d14098d10 con 0x7f1d1c1066b0 2023-12-11T17:45:01.172 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-12-11T17:45:01.173 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":86,"fsid":"d0a07eaa-984b-11ee-95a3-87774f69a715","created":"2023-12-11T17:37:14.526539+0000","modified":"2023-12-11T17:44:25.096094+0000","last_up_change":"2023-12-11T17:43:38.535503+0000","last_in_change":"2023-12-11T17:43:21.014888+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-11T17:40:26.596029+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-11T17:44:18.586290+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-11T17:44:19.432952+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-11T17:44:23.097308+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3c0b3218-4b2b-40f8-8bef-ed8cd5633607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":1040759398},{"type":"v1","addr":"172.21.15.59:6803","nonce":1040759398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":1040759398},{"type":"v1","addr":"172.21.15.59:6805","nonce":1040759398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":1040759398},{"type":"v1","addr":"172.21.15.59:6809","nonce":1040759398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":1040759398},{"type":"v1","addr":"172.21.15.59:6807","nonce":1040759398}]},"public_addr":"172.21.15.59:6803/1040759398","cluster_addr":"172.21.15.59:6805/1040759398","heartbeat_back_addr":"172.21.15.59:6809/1040759398","heartbeat_front_addr":"172.21.15.59:6807/1040759398","state":["exists","up"]},{"osd":1,"uuid":"2ead7578-765b-4f5a-a72c-863731ed69e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":2776354245},{"type":"v1","addr":"172.21.15.115:6801","nonce":2776354245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":2776354245},{"type":"v1","addr":"172.21.15.115:6803","nonce":2776354245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":2776354245},{"type":"v1","addr":"172.21.15.115:6807","nonce":2776354245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":2776354245},{"type":"v1","addr":"172.21.15.115:6805","nonce":2776354245}]},"public_addr":"172.21.15.115:6801/2776354245","cluster_addr":"172.21.15.115:6803/2776354245","heartbeat_back_addr":"172.21.15.115:6807/2776354245","heartbeat_front_addr":"172.21.15.115:6805/2776354245","state":["exists","up"]},{"osd":2,"uuid":"8cb1d7ec-53cc-4672-b0b6-70d73797c44f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4287675481},{"type":"v1","addr":"172.21.15.181:6801","nonce":4287675481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4287675481},{"type":"v1","addr":"172.21.15.181:6803","nonce":4287675481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4287675481},{"type":"v1","addr":"172.21.15.181:6807","nonce":4287675481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4287675481},{"type":"v1","addr":"172.21.15.181:6805","nonce":4287675481}]},"public_addr":"172.21.15.181:6801/4287675481","cluster_addr":"172.21.15.181:6803/4287675481","heartbeat_back_addr":"172.21.15.181:6807/4287675481","heartbeat_front_addr":"172.21.15.181:6805/4287675481","state":["exists","up"]},{"osd":3,"uuid":"5d64d9af-831b-4d0a-8ace-a452169351d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":219909995},{"type":"v1","addr":"172.21.15.59:6811","nonce":219909995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":219909995},{"type":"v1","addr":"172.21.15.59:6813","nonce":219909995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":219909995},{"type":"v1","addr":"172.21.15.59:6817","nonce":219909995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":219909995},{"type":"v1","addr":"172.21.15.59:6815","nonce":219909995}]},"public_addr":"172.21.15.59:6811/219909995","cluster_addr":"172.21.15.59:6813/219909995","heartbeat_back_addr":"172.21.15.59:6817/219909995","heartbeat_front_addr":"172.21.15.59:6815/219909995","state":["exists","up"]},{"osd":4,"uuid":"ef8fad28-1db5-4cba-b5f2-10e54e8e1549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":1021473826},{"type":"v1","addr":"172.21.15.115:6809","nonce":1021473826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":1021473826},{"type":"v1","addr":"172.21.15.115:6811","nonce":1021473826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":1021473826},{"type":"v1","addr":"172.21.15.115:6815","nonce":1021473826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":1021473826},{"type":"v1","addr":"172.21.15.115:6813","nonce":1021473826}]},"public_addr":"172.21.15.115:6809/1021473826","cluster_addr":"172.21.15.115:6811/1021473826","heartbeat_back_addr":"172.21.15.115:6815/1021473826","heartbeat_front_addr":"172.21.15.115:6813/1021473826","state":["exists","up"]},{"osd":5,"uuid":"4233a512-9e8f-4db8-aed2-6c3af706e589","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":2405172521},{"type":"v1","addr":"172.21.15.181:6809","nonce":2405172521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":2405172521},{"type":"v1","addr":"172.21.15.181:6811","nonce":2405172521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2405172521},{"type":"v1","addr":"172.21.15.181:6815","nonce":2405172521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":2405172521},{"type":"v1","addr":"172.21.15.181:6813","nonce":2405172521}]},"public_addr":"172.21.15.181:6809/2405172521","cluster_addr":"172.21.15.181:6811/2405172521","heartbeat_back_addr":"172.21.15.181:6815/2405172521","heartbeat_front_addr":"172.21.15.181:6813/2405172521","state":["exists","up"]},{"osd":6,"uuid":"8faab332-016e-4c77-8370-8bef95fc6f34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":815374853},{"type":"v1","addr":"172.21.15.59:6819","nonce":815374853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":815374853},{"type":"v1","addr":"172.21.15.59:6821","nonce":815374853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":815374853},{"type":"v1","addr":"172.21.15.59:6825","nonce":815374853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":815374853},{"type":"v1","addr":"172.21.15.59:6823","nonce":815374853}]},"public_addr":"172.21.15.59:6819/815374853","cluster_addr":"172.21.15.59:6821/815374853","heartbeat_back_addr":"172.21.15.59:6825/815374853","heartbeat_front_addr":"172.21.15.59:6823/815374853","state":["exists","up"]},{"osd":7,"uuid":"f20f470a-0d40-4ed3-8e0e-fd58675cf143","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":1314971870},{"type":"v1","addr":"172.21.15.115:6817","nonce":1314971870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":1314971870},{"type":"v1","addr":"172.21.15.115:6819","nonce":1314971870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":1314971870},{"type":"v1","addr":"172.21.15.115:6823","nonce":1314971870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":1314971870},{"type":"v1","addr":"172.21.15.115:6821","nonce":1314971870}]},"public_addr":"172.21.15.115:6817/1314971870","cluster_addr":"172.21.15.115:6819/1314971870","heartbeat_back_addr":"172.21.15.115:6823/1314971870","heartbeat_front_addr":"172.21.15.115:6821/1314971870","state":["exists","up"]},{"osd":8,"uuid":"eb657d32-1a95-4fd7-a293-d0a970837fc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":3954509873},{"type":"v1","addr":"172.21.15.181:6817","nonce":3954509873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":3954509873},{"type":"v1","addr":"172.21.15.181:6819","nonce":3954509873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":3954509873},{"type":"v1","addr":"172.21.15.181:6823","nonce":3954509873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":3954509873},{"type":"v1","addr":"172.21.15.181:6821","nonce":3954509873}]},"public_addr":"172.21.15.181:6817/3954509873","cluster_addr":"172.21.15.181:6819/3954509873","heartbeat_back_addr":"172.21.15.181:6823/3954509873","heartbeat_front_addr":"172.21.15.181:6821/3954509873","state":["exists","up"]},{"osd":9,"uuid":"8f7717d4-b6f2-4ec1-9b58-62feb0c9a146","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":1163095677},{"type":"v1","addr":"172.21.15.59:6827","nonce":1163095677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":1163095677},{"type":"v1","addr":"172.21.15.59:6829","nonce":1163095677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":1163095677},{"type":"v1","addr":"172.21.15.59:6833","nonce":1163095677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":1163095677},{"type":"v1","addr":"172.21.15.59:6831","nonce":1163095677}]},"public_addr":"172.21.15.59:6827/1163095677","cluster_addr":"172.21.15.59:6829/1163095677","heartbeat_back_addr":"172.21.15.59:6833/1163095677","heartbeat_front_addr":"172.21.15.59:6831/1163095677","state":["exists","up"]},{"osd":10,"uuid":"b75c179a-875d-43bc-b4c1-e8eb80fb64fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":2199610269},{"type":"v1","addr":"172.21.15.115:6825","nonce":2199610269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":2199610269},{"type":"v1","addr":"172.21.15.115:6827","nonce":2199610269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":2199610269},{"type":"v1","addr":"172.21.15.115:6831","nonce":2199610269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":2199610269},{"type":"v1","addr":"172.21.15.115:6829","nonce":2199610269}]},"public_addr":"172.21.15.115:6825/2199610269","cluster_addr":"172.21.15.115:6827/2199610269","heartbeat_back_addr":"172.21.15.115:6831/2199610269","heartbeat_front_addr":"172.21.15.115:6829/2199610269","state":["exists","up"]},{"osd":11,"uuid":"5e141dfe-8a00-4bbc-8204-2dc9be24e18d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":226506989},{"type":"v1","addr":"172.21.15.181:6825","nonce":226506989}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":226506989},{"type":"v1","addr":"172.21.15.181:6827","nonce":226506989}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":226506989},{"type":"v1","addr":"172.21.15.181:6831","nonce":226506989}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":226506989},{"type":"v1","addr":"172.21.15.181:6829","nonce":226506989}]},"public_addr":"172.21.15.181:6825/226506989","cluster_addr":"172.21.15.181:6827/226506989","heartbeat_back_addr":"172.21.15.181:6831/226506989","heartbeat_front_addr":"172.21.15.181:6829/226506989","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:39:42.687084+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:40:03.903448+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:40:24.899908+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:40:46.600251+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:41:07.690037+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:41:28.488855+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:41:49.459274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:42:10.863881+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:42:32.248391+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:42:54.112265+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:43:15.634872+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:43:36.922534+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.59:6800/1259916482":"2023-12-12T17:38:14.037922+0000","172.21.15.59:0/17457774":"2023-12-12T17:38:14.037922+0000","172.21.15.59:6801/1672530462":"2023-12-12T17:37:44.773925+0000","172.21.15.59:0/1053944547":"2023-12-12T17:38:14.037922+0000","172.21.15.59:6801/1259916482":"2023-12-12T17:38:14.037922+0000","172.21.15.59:6800/1672530462":"2023-12-12T17:37:44.773925+0000","172.21.15.59:0/3570767650":"2023-12-12T17:37:44.773925+0000","172.21.15.59:0/3853462899":"2023-12-12T17:37:44.773925+0000","172.21.15.59:0/2859415380":"2023-12-12T17:38:14.037922+0000","172.21.15.59:0/2192742819":"2023-12-12T17:37:44.773925+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-11T17:45:01.174 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:01.172+0000 7f1d2088d700 1 -- 172.21.15.59:0/1931471969 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f1d040a1590 msgr2=0x7f1d040a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:01.174 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:01.172+0000 7f1d2088d700 1 --2- 172.21.15.59:0/1931471969 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f1d040a1590 0x7f1d040a3a50 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f1d0c00b5c0 tx=0x7f1d0c005fd0 comp rx=0 tx=0).stop 2023-12-11T17:45:01.174 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:01.172+0000 7f1d2088d700 1 -- 172.21.15.59:0/1931471969 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1d1c1066b0 msgr2=0x7f1d1c19c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:01.174 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:01.172+0000 7f1d2088d700 1 --2- 172.21.15.59:0/1931471969 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1d1c1066b0 0x7f1d1c19c360 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f1d1400ada0 tx=0x7f1d1400c330 comp rx=0 tx=0).stop 2023-12-11T17:45:01.175 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:01.172+0000 7f1d2088d700 1 -- 172.21.15.59:0/1931471969 shutdown_connections 2023-12-11T17:45:01.175 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:01.172+0000 7f1d2088d700 1 --2- 172.21.15.59:0/1931471969 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f1d040a1590 0x7f1d040a3a50 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:01.175 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:01.172+0000 7f1d2088d700 1 --2- 172.21.15.59:0/1931471969 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1d1c1066b0 0x7f1d1c19c360 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:01.175 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:01.172+0000 7f1d2088d700 1 --2- 172.21.15.59:0/1931471969 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d1c0fff70 0x7f1d1c197a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:01.175 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:01.172+0000 7f1d2088d700 1 --2- 172.21.15.59:0/1931471969 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1d1c100910 0x7f1d1c197f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:01.175 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:01.172+0000 7f1d2088d700 1 -- 172.21.15.59:0/1931471969 >> 172.21.15.59:0/1931471969 conn(0x7f1d1c0757f0 msgr2=0x7f1d1c0fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:45:01.175 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:01.172+0000 7f1d2088d700 1 -- 172.21.15.59:0/1931471969 shutdown_connections 2023-12-11T17:45:01.175 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:01.172+0000 7f1d2088d700 1 -- 172.21.15.59:0/1931471969 wait complete. 2023-12-11T17:45:01.693 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell --fsid d0a07eaa-984b-11ee-95a3-87774f69a715 -- ceph tell osd.0 flush_pg_stats 2023-12-11T17:45:01.694 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell --fsid d0a07eaa-984b-11ee-95a3-87774f69a715 -- ceph tell osd.1 flush_pg_stats 2023-12-11T17:45:01.694 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell --fsid d0a07eaa-984b-11ee-95a3-87774f69a715 -- ceph tell osd.2 flush_pg_stats 2023-12-11T17:45:01.694 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell --fsid d0a07eaa-984b-11ee-95a3-87774f69a715 -- ceph tell osd.3 flush_pg_stats 2023-12-11T17:45:01.695 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell --fsid d0a07eaa-984b-11ee-95a3-87774f69a715 -- ceph tell osd.4 flush_pg_stats 2023-12-11T17:45:01.695 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell --fsid d0a07eaa-984b-11ee-95a3-87774f69a715 -- ceph tell osd.5 flush_pg_stats 2023-12-11T17:45:01.696 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell --fsid d0a07eaa-984b-11ee-95a3-87774f69a715 -- ceph tell osd.6 flush_pg_stats 2023-12-11T17:45:01.696 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell --fsid d0a07eaa-984b-11ee-95a3-87774f69a715 -- ceph tell osd.7 flush_pg_stats 2023-12-11T17:45:01.696 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell --fsid d0a07eaa-984b-11ee-95a3-87774f69a715 -- ceph tell osd.8 flush_pg_stats 2023-12-11T17:45:01.697 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell --fsid d0a07eaa-984b-11ee-95a3-87774f69a715 -- ceph tell osd.9 flush_pg_stats 2023-12-11T17:45:01.697 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell --fsid d0a07eaa-984b-11ee-95a3-87774f69a715 -- ceph tell osd.10 flush_pg_stats 2023-12-11T17:45:01.698 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell --fsid d0a07eaa-984b-11ee-95a3-87774f69a715 -- ceph tell osd.11 flush_pg_stats 2023-12-11T17:45:02.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:45:01 smithi181 ceph-mon[103627]: pgmap v279: 129 pgs: 129 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 7.3 KiB/s wr, 12 op/s 2023-12-11T17:45:02.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:45:01 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/1931471969' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-11T17:45:02.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:45:01 smithi059 ceph-mon[98847]: pgmap v279: 129 pgs: 129 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 7.3 KiB/s wr, 12 op/s 2023-12-11T17:45:02.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:45:01 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/1931471969' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-11T17:45:02.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:45:01 smithi115 ceph-mon[102748]: pgmap v279: 129 pgs: 129 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 7.3 KiB/s wr, 12 op/s 2023-12-11T17:45:02.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:45:01 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/1931471969' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-11T17:45:02.648 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/mon.a/config 2023-12-11T17:45:02.677 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/mon.a/config 2023-12-11T17:45:02.708 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/mon.a/config 2023-12-11T17:45:02.768 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/mon.a/config 2023-12-11T17:45:02.794 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/mon.a/config 2023-12-11T17:45:02.829 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/mon.a/config 2023-12-11T17:45:02.987 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/mon.a/config 2023-12-11T17:45:03.533 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/mon.a/config 2023-12-11T17:45:03.584 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/mon.a/config 2023-12-11T17:45:03.585 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/mon.a/config 2023-12-11T17:45:03.848 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:45:03 smithi059 ceph-mon[98847]: pgmap v280: 129 pgs: 129 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 6.2 KiB/s wr, 10 op/s 2023-12-11T17:45:03.913 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/mon.a/config 2023-12-11T17:45:03.948 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/mon.a/config 2023-12-11T17:45:04.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:45:03 smithi181 ceph-mon[103627]: pgmap v280: 129 pgs: 129 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 6.2 KiB/s wr, 10 op/s 2023-12-11T17:45:04.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:45:03 smithi115 ceph-mon[102748]: pgmap v280: 129 pgs: 129 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 6.2 KiB/s wr, 10 op/s 2023-12-11T17:45:06.063 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:45:05 smithi059 ceph-mon[98847]: pgmap v281: 129 pgs: 129 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.2 KiB/s wr, 8 op/s 2023-12-11T17:45:06.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:45:05 smithi181 ceph-mon[103627]: pgmap v281: 129 pgs: 129 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.2 KiB/s wr, 8 op/s 2023-12-11T17:45:06.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:45:05 smithi115 ceph-mon[102748]: pgmap v281: 129 pgs: 129 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.2 KiB/s wr, 8 op/s 2023-12-11T17:45:08.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:45:07 smithi181 ceph-mon[103627]: pgmap v282: 129 pgs: 129 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2023-12-11T17:45:08.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:45:07 smithi059 ceph-mon[98847]: pgmap v282: 129 pgs: 129 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2023-12-11T17:45:08.213 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:07.422+0000 7fadc8f6e700 1 -- 172.21.15.59:0/4213005481 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fadc40fff70 msgr2=0x7fadc41003d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:08.213 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:07.422+0000 7fadc8f6e700 1 --2- 172.21.15.59:0/4213005481 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fadc40fff70 0x7fadc41003d0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fadb8009990 tx=0x7fadb8009ca0 comp rx=0 tx=0).stop 2023-12-11T17:45:08.213 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:07.422+0000 7fadc8f6e700 1 -- 172.21.15.59:0/4213005481 shutdown_connections 2023-12-11T17:45:08.213 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:07.422+0000 7fadc8f6e700 1 --2- 172.21.15.59:0/4213005481 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fadc4100910 0x7fadc410e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:08.213 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:07.422+0000 7fadc8f6e700 1 --2- 172.21.15.59:0/4213005481 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fadc41066b0 0x7fadc4106a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:08.213 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:07.422+0000 7fadc8f6e700 1 --2- 172.21.15.59:0/4213005481 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fadc40fff70 0x7fadc41003d0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:08.213 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:07.422+0000 7fadc8f6e700 1 -- 172.21.15.59:0/4213005481 >> 172.21.15.59:0/4213005481 conn(0x7fadc40757f0 msgr2=0x7fadc4075c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:45:08.214 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:07.422+0000 7fadc8f6e700 1 -- 172.21.15.59:0/4213005481 shutdown_connections 2023-12-11T17:45:08.214 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:07.422+0000 7fadc8f6e700 1 -- 172.21.15.59:0/4213005481 wait complete. 2023-12-11T17:45:08.214 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:07.424+0000 7fadc8f6e700 1 Processor -- start 2023-12-11T17:45:08.214 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:07.424+0000 7fadc8f6e700 1 -- start start 2023-12-11T17:45:08.214 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:07.424+0000 7fadc8f6e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fadc40fff70 0x7fadc4101150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:08.214 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:07.424+0000 7fadc8f6e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fadc4100910 0x7fadc4101690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:08.214 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:07.424+0000 7fadc8f6e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fadc41066b0 0x7fadc4101bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:08.214 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:07.424+0000 7fadc8f6e700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadc41a5cb0 con 0x7fadc41066b0 2023-12-11T17:45:08.214 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:07.424+0000 7fadc8f6e700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadc41a5e20 con 0x7fadc4100910 2023-12-11T17:45:08.214 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:07.424+0000 7fadc8f6e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadc41a5fe0 con 0x7fadc40fff70 2023-12-11T17:45:08.214 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:07.424+0000 7fadc1d9b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fadc4100910 0x7fadc4101690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:08.214 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:07.424+0000 7fadc259c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fadc40fff70 0x7fadc4101150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:08.214 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:07.424+0000 7fadc1d9b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fadc4100910 0x7fadc4101690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.59:58272/0 (socket says 172.21.15.59:58272) 2023-12-11T17:45:08.215 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:07.424+0000 7fadc1d9b700 1 -- 172.21.15.59:0/1173312279 learned_addr learned my addr 172.21.15.59:0/1173312279 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:45:08.215 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:07.424+0000 7fadc259c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fadc40fff70 0x7fadc4101150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.59:38360/0 (socket says 172.21.15.59:38360) 2023-12-11T17:45:08.215 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:07.425+0000 7fadc1d9b700 1 -- 172.21.15.59:0/1173312279 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fadc40fff70 msgr2=0x7fadc4101150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:08.215 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:07.425+0000 7fadc1d9b700 1 --2- 172.21.15.59:0/1173312279 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fadc40fff70 0x7fadc4101150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:08.215 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:07.425+0000 7fadc1d9b700 1 -- 172.21.15.59:0/1173312279 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fadc41066b0 msgr2=0x7fadc4101bd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-11T17:45:08.215 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:07.425+0000 7fadc1d9b700 1 --2- 172.21.15.59:0/1173312279 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fadc41066b0 0x7fadc4101bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:08.215 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:07.425+0000 7fadc1d9b700 1 -- 172.21.15.59:0/1173312279 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadb80095f0 con 0x7fadc4100910 2023-12-11T17:45:08.215 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:07.425+0000 7fadc259c700 1 --2- 172.21.15.59:0/1173312279 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fadc40fff70 0x7fadc4101150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T17:45:08.215 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:07.425+0000 7fadc1d9b700 1 --2- 172.21.15.59:0/1173312279 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fadc4100910 0x7fadc4101690 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fadb8009fd0 tx=0x7fadb800f670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:08.215 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:07.425+0000 7fadb37fe700 1 -- 172.21.15.59:0/1173312279 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadb801d070 con 0x7fadc4100910 2023-12-11T17:45:08.215 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:07.425+0000 7fadb37fe700 1 -- 172.21.15.59:0/1173312279 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fadb8005510 con 0x7fadc4100910 2023-12-11T17:45:08.216 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:07.425+0000 7fadc8f6e700 1 -- 172.21.15.59:0/1173312279 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadc41a6260 con 0x7fadc4100910 2023-12-11T17:45:08.216 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:07.425+0000 7fadb37fe700 1 -- 172.21.15.59:0/1173312279 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadb8004a90 con 0x7fadc4100910 2023-12-11T17:45:08.216 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:07.425+0000 7fadc8f6e700 1 -- 172.21.15.59:0/1173312279 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadc41a6700 con 0x7fadc4100910 2023-12-11T17:45:08.216 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:07.426+0000 7fadc8f6e700 1 -- 172.21.15.59:0/1173312279 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fadc41117c0 con 0x7fadc4100910 2023-12-11T17:45:08.216 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:07.428+0000 7fadb37fe700 1 -- 172.21.15.59:0/1173312279 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7fadb8004c10 con 0x7fadc4100910 2023-12-11T17:45:08.216 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:07.429+0000 7fadb37fe700 1 --2- 172.21.15.59:0/1173312279 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fadac0a1510 0x7fadac0a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:08.216 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:07.429+0000 7fadb37fe700 1 -- 172.21.15.59:0/1173312279 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fadb80ce4a0 con 0x7fadc4100910 2023-12-11T17:45:08.216 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:07.429+0000 7fadc259c700 1 --2- 172.21.15.59:0/1173312279 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fadac0a1510 0x7fadac0a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:08.216 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:07.429+0000 7fadb37fe700 1 --2- 172.21.15.59:0/1173312279 >> [v2:172.21.15.59:6826/1163095677,v1:172.21.15.59:6827/1163095677] conn(0x7fadac0ac090 0x7fadac0ae4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:08.216 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:07.429+0000 7fadb37fe700 1 -- 172.21.15.59:0/1173312279 --> [v2:172.21.15.59:6826/1163095677,v1:172.21.15.59:6827/1163095677] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fadac0aeba0 con 0x7fadac0ac090 2023-12-11T17:45:08.216 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:07.429+0000 7fadb37fe700 1 -- 172.21.15.59:0/1173312279 <== mon.2 v2:172.21.15.115:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7fadb80ce8b0 con 0x7fadc4100910 2023-12-11T17:45:08.216 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:07.429+0000 7fadc2d9d700 1 --2- 172.21.15.59:0/1173312279 >> [v2:172.21.15.59:6826/1163095677,v1:172.21.15.59:6827/1163095677] conn(0x7fadac0ac090 0x7fadac0ae4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:08.216 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:07.430+0000 7fadc259c700 1 --2- 172.21.15.59:0/1173312279 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fadac0a1510 0x7fadac0a39d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fadc4068f40 tx=0x7fadb400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:08.217 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:07.430+0000 7fadc2d9d700 1 --2- 172.21.15.59:0/1173312279 >> [v2:172.21.15.59:6826/1163095677,v1:172.21.15.59:6827/1163095677] conn(0x7fadac0ac090 0x7fadac0ae4d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:08.217 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:07.431+0000 7fadb37fe700 1 -- 172.21.15.59:0/1173312279 <== osd.9 v2:172.21.15.59:6826/1163095677 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fadac0aeba0 con 0x7fadac0ac090 2023-12-11T17:45:08.217 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:07.447+0000 7fadc8f6e700 1 -- 172.21.15.59:0/1173312279 --> [v2:172.21.15.59:6826/1163095677,v1:172.21.15.59:6827/1163095677] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fadc4061c50 con 0x7fadac0ac090 2023-12-11T17:45:08.217 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:07.447+0000 7fadb37fe700 1 -- 172.21.15.59:0/1173312279 <== osd.9 v2:172.21.15.59:6826/1163095677 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fadc4061c50 con 0x7fadac0ac090 2023-12-11T17:45:08.217 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:07.448+0000 7fadc8f6e700 1 -- 172.21.15.59:0/1173312279 >> [v2:172.21.15.59:6826/1163095677,v1:172.21.15.59:6827/1163095677] conn(0x7fadac0ac090 msgr2=0x7fadac0ae4d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:08.217 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:07.448+0000 7fadc8f6e700 1 --2- 172.21.15.59:0/1173312279 >> [v2:172.21.15.59:6826/1163095677,v1:172.21.15.59:6827/1163095677] conn(0x7fadac0ac090 0x7fadac0ae4d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:08.217 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:07.448+0000 7fadc8f6e700 1 -- 172.21.15.59:0/1173312279 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fadac0a1510 msgr2=0x7fadac0a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:08.217 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:07.448+0000 7fadc8f6e700 1 --2- 172.21.15.59:0/1173312279 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fadac0a1510 0x7fadac0a39d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fadc4068f40 tx=0x7fadb400b410 comp rx=0 tx=0).stop 2023-12-11T17:45:08.218 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:07.448+0000 7fadc8f6e700 1 -- 172.21.15.59:0/1173312279 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fadc4100910 msgr2=0x7fadc4101690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:08.218 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:07.448+0000 7fadc8f6e700 1 --2- 172.21.15.59:0/1173312279 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fadc4100910 0x7fadc4101690 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fadb8009fd0 tx=0x7fadb800f670 comp rx=0 tx=0).stop 2023-12-11T17:45:08.218 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:07.448+0000 7fadc2d9d700 1 -- 172.21.15.59:0/1173312279 reap_dead start 2023-12-11T17:45:08.218 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:07.448+0000 7fadc8f6e700 1 -- 172.21.15.59:0/1173312279 shutdown_connections 2023-12-11T17:45:08.218 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:07.448+0000 7fadc8f6e700 1 -- 172.21.15.59:0/1173312279 >> 172.21.15.59:0/1173312279 conn(0x7fadc40757f0 msgr2=0x7fadc40feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:45:08.218 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:07.448+0000 7fadc8f6e700 1 -- 172.21.15.59:0/1173312279 shutdown_connections 2023-12-11T17:45:08.218 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:07.448+0000 7fadc8f6e700 1 -- 172.21.15.59:0/1173312279 wait complete. 2023-12-11T17:45:08.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:45:07 smithi115 ceph-mon[102748]: pgmap v282: 129 pgs: 129 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2023-12-11T17:45:08.670 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:08.666+0000 7f621b2ac700 1 -- 172.21.15.59:0/2846749711 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6214105470 msgr2=0x7f6214105850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:08.670 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:08.666+0000 7f621b2ac700 1 --2- 172.21.15.59:0/2846749711 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6214105470 0x7f6214105850 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f62100099c0 tx=0x7f6210009cd0 comp rx=0 tx=0).stop 2023-12-11T17:45:08.670 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:08.666+0000 7f621b2ac700 1 -- 172.21.15.59:0/2846749711 shutdown_connections 2023-12-11T17:45:08.670 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:08.667+0000 7f621b2ac700 1 --2- 172.21.15.59:0/2846749711 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6214100e80 0x7f621410d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:08.670 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:08.667+0000 7f621b2ac700 1 --2- 172.21.15.59:0/2846749711 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6214105470 0x7f6214105850 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:08.670 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:08.667+0000 7f621b2ac700 1 --2- 172.21.15.59:0/2846749711 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f62141004c0 0x7f6214100940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:08.670 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:08.667+0000 7f621b2ac700 1 -- 172.21.15.59:0/2846749711 >> 172.21.15.59:0/2846749711 conn(0x7f62140774a0 msgr2=0x7f62140778b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:45:08.671 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:08.667+0000 7f621b2ac700 1 -- 172.21.15.59:0/2846749711 shutdown_connections 2023-12-11T17:45:08.671 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:08.667+0000 7f621b2ac700 1 -- 172.21.15.59:0/2846749711 wait complete. 2023-12-11T17:45:08.674 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:08.667+0000 7f621b2ac700 1 Processor -- start 2023-12-11T17:45:08.674 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:08.667+0000 7f621b2ac700 1 -- start start 2023-12-11T17:45:08.675 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:08.668+0000 7f621b2ac700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f62141004c0 0x7f6214197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:08.675 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:08.668+0000 7f621b2ac700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6214100e80 0x7f6214197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:08.675 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:08.668+0000 7f621b2ac700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6214105470 0x7f621419c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:08.675 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:08.668+0000 7f621b2ac700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f621419c940 con 0x7f6214105470 2023-12-11T17:45:08.675 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:08.668+0000 7f621b2ac700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f621419cab0 con 0x7f6214100e80 2023-12-11T17:45:08.675 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:08.668+0000 7f621b2ac700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f621419cc70 con 0x7f62141004c0 2023-12-11T17:45:08.675 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:08.668+0000 7f6219849700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6214105470 0x7f621419c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:08.675 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:08.668+0000 7f6219849700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6214105470 0x7f621419c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33560/0 (socket says 172.21.15.59:33560) 2023-12-11T17:45:08.675 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:08.668+0000 7f6219849700 1 -- 172.21.15.59:0/691396887 learned_addr learned my addr 172.21.15.59:0/691396887 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:45:08.675 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:08.668+0000 7f6218847700 1 --2- 172.21.15.59:0/691396887 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6214100e80 0x7f6214197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:08.675 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:08.668+0000 7f6219048700 1 --2- 172.21.15.59:0/691396887 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f62141004c0 0x7f6214197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:08.676 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:08.668+0000 7f6219849700 1 -- 172.21.15.59:0/691396887 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f62141004c0 msgr2=0x7f6214197aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:08.676 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:08.668+0000 7f6219849700 1 --2- 172.21.15.59:0/691396887 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f62141004c0 0x7f6214197aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:08.676 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:08.668+0000 7f6219849700 1 -- 172.21.15.59:0/691396887 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6214100e80 msgr2=0x7f6214197fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:08.676 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:08.668+0000 7f6219849700 1 --2- 172.21.15.59:0/691396887 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6214100e80 0x7f6214197fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:08.676 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:08.668+0000 7f6219849700 1 -- 172.21.15.59:0/691396887 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62100095f0 con 0x7f6214105470 2023-12-11T17:45:08.676 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:08.668+0000 7f6219849700 1 --2- 172.21.15.59:0/691396887 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6214105470 0x7f621419c370 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f620c007ee0 tx=0x7f620c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:08.676 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:08.668+0000 7f620a7fc700 1 -- 172.21.15.59:0/691396887 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f620c017070 con 0x7f6214105470 2023-12-11T17:45:08.676 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:08.669+0000 7f621b2ac700 1 -- 172.21.15.59:0/691396887 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f621419cf50 con 0x7f6214105470 2023-12-11T17:45:08.676 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:08.669+0000 7f620a7fc700 1 -- 172.21.15.59:0/691396887 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f620c00f040 con 0x7f6214105470 2023-12-11T17:45:08.676 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:08.669+0000 7f620a7fc700 1 -- 172.21.15.59:0/691396887 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f620c005580 con 0x7f6214105470 2023-12-11T17:45:08.676 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:08.669+0000 7f621b2ac700 1 -- 172.21.15.59:0/691396887 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62141a47f0 con 0x7f6214105470 2023-12-11T17:45:08.676 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:08.670+0000 7f621b2ac700 1 -- 172.21.15.59:0/691396887 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f62141106c0 con 0x7f6214105470 2023-12-11T17:45:08.677 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:08.672+0000 7f620a7fc700 1 -- 172.21.15.59:0/691396887 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f620c00db20 con 0x7f6214105470 2023-12-11T17:45:08.678 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:08.672+0000 7f620a7fc700 1 --2- 172.21.15.59:0/691396887 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f62000a1510 0x7f62000a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:08.678 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:08.672+0000 7f620a7fc700 1 -- 172.21.15.59:0/691396887 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f620c0cce10 con 0x7f6214105470 2023-12-11T17:45:08.678 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:08.672+0000 7f6219048700 1 --2- 172.21.15.59:0/691396887 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f62000a1510 0x7f62000a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:08.678 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:08.673+0000 7f620a7fc700 1 --2- 172.21.15.59:0/691396887 >> [v2:172.21.15.59:6818/815374853,v1:172.21.15.59:6819/815374853] conn(0x7f62000ac090 0x7f62000ae4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:08.678 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:08.673+0000 7f620a7fc700 1 -- 172.21.15.59:0/691396887 --> [v2:172.21.15.59:6818/815374853,v1:172.21.15.59:6819/815374853] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f62000aeba0 con 0x7f62000ac090 2023-12-11T17:45:08.678 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:08.673+0000 7f620a7fc700 1 -- 172.21.15.59:0/691396887 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7f620c0cd220 con 0x7f6214105470 2023-12-11T17:45:08.678 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:08.673+0000 7f6218847700 1 --2- 172.21.15.59:0/691396887 >> [v2:172.21.15.59:6818/815374853,v1:172.21.15.59:6819/815374853] conn(0x7f62000ac090 0x7f62000ae4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:08.678 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:08.673+0000 7f6219048700 1 --2- 172.21.15.59:0/691396887 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f62000a1510 0x7f62000a39d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6210006010 tx=0x7f6210003680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:08.678 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:08.673+0000 7f6218847700 1 --2- 172.21.15.59:0/691396887 >> [v2:172.21.15.59:6818/815374853,v1:172.21.15.59:6819/815374853] conn(0x7f62000ac090 0x7f62000ae4d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:08.679 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:08.674+0000 7f620a7fc700 1 -- 172.21.15.59:0/691396887 <== osd.6 v2:172.21.15.59:6818/815374853 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f62000aeba0 con 0x7f62000ac090 2023-12-11T17:45:08.700 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:08.697+0000 7f621b2ac700 1 -- 172.21.15.59:0/691396887 --> [v2:172.21.15.59:6818/815374853,v1:172.21.15.59:6819/815374853] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f6214101bb0 con 0x7f62000ac090 2023-12-11T17:45:08.700 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:08.697+0000 7f620a7fc700 1 -- 172.21.15.59:0/691396887 <== osd.6 v2:172.21.15.59:6818/815374853 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f6214101bb0 con 0x7f62000ac090 2023-12-11T17:45:08.701 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:08.698+0000 7f621b2ac700 1 -- 172.21.15.59:0/691396887 >> [v2:172.21.15.59:6818/815374853,v1:172.21.15.59:6819/815374853] conn(0x7f62000ac090 msgr2=0x7f62000ae4d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:08.701 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:08.698+0000 7f621b2ac700 1 --2- 172.21.15.59:0/691396887 >> [v2:172.21.15.59:6818/815374853,v1:172.21.15.59:6819/815374853] conn(0x7f62000ac090 0x7f62000ae4d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:08.701 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:08.698+0000 7f621b2ac700 1 -- 172.21.15.59:0/691396887 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f62000a1510 msgr2=0x7f62000a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:08.701 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:08.698+0000 7f621b2ac700 1 --2- 172.21.15.59:0/691396887 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f62000a1510 0x7f62000a39d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6210006010 tx=0x7f6210003680 comp rx=0 tx=0).stop 2023-12-11T17:45:08.702 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:08.698+0000 7f621b2ac700 1 -- 172.21.15.59:0/691396887 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6214105470 msgr2=0x7f621419c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:08.702 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:08.698+0000 7f621b2ac700 1 --2- 172.21.15.59:0/691396887 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6214105470 0x7f621419c370 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f620c007ee0 tx=0x7f620c00d3b0 comp rx=0 tx=0).stop 2023-12-11T17:45:08.702 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:08.699+0000 7f6219849700 1 -- 172.21.15.59:0/691396887 reap_dead start 2023-12-11T17:45:08.702 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:08.699+0000 7f621b2ac700 1 -- 172.21.15.59:0/691396887 shutdown_connections 2023-12-11T17:45:08.702 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:08.699+0000 7f621b2ac700 1 -- 172.21.15.59:0/691396887 >> 172.21.15.59:0/691396887 conn(0x7f62140774a0 msgr2=0x7f6214069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:45:08.702 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:08.699+0000 7f621b2ac700 1 -- 172.21.15.59:0/691396887 shutdown_connections 2023-12-11T17:45:08.702 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:08.699+0000 7f621b2ac700 1 -- 172.21.15.59:0/691396887 wait complete. 2023-12-11T17:45:09.054 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:07.018+0000 7f75dd494700 1 -- 172.21.15.59:0/1680117229 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f75d8068f70 msgr2=0x7f75d80693f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:09.054 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:07.018+0000 7f75dd494700 1 --2- 172.21.15.59:0/1680117229 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f75d8068f70 0x7f75d80693f0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f75c00099c0 tx=0x7f75c0009cd0 comp rx=0 tx=0).stop 2023-12-11T17:45:09.054 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:07.018+0000 7f75dd494700 1 -- 172.21.15.59:0/1680117229 shutdown_connections 2023-12-11T17:45:09.054 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:07.018+0000 7f75dd494700 1 --2- 172.21.15.59:0/1680117229 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f75d8069930 0x7f75d810e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:09.054 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:07.018+0000 7f75dd494700 1 --2- 172.21.15.59:0/1680117229 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f75d8107620 0x7f75d8107a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:09.055 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:07.018+0000 7f75dd494700 1 --2- 172.21.15.59:0/1680117229 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f75d8068f70 0x7f75d80693f0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:09.055 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:07.018+0000 7f75dd494700 1 -- 172.21.15.59:0/1680117229 >> 172.21.15.59:0/1680117229 conn(0x7f75d8076570 msgr2=0x7f75d8076980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:45:09.055 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:07.018+0000 7f75dd494700 1 -- 172.21.15.59:0/1680117229 shutdown_connections 2023-12-11T17:45:09.055 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:07.018+0000 7f75dd494700 1 -- 172.21.15.59:0/1680117229 wait complete. 2023-12-11T17:45:09.055 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:07.019+0000 7f75dd494700 1 Processor -- start 2023-12-11T17:45:09.055 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:07.019+0000 7f75dd494700 1 -- start start 2023-12-11T17:45:09.055 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:07.019+0000 7f75dd494700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f75d8068f70 0x7f75d819d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:09.055 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:07.019+0000 7f75dd494700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f75d8069930 0x7f75d819dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:09.055 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:07.019+0000 7f75dd494700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f75d8107620 0x7f75d8197950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:09.055 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:07.019+0000 7f75dd494700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75d8197fb0 con 0x7f75d8107620 2023-12-11T17:45:09.055 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:07.019+0000 7f75dd494700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75d8198120 con 0x7f75d8069930 2023-12-11T17:45:09.055 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:07.019+0000 7f75dd494700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75d81982e0 con 0x7f75d8068f70 2023-12-11T17:45:09.055 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:07.020+0000 7f75d77fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f75d8107620 0x7f75d8197950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:09.055 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:07.020+0000 7f75d77fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f75d8107620 0x7f75d8197950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33530/0 (socket says 172.21.15.59:33530) 2023-12-11T17:45:09.055 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:07.020+0000 7f75d77fe700 1 -- 172.21.15.59:0/3506327079 learned_addr learned my addr 172.21.15.59:0/3506327079 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:45:09.056 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:07.020+0000 7f75d67fc700 1 --2- 172.21.15.59:0/3506327079 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f75d8069930 0x7f75d819dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:09.056 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:07.020+0000 7f75d6ffd700 1 --2- 172.21.15.59:0/3506327079 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f75d8068f70 0x7f75d819d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:09.056 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:07.020+0000 7f75d77fe700 1 -- 172.21.15.59:0/3506327079 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f75d8068f70 msgr2=0x7f75d819d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:09.056 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:07.020+0000 7f75d77fe700 1 --2- 172.21.15.59:0/3506327079 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f75d8068f70 0x7f75d819d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:09.056 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:07.020+0000 7f75d77fe700 1 -- 172.21.15.59:0/3506327079 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f75d8069930 msgr2=0x7f75d819dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:09.056 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:07.020+0000 7f75d77fe700 1 --2- 172.21.15.59:0/3506327079 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f75d8069930 0x7f75d819dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:09.056 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:07.020+0000 7f75d77fe700 1 -- 172.21.15.59:0/3506327079 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75c00095f0 con 0x7f75d8107620 2023-12-11T17:45:09.056 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:07.020+0000 7f75d67fc700 1 --2- 172.21.15.59:0/3506327079 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f75d8069930 0x7f75d819dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-11T17:45:09.056 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:07.020+0000 7f75d77fe700 1 --2- 172.21.15.59:0/3506327079 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f75d8107620 0x7f75d8197950 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f75d000b970 tx=0x7f75d000bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:09.056 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:07.020+0000 7f75cffff700 1 -- 172.21.15.59:0/3506327079 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d000c9b0 con 0x7f75d8107620 2023-12-11T17:45:09.056 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:07.020+0000 7f75cffff700 1 -- 172.21.15.59:0/3506327079 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f75d0014440 con 0x7f75d8107620 2023-12-11T17:45:09.056 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:07.020+0000 7f75cffff700 1 -- 172.21.15.59:0/3506327079 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d00075d0 con 0x7f75d8107620 2023-12-11T17:45:09.056 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:07.020+0000 7f75dd494700 1 -- 172.21.15.59:0/3506327079 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75d81985c0 con 0x7f75d8107620 2023-12-11T17:45:09.056 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:07.020+0000 7f75dd494700 1 -- 172.21.15.59:0/3506327079 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75d81a47b0 con 0x7f75d8107620 2023-12-11T17:45:09.057 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:07.021+0000 7f75dd494700 1 -- 172.21.15.59:0/3506327079 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f75d81118b0 con 0x7f75d8107620 2023-12-11T17:45:09.057 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:07.023+0000 7f75cffff700 1 -- 172.21.15.59:0/3506327079 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f75d00145b0 con 0x7f75d8107620 2023-12-11T17:45:09.057 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:07.023+0000 7f75cffff700 1 --2- 172.21.15.59:0/3506327079 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f75c40a1590 0x7f75c40a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:09.057 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:07.023+0000 7f75d6ffd700 1 --2- 172.21.15.59:0/3506327079 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f75c40a1590 0x7f75c40a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:09.057 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:07.023+0000 7f75cffff700 1 -- 172.21.15.59:0/3506327079 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f75d0006e40 con 0x7f75d8107620 2023-12-11T17:45:09.057 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:07.024+0000 7f75cffff700 1 --2- 172.21.15.59:0/3506327079 >> [v2:172.21.15.59:6810/219909995,v1:172.21.15.59:6811/219909995] conn(0x7f75c40ac110 0x7f75c40ae550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:09.057 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:07.024+0000 7f75cffff700 1 -- 172.21.15.59:0/3506327079 --> [v2:172.21.15.59:6810/219909995,v1:172.21.15.59:6811/219909995] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f75c40aec20 con 0x7f75c40ac110 2023-12-11T17:45:09.057 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:07.024+0000 7f75cffff700 1 -- 172.21.15.59:0/3506327079 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7f75d00ccd40 con 0x7f75d8107620 2023-12-11T17:45:09.057 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:07.024+0000 7f75d67fc700 1 --2- 172.21.15.59:0/3506327079 >> [v2:172.21.15.59:6810/219909995,v1:172.21.15.59:6811/219909995] conn(0x7f75c40ac110 0x7f75c40ae550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:09.058 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:07.024+0000 7f75d6ffd700 1 --2- 172.21.15.59:0/3506327079 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f75c40a1590 0x7f75c40a3a50 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f75c8005fd0 tx=0x7f75c800a560 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:09.058 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:07.024+0000 7f75d67fc700 1 --2- 172.21.15.59:0/3506327079 >> [v2:172.21.15.59:6810/219909995,v1:172.21.15.59:6811/219909995] conn(0x7f75c40ac110 0x7f75c40ae550 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:09.058 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:07.025+0000 7f75cffff700 1 -- 172.21.15.59:0/3506327079 <== osd.3 v2:172.21.15.59:6810/219909995 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f75c40aec20 con 0x7f75c40ac110 2023-12-11T17:45:09.058 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:07.044+0000 7f75dd494700 1 -- 172.21.15.59:0/3506327079 --> [v2:172.21.15.59:6810/219909995,v1:172.21.15.59:6811/219909995] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f75d8061cd0 con 0x7f75c40ac110 2023-12-11T17:45:09.058 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:07.045+0000 7f75cffff700 1 -- 172.21.15.59:0/3506327079 <== osd.3 v2:172.21.15.59:6810/219909995 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f75d8061cd0 con 0x7f75c40ac110 2023-12-11T17:45:09.058 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:07.045+0000 7f75dd494700 1 -- 172.21.15.59:0/3506327079 >> [v2:172.21.15.59:6810/219909995,v1:172.21.15.59:6811/219909995] conn(0x7f75c40ac110 msgr2=0x7f75c40ae550 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:09.058 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:07.045+0000 7f75dd494700 1 --2- 172.21.15.59:0/3506327079 >> [v2:172.21.15.59:6810/219909995,v1:172.21.15.59:6811/219909995] conn(0x7f75c40ac110 0x7f75c40ae550 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:09.058 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:07.045+0000 7f75dd494700 1 -- 172.21.15.59:0/3506327079 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f75c40a1590 msgr2=0x7f75c40a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:09.058 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:07.045+0000 7f75dd494700 1 --2- 172.21.15.59:0/3506327079 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f75c40a1590 0x7f75c40a3a50 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f75c8005fd0 tx=0x7f75c800a560 comp rx=0 tx=0).stop 2023-12-11T17:45:09.058 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:07.045+0000 7f75dd494700 1 -- 172.21.15.59:0/3506327079 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f75d8107620 msgr2=0x7f75d8197950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:09.058 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:07.045+0000 7f75dd494700 1 --2- 172.21.15.59:0/3506327079 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f75d8107620 0x7f75d8197950 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f75d000b970 tx=0x7f75d000bd30 comp rx=0 tx=0).stop 2023-12-11T17:45:09.058 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:07.045+0000 7f75d77fe700 1 -- 172.21.15.59:0/3506327079 reap_dead start 2023-12-11T17:45:09.058 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:07.045+0000 7f75dd494700 1 -- 172.21.15.59:0/3506327079 shutdown_connections 2023-12-11T17:45:09.058 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:07.045+0000 7f75dd494700 1 -- 172.21.15.59:0/3506327079 >> 172.21.15.59:0/3506327079 conn(0x7f75d8076570 msgr2=0x7f75d80ff5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:45:09.059 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:07.046+0000 7f75dd494700 1 -- 172.21.15.59:0/3506327079 shutdown_connections 2023-12-11T17:45:09.059 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:07.046+0000 7f75dd494700 1 -- 172.21.15.59:0/3506327079 wait complete. 2023-12-11T17:45:09.060 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:09.056+0000 7fe8bdf86700 1 -- 172.21.15.59:0/3087900902 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe8b8106560 msgr2=0x7fe8b8106940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:09.061 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:09.056+0000 7fe8bdf86700 1 --2- 172.21.15.59:0/3087900902 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe8b8106560 0x7fe8b8106940 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fe8a80099c0 tx=0x7fe8a8009cd0 comp rx=0 tx=0).stop 2023-12-11T17:45:09.061 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:09.056+0000 7fe8bdf86700 1 -- 172.21.15.59:0/3087900902 shutdown_connections 2023-12-11T17:45:09.061 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:09.056+0000 7fe8bdf86700 1 --2- 172.21.15.59:0/3087900902 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe8b8100ef0 0x7fe8b810d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:09.061 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:09.056+0000 7fe8bdf86700 1 --2- 172.21.15.59:0/3087900902 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe8b8106560 0x7fe8b8106940 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:09.061 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:09.056+0000 7fe8bdf86700 1 --2- 172.21.15.59:0/3087900902 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8b8100530 0x7fe8b81009b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:09.061 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:09.056+0000 7fe8bdf86700 1 -- 172.21.15.59:0/3087900902 >> 172.21.15.59:0/3087900902 conn(0x7fe8b8076570 msgr2=0x7fe8b8076980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:45:09.061 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:09.056+0000 7fe8bdf86700 1 -- 172.21.15.59:0/3087900902 shutdown_connections 2023-12-11T17:45:09.061 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:09.056+0000 7fe8bdf86700 1 -- 172.21.15.59:0/3087900902 wait complete. 2023-12-11T17:45:09.061 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:09.056+0000 7fe8bdf86700 1 Processor -- start 2023-12-11T17:45:09.061 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:09.057+0000 7fe8bdf86700 1 -- start start 2023-12-11T17:45:09.061 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:09.057+0000 7fe8bdf86700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe8b8100530 0x7fe8b81987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:09.061 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:09.057+0000 7fe8bdf86700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe8b8100ef0 0x7fe8b8198cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:09.061 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:09.057+0000 7fe8bdf86700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8b8106560 0x7fe8b81a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:09.061 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:09.057+0000 7fe8bdf86700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b81995b0 con 0x7fe8b8100530 2023-12-11T17:45:09.061 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:09.057+0000 7fe8bdf86700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b81a3d70 con 0x7fe8b8100ef0 2023-12-11T17:45:09.062 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:09.057+0000 7fe8bdf86700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b81a3eb0 con 0x7fe8b8106560 2023-12-11T17:45:09.062 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:09.057+0000 7fe8b77fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe8b8100530 0x7fe8b81987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:09.062 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:09.057+0000 7fe8b77fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe8b8100530 0x7fe8b81987b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33572/0 (socket says 172.21.15.59:33572) 2023-12-11T17:45:09.062 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:09.057+0000 7fe8b77fe700 1 -- 172.21.15.59:0/4229634245 learned_addr learned my addr 172.21.15.59:0/4229634245 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:45:09.062 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:09.057+0000 7fe8b6ffd700 1 --2- 172.21.15.59:0/4229634245 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe8b8100ef0 0x7fe8b8198cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:09.062 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:09.057+0000 7fe8b7fff700 1 --2- 172.21.15.59:0/4229634245 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8b8106560 0x7fe8b81a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:09.062 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:09.057+0000 7fe8b77fe700 1 -- 172.21.15.59:0/4229634245 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8b8106560 msgr2=0x7fe8b81a3830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:09.062 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:09.057+0000 7fe8b77fe700 1 --2- 172.21.15.59:0/4229634245 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8b8106560 0x7fe8b81a3830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:09.062 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:09.057+0000 7fe8b77fe700 1 -- 172.21.15.59:0/4229634245 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe8b8100ef0 msgr2=0x7fe8b8198cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:09.062 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:09.057+0000 7fe8b77fe700 1 --2- 172.21.15.59:0/4229634245 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe8b8100ef0 0x7fe8b8198cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:09.062 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:09.057+0000 7fe8b77fe700 1 -- 172.21.15.59:0/4229634245 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8a80095f0 con 0x7fe8b8100530 2023-12-11T17:45:09.062 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:09.057+0000 7fe8b7fff700 1 --2- 172.21.15.59:0/4229634245 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8b8106560 0x7fe8b81a3830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-11T17:45:09.062 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:09.057+0000 7fe8b6ffd700 1 --2- 172.21.15.59:0/4229634245 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe8b8100ef0 0x7fe8b8198cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-11T17:45:09.063 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:09.058+0000 7fe8b77fe700 1 --2- 172.21.15.59:0/4229634245 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe8b8100530 0x7fe8b81987b0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fe8a80098e0 tx=0x7fe8a80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:09.063 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:09.058+0000 7fe8b4ff9700 1 -- 172.21.15.59:0/4229634245 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a801d070 con 0x7fe8b8100530 2023-12-11T17:45:09.063 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:09.058+0000 7fe8bdf86700 1 -- 172.21.15.59:0/4229634245 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8b81a4130 con 0x7fe8b8100530 2023-12-11T17:45:09.063 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:09.058+0000 7fe8b4ff9700 1 -- 172.21.15.59:0/4229634245 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe8a800fbb0 con 0x7fe8b8100530 2023-12-11T17:45:09.063 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:09.058+0000 7fe8b4ff9700 1 -- 172.21.15.59:0/4229634245 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a8022d40 con 0x7fe8b8100530 2023-12-11T17:45:09.063 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:09.058+0000 7fe8bdf86700 1 -- 172.21.15.59:0/4229634245 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8b81a45a0 con 0x7fe8b8100530 2023-12-11T17:45:09.064 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:09.059+0000 7fe8bdf86700 1 -- 172.21.15.59:0/4229634245 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe8b8110690 con 0x7fe8b8100530 2023-12-11T17:45:09.064 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:09.060+0000 7fe8b4ff9700 1 -- 172.21.15.59:0/4229634245 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7fe8a800fd20 con 0x7fe8b8100530 2023-12-11T17:45:09.064 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:09.060+0000 7fe8b4ff9700 1 --2- 172.21.15.59:0/4229634245 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fe8a40a1590 0x7fe8a40a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:09.064 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:09.061+0000 7fe8b4ff9700 1 -- 172.21.15.59:0/4229634245 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fe8a80ced70 con 0x7fe8b8100530 2023-12-11T17:45:09.064 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:09.061+0000 7fe8b6ffd700 1 --2- 172.21.15.59:0/4229634245 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fe8a40a1590 0x7fe8a40a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:09.064 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:09.061+0000 7fe8b4ff9700 1 --2- 172.21.15.59:0/4229634245 >> [v2:172.21.15.115:6824/2199610269,v1:172.21.15.115:6825/2199610269] conn(0x7fe8a40ac110 0x7fe8a40ae550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:09.064 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:09.061+0000 7fe8b4ff9700 1 -- 172.21.15.59:0/4229634245 --> [v2:172.21.15.115:6824/2199610269,v1:172.21.15.115:6825/2199610269] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe8a40aec20 con 0x7fe8a40ac110 2023-12-11T17:45:09.065 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:09.061+0000 7fe8b4ff9700 1 -- 172.21.15.59:0/4229634245 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe8a8022ea0 con 0x7fe8b8100530 2023-12-11T17:45:09.065 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:09.061+0000 7fe8b6ffd700 1 --2- 172.21.15.59:0/4229634245 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fe8a40a1590 0x7fe8a40a3a50 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe8a0005fd0 tx=0x7fe8a0005e20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:09.065 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:09.061+0000 7fe8b7fff700 1 --2- 172.21.15.59:0/4229634245 >> [v2:172.21.15.115:6824/2199610269,v1:172.21.15.115:6825/2199610269] conn(0x7fe8a40ac110 0x7fe8a40ae550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:09.065 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:09.062+0000 7fe8b7fff700 1 --2- 172.21.15.59:0/4229634245 >> [v2:172.21.15.115:6824/2199610269,v1:172.21.15.115:6825/2199610269] conn(0x7fe8a40ac110 0x7fe8a40ae550 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:09.065 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:09.062+0000 7fe8b4ff9700 1 -- 172.21.15.59:0/4229634245 <== osd.10 v2:172.21.15.115:6824/2199610269 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fe8a40aec20 con 0x7fe8a40ac110 2023-12-11T17:45:09.082 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:09.079+0000 7fe8bdf86700 1 -- 172.21.15.59:0/4229634245 --> [v2:172.21.15.115:6824/2199610269,v1:172.21.15.115:6825/2199610269] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe8b8101bd0 con 0x7fe8a40ac110 2023-12-11T17:45:09.082 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:09.080+0000 7fe8b4ff9700 1 -- 172.21.15.59:0/4229634245 <== osd.10 v2:172.21.15.115:6824/2199610269 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fe8b8101bd0 con 0x7fe8a40ac110 2023-12-11T17:45:09.083 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:09.080+0000 7fe8bdf86700 1 -- 172.21.15.59:0/4229634245 >> [v2:172.21.15.115:6824/2199610269,v1:172.21.15.115:6825/2199610269] conn(0x7fe8a40ac110 msgr2=0x7fe8a40ae550 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:09.083 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:09.080+0000 7fe8bdf86700 1 --2- 172.21.15.59:0/4229634245 >> [v2:172.21.15.115:6824/2199610269,v1:172.21.15.115:6825/2199610269] conn(0x7fe8a40ac110 0x7fe8a40ae550 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:09.083 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:09.080+0000 7fe8bdf86700 1 -- 172.21.15.59:0/4229634245 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fe8a40a1590 msgr2=0x7fe8a40a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:09.083 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:09.080+0000 7fe8bdf86700 1 --2- 172.21.15.59:0/4229634245 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fe8a40a1590 0x7fe8a40a3a50 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe8a0005fd0 tx=0x7fe8a0005e20 comp rx=0 tx=0).stop 2023-12-11T17:45:09.083 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:09.080+0000 7fe8bdf86700 1 -- 172.21.15.59:0/4229634245 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe8b8100530 msgr2=0x7fe8b81987b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:09.083 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:09.080+0000 7fe8bdf86700 1 --2- 172.21.15.59:0/4229634245 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe8b8100530 0x7fe8b81987b0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fe8a80098e0 tx=0x7fe8a80057f0 comp rx=0 tx=0).stop 2023-12-11T17:45:09.083 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:09.080+0000 7fe8b7fff700 1 -- 172.21.15.59:0/4229634245 reap_dead start 2023-12-11T17:45:09.084 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:09.080+0000 7fe8bdf86700 1 -- 172.21.15.59:0/4229634245 shutdown_connections 2023-12-11T17:45:09.084 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:09.080+0000 7fe8bdf86700 1 -- 172.21.15.59:0/4229634245 >> 172.21.15.59:0/4229634245 conn(0x7fe8b8076570 msgr2=0x7fe8b80ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:45:09.084 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:09.081+0000 7fe8bdf86700 1 -- 172.21.15.59:0/4229634245 shutdown_connections 2023-12-11T17:45:09.084 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:09.081+0000 7fe8bdf86700 1 -- 172.21.15.59:0/4229634245 wait complete. 2023-12-11T17:45:10.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:45:09 smithi181 ceph-mon[103627]: pgmap v283: 129 pgs: 129 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.9 KiB/s wr, 7 op/s 2023-12-11T17:45:10.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:45:09 smithi059 ceph-mon[98847]: pgmap v283: 129 pgs: 129 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.9 KiB/s wr, 7 op/s 2023-12-11T17:45:10.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:45:09 smithi115 ceph-mon[102748]: pgmap v283: 129 pgs: 129 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.9 KiB/s wr, 7 op/s 2023-12-11T17:45:10.861 INFO:teuthology.orchestra.run.smithi059.stdout:279172874270 2023-12-11T17:45:10.861 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell --fsid d0a07eaa-984b-11ee-95a3-87774f69a715 -- ceph osd last-stat-seq osd.9 2023-12-11T17:45:10.972 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:10.969+0000 7f453f351700 1 -- 172.21.15.59:0/204454493 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4538101620 msgr2=0x7f4538101aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:10.972 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:10.969+0000 7f453f351700 1 --2- 172.21.15.59:0/204454493 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4538101620 0x7f4538101aa0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f4528009990 tx=0x7f4528009ca0 comp rx=0 tx=0).stop 2023-12-11T17:45:10.972 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:10.969+0000 7f453f351700 1 -- 172.21.15.59:0/204454493 shutdown_connections 2023-12-11T17:45:10.972 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:10.969+0000 7f453f351700 1 --2- 172.21.15.59:0/204454493 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4538101fe0 0x7f453810e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:10.973 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:10.969+0000 7f453f351700 1 --2- 172.21.15.59:0/204454493 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45381065a0 0x7f4538106980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:10.973 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:10.969+0000 7f453f351700 1 --2- 172.21.15.59:0/204454493 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4538101620 0x7f4538101aa0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:10.973 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:10.969+0000 7f453f351700 1 -- 172.21.15.59:0/204454493 >> 172.21.15.59:0/204454493 conn(0x7f4538078730 msgr2=0x7f4538078b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:45:10.973 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:10.970+0000 7f453f351700 1 -- 172.21.15.59:0/204454493 shutdown_connections 2023-12-11T17:45:10.973 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:10.970+0000 7f453f351700 1 -- 172.21.15.59:0/204454493 wait complete. 2023-12-11T17:45:10.975 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:10.970+0000 7f453f351700 1 Processor -- start 2023-12-11T17:45:10.975 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:10.970+0000 7f453f351700 1 -- start start 2023-12-11T17:45:10.975 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:10.970+0000 7f453f351700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4538101620 0x7f45380738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:10.975 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:10.970+0000 7f453f351700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4538101fe0 0x7f4538073e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:10.975 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:10.970+0000 7f453f351700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f45381065a0 0x7f4538074350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:10.975 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:10.970+0000 7f453f351700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45381a5cf0 con 0x7f4538101fe0 2023-12-11T17:45:10.975 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:10.970+0000 7f453f351700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45381a5e60 con 0x7f45381065a0 2023-12-11T17:45:10.975 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:10.970+0000 7f453f351700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45381a6020 con 0x7f4538101620 2023-12-11T17:45:10.975 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:10.971+0000 7f453c8ec700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4538101fe0 0x7f4538073e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:10.975 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:10.971+0000 7f453d0ed700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4538101620 0x7f45380738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:10.975 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:10.971+0000 7f453c8ec700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4538101fe0 0x7f4538073e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33592/0 (socket says 172.21.15.59:33592) 2023-12-11T17:45:10.975 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:10.971+0000 7f453c8ec700 1 -- 172.21.15.59:0/3272454207 learned_addr learned my addr 172.21.15.59:0/3272454207 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:45:10.975 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:10.971+0000 7f453d8ee700 1 --2- 172.21.15.59:0/3272454207 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f45381065a0 0x7f4538074350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:10.976 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:10.971+0000 7f453d0ed700 1 -- 172.21.15.59:0/3272454207 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f45381065a0 msgr2=0x7f4538074350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:10.976 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:10.971+0000 7f453d0ed700 1 --2- 172.21.15.59:0/3272454207 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f45381065a0 0x7f4538074350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:10.976 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:10.971+0000 7f453d0ed700 1 -- 172.21.15.59:0/3272454207 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4538101fe0 msgr2=0x7f4538073e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:10.976 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:10.971+0000 7f453d0ed700 1 --2- 172.21.15.59:0/3272454207 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4538101fe0 0x7f4538073e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:10.976 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:10.971+0000 7f453d0ed700 1 -- 172.21.15.59:0/3272454207 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45280095f0 con 0x7f4538101620 2023-12-11T17:45:10.976 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:10.971+0000 7f453d0ed700 1 --2- 172.21.15.59:0/3272454207 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4538101620 0x7f45380738d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f453400e9c0 tx=0x7f453400ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:10.976 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:10.971+0000 7f452e7fc700 1 -- 172.21.15.59:0/3272454207 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4534004220 con 0x7f4538101620 2023-12-11T17:45:10.976 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:10.971+0000 7f452e7fc700 1 -- 172.21.15.59:0/3272454207 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4534004380 con 0x7f4538101620 2023-12-11T17:45:10.976 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:10.971+0000 7f453f351700 1 -- 172.21.15.59:0/3272454207 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45381a6300 con 0x7f4538101620 2023-12-11T17:45:10.976 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:10.971+0000 7f452e7fc700 1 -- 172.21.15.59:0/3272454207 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4534003be0 con 0x7f4538101620 2023-12-11T17:45:10.976 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:10.972+0000 7f453f351700 1 -- 172.21.15.59:0/3272454207 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45381a6800 con 0x7f4538101620 2023-12-11T17:45:10.976 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:10.972+0000 7f453f351700 1 -- 172.21.15.59:0/3272454207 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4538111780 con 0x7f4538101620 2023-12-11T17:45:10.977 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:10.974+0000 7f452e7fc700 1 -- 172.21.15.59:0/3272454207 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f4534003d40 con 0x7f4538101620 2023-12-11T17:45:10.977 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:10.975+0000 7f452e7fc700 1 --2- 172.21.15.59:0/3272454207 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f45240a1510 0x7f45240a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:10.977 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:10.975+0000 7f453c8ec700 1 --2- 172.21.15.59:0/3272454207 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f45240a1510 0x7f45240a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:10.977 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:10.975+0000 7f452e7fc700 1 -- 172.21.15.59:0/3272454207 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f4534014070 con 0x7f4538101620 2023-12-11T17:45:10.977 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:10.975+0000 7f452e7fc700 1 --2- 172.21.15.59:0/3272454207 >> [v2:172.21.15.115:6808/1021473826,v1:172.21.15.115:6809/1021473826] conn(0x7f45240ac090 0x7f45240ae4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:10.977 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:10.975+0000 7f452e7fc700 1 -- 172.21.15.59:0/3272454207 --> [v2:172.21.15.115:6808/1021473826,v1:172.21.15.115:6809/1021473826] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f45240aeba0 con 0x7f45240ac090 2023-12-11T17:45:10.978 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:10.975+0000 7f452e7fc700 1 -- 172.21.15.59:0/3272454207 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7f45340cd7a0 con 0x7f4538101620 2023-12-11T17:45:10.978 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:10.975+0000 7f453c8ec700 1 --2- 172.21.15.59:0/3272454207 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f45240a1510 0x7f45240a39d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f452800b6f0 tx=0x7f4528005bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:10.978 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:10.975+0000 7f453d8ee700 1 --2- 172.21.15.59:0/3272454207 >> [v2:172.21.15.115:6808/1021473826,v1:172.21.15.115:6809/1021473826] conn(0x7f45240ac090 0x7f45240ae4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:10.978 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:10.976+0000 7f453d8ee700 1 --2- 172.21.15.59:0/3272454207 >> [v2:172.21.15.115:6808/1021473826,v1:172.21.15.115:6809/1021473826] conn(0x7f45240ac090 0x7f45240ae4d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:10.979 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:10.976+0000 7f452e7fc700 1 -- 172.21.15.59:0/3272454207 <== osd.4 v2:172.21.15.115:6808/1021473826 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f45240aeba0 con 0x7f45240ac090 2023-12-11T17:45:10.995 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:10.993+0000 7f453f351700 1 -- 172.21.15.59:0/3272454207 --> [v2:172.21.15.115:6808/1021473826,v1:172.21.15.115:6809/1021473826] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f4538102ca0 con 0x7f45240ac090 2023-12-11T17:45:10.996 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:10.994+0000 7f452e7fc700 1 -- 172.21.15.59:0/3272454207 <== osd.4 v2:172.21.15.115:6808/1021473826 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f4538102ca0 con 0x7f45240ac090 2023-12-11T17:45:10.997 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:10.994+0000 7f453f351700 1 -- 172.21.15.59:0/3272454207 >> [v2:172.21.15.115:6808/1021473826,v1:172.21.15.115:6809/1021473826] conn(0x7f45240ac090 msgr2=0x7f45240ae4d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:10.997 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:10.994+0000 7f453f351700 1 --2- 172.21.15.59:0/3272454207 >> [v2:172.21.15.115:6808/1021473826,v1:172.21.15.115:6809/1021473826] conn(0x7f45240ac090 0x7f45240ae4d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:10.997 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:10.994+0000 7f453f351700 1 -- 172.21.15.59:0/3272454207 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f45240a1510 msgr2=0x7f45240a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:10.997 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:10.994+0000 7f453f351700 1 --2- 172.21.15.59:0/3272454207 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f45240a1510 0x7f45240a39d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f452800b6f0 tx=0x7f4528005bc0 comp rx=0 tx=0).stop 2023-12-11T17:45:10.997 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:10.994+0000 7f453f351700 1 -- 172.21.15.59:0/3272454207 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4538101620 msgr2=0x7f45380738d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:10.997 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:10.994+0000 7f453f351700 1 --2- 172.21.15.59:0/3272454207 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4538101620 0x7f45380738d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f453400e9c0 tx=0x7f453400ecd0 comp rx=0 tx=0).stop 2023-12-11T17:45:10.997 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:10.994+0000 7f453d8ee700 1 -- 172.21.15.59:0/3272454207 reap_dead start 2023-12-11T17:45:10.997 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:10.994+0000 7f453f351700 1 -- 172.21.15.59:0/3272454207 shutdown_connections 2023-12-11T17:45:10.997 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:10.994+0000 7f453f351700 1 -- 172.21.15.59:0/3272454207 >> 172.21.15.59:0/3272454207 conn(0x7f4538078730 msgr2=0x7f45380ff6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:45:10.997 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:10.994+0000 7f453f351700 1 -- 172.21.15.59:0/3272454207 shutdown_connections 2023-12-11T17:45:10.997 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:10.994+0000 7f453f351700 1 -- 172.21.15.59:0/3272454207 wait complete. 2023-12-11T17:45:11.151 INFO:teuthology.orchestra.run.smithi059.stdout:107374182454 2023-12-11T17:45:11.152 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell --fsid d0a07eaa-984b-11ee-95a3-87774f69a715 -- ceph osd last-stat-seq osd.3 2023-12-11T17:45:11.465 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:11.462+0000 7ff402f25700 1 -- 172.21.15.59:0/4039178254 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff3fc107740 msgr2=0x7ff3fc107b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:11.465 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:11.462+0000 7ff402f25700 1 --2- 172.21.15.59:0/4039178254 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff3fc107740 0x7ff3fc107b20 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7ff3f0009990 tx=0x7ff3f0009ca0 comp rx=0 tx=0).stop 2023-12-11T17:45:11.465 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:11.462+0000 7ff402f25700 1 -- 172.21.15.59:0/4039178254 shutdown_connections 2023-12-11T17:45:11.465 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:11.462+0000 7ff402f25700 1 --2- 172.21.15.59:0/4039178254 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3fc101fe0 0x7ff3fc10e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:11.465 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:11.462+0000 7ff402f25700 1 --2- 172.21.15.59:0/4039178254 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff3fc107740 0x7ff3fc107b20 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:11.465 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:11.462+0000 7ff402f25700 1 --2- 172.21.15.59:0/4039178254 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff3fc101620 0x7ff3fc101aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:11.465 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:11.462+0000 7ff402f25700 1 -- 172.21.15.59:0/4039178254 >> 172.21.15.59:0/4039178254 conn(0x7ff3fc078730 msgr2=0x7ff3fc078b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:45:11.465 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:11.462+0000 7ff402f25700 1 -- 172.21.15.59:0/4039178254 shutdown_connections 2023-12-11T17:45:11.465 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:11.462+0000 7ff402f25700 1 -- 172.21.15.59:0/4039178254 wait complete. 2023-12-11T17:45:11.467 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:11.463+0000 7ff402f25700 1 Processor -- start 2023-12-11T17:45:11.467 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:11.463+0000 7ff402f25700 1 -- start start 2023-12-11T17:45:11.467 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:11.463+0000 7ff402f25700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff3fc101620 0x7ff3fc19d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:11.467 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:11.463+0000 7ff402f25700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff3fc101fe0 0x7ff3fc19dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:11.467 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:11.463+0000 7ff402f25700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3fc107740 0x7ff3fc197810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:11.467 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:11.463+0000 7ff402f25700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3fc197e80 con 0x7ff3fc101620 2023-12-11T17:45:11.467 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:11.463+0000 7ff402f25700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3fc197ff0 con 0x7ff3fc101fe0 2023-12-11T17:45:11.467 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:11.463+0000 7ff402f25700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3fc1981b0 con 0x7ff3fc107740 2023-12-11T17:45:11.467 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:11.463+0000 7ff400cc1700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff3fc101620 0x7ff3fc19d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:11.467 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:11.464+0000 7ff400cc1700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff3fc101620 0x7ff3fc19d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33604/0 (socket says 172.21.15.59:33604) 2023-12-11T17:45:11.467 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:11.464+0000 7ff400cc1700 1 -- 172.21.15.59:0/3661853074 learned_addr learned my addr 172.21.15.59:0/3661853074 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:45:11.468 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:11.464+0000 7ff3fbfff700 1 --2- 172.21.15.59:0/3661853074 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff3fc101fe0 0x7ff3fc19dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:11.468 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:11.464+0000 7ff400cc1700 1 -- 172.21.15.59:0/3661853074 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3fc107740 msgr2=0x7ff3fc197810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:11.468 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:11.464+0000 7ff400cc1700 1 --2- 172.21.15.59:0/3661853074 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3fc107740 0x7ff3fc197810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:11.468 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:11.464+0000 7ff400cc1700 1 -- 172.21.15.59:0/3661853074 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff3fc101fe0 msgr2=0x7ff3fc19dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:11.468 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:11.464+0000 7ff400cc1700 1 --2- 172.21.15.59:0/3661853074 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff3fc101fe0 0x7ff3fc19dc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:11.468 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:11.464+0000 7ff400cc1700 1 -- 172.21.15.59:0/3661853074 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3f00095f0 con 0x7ff3fc101620 2023-12-11T17:45:11.468 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:11.464+0000 7ff400cc1700 1 --2- 172.21.15.59:0/3661853074 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff3fc101620 0x7ff3fc19d6d0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ff3f0009fd0 tx=0x7ff3f000f720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:11.468 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:11.464+0000 7ff3f9ffb700 1 -- 172.21.15.59:0/3661853074 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3f001d070 con 0x7ff3fc101620 2023-12-11T17:45:11.468 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:11.464+0000 7ff3f9ffb700 1 -- 172.21.15.59:0/3661853074 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff3f0005510 con 0x7ff3fc101620 2023-12-11T17:45:11.468 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:11.464+0000 7ff402f25700 1 -- 172.21.15.59:0/3661853074 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3fc198430 con 0x7ff3fc101620 2023-12-11T17:45:11.468 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:11.464+0000 7ff3f9ffb700 1 -- 172.21.15.59:0/3661853074 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3f0004bd0 con 0x7ff3fc101620 2023-12-11T17:45:11.468 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:11.464+0000 7ff402f25700 1 -- 172.21.15.59:0/3661853074 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3fc1a46c0 con 0x7ff3fc101620 2023-12-11T17:45:11.469 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:11.466+0000 7ff402f25700 1 -- 172.21.15.59:0/3661853074 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff3fc111890 con 0x7ff3fc101620 2023-12-11T17:45:11.469 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:11.466+0000 7ff3f9ffb700 1 -- 172.21.15.59:0/3661853074 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7ff3f0004d30 con 0x7ff3fc101620 2023-12-11T17:45:11.469 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:11.467+0000 7ff3f9ffb700 1 --2- 172.21.15.59:0/3661853074 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7ff3ec0a1510 0x7ff3ec0a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:11.470 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:11.467+0000 7ff3fbfff700 1 --2- 172.21.15.59:0/3661853074 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7ff3ec0a1510 0x7ff3ec0a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:11.470 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:11.467+0000 7ff3f9ffb700 1 -- 172.21.15.59:0/3661853074 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7ff3f00cdeb0 con 0x7ff3fc101620 2023-12-11T17:45:11.470 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:11.467+0000 7ff3f9ffb700 1 --2- 172.21.15.59:0/3661853074 >> [v2:172.21.15.181:6824/226506989,v1:172.21.15.181:6825/226506989] conn(0x7ff3ec0ac090 0x7ff3ec0ae4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:11.470 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:11.467+0000 7ff3f9ffb700 1 -- 172.21.15.59:0/3661853074 --> [v2:172.21.15.181:6824/226506989,v1:172.21.15.181:6825/226506989] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff3ec0aeba0 con 0x7ff3ec0ac090 2023-12-11T17:45:11.470 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:11.467+0000 7ff3f9ffb700 1 -- 172.21.15.59:0/3661853074 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff3f00d0530 con 0x7ff3fc101620 2023-12-11T17:45:11.470 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:11.468+0000 7ff3fbfff700 1 --2- 172.21.15.59:0/3661853074 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7ff3ec0a1510 0x7ff3ec0a39d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff3e8005950 tx=0x7ff3e80058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:11.471 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:11.468+0000 7ff4014c2700 1 --2- 172.21.15.59:0/3661853074 >> [v2:172.21.15.181:6824/226506989,v1:172.21.15.181:6825/226506989] conn(0x7ff3ec0ac090 0x7ff3ec0ae4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:11.471 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:11.468+0000 7ff4014c2700 1 --2- 172.21.15.59:0/3661853074 >> [v2:172.21.15.181:6824/226506989,v1:172.21.15.181:6825/226506989] conn(0x7ff3ec0ac090 0x7ff3ec0ae4d0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:11.471 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:11.469+0000 7ff3f9ffb700 1 -- 172.21.15.59:0/3661853074 <== osd.11 v2:172.21.15.181:6824/226506989 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7ff3ec0aeba0 con 0x7ff3ec0ac090 2023-12-11T17:45:11.500 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:11.498+0000 7ff402f25700 1 -- 172.21.15.59:0/3661853074 --> [v2:172.21.15.181:6824/226506989,v1:172.21.15.181:6825/226506989] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff3fc061c50 con 0x7ff3ec0ac090 2023-12-11T17:45:11.501 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:11.498+0000 7ff3f9ffb700 1 -- 172.21.15.59:0/3661853074 <== osd.11 v2:172.21.15.181:6824/226506989 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7ff3fc061c50 con 0x7ff3ec0ac090 2023-12-11T17:45:11.501 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:11.499+0000 7ff402f25700 1 -- 172.21.15.59:0/3661853074 >> [v2:172.21.15.181:6824/226506989,v1:172.21.15.181:6825/226506989] conn(0x7ff3ec0ac090 msgr2=0x7ff3ec0ae4d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:11.501 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:11.499+0000 7ff402f25700 1 --2- 172.21.15.59:0/3661853074 >> [v2:172.21.15.181:6824/226506989,v1:172.21.15.181:6825/226506989] conn(0x7ff3ec0ac090 0x7ff3ec0ae4d0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:11.502 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:11.499+0000 7ff402f25700 1 -- 172.21.15.59:0/3661853074 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7ff3ec0a1510 msgr2=0x7ff3ec0a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:11.502 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:11.499+0000 7ff402f25700 1 --2- 172.21.15.59:0/3661853074 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7ff3ec0a1510 0x7ff3ec0a39d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff3e8005950 tx=0x7ff3e80058e0 comp rx=0 tx=0).stop 2023-12-11T17:45:11.502 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:11.499+0000 7ff402f25700 1 -- 172.21.15.59:0/3661853074 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff3fc101620 msgr2=0x7ff3fc19d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:11.502 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:11.499+0000 7ff402f25700 1 --2- 172.21.15.59:0/3661853074 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff3fc101620 0x7ff3fc19d6d0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ff3f0009fd0 tx=0x7ff3f000f720 comp rx=0 tx=0).stop 2023-12-11T17:45:11.502 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:11.499+0000 7ff4014c2700 1 -- 172.21.15.59:0/3661853074 reap_dead start 2023-12-11T17:45:11.502 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:11.499+0000 7ff402f25700 1 -- 172.21.15.59:0/3661853074 shutdown_connections 2023-12-11T17:45:11.502 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:11.499+0000 7ff402f25700 1 -- 172.21.15.59:0/3661853074 >> 172.21.15.59:0/3661853074 conn(0x7ff3fc078730 msgr2=0x7ff3fc100360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:45:11.502 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:11.499+0000 7ff402f25700 1 -- 172.21.15.59:0/3661853074 shutdown_connections 2023-12-11T17:45:11.502 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:11.499+0000 7ff402f25700 1 -- 172.21.15.59:0/3661853074 wait complete. 2023-12-11T17:45:11.512 INFO:teuthology.orchestra.run.smithi059.stdout:300647710744 2023-12-11T17:45:11.512 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell --fsid d0a07eaa-984b-11ee-95a3-87774f69a715 -- ceph osd last-stat-seq osd.10 2023-12-11T17:45:11.992 INFO:teuthology.orchestra.run.smithi059.stdout:137438953524 2023-12-11T17:45:11.992 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell --fsid d0a07eaa-984b-11ee-95a3-87774f69a715 -- ceph osd last-stat-seq osd.4 2023-12-11T17:45:12.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:45:11 smithi181 ceph-mon[103627]: pgmap v284: 129 pgs: 129 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 341 B/s wr, 6 op/s 2023-12-11T17:45:12.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:45:11 smithi059 ceph-mon[98847]: pgmap v284: 129 pgs: 129 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 341 B/s wr, 6 op/s 2023-12-11T17:45:12.246 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.244+0000 7fd733c0b700 1 -- 172.21.15.59:0/441773747 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd72c103fd0 msgr2=0x7fd72c104480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:12.247 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.244+0000 7fd733c0b700 1 --2- 172.21.15.59:0/441773747 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd72c103fd0 0x7fd72c104480 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fd72400b0a0 tx=0x7fd72400b3b0 comp rx=0 tx=0).stop 2023-12-11T17:45:12.247 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.244+0000 7fd733c0b700 1 -- 172.21.15.59:0/441773747 shutdown_connections 2023-12-11T17:45:12.247 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.244+0000 7fd733c0b700 1 --2- 172.21.15.59:0/441773747 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd72c103fd0 0x7fd72c104480 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:12.247 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.244+0000 7fd733c0b700 1 --2- 172.21.15.59:0/441773747 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd72c1065e0 0x7fd72c069ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:12.247 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.244+0000 7fd733c0b700 1 --2- 172.21.15.59:0/441773747 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd72c06a220 0x7fd72c06a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:12.247 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.244+0000 7fd733c0b700 1 -- 172.21.15.59:0/441773747 >> 172.21.15.59:0/441773747 conn(0x7fd72c0765b0 msgr2=0x7fd72c0769c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:45:12.247 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.244+0000 7fd733c0b700 1 -- 172.21.15.59:0/441773747 shutdown_connections 2023-12-11T17:45:12.247 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.244+0000 7fd733c0b700 1 -- 172.21.15.59:0/441773747 wait complete. 2023-12-11T17:45:12.249 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.245+0000 7fd733c0b700 1 Processor -- start 2023-12-11T17:45:12.249 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.245+0000 7fd733c0b700 1 -- start start 2023-12-11T17:45:12.249 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.245+0000 7fd733c0b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd72c06a220 0x7fd72c19def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:12.249 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.245+0000 7fd733c0b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd72c103fd0 0x7fd72c19e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:12.249 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.245+0000 7fd733c0b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd72c1065e0 0x7fd72c1a27e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:12.249 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.245+0000 7fd733c0b700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd72c1a2df0 con 0x7fd72c06a220 2023-12-11T17:45:12.249 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.245+0000 7fd733c0b700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd72c1a2f60 con 0x7fd72c1065e0 2023-12-11T17:45:12.249 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.245+0000 7fd733c0b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd72c1a3120 con 0x7fd72c103fd0 2023-12-11T17:45:12.249 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.245+0000 7fd7319a7700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd72c06a220 0x7fd72c19def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:12.249 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.245+0000 7fd7319a7700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd72c06a220 0x7fd72c19def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33636/0 (socket says 172.21.15.59:33636) 2023-12-11T17:45:12.249 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.245+0000 7fd7319a7700 1 -- 172.21.15.59:0/1468098793 learned_addr learned my addr 172.21.15.59:0/1468098793 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:45:12.249 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.245+0000 7fd7321a8700 1 --2- 172.21.15.59:0/1468098793 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd72c1065e0 0x7fd72c1a27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:12.249 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.245+0000 7fd7311a6700 1 --2- 172.21.15.59:0/1468098793 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd72c103fd0 0x7fd72c19e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:12.249 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.245+0000 7fd7319a7700 1 -- 172.21.15.59:0/1468098793 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd72c103fd0 msgr2=0x7fd72c19e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:12.250 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.245+0000 7fd7319a7700 1 --2- 172.21.15.59:0/1468098793 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd72c103fd0 0x7fd72c19e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:12.250 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.245+0000 7fd7319a7700 1 -- 172.21.15.59:0/1468098793 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd72c1065e0 msgr2=0x7fd72c1a27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:12.250 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.245+0000 7fd7319a7700 1 --2- 172.21.15.59:0/1468098793 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd72c1065e0 0x7fd72c1a27e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:12.250 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.245+0000 7fd7319a7700 1 -- 172.21.15.59:0/1468098793 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd724009d00 con 0x7fd72c06a220 2023-12-11T17:45:12.250 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.246+0000 7fd7319a7700 1 --2- 172.21.15.59:0/1468098793 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd72c06a220 0x7fd72c19def0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fd72800e9a0 tx=0x7fd72800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:12.250 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.246+0000 7fd722ffd700 1 -- 172.21.15.59:0/1468098793 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd728009790 con 0x7fd72c06a220 2023-12-11T17:45:12.250 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.246+0000 7fd733c0b700 1 -- 172.21.15.59:0/1468098793 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd72c1a3400 con 0x7fd72c06a220 2023-12-11T17:45:12.250 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.246+0000 7fd722ffd700 1 -- 172.21.15.59:0/1468098793 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd728010430 con 0x7fd72c06a220 2023-12-11T17:45:12.250 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.246+0000 7fd722ffd700 1 -- 172.21.15.59:0/1468098793 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd728018680 con 0x7fd72c06a220 2023-12-11T17:45:12.251 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.246+0000 7fd733c0b700 1 -- 172.21.15.59:0/1468098793 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd72c1aac60 con 0x7fd72c06a220 2023-12-11T17:45:12.251 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.247+0000 7fd733c0b700 1 -- 172.21.15.59:0/1468098793 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd72c117d80 con 0x7fd72c06a220 2023-12-11T17:45:12.251 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.248+0000 7fd722ffd700 1 -- 172.21.15.59:0/1468098793 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7fd7280105a0 con 0x7fd72c06a220 2023-12-11T17:45:12.252 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.249+0000 7fd722ffd700 1 --2- 172.21.15.59:0/1468098793 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fd7180a1510 0x7fd7180a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:12.252 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.249+0000 7fd7311a6700 1 --2- 172.21.15.59:0/1468098793 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fd7180a1510 0x7fd7180a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:12.252 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.249+0000 7fd722ffd700 1 -- 172.21.15.59:0/1468098793 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fd728014070 con 0x7fd72c06a220 2023-12-11T17:45:12.252 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.249+0000 7fd722ffd700 1 --2- 172.21.15.59:0/1468098793 >> [v2:172.21.15.181:6816/3954509873,v1:172.21.15.181:6817/3954509873] conn(0x7fd7180ac090 0x7fd7180ae4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:12.252 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.249+0000 7fd722ffd700 1 -- 172.21.15.59:0/1468098793 --> [v2:172.21.15.181:6816/3954509873,v1:172.21.15.181:6817/3954509873] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd7180aeba0 con 0x7fd7180ac090 2023-12-11T17:45:12.252 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.249+0000 7fd722ffd700 1 -- 172.21.15.59:0/1468098793 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd7280cd860 con 0x7fd72c06a220 2023-12-11T17:45:12.253 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.250+0000 7fd7321a8700 1 --2- 172.21.15.59:0/1468098793 >> [v2:172.21.15.181:6816/3954509873,v1:172.21.15.181:6817/3954509873] conn(0x7fd7180ac090 0x7fd7180ae4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:12.253 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.250+0000 7fd7311a6700 1 --2- 172.21.15.59:0/1468098793 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fd7180a1510 0x7fd7180a39d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd72c19f450 tx=0x7fd71c00a300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:12.253 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.250+0000 7fd7321a8700 1 --2- 172.21.15.59:0/1468098793 >> [v2:172.21.15.181:6816/3954509873,v1:172.21.15.181:6817/3954509873] conn(0x7fd7180ac090 0x7fd7180ae4d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:12.253 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.251+0000 7fd722ffd700 1 -- 172.21.15.59:0/1468098793 <== osd.8 v2:172.21.15.181:6816/3954509873 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fd7180aeba0 con 0x7fd7180ac090 2023-12-11T17:45:12.272 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.269+0000 7fd733c0b700 1 -- 172.21.15.59:0/1468098793 --> [v2:172.21.15.181:6816/3954509873,v1:172.21.15.181:6817/3954509873] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd72c100ab0 con 0x7fd7180ac090 2023-12-11T17:45:12.272 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.270+0000 7fd722ffd700 1 -- 172.21.15.59:0/1468098793 <== osd.8 v2:172.21.15.181:6816/3954509873 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fd72c100ab0 con 0x7fd7180ac090 2023-12-11T17:45:12.273 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.270+0000 7fd733c0b700 1 -- 172.21.15.59:0/1468098793 >> [v2:172.21.15.181:6816/3954509873,v1:172.21.15.181:6817/3954509873] conn(0x7fd7180ac090 msgr2=0x7fd7180ae4d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:12.273 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.270+0000 7fd733c0b700 1 --2- 172.21.15.59:0/1468098793 >> [v2:172.21.15.181:6816/3954509873,v1:172.21.15.181:6817/3954509873] conn(0x7fd7180ac090 0x7fd7180ae4d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:12.273 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.270+0000 7fd733c0b700 1 -- 172.21.15.59:0/1468098793 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fd7180a1510 msgr2=0x7fd7180a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:12.273 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.270+0000 7fd733c0b700 1 --2- 172.21.15.59:0/1468098793 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fd7180a1510 0x7fd7180a39d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd72c19f450 tx=0x7fd71c00a300 comp rx=0 tx=0).stop 2023-12-11T17:45:12.274 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.270+0000 7fd733c0b700 1 -- 172.21.15.59:0/1468098793 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd72c06a220 msgr2=0x7fd72c19def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:12.274 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.270+0000 7fd733c0b700 1 --2- 172.21.15.59:0/1468098793 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd72c06a220 0x7fd72c19def0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fd72800e9a0 tx=0x7fd72800ecb0 comp rx=0 tx=0).stop 2023-12-11T17:45:12.274 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.270+0000 7fd7321a8700 1 -- 172.21.15.59:0/1468098793 reap_dead start 2023-12-11T17:45:12.274 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.271+0000 7fd733c0b700 1 -- 172.21.15.59:0/1468098793 shutdown_connections 2023-12-11T17:45:12.274 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.271+0000 7fd733c0b700 1 -- 172.21.15.59:0/1468098793 >> 172.21.15.59:0/1468098793 conn(0x7fd72c0765b0 msgr2=0x7fd72c102fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:45:12.274 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.271+0000 7fd733c0b700 1 -- 172.21.15.59:0/1468098793 shutdown_connections 2023-12-11T17:45:12.274 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.271+0000 7fd733c0b700 1 -- 172.21.15.59:0/1468098793 wait complete. 2023-12-11T17:45:12.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:45:11 smithi115 ceph-mon[102748]: pgmap v284: 129 pgs: 129 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 341 B/s wr, 6 op/s 2023-12-11T17:45:12.339 INFO:teuthology.orchestra.run.smithi059.stdout:193273528363 2023-12-11T17:45:12.340 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell --fsid d0a07eaa-984b-11ee-95a3-87774f69a715 -- ceph osd last-stat-seq osd.6 2023-12-11T17:45:12.545 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.216+0000 7f4c31be4700 1 -- 172.21.15.59:0/942409917 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4c2c0fff70 msgr2=0x7f4c2c1003d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:12.546 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.216+0000 7f4c31be4700 1 --2- 172.21.15.59:0/942409917 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4c2c0fff70 0x7f4c2c1003d0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f4c20009990 tx=0x7f4c20009ca0 comp rx=0 tx=0).stop 2023-12-11T17:45:12.546 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.216+0000 7f4c31be4700 1 -- 172.21.15.59:0/942409917 shutdown_connections 2023-12-11T17:45:12.546 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.216+0000 7f4c31be4700 1 --2- 172.21.15.59:0/942409917 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4c2c100910 0x7f4c2c10e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:12.546 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.216+0000 7f4c31be4700 1 --2- 172.21.15.59:0/942409917 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c2c1066b0 0x7f4c2c106a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:12.546 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.216+0000 7f4c31be4700 1 --2- 172.21.15.59:0/942409917 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4c2c0fff70 0x7f4c2c1003d0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:12.546 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.216+0000 7f4c31be4700 1 -- 172.21.15.59:0/942409917 >> 172.21.15.59:0/942409917 conn(0x7f4c2c0757f0 msgr2=0x7f4c2c075c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:45:12.546 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.216+0000 7f4c31be4700 1 -- 172.21.15.59:0/942409917 shutdown_connections 2023-12-11T17:45:12.546 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.216+0000 7f4c31be4700 1 -- 172.21.15.59:0/942409917 wait complete. 2023-12-11T17:45:12.546 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.217+0000 7f4c31be4700 1 Processor -- start 2023-12-11T17:45:12.546 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.217+0000 7f4c31be4700 1 -- start start 2023-12-11T17:45:12.546 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.218+0000 7f4c31be4700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4c2c0fff70 0x7f4c2c19d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:12.546 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.218+0000 7f4c31be4700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4c2c100910 0x7f4c2c19dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:12.547 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.218+0000 7f4c31be4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c2c1066b0 0x7f4c2c1978f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:12.547 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.218+0000 7f4c31be4700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c2c1980a0 con 0x7f4c2c100910 2023-12-11T17:45:12.547 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.218+0000 7f4c31be4700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c2c198210 con 0x7f4c2c0fff70 2023-12-11T17:45:12.547 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.218+0000 7f4c31be4700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c2c1983d0 con 0x7f4c2c1066b0 2023-12-11T17:45:12.547 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.218+0000 7f4c2affd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4c2c100910 0x7f4c2c19dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:12.547 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.218+0000 7f4c2affd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4c2c100910 0x7f4c2c19dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33622/0 (socket says 172.21.15.59:33622) 2023-12-11T17:45:12.547 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.218+0000 7f4c2affd700 1 -- 172.21.15.59:0/3142560740 learned_addr learned my addr 172.21.15.59:0/3142560740 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:45:12.547 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.218+0000 7f4c2bfff700 1 --2- 172.21.15.59:0/3142560740 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c2c1066b0 0x7f4c2c1978f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:12.547 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.218+0000 7f4c2b7fe700 1 --2- 172.21.15.59:0/3142560740 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4c2c0fff70 0x7f4c2c19d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:12.547 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.218+0000 7f4c2affd700 1 -- 172.21.15.59:0/3142560740 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c2c1066b0 msgr2=0x7f4c2c1978f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:12.547 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.218+0000 7f4c2affd700 1 --2- 172.21.15.59:0/3142560740 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c2c1066b0 0x7f4c2c1978f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:12.547 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.218+0000 7f4c2affd700 1 -- 172.21.15.59:0/3142560740 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4c2c0fff70 msgr2=0x7f4c2c19d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:12.547 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.218+0000 7f4c2affd700 1 --2- 172.21.15.59:0/3142560740 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4c2c0fff70 0x7f4c2c19d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:12.547 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.218+0000 7f4c2affd700 1 -- 172.21.15.59:0/3142560740 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c200095f0 con 0x7f4c2c100910 2023-12-11T17:45:12.547 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.218+0000 7f4c2affd700 1 --2- 172.21.15.59:0/3142560740 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4c2c100910 0x7f4c2c19dd80 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f4c200095c0 tx=0x7f4c2000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:12.547 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.219+0000 7f4c28ff9700 1 -- 172.21.15.59:0/3142560740 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c2001d070 con 0x7f4c2c100910 2023-12-11T17:45:12.548 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.219+0000 7f4c31be4700 1 -- 172.21.15.59:0/3142560740 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c2c198650 con 0x7f4c2c100910 2023-12-11T17:45:12.548 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.219+0000 7f4c28ff9700 1 -- 172.21.15.59:0/3142560740 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4c20005050 con 0x7f4c2c100910 2023-12-11T17:45:12.548 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.219+0000 7f4c28ff9700 1 -- 172.21.15.59:0/3142560740 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c20004bb0 con 0x7f4c2c100910 2023-12-11T17:45:12.548 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.219+0000 7f4c31be4700 1 -- 172.21.15.59:0/3142560740 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c2c1a47c0 con 0x7f4c2c100910 2023-12-11T17:45:12.548 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.220+0000 7f4c31be4700 1 -- 172.21.15.59:0/3142560740 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4c2c111810 con 0x7f4c2c100910 2023-12-11T17:45:12.548 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.221+0000 7f4c28ff9700 1 -- 172.21.15.59:0/3142560740 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f4c20004d30 con 0x7f4c2c100910 2023-12-11T17:45:12.548 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.222+0000 7f4c28ff9700 1 --2- 172.21.15.59:0/3142560740 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f4c140a1510 0x7f4c140a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:12.548 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.222+0000 7f4c28ff9700 1 -- 172.21.15.59:0/3142560740 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f4c200cdf20 con 0x7f4c2c100910 2023-12-11T17:45:12.548 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.222+0000 7f4c2b7fe700 1 --2- 172.21.15.59:0/3142560740 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f4c140a1510 0x7f4c140a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:12.548 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.222+0000 7f4c28ff9700 1 --2- 172.21.15.59:0/3142560740 >> [v2:172.21.15.181:6808/2405172521,v1:172.21.15.181:6809/2405172521] conn(0x7f4c140ac090 0x7f4c140ae4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:12.549 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.222+0000 7f4c28ff9700 1 -- 172.21.15.59:0/3142560740 --> [v2:172.21.15.181:6808/2405172521,v1:172.21.15.181:6809/2405172521] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4c140aeba0 con 0x7f4c140ac090 2023-12-11T17:45:12.549 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.222+0000 7f4c28ff9700 1 -- 172.21.15.59:0/3142560740 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4c200d05a0 con 0x7f4c2c100910 2023-12-11T17:45:12.549 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.222+0000 7f4c2b7fe700 1 --2- 172.21.15.59:0/3142560740 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f4c140a1510 0x7f4c140a39d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f4c1c005950 tx=0x7f4c1c0058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:12.549 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.222+0000 7f4c2bfff700 1 --2- 172.21.15.59:0/3142560740 >> [v2:172.21.15.181:6808/2405172521,v1:172.21.15.181:6809/2405172521] conn(0x7f4c140ac090 0x7f4c140ae4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:12.549 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.223+0000 7f4c2bfff700 1 --2- 172.21.15.59:0/3142560740 >> [v2:172.21.15.181:6808/2405172521,v1:172.21.15.181:6809/2405172521] conn(0x7f4c140ac090 0x7f4c140ae4d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:12.549 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.224+0000 7f4c28ff9700 1 -- 172.21.15.59:0/3142560740 <== osd.5 v2:172.21.15.181:6808/2405172521 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f4c140aeba0 con 0x7f4c140ac090 2023-12-11T17:45:12.549 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.241+0000 7f4c31be4700 1 -- 172.21.15.59:0/3142560740 --> [v2:172.21.15.181:6808/2405172521,v1:172.21.15.181:6809/2405172521] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f4c2c061c50 con 0x7f4c140ac090 2023-12-11T17:45:12.549 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.241+0000 7f4c28ff9700 1 -- 172.21.15.59:0/3142560740 <== osd.5 v2:172.21.15.181:6808/2405172521 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f4c2c061c50 con 0x7f4c140ac090 2023-12-11T17:45:12.549 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.242+0000 7f4c31be4700 1 -- 172.21.15.59:0/3142560740 >> [v2:172.21.15.181:6808/2405172521,v1:172.21.15.181:6809/2405172521] conn(0x7f4c140ac090 msgr2=0x7f4c140ae4d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:12.549 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.242+0000 7f4c31be4700 1 --2- 172.21.15.59:0/3142560740 >> [v2:172.21.15.181:6808/2405172521,v1:172.21.15.181:6809/2405172521] conn(0x7f4c140ac090 0x7f4c140ae4d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:12.549 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.242+0000 7f4c31be4700 1 -- 172.21.15.59:0/3142560740 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f4c140a1510 msgr2=0x7f4c140a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:12.549 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.242+0000 7f4c31be4700 1 --2- 172.21.15.59:0/3142560740 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f4c140a1510 0x7f4c140a39d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f4c1c005950 tx=0x7f4c1c0058e0 comp rx=0 tx=0).stop 2023-12-11T17:45:12.549 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.242+0000 7f4c31be4700 1 -- 172.21.15.59:0/3142560740 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4c2c100910 msgr2=0x7f4c2c19dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:12.549 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.242+0000 7f4c31be4700 1 --2- 172.21.15.59:0/3142560740 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4c2c100910 0x7f4c2c19dd80 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f4c200095c0 tx=0x7f4c2000f690 comp rx=0 tx=0).stop 2023-12-11T17:45:12.549 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.242+0000 7f4c2bfff700 1 -- 172.21.15.59:0/3142560740 reap_dead start 2023-12-11T17:45:12.549 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.242+0000 7f4c31be4700 1 -- 172.21.15.59:0/3142560740 shutdown_connections 2023-12-11T17:45:12.550 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.242+0000 7f4c31be4700 1 -- 172.21.15.59:0/3142560740 >> 172.21.15.59:0/3142560740 conn(0x7f4c2c0757f0 msgr2=0x7f4c2c0fed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:45:12.550 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.242+0000 7f4c31be4700 1 -- 172.21.15.59:0/3142560740 shutdown_connections 2023-12-11T17:45:12.550 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.242+0000 7f4c31be4700 1 -- 172.21.15.59:0/3142560740 wait complete. 2023-12-11T17:45:12.753 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.705+0000 7f1abeb67700 1 -- 172.21.15.59:0/3957747053 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1ab81085d0 msgr2=0x7f1ab81089b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:12.753 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.705+0000 7f1abeb67700 1 --2- 172.21.15.59:0/3957747053 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1ab81085d0 0x7f1ab81089b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1aac0098f0 tx=0x7f1aac009c00 comp rx=0 tx=0).stop 2023-12-11T17:45:12.753 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.705+0000 7f1abeb67700 1 -- 172.21.15.59:0/3957747053 shutdown_connections 2023-12-11T17:45:12.753 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.705+0000 7f1abeb67700 1 --2- 172.21.15.59:0/3957747053 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ab8114720 0x7f1ab8116b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:12.753 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.705+0000 7f1abeb67700 1 --2- 172.21.15.59:0/3957747053 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1ab81085d0 0x7f1ab81089b0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:12.753 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.705+0000 7f1abeb67700 1 --2- 172.21.15.59:0/3957747053 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ab8108ef0 0x7f1ab81141e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:12.754 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.705+0000 7f1abeb67700 1 -- 172.21.15.59:0/3957747053 >> 172.21.15.59:0/3957747053 conn(0x7f1ab806c6a0 msgr2=0x7f1ab806cab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:45:12.754 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.705+0000 7f1abeb67700 1 -- 172.21.15.59:0/3957747053 shutdown_connections 2023-12-11T17:45:12.754 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.705+0000 7f1abeb67700 1 -- 172.21.15.59:0/3957747053 wait complete. 2023-12-11T17:45:12.754 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.705+0000 7f1abeb67700 1 Processor -- start 2023-12-11T17:45:12.754 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.706+0000 7f1abeb67700 1 -- start start 2023-12-11T17:45:12.754 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.706+0000 7f1abeb67700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ab81085d0 0x7f1ab810c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:12.754 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.706+0000 7f1abeb67700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1ab8108ef0 0x7f1ab810c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:12.754 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.706+0000 7f1abeb67700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ab8114720 0x7f1ab810ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:12.754 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.706+0000 7f1abeb67700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ab81b2ae0 con 0x7f1ab81085d0 2023-12-11T17:45:12.754 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.706+0000 7f1abeb67700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ab81b2c20 con 0x7f1ab8108ef0 2023-12-11T17:45:12.754 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.706+0000 7f1abeb67700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ab81a21f0 con 0x7f1ab8114720 2023-12-11T17:45:12.754 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.706+0000 7f1abc903700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ab81085d0 0x7f1ab810c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:12.754 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.706+0000 7f1abc903700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ab81085d0 0x7f1ab810c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33652/0 (socket says 172.21.15.59:33652) 2023-12-11T17:45:12.754 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.706+0000 7f1abc903700 1 -- 172.21.15.59:0/222620329 learned_addr learned my addr 172.21.15.59:0/222620329 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:45:12.755 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.706+0000 7f1ab7fff700 1 --2- 172.21.15.59:0/222620329 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1ab8108ef0 0x7f1ab810c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:12.755 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.706+0000 7f1abd104700 1 --2- 172.21.15.59:0/222620329 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ab8114720 0x7f1ab810ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:12.755 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.706+0000 7f1abc903700 1 -- 172.21.15.59:0/222620329 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ab8114720 msgr2=0x7f1ab810ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:12.755 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.706+0000 7f1abc903700 1 --2- 172.21.15.59:0/222620329 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ab8114720 0x7f1ab810ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:12.755 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.706+0000 7f1abc903700 1 -- 172.21.15.59:0/222620329 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1ab8108ef0 msgr2=0x7f1ab810c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:12.755 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.706+0000 7f1abc903700 1 --2- 172.21.15.59:0/222620329 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1ab8108ef0 0x7f1ab810c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:12.755 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.706+0000 7f1abc903700 1 -- 172.21.15.59:0/222620329 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1aac009520 con 0x7f1ab81085d0 2023-12-11T17:45:12.755 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.707+0000 7f1abc903700 1 --2- 172.21.15.59:0/222620329 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ab81085d0 0x7f1ab810c250 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f1aac005950 tx=0x7f1aac00fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:12.755 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.707+0000 7f1ab5ffb700 1 -- 172.21.15.59:0/222620329 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aac01c070 con 0x7f1ab81085d0 2023-12-11T17:45:12.755 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.707+0000 7f1abeb67700 1 -- 172.21.15.59:0/222620329 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ab81a2470 con 0x7f1ab81085d0 2023-12-11T17:45:12.755 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.707+0000 7f1ab5ffb700 1 -- 172.21.15.59:0/222620329 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1aac00bb20 con 0x7f1ab81085d0 2023-12-11T17:45:12.755 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.707+0000 7f1abeb67700 1 -- 172.21.15.59:0/222620329 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ab81a2910 con 0x7f1ab81085d0 2023-12-11T17:45:12.755 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.708+0000 7f1ab5ffb700 1 -- 172.21.15.59:0/222620329 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aac003880 con 0x7f1ab81085d0 2023-12-11T17:45:12.755 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.708+0000 7f1abeb67700 1 -- 172.21.15.59:0/222620329 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1ab8119fd0 con 0x7f1ab81085d0 2023-12-11T17:45:12.755 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.709+0000 7f1ab5ffb700 1 -- 172.21.15.59:0/222620329 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f1aac020030 con 0x7f1ab81085d0 2023-12-11T17:45:12.756 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.710+0000 7f1ab5ffb700 1 --2- 172.21.15.59:0/222620329 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f1aa80a1510 0x7f1aa80a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:12.756 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.710+0000 7f1ab7fff700 1 --2- 172.21.15.59:0/222620329 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f1aa80a1510 0x7f1aa80a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:12.756 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.710+0000 7f1ab5ffb700 1 -- 172.21.15.59:0/222620329 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f1aac0ceb10 con 0x7f1ab81085d0 2023-12-11T17:45:12.756 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.710+0000 7f1ab5ffb700 1 --2- 172.21.15.59:0/222620329 >> [v2:172.21.15.115:6800/2776354245,v1:172.21.15.115:6801/2776354245] conn(0x7f1aa80ac090 0x7f1aa80ae4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:12.756 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.710+0000 7f1ab5ffb700 1 -- 172.21.15.59:0/222620329 --> [v2:172.21.15.115:6800/2776354245,v1:172.21.15.115:6801/2776354245] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1aa80aeba0 con 0x7f1aa80ac090 2023-12-11T17:45:12.756 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.710+0000 7f1ab5ffb700 1 -- 172.21.15.59:0/222620329 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1aac0cef20 con 0x7f1ab81085d0 2023-12-11T17:45:12.756 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.710+0000 7f1ab7fff700 1 --2- 172.21.15.59:0/222620329 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f1aa80a1510 0x7f1aa80a39d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1ab810d980 tx=0x7f1aa4009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:12.756 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.710+0000 7f1abd104700 1 --2- 172.21.15.59:0/222620329 >> [v2:172.21.15.115:6800/2776354245,v1:172.21.15.115:6801/2776354245] conn(0x7f1aa80ac090 0x7f1aa80ae4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:12.756 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.711+0000 7f1abd104700 1 --2- 172.21.15.59:0/222620329 >> [v2:172.21.15.115:6800/2776354245,v1:172.21.15.115:6801/2776354245] conn(0x7f1aa80ac090 0x7f1aa80ae4d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:12.756 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.711+0000 7f1ab5ffb700 1 -- 172.21.15.59:0/222620329 <== osd.1 v2:172.21.15.115:6800/2776354245 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f1aa80aeba0 con 0x7f1aa80ac090 2023-12-11T17:45:12.757 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.729+0000 7f1abeb67700 1 -- 172.21.15.59:0/222620329 --> [v2:172.21.15.115:6800/2776354245,v1:172.21.15.115:6801/2776354245] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1ab806dc70 con 0x7f1aa80ac090 2023-12-11T17:45:12.757 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.729+0000 7f1ab5ffb700 1 -- 172.21.15.59:0/222620329 <== osd.1 v2:172.21.15.115:6800/2776354245 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f1ab806dc70 con 0x7f1aa80ac090 2023-12-11T17:45:12.757 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.730+0000 7f1abeb67700 1 -- 172.21.15.59:0/222620329 >> [v2:172.21.15.115:6800/2776354245,v1:172.21.15.115:6801/2776354245] conn(0x7f1aa80ac090 msgr2=0x7f1aa80ae4d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:12.757 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.730+0000 7f1abeb67700 1 --2- 172.21.15.59:0/222620329 >> [v2:172.21.15.115:6800/2776354245,v1:172.21.15.115:6801/2776354245] conn(0x7f1aa80ac090 0x7f1aa80ae4d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:12.757 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.730+0000 7f1abeb67700 1 -- 172.21.15.59:0/222620329 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f1aa80a1510 msgr2=0x7f1aa80a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:12.757 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.730+0000 7f1abeb67700 1 --2- 172.21.15.59:0/222620329 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f1aa80a1510 0x7f1aa80a39d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1ab810d980 tx=0x7f1aa4009310 comp rx=0 tx=0).stop 2023-12-11T17:45:12.757 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.730+0000 7f1abeb67700 1 -- 172.21.15.59:0/222620329 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ab81085d0 msgr2=0x7f1ab810c250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:12.757 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.730+0000 7f1abeb67700 1 --2- 172.21.15.59:0/222620329 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ab81085d0 0x7f1ab810c250 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f1aac005950 tx=0x7f1aac00fa00 comp rx=0 tx=0).stop 2023-12-11T17:45:12.757 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.730+0000 7f1abd104700 1 -- 172.21.15.59:0/222620329 reap_dead start 2023-12-11T17:45:12.757 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.730+0000 7f1abeb67700 1 -- 172.21.15.59:0/222620329 shutdown_connections 2023-12-11T17:45:12.757 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.730+0000 7f1abeb67700 1 -- 172.21.15.59:0/222620329 >> 172.21.15.59:0/222620329 conn(0x7f1ab806c6a0 msgr2=0x7f1ab810a070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:45:12.757 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.730+0000 7f1abeb67700 1 -- 172.21.15.59:0/222620329 shutdown_connections 2023-12-11T17:45:12.757 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.730+0000 7f1abeb67700 1 -- 172.21.15.59:0/222620329 wait complete. 2023-12-11T17:45:12.925 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.923+0000 7fd542d81700 1 -- 172.21.15.59:0/593575325 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd53c068f70 msgr2=0x7fd53c0693f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:12.926 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.923+0000 7fd542d81700 1 --2- 172.21.15.59:0/593575325 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd53c068f70 0x7fd53c0693f0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fd5280099c0 tx=0x7fd528009cd0 comp rx=0 tx=0).stop 2023-12-11T17:45:12.926 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.923+0000 7fd542d81700 1 -- 172.21.15.59:0/593575325 shutdown_connections 2023-12-11T17:45:12.926 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.923+0000 7fd542d81700 1 --2- 172.21.15.59:0/593575325 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd53c069930 0x7fd53c10e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:12.926 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.923+0000 7fd542d81700 1 --2- 172.21.15.59:0/593575325 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd53c107620 0x7fd53c107a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:12.926 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.923+0000 7fd542d81700 1 --2- 172.21.15.59:0/593575325 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd53c068f70 0x7fd53c0693f0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:12.926 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.923+0000 7fd542d81700 1 -- 172.21.15.59:0/593575325 >> 172.21.15.59:0/593575325 conn(0x7fd53c076570 msgr2=0x7fd53c076980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:45:12.926 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.923+0000 7fd542d81700 1 -- 172.21.15.59:0/593575325 shutdown_connections 2023-12-11T17:45:12.926 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.923+0000 7fd542d81700 1 -- 172.21.15.59:0/593575325 wait complete. 2023-12-11T17:45:12.928 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.923+0000 7fd542d81700 1 Processor -- start 2023-12-11T17:45:12.928 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.924+0000 7fd542d81700 1 -- start start 2023-12-11T17:45:12.928 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.924+0000 7fd542d81700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd53c068f70 0x7fd53c19f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:12.928 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.924+0000 7fd542d81700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd53c069930 0x7fd53c19fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:12.928 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.924+0000 7fd542d81700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd53c107620 0x7fd53c199a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:12.928 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.924+0000 7fd542d81700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd53c19a0a0 con 0x7fd53c069930 2023-12-11T17:45:12.929 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.924+0000 7fd542d81700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd53c19a210 con 0x7fd53c068f70 2023-12-11T17:45:12.929 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.924+0000 7fd542d81700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd53c19a3d0 con 0x7fd53c107620 2023-12-11T17:45:12.929 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.924+0000 7fd53bfff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd53c069930 0x7fd53c19fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:12.929 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.924+0000 7fd53bfff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd53c069930 0x7fd53c19fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33686/0 (socket says 172.21.15.59:33686) 2023-12-11T17:45:12.929 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.924+0000 7fd53bfff700 1 -- 172.21.15.59:0/2014428231 learned_addr learned my addr 172.21.15.59:0/2014428231 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:45:12.929 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.924+0000 7fd540b1d700 1 --2- 172.21.15.59:0/2014428231 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd53c068f70 0x7fd53c19f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:12.929 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.924+0000 7fd54131e700 1 --2- 172.21.15.59:0/2014428231 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd53c107620 0x7fd53c199a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:12.929 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.924+0000 7fd53bfff700 1 -- 172.21.15.59:0/2014428231 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd53c107620 msgr2=0x7fd53c199a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:12.929 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.924+0000 7fd53bfff700 1 --2- 172.21.15.59:0/2014428231 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd53c107620 0x7fd53c199a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:12.929 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.924+0000 7fd53bfff700 1 -- 172.21.15.59:0/2014428231 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd53c068f70 msgr2=0x7fd53c19f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:12.929 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.924+0000 7fd53bfff700 1 --2- 172.21.15.59:0/2014428231 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd53c068f70 0x7fd53c19f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:12.929 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.924+0000 7fd53bfff700 1 -- 172.21.15.59:0/2014428231 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5280095f0 con 0x7fd53c069930 2023-12-11T17:45:12.929 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.924+0000 7fd53bfff700 1 --2- 172.21.15.59:0/2014428231 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd53c069930 0x7fd53c19fda0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fd528005720 tx=0x7fd528005750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:12.929 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.925+0000 7fd539ffb700 1 -- 172.21.15.59:0/2014428231 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd52801d070 con 0x7fd53c069930 2023-12-11T17:45:12.929 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.925+0000 7fd542d81700 1 -- 172.21.15.59:0/2014428231 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd53c19a650 con 0x7fd53c069930 2023-12-11T17:45:12.930 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.925+0000 7fd539ffb700 1 -- 172.21.15.59:0/2014428231 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd52800fb30 con 0x7fd53c069930 2023-12-11T17:45:12.930 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.925+0000 7fd539ffb700 1 -- 172.21.15.59:0/2014428231 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd528022bc0 con 0x7fd53c069930 2023-12-11T17:45:12.930 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.925+0000 7fd542d81700 1 -- 172.21.15.59:0/2014428231 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd53c1a6850 con 0x7fd53c069930 2023-12-11T17:45:12.930 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.926+0000 7fd542d81700 1 -- 172.21.15.59:0/2014428231 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd53c1117c0 con 0x7fd53c069930 2023-12-11T17:45:12.931 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.927+0000 7fd539ffb700 1 -- 172.21.15.59:0/2014428231 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7fd52800f670 con 0x7fd53c069930 2023-12-11T17:45:12.931 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.928+0000 7fd539ffb700 1 --2- 172.21.15.59:0/2014428231 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fd52c0a1510 0x7fd52c0a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:12.931 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.928+0000 7fd540b1d700 1 --2- 172.21.15.59:0/2014428231 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fd52c0a1510 0x7fd52c0a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:12.931 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.928+0000 7fd539ffb700 1 -- 172.21.15.59:0/2014428231 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fd5280ceec0 con 0x7fd53c069930 2023-12-11T17:45:12.931 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.928+0000 7fd539ffb700 1 --2- 172.21.15.59:0/2014428231 >> [v2:172.21.15.59:6802/1040759398,v1:172.21.15.59:6803/1040759398] conn(0x7fd52c0ac090 0x7fd52c0ae4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:12.931 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.928+0000 7fd539ffb700 1 -- 172.21.15.59:0/2014428231 --> [v2:172.21.15.59:6802/1040759398,v1:172.21.15.59:6803/1040759398] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd52c0aeba0 con 0x7fd52c0ac090 2023-12-11T17:45:12.931 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.928+0000 7fd539ffb700 1 -- 172.21.15.59:0/2014428231 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd5280cf2d0 con 0x7fd53c069930 2023-12-11T17:45:12.931 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.928+0000 7fd54131e700 1 --2- 172.21.15.59:0/2014428231 >> [v2:172.21.15.59:6802/1040759398,v1:172.21.15.59:6803/1040759398] conn(0x7fd52c0ac090 0x7fd52c0ae4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:12.931 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.928+0000 7fd540b1d700 1 --2- 172.21.15.59:0/2014428231 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fd52c0a1510 0x7fd52c0a39d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd530005950 tx=0x7fd53000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:12.932 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.929+0000 7fd54131e700 1 --2- 172.21.15.59:0/2014428231 >> [v2:172.21.15.59:6802/1040759398,v1:172.21.15.59:6803/1040759398] conn(0x7fd52c0ac090 0x7fd52c0ae4d0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:12.932 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.929+0000 7fd539ffb700 1 -- 172.21.15.59:0/2014428231 <== osd.0 v2:172.21.15.59:6802/1040759398 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fd52c0aeba0 con 0x7fd52c0ac090 2023-12-11T17:45:12.948 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.946+0000 7fd542d81700 1 -- 172.21.15.59:0/2014428231 --> [v2:172.21.15.59:6802/1040759398,v1:172.21.15.59:6803/1040759398] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd53c0ffaa0 con 0x7fd52c0ac090 2023-12-11T17:45:12.949 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.947+0000 7fd539ffb700 1 -- 172.21.15.59:0/2014428231 <== osd.0 v2:172.21.15.59:6802/1040759398 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fd53c0ffaa0 con 0x7fd52c0ac090 2023-12-11T17:45:12.950 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.947+0000 7fd542d81700 1 -- 172.21.15.59:0/2014428231 >> [v2:172.21.15.59:6802/1040759398,v1:172.21.15.59:6803/1040759398] conn(0x7fd52c0ac090 msgr2=0x7fd52c0ae4d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:12.950 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.947+0000 7fd542d81700 1 --2- 172.21.15.59:0/2014428231 >> [v2:172.21.15.59:6802/1040759398,v1:172.21.15.59:6803/1040759398] conn(0x7fd52c0ac090 0x7fd52c0ae4d0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:12.950 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.947+0000 7fd542d81700 1 -- 172.21.15.59:0/2014428231 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fd52c0a1510 msgr2=0x7fd52c0a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:12.950 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.947+0000 7fd542d81700 1 --2- 172.21.15.59:0/2014428231 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fd52c0a1510 0x7fd52c0a39d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd530005950 tx=0x7fd53000b410 comp rx=0 tx=0).stop 2023-12-11T17:45:12.950 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.947+0000 7fd542d81700 1 -- 172.21.15.59:0/2014428231 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd53c069930 msgr2=0x7fd53c19fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:12.950 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.947+0000 7fd542d81700 1 --2- 172.21.15.59:0/2014428231 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd53c069930 0x7fd53c19fda0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fd528005720 tx=0x7fd528005750 comp rx=0 tx=0).stop 2023-12-11T17:45:12.950 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.947+0000 7fd54131e700 1 -- 172.21.15.59:0/2014428231 reap_dead start 2023-12-11T17:45:12.950 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.947+0000 7fd542d81700 1 -- 172.21.15.59:0/2014428231 shutdown_connections 2023-12-11T17:45:12.950 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.947+0000 7fd542d81700 1 -- 172.21.15.59:0/2014428231 >> 172.21.15.59:0/2014428231 conn(0x7fd53c076570 msgr2=0x7fd53c0ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:45:12.950 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.947+0000 7fd542d81700 1 -- 172.21.15.59:0/2014428231 shutdown_connections 2023-12-11T17:45:12.951 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.947+0000 7fd542d81700 1 -- 172.21.15.59:0/2014428231 wait complete. 2023-12-11T17:45:13.630 INFO:teuthology.orchestra.run.smithi059.stdout:330712481814 2023-12-11T17:45:13.631 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell --fsid d0a07eaa-984b-11ee-95a3-87774f69a715 -- ceph osd last-stat-seq osd.11 2023-12-11T17:45:13.732 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:13.729+0000 7fa368781700 1 -- 172.21.15.59:0/1269370840 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa360100e80 msgr2=0x7fa36010d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:13.732 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:13.729+0000 7fa368781700 1 --2- 172.21.15.59:0/1269370840 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa360100e80 0x7fa36010d370 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fa35800b0d0 tx=0x7fa35800b3e0 comp rx=0 tx=0).stop 2023-12-11T17:45:13.732 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:13.729+0000 7fa368781700 1 -- 172.21.15.59:0/1269370840 shutdown_connections 2023-12-11T17:45:13.732 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:13.729+0000 7fa368781700 1 --2- 172.21.15.59:0/1269370840 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa360100e80 0x7fa36010d370 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:13.732 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:13.729+0000 7fa368781700 1 --2- 172.21.15.59:0/1269370840 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa360105470 0x7fa360105850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:13.732 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:13.729+0000 7fa368781700 1 --2- 172.21.15.59:0/1269370840 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3601004c0 0x7fa360100940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:13.732 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:13.729+0000 7fa368781700 1 -- 172.21.15.59:0/1269370840 >> 172.21.15.59:0/1269370840 conn(0x7fa3600774a0 msgr2=0x7fa3600778b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:45:13.732 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:13.729+0000 7fa368781700 1 -- 172.21.15.59:0/1269370840 shutdown_connections 2023-12-11T17:45:13.732 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:13.729+0000 7fa368781700 1 -- 172.21.15.59:0/1269370840 wait complete. 2023-12-11T17:45:13.734 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:13.730+0000 7fa368781700 1 Processor -- start 2023-12-11T17:45:13.734 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:13.730+0000 7fa368781700 1 -- start start 2023-12-11T17:45:13.734 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:13.730+0000 7fa368781700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa3601004c0 0x7fa36019d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:13.734 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:13.730+0000 7fa368781700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa360100e80 0x7fa36019dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:13.734 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:13.730+0000 7fa368781700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa360105470 0x7fa360197920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:13.734 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:13.730+0000 7fa368781700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa360197f80 con 0x7fa3601004c0 2023-12-11T17:45:13.735 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:13.730+0000 7fa368781700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3601980f0 con 0x7fa360100e80 2023-12-11T17:45:13.735 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:13.730+0000 7fa368781700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3601982b0 con 0x7fa360105470 2023-12-11T17:45:13.735 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:13.730+0000 7fa36651d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa3601004c0 0x7fa36019d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:13.735 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:13.730+0000 7fa36651d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa3601004c0 0x7fa36019d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33700/0 (socket says 172.21.15.59:33700) 2023-12-11T17:45:13.735 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:13.730+0000 7fa36651d700 1 -- 172.21.15.59:0/736662593 learned_addr learned my addr 172.21.15.59:0/736662593 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:45:13.735 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:13.730+0000 7fa365d1c700 1 --2- 172.21.15.59:0/736662593 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa360100e80 0x7fa36019dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:13.735 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:13.730+0000 7fa366d1e700 1 --2- 172.21.15.59:0/736662593 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa360105470 0x7fa360197920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:13.735 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:13.730+0000 7fa36651d700 1 -- 172.21.15.59:0/736662593 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa360105470 msgr2=0x7fa360197920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:13.735 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:13.730+0000 7fa36651d700 1 --2- 172.21.15.59:0/736662593 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa360105470 0x7fa360197920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:13.735 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:13.730+0000 7fa36651d700 1 -- 172.21.15.59:0/736662593 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa360100e80 msgr2=0x7fa36019dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:13.735 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:13.730+0000 7fa36651d700 1 --2- 172.21.15.59:0/736662593 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa360100e80 0x7fa36019dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:13.735 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:13.731+0000 7fa36651d700 1 -- 172.21.15.59:0/736662593 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa358009d00 con 0x7fa3601004c0 2023-12-11T17:45:13.736 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:13.731+0000 7fa36651d700 1 --2- 172.21.15.59:0/736662593 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa3601004c0 0x7fa36019d7e0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fa35c00e970 tx=0x7fa35c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:13.736 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:13.731+0000 7fa3577fe700 1 -- 172.21.15.59:0/736662593 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa35c009790 con 0x7fa3601004c0 2023-12-11T17:45:13.736 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:13.731+0000 7fa3577fe700 1 -- 172.21.15.59:0/736662593 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa35c00c7e0 con 0x7fa3601004c0 2023-12-11T17:45:13.736 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:13.731+0000 7fa368781700 1 -- 172.21.15.59:0/736662593 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa360198590 con 0x7fa3601004c0 2023-12-11T17:45:13.736 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:13.731+0000 7fa3577fe700 1 -- 172.21.15.59:0/736662593 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa35c003c70 con 0x7fa3601004c0 2023-12-11T17:45:13.736 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:13.731+0000 7fa368781700 1 -- 172.21.15.59:0/736662593 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3601a47d0 con 0x7fa3601004c0 2023-12-11T17:45:13.736 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:13.732+0000 7fa368781700 1 -- 172.21.15.59:0/736662593 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa360111820 con 0x7fa3601004c0 2023-12-11T17:45:13.737 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:13.733+0000 7fa3577fe700 1 -- 172.21.15.59:0/736662593 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7fa35c00cc50 con 0x7fa3601004c0 2023-12-11T17:45:13.737 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:13.734+0000 7fa3577fe700 1 --2- 172.21.15.59:0/736662593 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fa34c0a1510 0x7fa34c0a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:13.737 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:13.734+0000 7fa3577fe700 1 -- 172.21.15.59:0/736662593 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fa35c014070 con 0x7fa3601004c0 2023-12-11T17:45:13.738 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:13.734+0000 7fa365d1c700 1 --2- 172.21.15.59:0/736662593 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fa34c0a1510 0x7fa34c0a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:13.738 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:13.734+0000 7fa3577fe700 1 --2- 172.21.15.59:0/736662593 >> [v2:172.21.15.181:6800/4287675481,v1:172.21.15.181:6801/4287675481] conn(0x7fa34c0ac090 0x7fa34c0ae4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:13.738 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:13.734+0000 7fa3577fe700 1 -- 172.21.15.59:0/736662593 --> [v2:172.21.15.181:6800/4287675481,v1:172.21.15.181:6801/4287675481] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa34c0aeba0 con 0x7fa34c0ac090 2023-12-11T17:45:13.738 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:13.734+0000 7fa3577fe700 1 -- 172.21.15.59:0/736662593 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa35c0cd6d0 con 0x7fa3601004c0 2023-12-11T17:45:13.738 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:13.734+0000 7fa365d1c700 1 --2- 172.21.15.59:0/736662593 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fa34c0a1510 0x7fa34c0a39d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa360198ae0 tx=0x7fa35000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:13.738 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:13.734+0000 7fa366d1e700 1 --2- 172.21.15.59:0/736662593 >> [v2:172.21.15.181:6800/4287675481,v1:172.21.15.181:6801/4287675481] conn(0x7fa34c0ac090 0x7fa34c0ae4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:13.738 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:13.735+0000 7fa366d1e700 1 --2- 172.21.15.59:0/736662593 >> [v2:172.21.15.181:6800/4287675481,v1:172.21.15.181:6801/4287675481] conn(0x7fa34c0ac090 0x7fa34c0ae4d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:13.739 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:13.736+0000 7fa3577fe700 1 -- 172.21.15.59:0/736662593 <== osd.2 v2:172.21.15.181:6800/4287675481 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fa34c0aeba0 con 0x7fa34c0ac090 2023-12-11T17:45:13.757 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:13.753+0000 7fa368781700 1 -- 172.21.15.59:0/736662593 --> [v2:172.21.15.181:6800/4287675481,v1:172.21.15.181:6801/4287675481] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa360061cd0 con 0x7fa34c0ac090 2023-12-11T17:45:13.757 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:13.754+0000 7fa3577fe700 1 -- 172.21.15.59:0/736662593 <== osd.2 v2:172.21.15.181:6800/4287675481 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fa360061cd0 con 0x7fa34c0ac090 2023-12-11T17:45:13.758 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:13.755+0000 7fa368781700 1 -- 172.21.15.59:0/736662593 >> [v2:172.21.15.181:6800/4287675481,v1:172.21.15.181:6801/4287675481] conn(0x7fa34c0ac090 msgr2=0x7fa34c0ae4d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:13.758 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:13.755+0000 7fa368781700 1 --2- 172.21.15.59:0/736662593 >> [v2:172.21.15.181:6800/4287675481,v1:172.21.15.181:6801/4287675481] conn(0x7fa34c0ac090 0x7fa34c0ae4d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:13.758 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:13.755+0000 7fa368781700 1 -- 172.21.15.59:0/736662593 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fa34c0a1510 msgr2=0x7fa34c0a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:13.759 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:13.755+0000 7fa368781700 1 --2- 172.21.15.59:0/736662593 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fa34c0a1510 0x7fa34c0a39d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa360198ae0 tx=0x7fa35000b410 comp rx=0 tx=0).stop 2023-12-11T17:45:13.759 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:13.755+0000 7fa368781700 1 -- 172.21.15.59:0/736662593 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa3601004c0 msgr2=0x7fa36019d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:13.759 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:13.755+0000 7fa368781700 1 --2- 172.21.15.59:0/736662593 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa3601004c0 0x7fa36019d7e0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fa35c00e970 tx=0x7fa35c00ec80 comp rx=0 tx=0).stop 2023-12-11T17:45:13.759 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:13.755+0000 7fa366d1e700 1 -- 172.21.15.59:0/736662593 reap_dead start 2023-12-11T17:45:13.759 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:13.755+0000 7fa368781700 1 -- 172.21.15.59:0/736662593 shutdown_connections 2023-12-11T17:45:13.759 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:13.755+0000 7fa368781700 1 -- 172.21.15.59:0/736662593 >> 172.21.15.59:0/736662593 conn(0x7fa3600774a0 msgr2=0x7fa3600fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:45:13.759 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:13.755+0000 7fa368781700 1 -- 172.21.15.59:0/736662593 shutdown_connections 2023-12-11T17:45:13.759 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:13.755+0000 7fa368781700 1 -- 172.21.15.59:0/736662593 wait complete. 2023-12-11T17:45:13.815 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.786+0000 7f80fa58b700 1 -- 172.21.15.59:0/1410887314 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80f4106560 msgr2=0x7f80f4106940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:13.815 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.786+0000 7f80fa58b700 1 --2- 172.21.15.59:0/1410887314 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80f4106560 0x7f80f4106940 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f80e40099c0 tx=0x7f80e4009cd0 comp rx=0 tx=0).stop 2023-12-11T17:45:13.816 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.787+0000 7f80fa58b700 1 -- 172.21.15.59:0/1410887314 shutdown_connections 2023-12-11T17:45:13.816 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.787+0000 7f80fa58b700 1 --2- 172.21.15.59:0/1410887314 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f80f4100ef0 0x7f80f410d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:13.816 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.787+0000 7f80fa58b700 1 --2- 172.21.15.59:0/1410887314 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80f4106560 0x7f80f4106940 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:13.816 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.787+0000 7f80fa58b700 1 --2- 172.21.15.59:0/1410887314 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f80f4100530 0x7f80f41009b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:13.816 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.787+0000 7f80fa58b700 1 -- 172.21.15.59:0/1410887314 >> 172.21.15.59:0/1410887314 conn(0x7f80f4076570 msgr2=0x7f80f4076980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:45:13.816 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.787+0000 7f80fa58b700 1 -- 172.21.15.59:0/1410887314 shutdown_connections 2023-12-11T17:45:13.816 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.787+0000 7f80fa58b700 1 -- 172.21.15.59:0/1410887314 wait complete. 2023-12-11T17:45:13.816 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.787+0000 7f80fa58b700 1 Processor -- start 2023-12-11T17:45:13.816 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.787+0000 7f80fa58b700 1 -- start start 2023-12-11T17:45:13.816 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.788+0000 7f80fa58b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f80f4100530 0x7f80f419f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:13.816 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.788+0000 7f80fa58b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f80f4100ef0 0x7f80f419fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:13.816 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.788+0000 7f80fa58b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80f4106560 0x7f80f4199980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:13.816 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.788+0000 7f80fa58b700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80f419a130 con 0x7f80f4106560 2023-12-11T17:45:13.817 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.788+0000 7f80fa58b700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80f419a2a0 con 0x7f80f4100530 2023-12-11T17:45:13.817 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.788+0000 7f80fa58b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80f419a460 con 0x7f80f4100ef0 2023-12-11T17:45:13.817 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.788+0000 7f80f8b28700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80f4106560 0x7f80f4199980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:13.817 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.788+0000 7f80f8b28700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80f4106560 0x7f80f4199980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33672/0 (socket says 172.21.15.59:33672) 2023-12-11T17:45:13.817 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.788+0000 7f80f8b28700 1 -- 172.21.15.59:0/2071558011 learned_addr learned my addr 172.21.15.59:0/2071558011 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:45:13.817 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.788+0000 7f80f3fff700 1 --2- 172.21.15.59:0/2071558011 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f80f4100530 0x7f80f419f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:13.817 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.788+0000 7f80f37fe700 1 --2- 172.21.15.59:0/2071558011 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f80f4100ef0 0x7f80f419fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:13.817 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.788+0000 7f80f8b28700 1 -- 172.21.15.59:0/2071558011 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f80f4100ef0 msgr2=0x7f80f419fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:13.817 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.788+0000 7f80f8b28700 1 --2- 172.21.15.59:0/2071558011 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f80f4100ef0 0x7f80f419fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:13.817 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.788+0000 7f80f8b28700 1 -- 172.21.15.59:0/2071558011 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f80f4100530 msgr2=0x7f80f419f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:13.817 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.788+0000 7f80f8b28700 1 --2- 172.21.15.59:0/2071558011 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f80f4100530 0x7f80f419f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:13.817 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.788+0000 7f80f8b28700 1 -- 172.21.15.59:0/2071558011 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80e40095f0 con 0x7f80f4106560 2023-12-11T17:45:13.817 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.788+0000 7f80f8b28700 1 --2- 172.21.15.59:0/2071558011 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80f4106560 0x7f80f4199980 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f80ec00ada0 tx=0x7f80ec00c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:13.817 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.789+0000 7f80f17fa700 1 -- 172.21.15.59:0/2071558011 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80ec00f070 con 0x7f80f4106560 2023-12-11T17:45:13.817 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.789+0000 7f80f17fa700 1 -- 172.21.15.59:0/2071558011 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f80ec007b30 con 0x7f80f4106560 2023-12-11T17:45:13.817 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.789+0000 7f80fa58b700 1 -- 172.21.15.59:0/2071558011 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80f419a740 con 0x7f80f4106560 2023-12-11T17:45:13.818 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.789+0000 7f80f17fa700 1 -- 172.21.15.59:0/2071558011 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80ec005280 con 0x7f80f4106560 2023-12-11T17:45:13.818 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.789+0000 7f80fa58b700 1 -- 172.21.15.59:0/2071558011 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80f41a6850 con 0x7f80f4106560 2023-12-11T17:45:13.818 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.789+0000 7f80fa58b700 1 -- 172.21.15.59:0/2071558011 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f80f41117e0 con 0x7f80f4106560 2023-12-11T17:45:13.818 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.791+0000 7f80f17fa700 1 -- 172.21.15.59:0/2071558011 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f80ec00a080 con 0x7f80f4106560 2023-12-11T17:45:13.818 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.791+0000 7f80f17fa700 1 --2- 172.21.15.59:0/2071558011 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f80dc0a1510 0x7f80dc0a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:13.818 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.791+0000 7f80f17fa700 1 -- 172.21.15.59:0/2071558011 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f80ec0ccfa0 con 0x7f80f4106560 2023-12-11T17:45:13.818 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.792+0000 7f80f3fff700 1 --2- 172.21.15.59:0/2071558011 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f80dc0a1510 0x7f80dc0a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:13.818 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.792+0000 7f80f17fa700 1 --2- 172.21.15.59:0/2071558011 >> [v2:172.21.15.115:6816/1314971870,v1:172.21.15.115:6817/1314971870] conn(0x7f80dc0ac080 0x7f80dc0ae4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:13.818 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.792+0000 7f80f17fa700 1 -- 172.21.15.59:0/2071558011 --> [v2:172.21.15.115:6816/1314971870,v1:172.21.15.115:6817/1314971870] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f80dc0aeb90 con 0x7f80dc0ac080 2023-12-11T17:45:13.818 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.792+0000 7f80f17fa700 1 -- 172.21.15.59:0/2071558011 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7f80ec0cd3b0 con 0x7f80f4106560 2023-12-11T17:45:13.819 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.792+0000 7f80f3fff700 1 --2- 172.21.15.59:0/2071558011 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f80dc0a1510 0x7f80dc0a39d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f80e4009f50 tx=0x7f80e4005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:13.819 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.792+0000 7f80f37fe700 1 --2- 172.21.15.59:0/2071558011 >> [v2:172.21.15.115:6816/1314971870,v1:172.21.15.115:6817/1314971870] conn(0x7f80dc0ac080 0x7f80dc0ae4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:13.819 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.792+0000 7f80f37fe700 1 --2- 172.21.15.59:0/2071558011 >> [v2:172.21.15.115:6816/1314971870,v1:172.21.15.115:6817/1314971870] conn(0x7f80dc0ac080 0x7f80dc0ae4c0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:13.819 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.793+0000 7f80f17fa700 1 -- 172.21.15.59:0/2071558011 <== osd.7 v2:172.21.15.115:6816/1314971870 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f80dc0aeb90 con 0x7f80dc0ac080 2023-12-11T17:45:13.819 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.813+0000 7f80fa58b700 1 -- 172.21.15.59:0/2071558011 --> [v2:172.21.15.115:6816/1314971870,v1:172.21.15.115:6817/1314971870] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f80f410d360 con 0x7f80dc0ac080 2023-12-11T17:45:13.819 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.813+0000 7f80f17fa700 1 -- 172.21.15.59:0/2071558011 <== osd.7 v2:172.21.15.115:6816/1314971870 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f80f410d360 con 0x7f80dc0ac080 2023-12-11T17:45:13.819 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.814+0000 7f80fa58b700 1 -- 172.21.15.59:0/2071558011 >> [v2:172.21.15.115:6816/1314971870,v1:172.21.15.115:6817/1314971870] conn(0x7f80dc0ac080 msgr2=0x7f80dc0ae4c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:13.819 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.814+0000 7f80fa58b700 1 --2- 172.21.15.59:0/2071558011 >> [v2:172.21.15.115:6816/1314971870,v1:172.21.15.115:6817/1314971870] conn(0x7f80dc0ac080 0x7f80dc0ae4c0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:13.820 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.814+0000 7f80fa58b700 1 -- 172.21.15.59:0/2071558011 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f80dc0a1510 msgr2=0x7f80dc0a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:13.820 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.814+0000 7f80fa58b700 1 --2- 172.21.15.59:0/2071558011 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f80dc0a1510 0x7f80dc0a39d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f80e4009f50 tx=0x7f80e4005fb0 comp rx=0 tx=0).stop 2023-12-11T17:45:13.820 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.814+0000 7f80fa58b700 1 -- 172.21.15.59:0/2071558011 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80f4106560 msgr2=0x7f80f4199980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:13.820 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.814+0000 7f80fa58b700 1 --2- 172.21.15.59:0/2071558011 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80f4106560 0x7f80f4199980 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f80ec00ada0 tx=0x7f80ec00c330 comp rx=0 tx=0).stop 2023-12-11T17:45:13.820 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.814+0000 7f80f8b28700 1 -- 172.21.15.59:0/2071558011 reap_dead start 2023-12-11T17:45:13.820 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.814+0000 7f80fa58b700 1 -- 172.21.15.59:0/2071558011 shutdown_connections 2023-12-11T17:45:13.820 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.814+0000 7f80fa58b700 1 -- 172.21.15.59:0/2071558011 >> 172.21.15.59:0/2071558011 conn(0x7f80f4076570 msgr2=0x7f80f410b980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:45:13.820 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.814+0000 7f80fa58b700 1 -- 172.21.15.59:0/2071558011 shutdown_connections 2023-12-11T17:45:13.820 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:12.814+0000 7f80fa58b700 1 -- 172.21.15.59:0/2071558011 wait complete. 2023-12-11T17:45:13.821 INFO:teuthology.orchestra.run.smithi059.stdout:34359738437 2023-12-11T17:45:13.821 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell --fsid d0a07eaa-984b-11ee-95a3-87774f69a715 -- ceph osd last-stat-seq osd.0 2023-12-11T17:45:14.026 INFO:teuthology.orchestra.run.smithi059.stdout:158913789999 2023-12-11T17:45:14.027 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell --fsid d0a07eaa-984b-11ee-95a3-87774f69a715 -- ceph osd last-stat-seq osd.5 2023-12-11T17:45:14.086 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:45:13 smithi059 ceph-mon[98847]: pgmap v285: 129 pgs: 129 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 341 B/s wr, 6 op/s 2023-12-11T17:45:14.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:45:13 smithi181 ceph-mon[103627]: pgmap v285: 129 pgs: 129 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 341 B/s wr, 6 op/s 2023-12-11T17:45:14.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:45:13 smithi115 ceph-mon[102748]: pgmap v285: 129 pgs: 129 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 341 B/s wr, 6 op/s 2023-12-11T17:45:14.344 INFO:teuthology.orchestra.run.smithi059.stdout:55834574913 2023-12-11T17:45:14.344 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell --fsid d0a07eaa-984b-11ee-95a3-87774f69a715 -- ceph osd last-stat-seq osd.1 2023-12-11T17:45:14.840 INFO:teuthology.orchestra.run.smithi059.stdout:244813135907 2023-12-11T17:45:14.841 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell --fsid d0a07eaa-984b-11ee-95a3-87774f69a715 -- ceph osd last-stat-seq osd.8 2023-12-11T17:45:14.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:45:14 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]: dispatch 2023-12-11T17:45:14.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:45:14 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 1]}]: dispatch 2023-12-11T17:45:14.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:45:14 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2023-12-11T17:45:14.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:45:14 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 11]}]: dispatch 2023-12-11T17:45:14.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:45:14 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]: dispatch 2023-12-11T17:45:14.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:45:14 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 7]}]: dispatch 2023-12-11T17:45:14.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:45:14 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 6, 3, 8]}]: dispatch 2023-12-11T17:45:14.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:45:14 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2023-12-11T17:45:14.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:45:14 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]: dispatch 2023-12-11T17:45:14.960 INFO:teuthology.orchestra.run.smithi059.stdout:77309411389 2023-12-11T17:45:14.960 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell --fsid d0a07eaa-984b-11ee-95a3-87774f69a715 -- ceph osd last-stat-seq osd.2 2023-12-11T17:45:15.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:45:14 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]: dispatch 2023-12-11T17:45:15.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:45:14 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 1]}]: dispatch 2023-12-11T17:45:15.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:45:14 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2023-12-11T17:45:15.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:45:14 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 11]}]: dispatch 2023-12-11T17:45:15.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:45:14 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]: dispatch 2023-12-11T17:45:15.146 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:45:14 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 7]}]: dispatch 2023-12-11T17:45:15.146 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:45:14 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 6, 3, 8]}]: dispatch 2023-12-11T17:45:15.146 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:45:14 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2023-12-11T17:45:15.146 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:45:14 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]: dispatch 2023-12-11T17:45:15.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:45:14 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]: dispatch 2023-12-11T17:45:15.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:45:14 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 1]}]: dispatch 2023-12-11T17:45:15.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:45:14 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2023-12-11T17:45:15.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:45:14 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 11]}]: dispatch 2023-12-11T17:45:15.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:45:14 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]: dispatch 2023-12-11T17:45:15.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:45:14 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 7]}]: dispatch 2023-12-11T17:45:15.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:45:14 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 6, 3, 8]}]: dispatch 2023-12-11T17:45:15.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:45:14 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2023-12-11T17:45:15.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:45:14 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]: dispatch 2023-12-11T17:45:15.422 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/mon.a/config 2023-12-11T17:45:15.474 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/mon.a/config 2023-12-11T17:45:15.499 INFO:teuthology.orchestra.run.smithi059.stdout:214748364839 2023-12-11T17:45:15.499 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell --fsid d0a07eaa-984b-11ee-95a3-87774f69a715 -- ceph osd last-stat-seq osd.7 2023-12-11T17:45:15.577 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/mon.a/config 2023-12-11T17:45:15.578 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/mon.a/config 2023-12-11T17:45:15.606 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/mon.a/config 2023-12-11T17:45:15.606 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/mon.a/config 2023-12-11T17:45:15.847 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/mon.a/config 2023-12-11T17:45:16.030 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:45:15 smithi059 ceph-mon[98847]: pgmap v286: 129 pgs: 129 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 341 B/s wr, 6 op/s 2023-12-11T17:45:16.030 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:45:15 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]': finished 2023-12-11T17:45:16.030 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:45:15 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 1]}]': finished 2023-12-11T17:45:16.030 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:45:15 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2023-12-11T17:45:16.030 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:45:15 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 11]}]': finished 2023-12-11T17:45:16.030 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:45:15 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]': finished 2023-12-11T17:45:16.030 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:45:15 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 7]}]': finished 2023-12-11T17:45:16.030 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:45:15 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 6, 3, 8]}]': finished 2023-12-11T17:45:16.030 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:45:15 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2023-12-11T17:45:16.030 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:45:15 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]': finished 2023-12-11T17:45:16.031 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:45:15 smithi059 ceph-mon[98847]: osdmap e87: 12 total, 12 up, 12 in 2023-12-11T17:45:16.039 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/mon.a/config 2023-12-11T17:45:16.040 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/mon.a/config 2023-12-11T17:45:16.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:45:15 smithi181 ceph-mon[103627]: pgmap v286: 129 pgs: 129 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 341 B/s wr, 6 op/s 2023-12-11T17:45:16.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:45:15 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]': finished 2023-12-11T17:45:16.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:45:15 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 1]}]': finished 2023-12-11T17:45:16.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:45:15 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2023-12-11T17:45:16.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:45:15 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 11]}]': finished 2023-12-11T17:45:16.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:45:15 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]': finished 2023-12-11T17:45:16.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:45:15 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 7]}]': finished 2023-12-11T17:45:16.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:45:15 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 6, 3, 8]}]': finished 2023-12-11T17:45:16.146 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:45:15 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2023-12-11T17:45:16.146 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:45:15 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]': finished 2023-12-11T17:45:16.146 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:45:15 smithi181 ceph-mon[103627]: osdmap e87: 12 total, 12 up, 12 in 2023-12-11T17:45:16.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:45:15 smithi115 ceph-mon[102748]: pgmap v286: 129 pgs: 129 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 341 B/s wr, 6 op/s 2023-12-11T17:45:16.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:45:15 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]': finished 2023-12-11T17:45:16.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:45:15 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 1]}]': finished 2023-12-11T17:45:16.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:45:15 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2023-12-11T17:45:16.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:45:15 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 11]}]': finished 2023-12-11T17:45:16.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:45:15 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]': finished 2023-12-11T17:45:16.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:45:15 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 7]}]': finished 2023-12-11T17:45:16.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:45:15 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 6, 3, 8]}]': finished 2023-12-11T17:45:16.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:45:15 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2023-12-11T17:45:16.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:45:15 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]': finished 2023-12-11T17:45:16.284 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:45:15 smithi115 ceph-mon[102748]: osdmap e87: 12 total, 12 up, 12 in 2023-12-11T17:45:17.128 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/mon.a/config 2023-12-11T17:45:17.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:45:16 smithi181 ceph-mon[103627]: osdmap e88: 12 total, 12 up, 12 in 2023-12-11T17:45:17.155 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:45:16 smithi059 ceph-mon[98847]: osdmap e88: 12 total, 12 up, 12 in 2023-12-11T17:45:17.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:45:16 smithi115 ceph-mon[102748]: osdmap e88: 12 total, 12 up, 12 in 2023-12-11T17:45:18.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:45:17 smithi181 ceph-mon[103627]: pgmap v289: 129 pgs: 2 peering, 127 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 4 op/s 2023-12-11T17:45:18.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:45:17 smithi059 ceph-mon[98847]: pgmap v289: 129 pgs: 2 peering, 127 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 4 op/s 2023-12-11T17:45:18.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:45:17 smithi115 ceph-mon[102748]: pgmap v289: 129 pgs: 2 peering, 127 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 4 op/s 2023-12-11T17:45:18.890 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/mon.a/config 2023-12-11T17:45:18.986 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:18.981+0000 7f09dde8f700 1 -- 172.21.15.59:0/1578426251 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f09d8100930 msgr2=0x7f09d810caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:18.986 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:18.981+0000 7f09dde8f700 1 --2- 172.21.15.59:0/1578426251 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f09d8100930 0x7f09d810caf0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f09d000b0a0 tx=0x7f09d000b3b0 comp rx=0 tx=0).stop 2023-12-11T17:45:18.986 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:18.981+0000 7f09dde8f700 1 -- 172.21.15.59:0/1578426251 shutdown_connections 2023-12-11T17:45:18.986 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:18.981+0000 7f09dde8f700 1 --2- 172.21.15.59:0/1578426251 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f09d8100930 0x7f09d810caf0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:18.986 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:18.981+0000 7f09dde8f700 1 --2- 172.21.15.59:0/1578426251 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f09d8104c50 0x7f09d8105030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:18.987 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:18.981+0000 7f09dde8f700 1 --2- 172.21.15.59:0/1578426251 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09d80fff70 0x7f09d81003f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:18.987 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:18.981+0000 7f09dde8f700 1 -- 172.21.15.59:0/1578426251 >> 172.21.15.59:0/1578426251 conn(0x7f09d80757f0 msgr2=0x7f09d8075c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:45:18.987 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:18.981+0000 7f09dde8f700 1 -- 172.21.15.59:0/1578426251 shutdown_connections 2023-12-11T17:45:18.987 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:18.981+0000 7f09dde8f700 1 -- 172.21.15.59:0/1578426251 wait complete. 2023-12-11T17:45:18.987 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:18.982+0000 7f09dde8f700 1 Processor -- start 2023-12-11T17:45:18.987 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:18.982+0000 7f09dde8f700 1 -- start start 2023-12-11T17:45:18.987 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:18.982+0000 7f09dde8f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f09d80fff70 0x7f09d8101950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:18.987 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:18.982+0000 7f09dde8f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09d8100930 0x7f09d8101e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:18.988 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:18.982+0000 7f09dde8f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f09d8104c50 0x7f09d81023d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:18.988 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:18.982+0000 7f09dde8f700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09d81a5f00 con 0x7f09d80fff70 2023-12-11T17:45:18.988 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:18.982+0000 7f09dde8f700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09d81a6070 con 0x7f09d8104c50 2023-12-11T17:45:18.988 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:18.982+0000 7f09dde8f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09d81a6230 con 0x7f09d8100930 2023-12-11T17:45:18.988 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:18.982+0000 7f09d77fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f09d80fff70 0x7f09d8101950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:18.988 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:18.982+0000 7f09d77fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f09d80fff70 0x7f09d8101950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33716/0 (socket says 172.21.15.59:33716) 2023-12-11T17:45:18.988 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:18.982+0000 7f09d77fe700 1 -- 172.21.15.59:0/1441713082 learned_addr learned my addr 172.21.15.59:0/1441713082 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:45:18.989 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:18.982+0000 7f09d7fff700 1 --2- 172.21.15.59:0/1441713082 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f09d8104c50 0x7f09d81023d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:18.989 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:18.982+0000 7f09d6ffd700 1 --2- 172.21.15.59:0/1441713082 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09d8100930 0x7f09d8101e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:18.989 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:18.982+0000 7f09d77fe700 1 -- 172.21.15.59:0/1441713082 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09d8100930 msgr2=0x7f09d8101e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:18.989 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:18.982+0000 7f09d77fe700 1 --2- 172.21.15.59:0/1441713082 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09d8100930 0x7f09d8101e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:18.989 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:18.982+0000 7f09d77fe700 1 -- 172.21.15.59:0/1441713082 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f09d8104c50 msgr2=0x7f09d81023d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:18.989 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:18.982+0000 7f09d77fe700 1 --2- 172.21.15.59:0/1441713082 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f09d8104c50 0x7f09d81023d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:18.989 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:18.983+0000 7f09d77fe700 1 -- 172.21.15.59:0/1441713082 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09d0009d00 con 0x7f09d80fff70 2023-12-11T17:45:18.989 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:18.983+0000 7f09d6ffd700 1 --2- 172.21.15.59:0/1441713082 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09d8100930 0x7f09d8101e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T17:45:18.989 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:18.983+0000 7f09d77fe700 1 --2- 172.21.15.59:0/1441713082 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f09d80fff70 0x7f09d8101950 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f09c800e970 tx=0x7f09c800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:18.989 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:18.983+0000 7f09d4ff9700 1 -- 172.21.15.59:0/1441713082 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09c8009790 con 0x7f09d80fff70 2023-12-11T17:45:18.989 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:18.983+0000 7f09d4ff9700 1 -- 172.21.15.59:0/1441713082 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f09c800eec0 con 0x7f09d80fff70 2023-12-11T17:45:18.990 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:18.983+0000 7f09dde8f700 1 -- 172.21.15.59:0/1441713082 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09d81a6510 con 0x7f09d80fff70 2023-12-11T17:45:18.990 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:18.983+0000 7f09d4ff9700 1 -- 172.21.15.59:0/1441713082 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09c8003cf0 con 0x7f09d80fff70 2023-12-11T17:45:18.990 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:18.983+0000 7f09dde8f700 1 -- 172.21.15.59:0/1441713082 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09d81a6a10 con 0x7f09d80fff70 2023-12-11T17:45:18.991 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:18.984+0000 7f09dde8f700 1 -- 172.21.15.59:0/1441713082 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09d8109170 con 0x7f09d80fff70 2023-12-11T17:45:18.991 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:18.985+0000 7f09d4ff9700 1 -- 172.21.15.59:0/1441713082 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f09c800ccd0 con 0x7f09d80fff70 2023-12-11T17:45:18.991 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:18.986+0000 7f09d4ff9700 1 --2- 172.21.15.59:0/1441713082 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f09c00a1510 0x7f09c00a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:18.991 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:18.986+0000 7f09d6ffd700 1 --2- 172.21.15.59:0/1441713082 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f09c00a1510 0x7f09c00a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:18.991 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:18.986+0000 7f09d4ff9700 1 -- 172.21.15.59:0/1441713082 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f09c8014070 con 0x7f09d80fff70 2023-12-11T17:45:18.991 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:18.986+0000 7f09d6ffd700 1 --2- 172.21.15.59:0/1441713082 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f09c00a1510 0x7f09c00a39d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f09cc005950 tx=0x7f09cc0058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:18.992 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:18.990+0000 7f09d4ff9700 1 -- 172.21.15.59:0/1441713082 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f09c8096590 con 0x7f09d80fff70 2023-12-11T17:45:19.197 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:19.194+0000 7f09dde8f700 1 -- 172.21.15.59:0/1441713082 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) v1 -- 0x7f09d8101390 con 0x7f09d80fff70 2023-12-11T17:45:19.197 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:19.195+0000 7f09d4ff9700 1 -- 172.21.15.59:0/1441713082 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f09c8094e60 con 0x7f09d80fff70 2023-12-11T17:45:19.197 INFO:teuthology.orchestra.run.smithi059.stdout:279172874272 2023-12-11T17:45:19.199 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:19.196+0000 7f09dde8f700 1 -- 172.21.15.59:0/1441713082 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f09c00a1510 msgr2=0x7f09c00a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:19.200 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:19.196+0000 7f09dde8f700 1 --2- 172.21.15.59:0/1441713082 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f09c00a1510 0x7f09c00a39d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f09cc005950 tx=0x7f09cc0058e0 comp rx=0 tx=0).stop 2023-12-11T17:45:19.200 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:19.196+0000 7f09dde8f700 1 -- 172.21.15.59:0/1441713082 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f09d80fff70 msgr2=0x7f09d8101950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:19.200 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:19.196+0000 7f09dde8f700 1 --2- 172.21.15.59:0/1441713082 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f09d80fff70 0x7f09d8101950 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f09c800e970 tx=0x7f09c800ec80 comp rx=0 tx=0).stop 2023-12-11T17:45:19.200 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:19.197+0000 7f09dde8f700 1 -- 172.21.15.59:0/1441713082 shutdown_connections 2023-12-11T17:45:19.200 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:19.197+0000 7f09dde8f700 1 --2- 172.21.15.59:0/1441713082 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f09c00a1510 0x7f09c00a39d0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:19.200 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:19.197+0000 7f09dde8f700 1 --2- 172.21.15.59:0/1441713082 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f09d8104c50 0x7f09d81023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:19.200 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:19.197+0000 7f09dde8f700 1 --2- 172.21.15.59:0/1441713082 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f09d80fff70 0x7f09d8101950 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:19.200 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:19.197+0000 7f09dde8f700 1 --2- 172.21.15.59:0/1441713082 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09d8100930 0x7f09d8101e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:19.200 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:19.197+0000 7f09dde8f700 1 -- 172.21.15.59:0/1441713082 >> 172.21.15.59:0/1441713082 conn(0x7f09d80757f0 msgr2=0x7f09d810dc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:45:19.200 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:19.197+0000 7f09dde8f700 1 -- 172.21.15.59:0/1441713082 shutdown_connections 2023-12-11T17:45:19.200 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:19.197+0000 7f09dde8f700 1 -- 172.21.15.59:0/1441713082 wait complete. 2023-12-11T17:45:19.996 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:45:19 smithi059 ceph-mon[98847]: pgmap v290: 129 pgs: 2 peering, 127 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 383 B/s wr, 6 op/s; 72 B/s, 0 objects/s recovering 2023-12-11T17:45:19.996 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:45:19 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/1441713082' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-12-11T17:45:20.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:45:19 smithi181 ceph-mon[103627]: pgmap v290: 129 pgs: 2 peering, 127 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 383 B/s wr, 6 op/s; 72 B/s, 0 objects/s recovering 2023-12-11T17:45:20.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:45:19 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/1441713082' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-12-11T17:45:20.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:45:19 smithi115 ceph-mon[102748]: pgmap v290: 129 pgs: 2 peering, 127 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 383 B/s wr, 6 op/s; 72 B/s, 0 objects/s recovering 2023-12-11T17:45:20.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:45:19 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/1441713082' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-12-11T17:45:20.337 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:20.333+0000 7f1642bdd700 1 -- 172.21.15.59:0/1239925891 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f163c100e80 msgr2=0x7f163c10d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:20.337 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:20.333+0000 7f1642bdd700 1 --2- 172.21.15.59:0/1239925891 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f163c100e80 0x7f163c10d370 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f163400b0d0 tx=0x7f163400b3e0 comp rx=0 tx=0).stop 2023-12-11T17:45:20.338 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:20.333+0000 7f1642bdd700 1 -- 172.21.15.59:0/1239925891 shutdown_connections 2023-12-11T17:45:20.338 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:20.333+0000 7f1642bdd700 1 --2- 172.21.15.59:0/1239925891 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f163c100e80 0x7f163c10d370 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:20.338 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:20.333+0000 7f1642bdd700 1 --2- 172.21.15.59:0/1239925891 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f163c105470 0x7f163c105850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:20.338 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:20.333+0000 7f1642bdd700 1 --2- 172.21.15.59:0/1239925891 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f163c1004c0 0x7f163c100940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:20.338 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:20.333+0000 7f1642bdd700 1 -- 172.21.15.59:0/1239925891 >> 172.21.15.59:0/1239925891 conn(0x7f163c0774a0 msgr2=0x7f163c0778b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:45:20.338 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:20.333+0000 7f1642bdd700 1 -- 172.21.15.59:0/1239925891 shutdown_connections 2023-12-11T17:45:20.338 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:20.333+0000 7f1642bdd700 1 -- 172.21.15.59:0/1239925891 wait complete. 2023-12-11T17:45:20.338 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:20.334+0000 7f1642bdd700 1 Processor -- start 2023-12-11T17:45:20.338 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:20.334+0000 7f1642bdd700 1 -- start start 2023-12-11T17:45:20.338 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:20.334+0000 7f1642bdd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f163c1004c0 0x7f163c1989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:20.338 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:20.334+0000 7f1642bdd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f163c100e80 0x7f163c198ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:20.338 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:20.334+0000 7f1642bdd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f163c105470 0x7f163c1a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:20.339 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:20.334+0000 7f1642bdd700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f163c199730 con 0x7f163c1004c0 2023-12-11T17:45:20.339 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:20.334+0000 7f1642bdd700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f163c1a3f60 con 0x7f163c105470 2023-12-11T17:45:20.339 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:20.334+0000 7f1642bdd700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f163c1a40a0 con 0x7f163c100e80 2023-12-11T17:45:20.339 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:20.334+0000 7f1640979700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f163c1004c0 0x7f163c1989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:20.339 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:20.334+0000 7f1640979700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f163c1004c0 0x7f163c1989a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33726/0 (socket says 172.21.15.59:33726) 2023-12-11T17:45:20.339 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:20.334+0000 7f1640979700 1 -- 172.21.15.59:0/2893281172 learned_addr learned my addr 172.21.15.59:0/2893281172 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:45:20.339 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:20.334+0000 7f164117a700 1 --2- 172.21.15.59:0/2893281172 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f163c105470 0x7f163c1a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:20.339 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:20.334+0000 7f163bfff700 1 --2- 172.21.15.59:0/2893281172 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f163c100e80 0x7f163c198ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:20.339 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:20.334+0000 7f1640979700 1 -- 172.21.15.59:0/2893281172 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f163c100e80 msgr2=0x7f163c198ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:20.339 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:20.334+0000 7f1640979700 1 --2- 172.21.15.59:0/2893281172 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f163c100e80 0x7f163c198ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:20.339 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:20.334+0000 7f1640979700 1 -- 172.21.15.59:0/2893281172 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f163c105470 msgr2=0x7f163c1a3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:20.339 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:20.334+0000 7f1640979700 1 --2- 172.21.15.59:0/2893281172 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f163c105470 0x7f163c1a3a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:20.339 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:20.334+0000 7f1640979700 1 -- 172.21.15.59:0/2893281172 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1634009d00 con 0x7f163c1004c0 2023-12-11T17:45:20.339 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:20.335+0000 7f1640979700 1 --2- 172.21.15.59:0/2893281172 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f163c1004c0 0x7f163c1989a0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f163000b530 tx=0x7f163000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:20.340 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:20.335+0000 7f1639ffb700 1 -- 172.21.15.59:0/2893281172 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1630011830 con 0x7f163c1004c0 2023-12-11T17:45:20.340 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:20.335+0000 7f1642bdd700 1 -- 172.21.15.59:0/2893281172 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f163c1a4300 con 0x7f163c1004c0 2023-12-11T17:45:20.340 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:20.335+0000 7f1639ffb700 1 -- 172.21.15.59:0/2893281172 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1630011e50 con 0x7f163c1004c0 2023-12-11T17:45:20.340 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:20.335+0000 7f1639ffb700 1 -- 172.21.15.59:0/2893281172 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1630010de0 con 0x7f163c1004c0 2023-12-11T17:45:20.340 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:20.335+0000 7f1642bdd700 1 -- 172.21.15.59:0/2893281172 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f163c1a4800 con 0x7f163c1004c0 2023-12-11T17:45:20.341 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:20.336+0000 7f1642bdd700 1 -- 172.21.15.59:0/2893281172 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f163c108a10 con 0x7f163c1004c0 2023-12-11T17:45:20.341 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:20.337+0000 7f1639ffb700 1 -- 172.21.15.59:0/2893281172 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f1630010480 con 0x7f163c1004c0 2023-12-11T17:45:20.341 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:20.337+0000 7f1639ffb700 1 --2- 172.21.15.59:0/2893281172 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f162c0a1510 0x7f162c0a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:20.341 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:20.338+0000 7f163bfff700 1 --2- 172.21.15.59:0/2893281172 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f162c0a1510 0x7f162c0a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:20.341 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:20.338+0000 7f1639ffb700 1 -- 172.21.15.59:0/2893281172 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f16300ccab0 con 0x7f163c1004c0 2023-12-11T17:45:20.341 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:20.338+0000 7f163bfff700 1 --2- 172.21.15.59:0/2893281172 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f162c0a1510 0x7f162c0a39d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f163c06a000 tx=0x7f162800a300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:20.344 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:20.342+0000 7f1639ffb700 1 -- 172.21.15.59:0/2893281172 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f1630095dc0 con 0x7f163c1004c0 2023-12-11T17:45:20.547 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:20.544+0000 7f1642bdd700 1 -- 172.21.15.59:0/2893281172 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) v1 -- 0x7f163c04f290 con 0x7f163c1004c0 2023-12-11T17:45:20.547 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:20.545+0000 7f1639ffb700 1 -- 172.21.15.59:0/2893281172 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7f16300993e0 con 0x7f163c1004c0 2023-12-11T17:45:20.547 INFO:teuthology.orchestra.run.smithi059.stdout:300647710746 2023-12-11T17:45:20.549 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:20.547+0000 7f1642bdd700 1 -- 172.21.15.59:0/2893281172 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f162c0a1510 msgr2=0x7f162c0a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:20.550 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:20.547+0000 7f1642bdd700 1 --2- 172.21.15.59:0/2893281172 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f162c0a1510 0x7f162c0a39d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f163c06a000 tx=0x7f162800a300 comp rx=0 tx=0).stop 2023-12-11T17:45:20.550 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:20.547+0000 7f1642bdd700 1 -- 172.21.15.59:0/2893281172 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f163c1004c0 msgr2=0x7f163c1989a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:20.550 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:20.547+0000 7f1642bdd700 1 --2- 172.21.15.59:0/2893281172 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f163c1004c0 0x7f163c1989a0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f163000b530 tx=0x7f163000b840 comp rx=0 tx=0).stop 2023-12-11T17:45:20.550 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:20.547+0000 7f1642bdd700 1 -- 172.21.15.59:0/2893281172 shutdown_connections 2023-12-11T17:45:20.550 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:20.547+0000 7f1642bdd700 1 --2- 172.21.15.59:0/2893281172 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f162c0a1510 0x7f162c0a39d0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:20.550 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:20.547+0000 7f1642bdd700 1 --2- 172.21.15.59:0/2893281172 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f163c105470 0x7f163c1a3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:20.551 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:20.547+0000 7f1642bdd700 1 --2- 172.21.15.59:0/2893281172 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f163c1004c0 0x7f163c1989a0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:20.551 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:20.547+0000 7f1642bdd700 1 --2- 172.21.15.59:0/2893281172 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f163c100e80 0x7f163c198ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:20.551 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:20.547+0000 7f1642bdd700 1 -- 172.21.15.59:0/2893281172 >> 172.21.15.59:0/2893281172 conn(0x7f163c0774a0 msgr2=0x7f163c0fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:45:20.551 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:20.547+0000 7f1642bdd700 1 -- 172.21.15.59:0/2893281172 shutdown_connections 2023-12-11T17:45:20.551 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:20.547+0000 7f1642bdd700 1 -- 172.21.15.59:0/2893281172 wait complete. 2023-12-11T17:45:21.022 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:45:20 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/2893281172' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-12-11T17:45:21.038 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:21.035+0000 7f2930111700 1 -- 172.21.15.59:0/509220814 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2928106560 msgr2=0x7f2928106940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:21.038 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:21.035+0000 7f2930111700 1 --2- 172.21.15.59:0/509220814 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2928106560 0x7f2928106940 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f291800b0d0 tx=0x7f291800b3e0 comp rx=0 tx=0).stop 2023-12-11T17:45:21.038 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:21.035+0000 7f2930111700 1 -- 172.21.15.59:0/509220814 shutdown_connections 2023-12-11T17:45:21.038 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:21.035+0000 7f2930111700 1 --2- 172.21.15.59:0/509220814 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2928100ef0 0x7f292810d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:21.038 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:21.035+0000 7f2930111700 1 --2- 172.21.15.59:0/509220814 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2928106560 0x7f2928106940 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:21.038 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:21.035+0000 7f2930111700 1 --2- 172.21.15.59:0/509220814 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2928100530 0x7f29281009b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:21.038 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:21.035+0000 7f2930111700 1 -- 172.21.15.59:0/509220814 >> 172.21.15.59:0/509220814 conn(0x7f2928076570 msgr2=0x7f2928076980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:45:21.039 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:21.035+0000 7f2930111700 1 -- 172.21.15.59:0/509220814 shutdown_connections 2023-12-11T17:45:21.039 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:21.035+0000 7f2930111700 1 -- 172.21.15.59:0/509220814 wait complete. 2023-12-11T17:45:21.041 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:21.036+0000 7f2930111700 1 Processor -- start 2023-12-11T17:45:21.041 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:21.036+0000 7f2930111700 1 -- start start 2023-12-11T17:45:21.041 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:21.037+0000 7f2930111700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2928100530 0x7f292819c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:21.041 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:21.037+0000 7f2930111700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2928100ef0 0x7f292819cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:21.041 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:21.037+0000 7f2930111700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2928106560 0x7f2928196830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:21.041 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:21.037+0000 7f2930111700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2928196ea0 con 0x7f2928100ef0 2023-12-11T17:45:21.041 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:21.037+0000 7f2930111700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2928197010 con 0x7f2928100530 2023-12-11T17:45:21.041 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:21.037+0000 7f2930111700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29281971d0 con 0x7f2928106560 2023-12-11T17:45:21.041 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:21.037+0000 7f292e6ae700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2928106560 0x7f2928196830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:21.041 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:21.037+0000 7f292e6ae700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2928106560 0x7f2928196830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.59:38558/0 (socket says 172.21.15.59:38558) 2023-12-11T17:45:21.041 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:21.037+0000 7f292e6ae700 1 -- 172.21.15.59:0/1382377916 learned_addr learned my addr 172.21.15.59:0/1382377916 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:45:21.041 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:21.037+0000 7f292dead700 1 --2- 172.21.15.59:0/1382377916 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2928100530 0x7f292819c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:21.041 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:21.037+0000 7f292d6ac700 1 --2- 172.21.15.59:0/1382377916 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2928100ef0 0x7f292819cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:21.042 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:21.037+0000 7f292e6ae700 1 -- 172.21.15.59:0/1382377916 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2928100530 msgr2=0x7f292819c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:21.042 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:21.037+0000 7f292e6ae700 1 --2- 172.21.15.59:0/1382377916 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2928100530 0x7f292819c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:21.042 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:21.037+0000 7f292e6ae700 1 -- 172.21.15.59:0/1382377916 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2928100ef0 msgr2=0x7f292819cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:21.042 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:21.037+0000 7f292e6ae700 1 --2- 172.21.15.59:0/1382377916 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2928100ef0 0x7f292819cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:21.042 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:21.037+0000 7f292e6ae700 1 -- 172.21.15.59:0/1382377916 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2918009d00 con 0x7f2928106560 2023-12-11T17:45:21.042 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:21.038+0000 7f292e6ae700 1 --2- 172.21.15.59:0/1382377916 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2928106560 0x7f2928196830 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f292000d980 tx=0x7f292000dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:21.042 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:21.038+0000 7f2926ffd700 1 -- 172.21.15.59:0/1382377916 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2920004a30 con 0x7f2928106560 2023-12-11T17:45:21.042 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:21.038+0000 7f2926ffd700 1 -- 172.21.15.59:0/1382377916 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f292000b5c0 con 0x7f2928106560 2023-12-11T17:45:21.042 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:21.038+0000 7f2930111700 1 -- 172.21.15.59:0/1382377916 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29281974b0 con 0x7f2928106560 2023-12-11T17:45:21.042 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:21.038+0000 7f2926ffd700 1 -- 172.21.15.59:0/1382377916 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2920009b50 con 0x7f2928106560 2023-12-11T17:45:21.042 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:21.038+0000 7f2930111700 1 -- 172.21.15.59:0/1382377916 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2928069140 con 0x7f2928106560 2023-12-11T17:45:21.043 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:21.039+0000 7f2930111700 1 -- 172.21.15.59:0/1382377916 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2928101bd0 con 0x7f2928106560 2023-12-11T17:45:21.044 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:21.041+0000 7f2926ffd700 1 -- 172.21.15.59:0/1382377916 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f2920009cb0 con 0x7f2928106560 2023-12-11T17:45:21.044 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:21.042+0000 7f2926ffd700 1 --2- 172.21.15.59:0/1382377916 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f29100a1590 0x7f29100a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:21.044 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:21.042+0000 7f2926ffd700 1 -- 172.21.15.59:0/1382377916 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f29200cd310 con 0x7f2928106560 2023-12-11T17:45:21.044 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:21.042+0000 7f292dead700 1 --2- 172.21.15.59:0/1382377916 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f29100a1590 0x7f29100a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:21.045 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:21.042+0000 7f292dead700 1 --2- 172.21.15.59:0/1382377916 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f29100a1590 0x7f29100a3a50 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2918009fd0 tx=0x7f291800b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:21.047 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:21.045+0000 7f2926ffd700 1 -- 172.21.15.59:0/1382377916 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f2920096590 con 0x7f2928106560 2023-12-11T17:45:21.098 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/mon.a/config 2023-12-11T17:45:21.252 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:21.249+0000 7f2930111700 1 -- 172.21.15.59:0/1382377916 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f2928197d20 con 0x7f2928106560 2023-12-11T17:45:21.252 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:21.250+0000 7f2926ffd700 1 -- 172.21.15.59:0/1382377916 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2920017030 con 0x7f2928106560 2023-12-11T17:45:21.252 INFO:teuthology.orchestra.run.smithi059.stdout:107374182457 2023-12-11T17:45:21.255 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:21.252+0000 7f2930111700 1 -- 172.21.15.59:0/1382377916 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f29100a1590 msgr2=0x7f29100a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:21.255 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:21.252+0000 7f2930111700 1 --2- 172.21.15.59:0/1382377916 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f29100a1590 0x7f29100a3a50 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2918009fd0 tx=0x7f291800b040 comp rx=0 tx=0).stop 2023-12-11T17:45:21.255 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:21.252+0000 7f2930111700 1 -- 172.21.15.59:0/1382377916 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2928106560 msgr2=0x7f2928196830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:21.255 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:21.252+0000 7f2930111700 1 --2- 172.21.15.59:0/1382377916 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2928106560 0x7f2928196830 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f292000d980 tx=0x7f292000dc90 comp rx=0 tx=0).stop 2023-12-11T17:45:21.255 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:21.252+0000 7f2930111700 1 -- 172.21.15.59:0/1382377916 shutdown_connections 2023-12-11T17:45:21.255 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:21.252+0000 7f2930111700 1 --2- 172.21.15.59:0/1382377916 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f29100a1590 0x7f29100a3a50 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:21.255 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:21.252+0000 7f2930111700 1 --2- 172.21.15.59:0/1382377916 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2928106560 0x7f2928196830 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:21.255 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:21.252+0000 7f2930111700 1 --2- 172.21.15.59:0/1382377916 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2928100530 0x7f292819c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:21.255 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:21.252+0000 7f2930111700 1 --2- 172.21.15.59:0/1382377916 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2928100ef0 0x7f292819cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:21.255 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:21.252+0000 7f2930111700 1 -- 172.21.15.59:0/1382377916 >> 172.21.15.59:0/1382377916 conn(0x7f2928076570 msgr2=0x7f29280ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:45:21.255 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:21.252+0000 7f2930111700 1 -- 172.21.15.59:0/1382377916 shutdown_connections 2023-12-11T17:45:21.255 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:21.252+0000 7f2930111700 1 -- 172.21.15.59:0/1382377916 wait complete. 2023-12-11T17:45:21.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:45:20 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/2893281172' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-12-11T17:45:21.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:45:20 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/2893281172' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-12-11T17:45:21.527 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:21.496+0000 7f9b17c58700 1 -- 172.21.15.59:0/2414824941 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9b10100ef0 msgr2=0x7f9b1010d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:21.527 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:21.496+0000 7f9b17c58700 1 --2- 172.21.15.59:0/2414824941 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9b10100ef0 0x7f9b1010d330 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f9b0800b0d0 tx=0x7f9b0800b3e0 comp rx=0 tx=0).stop 2023-12-11T17:45:21.527 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:21.496+0000 7f9b17c58700 1 -- 172.21.15.59:0/2414824941 shutdown_connections 2023-12-11T17:45:21.527 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:21.496+0000 7f9b17c58700 1 --2- 172.21.15.59:0/2414824941 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9b10100ef0 0x7f9b1010d330 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:21.527 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:21.496+0000 7f9b17c58700 1 --2- 172.21.15.59:0/2414824941 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b10106560 0x7f9b10106940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:21.527 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:21.496+0000 7f9b17c58700 1 --2- 172.21.15.59:0/2414824941 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9b10100530 0x7f9b101009b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:21.528 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:21.496+0000 7f9b17c58700 1 -- 172.21.15.59:0/2414824941 >> 172.21.15.59:0/2414824941 conn(0x7f9b10076570 msgr2=0x7f9b10076980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:45:21.528 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:21.497+0000 7f9b17c58700 1 -- 172.21.15.59:0/2414824941 shutdown_connections 2023-12-11T17:45:21.528 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:21.497+0000 7f9b17c58700 1 -- 172.21.15.59:0/2414824941 wait complete. 2023-12-11T17:45:21.528 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:21.497+0000 7f9b17c58700 1 Processor -- start 2023-12-11T17:45:21.528 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:21.497+0000 7f9b17c58700 1 -- start start 2023-12-11T17:45:21.528 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:21.498+0000 7f9b17c58700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b10100530 0x7f9b10198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:21.528 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:21.498+0000 7f9b17c58700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9b10100ef0 0x7f9b10198da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:21.528 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:21.498+0000 7f9b17c58700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9b10106560 0x7f9b101a38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:21.528 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:21.498+0000 7f9b17c58700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b101995f0 con 0x7f9b10106560 2023-12-11T17:45:21.528 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:21.498+0000 7f9b17c58700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b101a3e20 con 0x7f9b10100ef0 2023-12-11T17:45:21.528 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:21.498+0000 7f9b17c58700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b101a3f60 con 0x7f9b10100530 2023-12-11T17:45:21.528 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:21.498+0000 7f9b161f5700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9b10106560 0x7f9b101a38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:21.528 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:21.498+0000 7f9b159f4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b10100530 0x7f9b10198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:21.528 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:21.498+0000 7f9b161f5700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9b10106560 0x7f9b101a38e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33756/0 (socket says 172.21.15.59:33756) 2023-12-11T17:45:21.528 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:21.498+0000 7f9b161f5700 1 -- 172.21.15.59:0/166851880 learned_addr learned my addr 172.21.15.59:0/166851880 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:45:21.529 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:21.498+0000 7f9b151f3700 1 --2- 172.21.15.59:0/166851880 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9b10100ef0 0x7f9b10198da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:21.529 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:21.498+0000 7f9b161f5700 1 -- 172.21.15.59:0/166851880 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b10100530 msgr2=0x7f9b10198860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:21.529 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:21.498+0000 7f9b161f5700 1 --2- 172.21.15.59:0/166851880 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b10100530 0x7f9b10198860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:21.529 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:21.498+0000 7f9b161f5700 1 -- 172.21.15.59:0/166851880 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9b10100ef0 msgr2=0x7f9b10198da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:21.529 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:21.498+0000 7f9b161f5700 1 --2- 172.21.15.59:0/166851880 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9b10100ef0 0x7f9b10198da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:21.529 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:21.498+0000 7f9b161f5700 1 -- 172.21.15.59:0/166851880 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b08009d00 con 0x7f9b10106560 2023-12-11T17:45:21.529 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:21.498+0000 7f9b161f5700 1 --2- 172.21.15.59:0/166851880 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9b10106560 0x7f9b101a38e0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f9b0800bce0 tx=0x7f9b0800bd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:21.529 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:21.499+0000 7f9b06ffd700 1 -- 172.21.15.59:0/166851880 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b0800e040 con 0x7f9b10106560 2023-12-11T17:45:21.529 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:21.499+0000 7f9b06ffd700 1 -- 172.21.15.59:0/166851880 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9b08005070 con 0x7f9b10106560 2023-12-11T17:45:21.529 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:21.499+0000 7f9b17c58700 1 -- 172.21.15.59:0/166851880 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b101a4160 con 0x7f9b10106560 2023-12-11T17:45:21.529 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:21.499+0000 7f9b06ffd700 1 -- 172.21.15.59:0/166851880 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b0801b6a0 con 0x7f9b10106560 2023-12-11T17:45:21.529 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:21.499+0000 7f9b17c58700 1 -- 172.21.15.59:0/166851880 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b101a4570 con 0x7f9b10106560 2023-12-11T17:45:21.529 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:21.500+0000 7f9b17c58700 1 -- 172.21.15.59:0/166851880 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b10101bd0 con 0x7f9b10106560 2023-12-11T17:45:21.529 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:21.501+0000 7f9b06ffd700 1 -- 172.21.15.59:0/166851880 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f9b080054c0 con 0x7f9b10106560 2023-12-11T17:45:21.529 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:21.501+0000 7f9b06ffd700 1 --2- 172.21.15.59:0/166851880 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f9afc0a1510 0x7f9afc0a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:21.530 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:21.501+0000 7f9b06ffd700 1 -- 172.21.15.59:0/166851880 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f9b080cdfa0 con 0x7f9b10106560 2023-12-11T17:45:21.530 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:21.501+0000 7f9b159f4700 1 --2- 172.21.15.59:0/166851880 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f9afc0a1510 0x7f9afc0a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:21.530 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:21.502+0000 7f9b159f4700 1 --2- 172.21.15.59:0/166851880 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f9afc0a1510 0x7f9afc0a39d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c009c60 tx=0x7f9b0c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:21.530 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:21.505+0000 7f9b06ffd700 1 -- 172.21.15.59:0/166851880 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f9b08097230 con 0x7f9b10106560 2023-12-11T17:45:21.711 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:21.709+0000 7f9b17c58700 1 -- 172.21.15.59:0/166851880 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) v1 -- 0x7f9b10110420 con 0x7f9b10106560 2023-12-11T17:45:21.712 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:21.709+0000 7f9b06ffd700 1 -- 172.21.15.59:0/166851880 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9b0809a850 con 0x7f9b10106560 2023-12-11T17:45:21.712 INFO:teuthology.orchestra.run.smithi059.stdout:244813135909 2023-12-11T17:45:21.714 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:21.711+0000 7f9b17c58700 1 -- 172.21.15.59:0/166851880 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f9afc0a1510 msgr2=0x7f9afc0a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:21.714 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:21.711+0000 7f9b17c58700 1 --2- 172.21.15.59:0/166851880 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f9afc0a1510 0x7f9afc0a39d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c009c60 tx=0x7f9b0c009260 comp rx=0 tx=0).stop 2023-12-11T17:45:21.714 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:21.711+0000 7f9b17c58700 1 -- 172.21.15.59:0/166851880 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9b10106560 msgr2=0x7f9b101a38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:21.714 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:21.711+0000 7f9b17c58700 1 --2- 172.21.15.59:0/166851880 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9b10106560 0x7f9b101a38e0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f9b0800bce0 tx=0x7f9b0800bd10 comp rx=0 tx=0).stop 2023-12-11T17:45:21.714 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:21.712+0000 7f9b17c58700 1 -- 172.21.15.59:0/166851880 shutdown_connections 2023-12-11T17:45:21.714 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:21.712+0000 7f9b17c58700 1 --2- 172.21.15.59:0/166851880 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f9afc0a1510 0x7f9afc0a39d0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:21.715 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:21.712+0000 7f9b17c58700 1 --2- 172.21.15.59:0/166851880 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9b10106560 0x7f9b101a38e0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:21.715 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:21.712+0000 7f9b17c58700 1 --2- 172.21.15.59:0/166851880 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b10100530 0x7f9b10198860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:21.715 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:21.712+0000 7f9b17c58700 1 --2- 172.21.15.59:0/166851880 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9b10100ef0 0x7f9b10198da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:21.715 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:21.712+0000 7f9b17c58700 1 -- 172.21.15.59:0/166851880 >> 172.21.15.59:0/166851880 conn(0x7f9b10076570 msgr2=0x7f9b100fefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:45:21.715 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:21.712+0000 7f9b17c58700 1 -- 172.21.15.59:0/166851880 shutdown_connections 2023-12-11T17:45:21.715 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:21.712+0000 7f9b17c58700 1 -- 172.21.15.59:0/166851880 wait complete. 2023-12-11T17:45:21.968 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:45:21 smithi059 ceph-mon[98847]: pgmap v291: 129 pgs: 1 peering, 128 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 383 B/s wr, 4 op/s; 199 B/s, 1 objects/s recovering 2023-12-11T17:45:21.968 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:45:21 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/1382377916' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-12-11T17:45:21.969 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:45:21 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/166851880' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-12-11T17:45:22.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:45:21 smithi115 ceph-mon[102748]: pgmap v291: 129 pgs: 1 peering, 128 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 383 B/s wr, 4 op/s; 199 B/s, 1 objects/s recovering 2023-12-11T17:45:22.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:45:21 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/1382377916' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-12-11T17:45:22.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:45:21 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/166851880' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-12-11T17:45:22.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:45:21 smithi181 ceph-mon[103627]: pgmap v291: 129 pgs: 1 peering, 128 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 383 B/s wr, 4 op/s; 199 B/s, 1 objects/s recovering 2023-12-11T17:45:22.396 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:45:21 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/1382377916' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-12-11T17:45:22.396 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:45:21 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/166851880' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-12-11T17:45:22.411 INFO:tasks.cephadm.ceph_manager.ceph:need seq 300647710744 got 300647710746 for osd.10 2023-12-11T17:45:22.412 DEBUG:teuthology.parallel:result is None 2023-12-11T17:45:22.970 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:45:22 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:45:23.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:45:22 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:45:23.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:45:22 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:45:23.401 INFO:tasks.cephadm.ceph_manager.ceph:need seq 279172874270 got 279172874272 for osd.9 2023-12-11T17:45:23.402 DEBUG:teuthology.parallel:result is None 2023-12-11T17:45:23.422 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:23.420+0000 7f10a36c6700 1 -- 172.21.15.59:0/2925701540 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f109c100ec0 msgr2=0x7f109c10d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:23.423 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:23.420+0000 7f10a36c6700 1 --2- 172.21.15.59:0/2925701540 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f109c100ec0 0x7f109c10d310 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f10940099c0 tx=0x7f1094009cd0 comp rx=0 tx=0).stop 2023-12-11T17:45:23.423 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:23.420+0000 7f10a36c6700 1 -- 172.21.15.59:0/2925701540 shutdown_connections 2023-12-11T17:45:23.423 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:23.420+0000 7f10a36c6700 1 --2- 172.21.15.59:0/2925701540 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f109c100ec0 0x7f109c10d310 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:23.423 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:23.420+0000 7f10a36c6700 1 --2- 172.21.15.59:0/2925701540 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f109c106540 0x7f109c106920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:23.423 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:23.420+0000 7f10a36c6700 1 --2- 172.21.15.59:0/2925701540 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f109c100500 0x7f109c100980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:23.423 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:23.420+0000 7f10a36c6700 1 -- 172.21.15.59:0/2925701540 >> 172.21.15.59:0/2925701540 conn(0x7f109c076580 msgr2=0x7f109c076990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:45:23.423 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:23.420+0000 7f10a36c6700 1 -- 172.21.15.59:0/2925701540 shutdown_connections 2023-12-11T17:45:23.423 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:23.420+0000 7f10a36c6700 1 -- 172.21.15.59:0/2925701540 wait complete. 2023-12-11T17:45:23.425 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:23.420+0000 7f10a36c6700 1 Processor -- start 2023-12-11T17:45:23.425 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:23.420+0000 7f10a36c6700 1 -- start start 2023-12-11T17:45:23.425 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:23.421+0000 7f10a36c6700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f109c100500 0x7f109c1987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:23.425 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:23.421+0000 7f10a36c6700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f109c100ec0 0x7f109c198cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:23.425 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:23.421+0000 7f10a36c6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f109c106540 0x7f109c199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:23.425 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:23.421+0000 7f10a36c6700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f109c1a3ac0 con 0x7f109c100500 2023-12-11T17:45:23.425 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:23.421+0000 7f10a36c6700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f109c1a3c30 con 0x7f109c100ec0 2023-12-11T17:45:23.425 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:23.421+0000 7f10a36c6700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f109c1a3df0 con 0x7f109c106540 2023-12-11T17:45:23.426 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:23.421+0000 7f10a1462700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f109c100500 0x7f109c1987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:23.426 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:23.421+0000 7f10a1462700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f109c100500 0x7f109c1987b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33766/0 (socket says 172.21.15.59:33766) 2023-12-11T17:45:23.426 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:23.421+0000 7f10a1462700 1 -- 172.21.15.59:0/3770516581 learned_addr learned my addr 172.21.15.59:0/3770516581 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:45:23.426 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:23.421+0000 7f10a1c63700 1 --2- 172.21.15.59:0/3770516581 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f109c106540 0x7f109c199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:23.426 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:23.421+0000 7f10a0c61700 1 --2- 172.21.15.59:0/3770516581 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f109c100ec0 0x7f109c198cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:23.426 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:23.421+0000 7f10a1462700 1 -- 172.21.15.59:0/3770516581 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f109c106540 msgr2=0x7f109c199270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:23.426 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:23.421+0000 7f10a1462700 1 --2- 172.21.15.59:0/3770516581 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f109c106540 0x7f109c199270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:23.426 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:23.421+0000 7f10a1462700 1 -- 172.21.15.59:0/3770516581 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f109c100ec0 msgr2=0x7f109c198cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:23.426 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:23.421+0000 7f10a1462700 1 --2- 172.21.15.59:0/3770516581 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f109c100ec0 0x7f109c198cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:23.426 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:23.421+0000 7f10a1462700 1 -- 172.21.15.59:0/3770516581 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10940095f0 con 0x7f109c100500 2023-12-11T17:45:23.426 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:23.421+0000 7f10a1462700 1 --2- 172.21.15.59:0/3770516581 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f109c100500 0x7f109c1987b0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f108c0060b0 tx=0x7f108c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:23.426 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:23.421+0000 7f109a7fc700 1 -- 172.21.15.59:0/3770516581 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f108c00dd20 con 0x7f109c100500 2023-12-11T17:45:23.426 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:23.422+0000 7f109a7fc700 1 -- 172.21.15.59:0/3770516581 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f108c00f040 con 0x7f109c100500 2023-12-11T17:45:23.426 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:23.422+0000 7f10a36c6700 1 -- 172.21.15.59:0/3770516581 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f109c1a40d0 con 0x7f109c100500 2023-12-11T17:45:23.426 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:23.422+0000 7f109a7fc700 1 -- 172.21.15.59:0/3770516581 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f108c014600 con 0x7f109c100500 2023-12-11T17:45:23.427 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:23.422+0000 7f10a36c6700 1 -- 172.21.15.59:0/3770516581 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f109c1a4620 con 0x7f109c100500 2023-12-11T17:45:23.427 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:23.423+0000 7f10a36c6700 1 -- 172.21.15.59:0/3770516581 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10880052f0 con 0x7f109c100500 2023-12-11T17:45:23.427 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:23.424+0000 7f109a7fc700 1 -- 172.21.15.59:0/3770516581 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f108c014760 con 0x7f109c100500 2023-12-11T17:45:23.427 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:23.424+0000 7f109a7fc700 1 --2- 172.21.15.59:0/3770516581 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f10840a1590 0x7f10840a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:23.427 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:23.424+0000 7f10a0c61700 1 --2- 172.21.15.59:0/3770516581 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f10840a1590 0x7f10840a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:23.427 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:23.424+0000 7f109a7fc700 1 -- 172.21.15.59:0/3770516581 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f108c0cc910 con 0x7f109c100500 2023-12-11T17:45:23.428 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:23.425+0000 7f10a0c61700 1 --2- 172.21.15.59:0/3770516581 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f10840a1590 0x7f10840a3a50 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f109c06a420 tx=0x7f109000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:23.431 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:23.429+0000 7f109a7fc700 1 -- 172.21.15.59:0/3770516581 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f108c095b10 con 0x7f109c100500 2023-12-11T17:45:23.563 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182454 got 107374182457 for osd.3 2023-12-11T17:45:23.563 DEBUG:teuthology.parallel:result is None 2023-12-11T17:45:23.671 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:23.669+0000 7f10a36c6700 1 -- 172.21.15.59:0/3770516581 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f1088005160 con 0x7f109c100500 2023-12-11T17:45:23.673 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:23.671+0000 7f109a7fc700 1 -- 172.21.15.59:0/3770516581 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f108c099130 con 0x7f109c100500 2023-12-11T17:45:23.674 INFO:teuthology.orchestra.run.smithi059.stdout:77309411391 2023-12-11T17:45:23.676 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:23.673+0000 7f10a36c6700 1 -- 172.21.15.59:0/3770516581 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f10840a1590 msgr2=0x7f10840a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:23.676 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:23.673+0000 7f10a36c6700 1 --2- 172.21.15.59:0/3770516581 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f10840a1590 0x7f10840a3a50 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f109c06a420 tx=0x7f109000b410 comp rx=0 tx=0).stop 2023-12-11T17:45:23.676 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:23.673+0000 7f10a36c6700 1 -- 172.21.15.59:0/3770516581 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f109c100500 msgr2=0x7f109c1987b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:23.676 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:23.673+0000 7f10a36c6700 1 --2- 172.21.15.59:0/3770516581 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f109c100500 0x7f109c1987b0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f108c0060b0 tx=0x7f108c00d3b0 comp rx=0 tx=0).stop 2023-12-11T17:45:23.676 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:23.673+0000 7f10a36c6700 1 -- 172.21.15.59:0/3770516581 shutdown_connections 2023-12-11T17:45:23.677 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:23.674+0000 7f10a36c6700 1 --2- 172.21.15.59:0/3770516581 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f10840a1590 0x7f10840a3a50 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:23.677 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:23.674+0000 7f10a36c6700 1 --2- 172.21.15.59:0/3770516581 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f109c106540 0x7f109c199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:23.677 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:23.674+0000 7f10a36c6700 1 --2- 172.21.15.59:0/3770516581 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f109c100500 0x7f109c1987b0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:23.677 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:23.674+0000 7f10a36c6700 1 --2- 172.21.15.59:0/3770516581 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f109c100ec0 0x7f109c198cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:23.677 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:23.674+0000 7f10a36c6700 1 -- 172.21.15.59:0/3770516581 >> 172.21.15.59:0/3770516581 conn(0x7f109c076580 msgr2=0x7f109c0ff1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:45:23.677 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:23.674+0000 7f10a36c6700 1 -- 172.21.15.59:0/3770516581 shutdown_connections 2023-12-11T17:45:23.677 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:23.674+0000 7f10a36c6700 1 -- 172.21.15.59:0/3770516581 wait complete. 2023-12-11T17:45:24.047 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.044+0000 7f3c2fd17700 1 -- 172.21.15.59:0/2589816780 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3c281004c0 msgr2=0x7f3c28100940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:24.047 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.044+0000 7f3c2fd17700 1 --2- 172.21.15.59:0/2589816780 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3c281004c0 0x7f3c28100940 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c0099c0 tx=0x7f3c1c009cd0 comp rx=0 tx=0).stop 2023-12-11T17:45:24.047 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.044+0000 7f3c2fd17700 1 -- 172.21.15.59:0/2589816780 shutdown_connections 2023-12-11T17:45:24.047 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.044+0000 7f3c2fd17700 1 --2- 172.21.15.59:0/2589816780 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c28100e80 0x7f3c2810d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:24.047 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.044+0000 7f3c2fd17700 1 --2- 172.21.15.59:0/2589816780 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3c28105470 0x7f3c28105850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:24.047 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.044+0000 7f3c2fd17700 1 --2- 172.21.15.59:0/2589816780 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3c281004c0 0x7f3c28100940 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:24.047 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.044+0000 7f3c2fd17700 1 -- 172.21.15.59:0/2589816780 >> 172.21.15.59:0/2589816780 conn(0x7f3c280774a0 msgr2=0x7f3c280778b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:45:24.047 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.044+0000 7f3c2fd17700 1 -- 172.21.15.59:0/2589816780 shutdown_connections 2023-12-11T17:45:24.047 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.044+0000 7f3c2fd17700 1 -- 172.21.15.59:0/2589816780 wait complete. 2023-12-11T17:45:24.049 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.045+0000 7f3c2fd17700 1 Processor -- start 2023-12-11T17:45:24.049 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.045+0000 7f3c2fd17700 1 -- start start 2023-12-11T17:45:24.050 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.045+0000 7f3c2fd17700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3c281004c0 0x7f3c2819c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:24.050 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.045+0000 7f3c2fd17700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c28100e80 0x7f3c2819cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:24.050 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.045+0000 7f3c2fd17700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3c28105470 0x7f3c28196700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:24.050 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.045+0000 7f3c2fd17700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c28196d60 con 0x7f3c28105470 2023-12-11T17:45:24.050 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.045+0000 7f3c2fd17700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c28196ed0 con 0x7f3c281004c0 2023-12-11T17:45:24.050 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.045+0000 7f3c2fd17700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c28197090 con 0x7f3c28100e80 2023-12-11T17:45:24.050 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.045+0000 7f3c2e2b4700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3c28105470 0x7f3c28196700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:24.050 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.045+0000 7f3c2e2b4700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3c28105470 0x7f3c28196700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33798/0 (socket says 172.21.15.59:33798) 2023-12-11T17:45:24.050 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.045+0000 7f3c2e2b4700 1 -- 172.21.15.59:0/729720105 learned_addr learned my addr 172.21.15.59:0/729720105 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:45:24.050 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.045+0000 7f3c2d2b2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c28100e80 0x7f3c2819cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:24.050 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.045+0000 7f3c2dab3700 1 --2- 172.21.15.59:0/729720105 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3c281004c0 0x7f3c2819c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:24.050 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.046+0000 7f3c2d2b2700 1 -- 172.21.15.59:0/729720105 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3c281004c0 msgr2=0x7f3c2819c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:24.050 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.046+0000 7f3c2d2b2700 1 --2- 172.21.15.59:0/729720105 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3c281004c0 0x7f3c2819c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:24.050 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.046+0000 7f3c2d2b2700 1 -- 172.21.15.59:0/729720105 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3c28105470 msgr2=0x7f3c28196700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:24.051 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.046+0000 7f3c2d2b2700 1 --2- 172.21.15.59:0/729720105 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3c28105470 0x7f3c28196700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:24.051 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.046+0000 7f3c2d2b2700 1 -- 172.21.15.59:0/729720105 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c1c0095f0 con 0x7f3c28100e80 2023-12-11T17:45:24.051 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.046+0000 7f3c2e2b4700 1 --2- 172.21.15.59:0/729720105 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3c28105470 0x7f3c28196700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T17:45:24.051 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.046+0000 7f3c2d2b2700 1 --2- 172.21.15.59:0/729720105 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c28100e80 0x7f3c2819cb00 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c0098e0 tx=0x7f3c1c00b9c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:24.051 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.046+0000 7f3c1affd700 1 -- 172.21.15.59:0/729720105 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c1c01d070 con 0x7f3c28100e80 2023-12-11T17:45:24.051 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.046+0000 7f3c2fd17700 1 -- 172.21.15.59:0/729720105 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c28197310 con 0x7f3c28100e80 2023-12-11T17:45:24.051 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.046+0000 7f3c1affd700 1 -- 172.21.15.59:0/729720105 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3c1c00be80 con 0x7f3c28100e80 2023-12-11T17:45:24.051 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.046+0000 7f3c1affd700 1 -- 172.21.15.59:0/729720105 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c1c00fa60 con 0x7f3c28100e80 2023-12-11T17:45:24.051 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.046+0000 7f3c2fd17700 1 -- 172.21.15.59:0/729720105 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c280699d0 con 0x7f3c28100e80 2023-12-11T17:45:24.051 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.047+0000 7f3c2fd17700 1 -- 172.21.15.59:0/729720105 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c28101bb0 con 0x7f3c28100e80 2023-12-11T17:45:24.052 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.048+0000 7f3c1affd700 1 -- 172.21.15.59:0/729720105 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f3c1c003680 con 0x7f3c28100e80 2023-12-11T17:45:24.052 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.049+0000 7f3c1affd700 1 --2- 172.21.15.59:0/729720105 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f3c140a1510 0x7f3c140a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:24.052 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.049+0000 7f3c1affd700 1 -- 172.21.15.59:0/729720105 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f3c1c0ce1b0 con 0x7f3c28100e80 2023-12-11T17:45:24.052 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.049+0000 7f3c2dab3700 1 --2- 172.21.15.59:0/729720105 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f3c140a1510 0x7f3c140a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:24.052 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.049+0000 7f3c2dab3700 1 --2- 172.21.15.59:0/729720105 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f3c140a1510 0x7f3c140a39d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f3c281016d0 tx=0x7f3c24009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:24.055 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.052+0000 7f3c1affd700 1 -- 172.21.15.59:0/729720105 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f3c1c0973e0 con 0x7f3c28100e80 2023-12-11T17:45:24.162 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:45:23 smithi059 ceph-mon[98847]: pgmap v292: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 383 B/s wr, 4 op/s; 199 B/s, 1 objects/s recovering 2023-12-11T17:45:24.162 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:45:23 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/3770516581' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-12-11T17:45:24.266 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.264+0000 7f3c2fd17700 1 -- 172.21.15.59:0/729720105 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) v1 -- 0x7f3c2804f290 con 0x7f3c28100e80 2023-12-11T17:45:24.267 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.265+0000 7f3c1affd700 1 -- 172.21.15.59:0/729720105 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7f3c1c09aa00 con 0x7f3c28100e80 2023-12-11T17:45:24.267 INFO:teuthology.orchestra.run.smithi059.stdout:330712481816 2023-12-11T17:45:24.269 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.267+0000 7f3c2fd17700 1 -- 172.21.15.59:0/729720105 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f3c140a1510 msgr2=0x7f3c140a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:24.269 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.267+0000 7f3c2fd17700 1 --2- 172.21.15.59:0/729720105 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f3c140a1510 0x7f3c140a39d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f3c281016d0 tx=0x7f3c24009260 comp rx=0 tx=0).stop 2023-12-11T17:45:24.270 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.267+0000 7f3c2fd17700 1 -- 172.21.15.59:0/729720105 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c28100e80 msgr2=0x7f3c2819cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:24.270 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.267+0000 7f3c2fd17700 1 --2- 172.21.15.59:0/729720105 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c28100e80 0x7f3c2819cb00 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c0098e0 tx=0x7f3c1c00b9c0 comp rx=0 tx=0).stop 2023-12-11T17:45:24.270 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.267+0000 7f3c2fd17700 1 -- 172.21.15.59:0/729720105 shutdown_connections 2023-12-11T17:45:24.270 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.267+0000 7f3c2fd17700 1 --2- 172.21.15.59:0/729720105 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f3c140a1510 0x7f3c140a39d0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:24.270 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.267+0000 7f3c2fd17700 1 --2- 172.21.15.59:0/729720105 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3c28105470 0x7f3c28196700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:24.270 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.267+0000 7f3c2fd17700 1 --2- 172.21.15.59:0/729720105 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3c281004c0 0x7f3c2819c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:24.270 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.267+0000 7f3c2fd17700 1 --2- 172.21.15.59:0/729720105 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c28100e80 0x7f3c2819cb00 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:24.270 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.267+0000 7f3c2fd17700 1 -- 172.21.15.59:0/729720105 >> 172.21.15.59:0/729720105 conn(0x7f3c280774a0 msgr2=0x7f3c280fe670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:45:24.270 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.267+0000 7f3c2fd17700 1 -- 172.21.15.59:0/729720105 shutdown_connections 2023-12-11T17:45:24.270 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.267+0000 7f3c2fd17700 1 -- 172.21.15.59:0/729720105 wait complete. 2023-12-11T17:45:24.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:45:23 smithi115 ceph-mon[102748]: pgmap v292: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 383 B/s wr, 4 op/s; 199 B/s, 1 objects/s recovering 2023-12-11T17:45:24.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:45:23 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/3770516581' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-12-11T17:45:24.343 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:23.909+0000 7f7e3e19f700 1 -- 172.21.15.59:0/2372551006 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e38106560 msgr2=0x7f7e38106940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:24.344 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:23.909+0000 7f7e3e19f700 1 --2- 172.21.15.59:0/2372551006 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e38106560 0x7f7e38106940 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7e280098f0 tx=0x7f7e28009c00 comp rx=0 tx=0).stop 2023-12-11T17:45:24.344 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:23.909+0000 7f7e3e19f700 1 -- 172.21.15.59:0/2372551006 shutdown_connections 2023-12-11T17:45:24.344 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:23.909+0000 7f7e3e19f700 1 --2- 172.21.15.59:0/2372551006 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7e38100ef0 0x7f7e3810d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:24.344 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:23.909+0000 7f7e3e19f700 1 --2- 172.21.15.59:0/2372551006 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e38106560 0x7f7e38106940 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:24.344 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:23.909+0000 7f7e3e19f700 1 --2- 172.21.15.59:0/2372551006 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7e38100530 0x7f7e381009b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:24.344 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:23.909+0000 7f7e3e19f700 1 -- 172.21.15.59:0/2372551006 >> 172.21.15.59:0/2372551006 conn(0x7f7e38076570 msgr2=0x7f7e38076980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:45:24.344 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:23.909+0000 7f7e3e19f700 1 -- 172.21.15.59:0/2372551006 shutdown_connections 2023-12-11T17:45:24.344 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:23.909+0000 7f7e3e19f700 1 -- 172.21.15.59:0/2372551006 wait complete. 2023-12-11T17:45:24.344 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:23.910+0000 7f7e3e19f700 1 Processor -- start 2023-12-11T17:45:24.344 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:23.910+0000 7f7e3e19f700 1 -- start start 2023-12-11T17:45:24.344 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:23.910+0000 7f7e3e19f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7e38100530 0x7f7e3819f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:24.344 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:23.910+0000 7f7e3e19f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e38100ef0 0x7f7e3819ff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:24.344 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:23.910+0000 7f7e3e19f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7e38106560 0x7f7e38199ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:24.345 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:23.910+0000 7f7e3e19f700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e3819a280 con 0x7f7e38100530 2023-12-11T17:45:24.345 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:23.910+0000 7f7e3e19f700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e3819a3f0 con 0x7f7e38106560 2023-12-11T17:45:24.345 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:23.910+0000 7f7e3e19f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e3819a5b0 con 0x7f7e38100ef0 2023-12-11T17:45:24.345 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:23.910+0000 7f7e377fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7e38100530 0x7f7e3819f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:24.345 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:23.910+0000 7f7e377fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7e38100530 0x7f7e3819f9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33780/0 (socket says 172.21.15.59:33780) 2023-12-11T17:45:24.345 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:23.910+0000 7f7e377fe700 1 -- 172.21.15.59:0/1625058837 learned_addr learned my addr 172.21.15.59:0/1625058837 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:45:24.345 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:23.910+0000 7f7e37fff700 1 --2- 172.21.15.59:0/1625058837 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7e38106560 0x7f7e38199ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:24.345 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:23.911+0000 7f7e36ffd700 1 --2- 172.21.15.59:0/1625058837 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e38100ef0 0x7f7e3819ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:24.345 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:23.911+0000 7f7e377fe700 1 -- 172.21.15.59:0/1625058837 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e38100ef0 msgr2=0x7f7e3819ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:24.345 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:23.911+0000 7f7e377fe700 1 --2- 172.21.15.59:0/1625058837 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e38100ef0 0x7f7e3819ff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:24.345 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:23.911+0000 7f7e377fe700 1 -- 172.21.15.59:0/1625058837 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7e38106560 msgr2=0x7f7e38199ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:24.345 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:23.911+0000 7f7e377fe700 1 --2- 172.21.15.59:0/1625058837 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7e38106560 0x7f7e38199ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:24.345 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:23.911+0000 7f7e377fe700 1 -- 172.21.15.59:0/1625058837 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e28009520 con 0x7f7e38100530 2023-12-11T17:45:24.345 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:23.911+0000 7f7e377fe700 1 --2- 172.21.15.59:0/1625058837 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7e38100530 0x7f7e3819f9d0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f7e28005950 tx=0x7f7e2800fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:24.346 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:23.911+0000 7f7e34ff9700 1 -- 172.21.15.59:0/1625058837 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e2801c070 con 0x7f7e38100530 2023-12-11T17:45:24.346 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:23.911+0000 7f7e3e19f700 1 -- 172.21.15.59:0/1625058837 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e3819a830 con 0x7f7e38100530 2023-12-11T17:45:24.346 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:23.911+0000 7f7e34ff9700 1 -- 172.21.15.59:0/1625058837 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7e2800bb20 con 0x7f7e38100530 2023-12-11T17:45:24.346 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:23.911+0000 7f7e34ff9700 1 -- 172.21.15.59:0/1625058837 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e28003970 con 0x7f7e38100530 2023-12-11T17:45:24.346 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:23.911+0000 7f7e3e19f700 1 -- 172.21.15.59:0/1625058837 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e381a6950 con 0x7f7e38100530 2023-12-11T17:45:24.346 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:23.912+0000 7f7e3e19f700 1 -- 172.21.15.59:0/1625058837 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e38101bd0 con 0x7f7e38100530 2023-12-11T17:45:24.346 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:23.913+0000 7f7e34ff9700 1 -- 172.21.15.59:0/1625058837 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f7e28020030 con 0x7f7e38100530 2023-12-11T17:45:24.346 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:23.914+0000 7f7e34ff9700 1 --2- 172.21.15.59:0/1625058837 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f7e200a1510 0x7f7e200a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:24.346 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:23.914+0000 7f7e36ffd700 1 --2- 172.21.15.59:0/1625058837 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f7e200a1510 0x7f7e200a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:24.346 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:23.914+0000 7f7e34ff9700 1 -- 172.21.15.59:0/1625058837 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f7e280ce2a0 con 0x7f7e38100530 2023-12-11T17:45:24.346 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:23.914+0000 7f7e36ffd700 1 --2- 172.21.15.59:0/1625058837 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f7e200a1510 0x7f7e200a39d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7e3819b1f0 tx=0x7f7e2c009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:24.347 INFO:teuthology.orchestra.run.smithi059.stdout:158913790001 2023-12-11T17:45:24.347 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:23.917+0000 7f7e34ff9700 1 -- 172.21.15.59:0/1625058837 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f7e280975d0 con 0x7f7e38100530 2023-12-11T17:45:24.347 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.135+0000 7f7e3e19f700 1 -- 172.21.15.59:0/1625058837 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f7e3819af60 con 0x7f7e38100530 2023-12-11T17:45:24.347 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.135+0000 7f7e34ff9700 1 -- 172.21.15.59:0/1625058837 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7e2809abf0 con 0x7f7e38100530 2023-12-11T17:45:24.347 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.137+0000 7f7e3e19f700 1 -- 172.21.15.59:0/1625058837 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f7e200a1510 msgr2=0x7f7e200a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:24.347 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.137+0000 7f7e3e19f700 1 --2- 172.21.15.59:0/1625058837 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f7e200a1510 0x7f7e200a39d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7e3819b1f0 tx=0x7f7e2c009310 comp rx=0 tx=0).stop 2023-12-11T17:45:24.347 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.137+0000 7f7e3e19f700 1 -- 172.21.15.59:0/1625058837 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7e38100530 msgr2=0x7f7e3819f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:24.348 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.137+0000 7f7e3e19f700 1 --2- 172.21.15.59:0/1625058837 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7e38100530 0x7f7e3819f9d0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f7e28005950 tx=0x7f7e2800fa00 comp rx=0 tx=0).stop 2023-12-11T17:45:24.348 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.137+0000 7f7e3e19f700 1 -- 172.21.15.59:0/1625058837 shutdown_connections 2023-12-11T17:45:24.348 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.137+0000 7f7e3e19f700 1 --2- 172.21.15.59:0/1625058837 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f7e200a1510 0x7f7e200a39d0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:24.348 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.137+0000 7f7e3e19f700 1 --2- 172.21.15.59:0/1625058837 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7e38106560 0x7f7e38199ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:24.348 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.137+0000 7f7e3e19f700 1 --2- 172.21.15.59:0/1625058837 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7e38100530 0x7f7e3819f9d0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:24.348 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.137+0000 7f7e3e19f700 1 --2- 172.21.15.59:0/1625058837 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e38100ef0 0x7f7e3819ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:24.348 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.137+0000 7f7e3e19f700 1 -- 172.21.15.59:0/1625058837 >> 172.21.15.59:0/1625058837 conn(0x7f7e38076570 msgr2=0x7f7e380fefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:45:24.348 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.137+0000 7f7e3e19f700 1 -- 172.21.15.59:0/1625058837 shutdown_connections 2023-12-11T17:45:24.349 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.137+0000 7f7e3e19f700 1 -- 172.21.15.59:0/1625058837 wait complete. 2023-12-11T17:45:24.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:45:23 smithi181 ceph-mon[103627]: pgmap v292: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 383 B/s wr, 4 op/s; 199 B/s, 1 objects/s recovering 2023-12-11T17:45:24.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:45:23 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/3770516581' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-12-11T17:45:24.794 INFO:tasks.cephadm.ceph_manager.ceph:need seq 244813135907 got 244813135909 for osd.8 2023-12-11T17:45:24.794 DEBUG:teuthology.parallel:result is None 2023-12-11T17:45:24.996 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.993+0000 7f1e8fd50700 1 -- 172.21.15.59:0/2222242541 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1e88100ef0 msgr2=0x7f1e8810d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:24.996 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.993+0000 7f1e8fd50700 1 --2- 172.21.15.59:0/2222242541 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1e88100ef0 0x7f1e8810d310 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f1e8000b0d0 tx=0x7f1e8000b3e0 comp rx=0 tx=0).stop 2023-12-11T17:45:24.997 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.994+0000 7f1e8fd50700 1 -- 172.21.15.59:0/2222242541 shutdown_connections 2023-12-11T17:45:24.997 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.994+0000 7f1e8fd50700 1 --2- 172.21.15.59:0/2222242541 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1e88100ef0 0x7f1e8810d310 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:24.997 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.994+0000 7f1e8fd50700 1 --2- 172.21.15.59:0/2222242541 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1e88106540 0x7f1e88106920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:24.997 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.994+0000 7f1e8fd50700 1 --2- 172.21.15.59:0/2222242541 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1e88100530 0x7f1e881009b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:24.997 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.994+0000 7f1e8fd50700 1 -- 172.21.15.59:0/2222242541 >> 172.21.15.59:0/2222242541 conn(0x7f1e880765b0 msgr2=0x7f1e880769c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:45:24.997 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.994+0000 7f1e8fd50700 1 -- 172.21.15.59:0/2222242541 shutdown_connections 2023-12-11T17:45:24.997 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.994+0000 7f1e8fd50700 1 -- 172.21.15.59:0/2222242541 wait complete. 2023-12-11T17:45:24.999 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.994+0000 7f1e8fd50700 1 Processor -- start 2023-12-11T17:45:24.999 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.994+0000 7f1e8fd50700 1 -- start start 2023-12-11T17:45:24.999 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.995+0000 7f1e8fd50700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1e88100530 0x7f1e8819f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:24.999 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.995+0000 7f1e8fd50700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1e88100ef0 0x7f1e8819fec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:24.999 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.995+0000 7f1e8fd50700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1e88106540 0x7f1e88199ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:24.999 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.995+0000 7f1e8fd50700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e8819a120 con 0x7f1e88100530 2023-12-11T17:45:24.999 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.995+0000 7f1e8fd50700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e8819a290 con 0x7f1e88100ef0 2023-12-11T17:45:24.999 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.995+0000 7f1e8fd50700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e8819a450 con 0x7f1e88106540 2023-12-11T17:45:24.999 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.995+0000 7f1e8daec700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1e88100530 0x7f1e8819f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:24.999 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.995+0000 7f1e8daec700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1e88100530 0x7f1e8819f980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33822/0 (socket says 172.21.15.59:33822) 2023-12-11T17:45:24.999 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.995+0000 7f1e8daec700 1 -- 172.21.15.59:0/3980864536 learned_addr learned my addr 172.21.15.59:0/3980864536 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:45:25.000 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.995+0000 7f1e8d2eb700 1 --2- 172.21.15.59:0/3980864536 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1e88100ef0 0x7f1e8819fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:25.000 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.995+0000 7f1e8e2ed700 1 --2- 172.21.15.59:0/3980864536 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1e88106540 0x7f1e88199ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:25.000 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.995+0000 7f1e8d2eb700 1 -- 172.21.15.59:0/3980864536 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1e88106540 msgr2=0x7f1e88199ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:25.000 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.995+0000 7f1e8d2eb700 1 --2- 172.21.15.59:0/3980864536 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1e88106540 0x7f1e88199ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:25.000 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.995+0000 7f1e8d2eb700 1 -- 172.21.15.59:0/3980864536 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1e88100530 msgr2=0x7f1e8819f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:25.000 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.995+0000 7f1e8d2eb700 1 --2- 172.21.15.59:0/3980864536 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1e88100530 0x7f1e8819f980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:25.000 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.995+0000 7f1e8d2eb700 1 -- 172.21.15.59:0/3980864536 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e80009d00 con 0x7f1e88100ef0 2023-12-11T17:45:25.000 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.995+0000 7f1e8daec700 1 --2- 172.21.15.59:0/3980864536 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1e88100530 0x7f1e8819f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T17:45:25.000 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.995+0000 7f1e8d2eb700 1 --2- 172.21.15.59:0/3980864536 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1e88100ef0 0x7f1e8819fec0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1e7800e9f0 tx=0x7f1e7800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:25.000 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.995+0000 7f1e7effd700 1 -- 172.21.15.59:0/3980864536 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e78004220 con 0x7f1e88100ef0 2023-12-11T17:45:25.000 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.996+0000 7f1e8fd50700 1 -- 172.21.15.59:0/3980864536 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e8819a730 con 0x7f1e88100ef0 2023-12-11T17:45:25.000 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.996+0000 7f1e7effd700 1 -- 172.21.15.59:0/3980864536 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1e78004cf0 con 0x7f1e88100ef0 2023-12-11T17:45:25.000 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.996+0000 7f1e7effd700 1 -- 172.21.15.59:0/3980864536 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e7800ce70 con 0x7f1e88100ef0 2023-12-11T17:45:25.000 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.996+0000 7f1e8fd50700 1 -- 172.21.15.59:0/3980864536 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e881a6970 con 0x7f1e88100ef0 2023-12-11T17:45:25.001 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.996+0000 7f1e8fd50700 1 -- 172.21.15.59:0/3980864536 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e88101bb0 con 0x7f1e88100ef0 2023-12-11T17:45:25.001 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.998+0000 7f1e7effd700 1 -- 172.21.15.59:0/3980864536 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f1e78004750 con 0x7f1e88100ef0 2023-12-11T17:45:25.001 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.999+0000 7f1e7effd700 1 --2- 172.21.15.59:0/3980864536 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f1e740a1510 0x7f1e740a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:25.002 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.999+0000 7f1e8daec700 1 --2- 172.21.15.59:0/3980864536 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f1e740a1510 0x7f1e740a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:25.002 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.999+0000 7f1e7effd700 1 -- 172.21.15.59:0/3980864536 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f1e78014070 con 0x7f1e88100ef0 2023-12-11T17:45:25.002 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:25.000+0000 7f1e8daec700 1 --2- 172.21.15.59:0/3980864536 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f1e740a1510 0x7f1e740a39d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f1e840104e0 tx=0x7f1e84007480 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:25.004 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:25.001+0000 7f1e7effd700 1 -- 172.21.15.59:0/3980864536 <== mon.2 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f1e78096240 con 0x7f1e88100ef0 2023-12-11T17:45:25.091 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:45:24 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/1625058837' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-12-11T17:45:25.092 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:45:24 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/729720105' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-12-11T17:45:25.219 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:25.216+0000 7f1e8fd50700 1 -- 172.21.15.59:0/3980864536 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f1e88069490 con 0x7f1e88100ef0 2023-12-11T17:45:25.220 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:25.217+0000 7f1e7effd700 1 -- 172.21.15.59:0/3980864536 <== mon.2 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1e78004e60 con 0x7f1e88100ef0 2023-12-11T17:45:25.220 INFO:teuthology.orchestra.run.smithi059.stdout:137438953526 2023-12-11T17:45:25.222 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:25.219+0000 7f1e8fd50700 1 -- 172.21.15.59:0/3980864536 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f1e740a1510 msgr2=0x7f1e740a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:25.223 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:25.219+0000 7f1e8fd50700 1 --2- 172.21.15.59:0/3980864536 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f1e740a1510 0x7f1e740a39d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f1e840104e0 tx=0x7f1e84007480 comp rx=0 tx=0).stop 2023-12-11T17:45:25.223 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:25.219+0000 7f1e8fd50700 1 -- 172.21.15.59:0/3980864536 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1e88100ef0 msgr2=0x7f1e8819fec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:25.223 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:25.219+0000 7f1e8fd50700 1 --2- 172.21.15.59:0/3980864536 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1e88100ef0 0x7f1e8819fec0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1e7800e9f0 tx=0x7f1e7800ed00 comp rx=0 tx=0).stop 2023-12-11T17:45:25.223 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:25.220+0000 7f1e8fd50700 1 -- 172.21.15.59:0/3980864536 shutdown_connections 2023-12-11T17:45:25.223 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:25.220+0000 7f1e8fd50700 1 --2- 172.21.15.59:0/3980864536 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f1e740a1510 0x7f1e740a39d0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:25.223 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:25.220+0000 7f1e8fd50700 1 --2- 172.21.15.59:0/3980864536 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1e88106540 0x7f1e88199ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:25.224 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:25.220+0000 7f1e8fd50700 1 --2- 172.21.15.59:0/3980864536 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1e88100530 0x7f1e8819f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:25.224 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:25.220+0000 7f1e8fd50700 1 --2- 172.21.15.59:0/3980864536 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1e88100ef0 0x7f1e8819fec0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:25.224 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:25.220+0000 7f1e8fd50700 1 -- 172.21.15.59:0/3980864536 >> 172.21.15.59:0/3980864536 conn(0x7f1e880765b0 msgr2=0x7f1e88069c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:45:25.224 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:25.220+0000 7f1e8fd50700 1 -- 172.21.15.59:0/3980864536 shutdown_connections 2023-12-11T17:45:25.224 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:25.220+0000 7f1e8fd50700 1 -- 172.21.15.59:0/3980864536 wait complete. 2023-12-11T17:45:25.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:45:24 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/1625058837' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-12-11T17:45:25.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:45:24 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/729720105' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-12-11T17:45:25.285 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.892+0000 7f617a961700 1 -- 172.21.15.59:0/912127026 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6174101620 msgr2=0x7f6174101a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:25.285 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.892+0000 7f617a961700 1 --2- 172.21.15.59:0/912127026 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6174101620 0x7f6174101a80 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f6168009990 tx=0x7f6168009ca0 comp rx=0 tx=0).stop 2023-12-11T17:45:25.285 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.892+0000 7f617a961700 1 -- 172.21.15.59:0/912127026 shutdown_connections 2023-12-11T17:45:25.285 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.893+0000 7f617a961700 1 --2- 172.21.15.59:0/912127026 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6174101fc0 0x7f617410e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:25.285 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.893+0000 7f617a961700 1 --2- 172.21.15.59:0/912127026 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6174106530 0x7f6174106910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:25.285 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.893+0000 7f617a961700 1 --2- 172.21.15.59:0/912127026 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6174101620 0x7f6174101a80 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:25.285 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.893+0000 7f617a961700 1 -- 172.21.15.59:0/912127026 >> 172.21.15.59:0/912127026 conn(0x7f61740787c0 msgr2=0x7f6174078bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:45:25.285 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.893+0000 7f617a961700 1 -- 172.21.15.59:0/912127026 shutdown_connections 2023-12-11T17:45:25.285 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.893+0000 7f617a961700 1 -- 172.21.15.59:0/912127026 wait complete. 2023-12-11T17:45:25.286 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.893+0000 7f617a961700 1 Processor -- start 2023-12-11T17:45:25.286 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.893+0000 7f617a961700 1 -- start start 2023-12-11T17:45:25.286 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.894+0000 7f617a961700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6174101620 0x7f61740738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:25.286 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.894+0000 7f617a961700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6174101fc0 0x7f6174073e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:25.286 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.894+0000 7f617a961700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6174106530 0x7f6174074350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:25.286 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.894+0000 7f617a961700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61741a5d90 con 0x7f6174101620 2023-12-11T17:45:25.286 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.894+0000 7f617a961700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61741a5f00 con 0x7f6174101fc0 2023-12-11T17:45:25.286 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.894+0000 7f617a961700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61741a60c0 con 0x7f6174106530 2023-12-11T17:45:25.286 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.894+0000 7f6173fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6174101620 0x7f61740738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:25.286 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.894+0000 7f6173fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6174101620 0x7f61740738d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33808/0 (socket says 172.21.15.59:33808) 2023-12-11T17:45:25.286 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.894+0000 7f6173fff700 1 -- 172.21.15.59:0/2463510914 learned_addr learned my addr 172.21.15.59:0/2463510914 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:45:25.286 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.894+0000 7f61737fe700 1 --2- 172.21.15.59:0/2463510914 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6174101fc0 0x7f6174073e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:25.286 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.894+0000 7f6178efe700 1 --2- 172.21.15.59:0/2463510914 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6174106530 0x7f6174074350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:25.286 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.894+0000 7f6173fff700 1 -- 172.21.15.59:0/2463510914 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6174106530 msgr2=0x7f6174074350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:25.287 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.894+0000 7f6173fff700 1 --2- 172.21.15.59:0/2463510914 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6174106530 0x7f6174074350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:25.287 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.894+0000 7f6173fff700 1 -- 172.21.15.59:0/2463510914 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6174101fc0 msgr2=0x7f6174073e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:25.287 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.894+0000 7f6173fff700 1 --2- 172.21.15.59:0/2463510914 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6174101fc0 0x7f6174073e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:25.287 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.894+0000 7f6173fff700 1 -- 172.21.15.59:0/2463510914 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61680095f0 con 0x7f6174101620 2023-12-11T17:45:25.287 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.894+0000 7f6173fff700 1 --2- 172.21.15.59:0/2463510914 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6174101620 0x7f61740738d0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f616400d980 tx=0x7f616400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:25.287 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.894+0000 7f61717fa700 1 -- 172.21.15.59:0/2463510914 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6164004020 con 0x7f6174101620 2023-12-11T17:45:25.287 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.894+0000 7f61717fa700 1 -- 172.21.15.59:0/2463510914 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6164009730 con 0x7f6174101620 2023-12-11T17:45:25.287 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.894+0000 7f61717fa700 1 -- 172.21.15.59:0/2463510914 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6164003a80 con 0x7f6174101620 2023-12-11T17:45:25.287 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.894+0000 7f617a961700 1 -- 172.21.15.59:0/2463510914 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61741a63a0 con 0x7f6174101620 2023-12-11T17:45:25.287 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.895+0000 7f617a961700 1 -- 172.21.15.59:0/2463510914 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61741a6820 con 0x7f6174101620 2023-12-11T17:45:25.287 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.895+0000 7f617a961700 1 -- 172.21.15.59:0/2463510914 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6174109b80 con 0x7f6174101620 2023-12-11T17:45:25.287 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.897+0000 7f61717fa700 1 -- 172.21.15.59:0/2463510914 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f61640098a0 con 0x7f6174101620 2023-12-11T17:45:25.287 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.897+0000 7f61717fa700 1 --2- 172.21.15.59:0/2463510914 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f615c0a1510 0x7f615c0a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:25.287 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.897+0000 7f61737fe700 1 --2- 172.21.15.59:0/2463510914 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f615c0a1510 0x7f615c0a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:25.288 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.897+0000 7f61717fa700 1 -- 172.21.15.59:0/2463510914 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f61640cce30 con 0x7f6174101620 2023-12-11T17:45:25.288 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.898+0000 7f61737fe700 1 --2- 172.21.15.59:0/2463510914 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f615c0a1510 0x7f615c0a39d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f616800b2e0 tx=0x7f6168005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:25.288 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:24.902+0000 7f61717fa700 1 -- 172.21.15.59:0/2463510914 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f6164096030 con 0x7f6174101620 2023-12-11T17:45:25.288 INFO:teuthology.orchestra.run.smithi059.stdout:34359738439 2023-12-11T17:45:25.288 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:25.116+0000 7f617a961700 1 -- 172.21.15.59:0/2463510914 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f6174102ce0 con 0x7f6174101620 2023-12-11T17:45:25.288 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:25.116+0000 7f61717fa700 1 -- 172.21.15.59:0/2463510914 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6164099650 con 0x7f6174101620 2023-12-11T17:45:25.288 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:25.118+0000 7f617a961700 1 -- 172.21.15.59:0/2463510914 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f615c0a1510 msgr2=0x7f615c0a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:25.289 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:25.118+0000 7f617a961700 1 --2- 172.21.15.59:0/2463510914 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f615c0a1510 0x7f615c0a39d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f616800b2e0 tx=0x7f6168005960 comp rx=0 tx=0).stop 2023-12-11T17:45:25.289 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:25.118+0000 7f617a961700 1 -- 172.21.15.59:0/2463510914 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6174101620 msgr2=0x7f61740738d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:25.289 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:25.118+0000 7f617a961700 1 --2- 172.21.15.59:0/2463510914 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6174101620 0x7f61740738d0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f616400d980 tx=0x7f616400dc90 comp rx=0 tx=0).stop 2023-12-11T17:45:25.289 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:25.119+0000 7f617a961700 1 -- 172.21.15.59:0/2463510914 shutdown_connections 2023-12-11T17:45:25.289 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:25.119+0000 7f617a961700 1 --2- 172.21.15.59:0/2463510914 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f615c0a1510 0x7f615c0a39d0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:25.289 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:25.119+0000 7f617a961700 1 --2- 172.21.15.59:0/2463510914 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6174106530 0x7f6174074350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:25.290 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:25.119+0000 7f617a961700 1 --2- 172.21.15.59:0/2463510914 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6174101620 0x7f61740738d0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:25.290 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:25.119+0000 7f617a961700 1 --2- 172.21.15.59:0/2463510914 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6174101fc0 0x7f6174073e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:25.290 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:25.119+0000 7f617a961700 1 -- 172.21.15.59:0/2463510914 >> 172.21.15.59:0/2463510914 conn(0x7f61740787c0 msgr2=0x7f61740ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:45:25.290 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:25.119+0000 7f617a961700 1 -- 172.21.15.59:0/2463510914 shutdown_connections 2023-12-11T17:45:25.290 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:25.119+0000 7f617a961700 1 -- 172.21.15.59:0/2463510914 wait complete. 2023-12-11T17:45:25.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:45:24 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/1625058837' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-12-11T17:45:25.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:45:24 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/729720105' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-12-11T17:45:25.451 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:25.449+0000 7f0f21a33700 1 -- 172.21.15.59:0/2801583140 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0f1c068f70 msgr2=0x7f0f1c0693f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:25.451 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:25.449+0000 7f0f21a33700 1 --2- 172.21.15.59:0/2801583140 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0f1c068f70 0x7f0f1c0693f0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f0f100099c0 tx=0x7f0f10009cd0 comp rx=0 tx=0).stop 2023-12-11T17:45:25.452 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:25.449+0000 7f0f21a33700 1 -- 172.21.15.59:0/2801583140 shutdown_connections 2023-12-11T17:45:25.452 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:25.449+0000 7f0f21a33700 1 --2- 172.21.15.59:0/2801583140 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0f1c069930 0x7f0f1c10e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:25.452 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:25.449+0000 7f0f21a33700 1 --2- 172.21.15.59:0/2801583140 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0f1c107620 0x7f0f1c107a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:25.452 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:25.449+0000 7f0f21a33700 1 --2- 172.21.15.59:0/2801583140 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0f1c068f70 0x7f0f1c0693f0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:25.452 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:25.449+0000 7f0f21a33700 1 -- 172.21.15.59:0/2801583140 >> 172.21.15.59:0/2801583140 conn(0x7f0f1c076570 msgr2=0x7f0f1c076980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:45:25.452 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:25.449+0000 7f0f21a33700 1 -- 172.21.15.59:0/2801583140 shutdown_connections 2023-12-11T17:45:25.452 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:25.449+0000 7f0f21a33700 1 -- 172.21.15.59:0/2801583140 wait complete. 2023-12-11T17:45:25.453 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:25.449+0000 7f0f21a33700 1 Processor -- start 2023-12-11T17:45:25.453 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:25.450+0000 7f0f21a33700 1 -- start start 2023-12-11T17:45:25.454 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:25.450+0000 7f0f21a33700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0f1c068f70 0x7f0f1c19d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:25.454 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:25.450+0000 7f0f21a33700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0f1c069930 0x7f0f1c19dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:25.454 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:25.450+0000 7f0f21a33700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0f1c107620 0x7f0f1c197840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:25.454 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:25.450+0000 7f0f21a33700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f1c197eb0 con 0x7f0f1c069930 2023-12-11T17:45:25.454 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:25.450+0000 7f0f21a33700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f1c198020 con 0x7f0f1c068f70 2023-12-11T17:45:25.454 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:25.450+0000 7f0f21a33700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f1c1981e0 con 0x7f0f1c107620 2023-12-11T17:45:25.454 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:25.450+0000 7f0f1a7fc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0f1c069930 0x7f0f1c19dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:25.454 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:25.450+0000 7f0f1affd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0f1c068f70 0x7f0f1c19d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:25.454 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:25.450+0000 7f0f1a7fc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0f1c069930 0x7f0f1c19dc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33838/0 (socket says 172.21.15.59:33838) 2023-12-11T17:45:25.454 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:25.450+0000 7f0f1a7fc700 1 -- 172.21.15.59:0/3531885261 learned_addr learned my addr 172.21.15.59:0/3531885261 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:45:25.454 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:25.450+0000 7f0f1b7fe700 1 --2- 172.21.15.59:0/3531885261 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0f1c107620 0x7f0f1c197840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:25.454 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:25.450+0000 7f0f1a7fc700 1 -- 172.21.15.59:0/3531885261 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0f1c107620 msgr2=0x7f0f1c197840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:25.454 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:25.450+0000 7f0f1a7fc700 1 --2- 172.21.15.59:0/3531885261 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0f1c107620 0x7f0f1c197840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:25.454 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:25.450+0000 7f0f1a7fc700 1 -- 172.21.15.59:0/3531885261 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0f1c068f70 msgr2=0x7f0f1c19d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:25.455 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:25.450+0000 7f0f1a7fc700 1 --2- 172.21.15.59:0/3531885261 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0f1c068f70 0x7f0f1c19d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:25.455 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:25.450+0000 7f0f1a7fc700 1 -- 172.21.15.59:0/3531885261 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f100095f0 con 0x7f0f1c069930 2023-12-11T17:45:25.455 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:25.450+0000 7f0f1affd700 1 --2- 172.21.15.59:0/3531885261 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0f1c068f70 0x7f0f1c19d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T17:45:25.455 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:25.451+0000 7f0f1a7fc700 1 --2- 172.21.15.59:0/3531885261 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0f1c069930 0x7f0f1c19dc40 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f0f10000c00 tx=0x7f0f1000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:25.455 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:25.451+0000 7f0f20a31700 1 -- 172.21.15.59:0/3531885261 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f1001d070 con 0x7f0f1c069930 2023-12-11T17:45:25.455 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:25.451+0000 7f0f20a31700 1 -- 172.21.15.59:0/3531885261 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0f10022470 con 0x7f0f1c069930 2023-12-11T17:45:25.455 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:25.451+0000 7f0f21a33700 1 -- 172.21.15.59:0/3531885261 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f1c198460 con 0x7f0f1c069930 2023-12-11T17:45:25.455 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:25.451+0000 7f0f20a31700 1 -- 172.21.15.59:0/3531885261 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f10017650 con 0x7f0f1c069930 2023-12-11T17:45:25.458 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:25.451+0000 7f0f21a33700 1 -- 172.21.15.59:0/3531885261 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f1c1a46f0 con 0x7f0f1c069930 2023-12-11T17:45:25.458 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:25.452+0000 7f0f21a33700 1 -- 172.21.15.59:0/3531885261 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f1c109a00 con 0x7f0f1c069930 2023-12-11T17:45:25.458 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:25.453+0000 7f0f20a31700 1 -- 172.21.15.59:0/3531885261 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f0f10022850 con 0x7f0f1c069930 2023-12-11T17:45:25.458 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:25.454+0000 7f0f20a31700 1 --2- 172.21.15.59:0/3531885261 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f0f040a1510 0x7f0f040a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:25.458 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:25.454+0000 7f0f20a31700 1 -- 172.21.15.59:0/3531885261 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f0f100cdc60 con 0x7f0f1c069930 2023-12-11T17:45:25.458 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:25.454+0000 7f0f1affd700 1 --2- 172.21.15.59:0/3531885261 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f0f040a1510 0x7f0f040a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:25.459 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:25.454+0000 7f0f1affd700 1 --2- 172.21.15.59:0/3531885261 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f0f040a1510 0x7f0f040a39d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f0f0c009fd0 tx=0x7f0f0c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:25.461 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:25.458+0000 7f0f20a31700 1 -- 172.21.15.59:0/3531885261 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f0f10097000 con 0x7f0f1c069930 2023-12-11T17:45:25.600 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411389 got 77309411391 for osd.2 2023-12-11T17:45:25.600 DEBUG:teuthology.parallel:result is None 2023-12-11T17:45:25.684 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:25.681+0000 7f0f21a33700 1 -- 172.21.15.59:0/3531885261 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f0f1c102cc0 con 0x7f0f1c069930 2023-12-11T17:45:25.684 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:25.681+0000 7f0f20a31700 1 -- 172.21.15.59:0/3531885261 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0f1009a620 con 0x7f0f1c069930 2023-12-11T17:45:25.684 INFO:teuthology.orchestra.run.smithi059.stdout:193273528366 2023-12-11T17:45:25.686 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:25.683+0000 7f0f21a33700 1 -- 172.21.15.59:0/3531885261 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f0f040a1510 msgr2=0x7f0f040a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:25.686 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:25.683+0000 7f0f21a33700 1 --2- 172.21.15.59:0/3531885261 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f0f040a1510 0x7f0f040a39d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f0f0c009fd0 tx=0x7f0f0c009260 comp rx=0 tx=0).stop 2023-12-11T17:45:25.686 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:25.683+0000 7f0f21a33700 1 -- 172.21.15.59:0/3531885261 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0f1c069930 msgr2=0x7f0f1c19dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:25.686 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:25.683+0000 7f0f21a33700 1 --2- 172.21.15.59:0/3531885261 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0f1c069930 0x7f0f1c19dc40 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f0f10000c00 tx=0x7f0f1000f690 comp rx=0 tx=0).stop 2023-12-11T17:45:25.686 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:25.684+0000 7f0f21a33700 1 -- 172.21.15.59:0/3531885261 shutdown_connections 2023-12-11T17:45:25.686 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:25.684+0000 7f0f21a33700 1 --2- 172.21.15.59:0/3531885261 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f0f040a1510 0x7f0f040a39d0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:25.686 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:25.684+0000 7f0f21a33700 1 --2- 172.21.15.59:0/3531885261 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0f1c107620 0x7f0f1c197840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:25.686 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:25.684+0000 7f0f21a33700 1 --2- 172.21.15.59:0/3531885261 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0f1c068f70 0x7f0f1c19d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:25.687 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:25.684+0000 7f0f21a33700 1 --2- 172.21.15.59:0/3531885261 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0f1c069930 0x7f0f1c19dc40 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:25.687 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:25.684+0000 7f0f21a33700 1 -- 172.21.15.59:0/3531885261 >> 172.21.15.59:0/3531885261 conn(0x7f0f1c076570 msgr2=0x7f0f1c0ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:45:25.687 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:25.684+0000 7f0f21a33700 1 -- 172.21.15.59:0/3531885261 shutdown_connections 2023-12-11T17:45:25.687 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:25.684+0000 7f0f21a33700 1 -- 172.21.15.59:0/3531885261 wait complete. 2023-12-11T17:45:26.266 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:45:25 smithi059 ceph-mon[98847]: pgmap v293: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 331 B/s wr, 5 op/s; 172 B/s, 0 objects/s recovering 2023-12-11T17:45:26.266 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:45:25 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/2463510914' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-12-11T17:45:26.267 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:45:25 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/3980864536' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-12-11T17:45:26.267 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:45:25 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:45:26.267 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:45:25 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:45:26.267 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:45:25 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:45:26.267 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:45:25 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:45:26.267 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:45:25 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/3531885261' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-12-11T17:45:26.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:45:25 smithi115 ceph-mon[102748]: pgmap v293: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 331 B/s wr, 5 op/s; 172 B/s, 0 objects/s recovering 2023-12-11T17:45:26.284 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:45:25 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/2463510914' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-12-11T17:45:26.284 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:45:25 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/3980864536' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-12-11T17:45:26.284 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:45:25 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:45:26.284 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:45:25 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:45:26.284 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:45:25 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:45:26.284 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:45:25 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:45:26.284 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:45:25 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/3531885261' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-12-11T17:45:26.351 INFO:tasks.cephadm.ceph_manager.ceph:need seq 137438953524 got 137438953526 for osd.4 2023-12-11T17:45:26.351 DEBUG:teuthology.parallel:result is None 2023-12-11T17:45:26.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:45:25 smithi181 ceph-mon[103627]: pgmap v293: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 331 B/s wr, 5 op/s; 172 B/s, 0 objects/s recovering 2023-12-11T17:45:26.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:45:25 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/2463510914' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-12-11T17:45:26.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:45:25 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/3980864536' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-12-11T17:45:26.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:45:25 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:45:26.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:45:25 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:45:26.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:45:25 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:45:26.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:45:25 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:45:26.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:45:25 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/3531885261' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-12-11T17:45:26.568 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738437 got 34359738439 for osd.0 2023-12-11T17:45:26.568 DEBUG:teuthology.parallel:result is None 2023-12-11T17:45:26.569 INFO:tasks.cephadm.ceph_manager.ceph:need seq 193273528363 got 193273528366 for osd.6 2023-12-11T17:45:26.569 DEBUG:teuthology.parallel:result is None 2023-12-11T17:45:26.925 INFO:tasks.cephadm.ceph_manager.ceph:need seq 330712481814 got 330712481816 for osd.11 2023-12-11T17:45:26.926 DEBUG:teuthology.parallel:result is None 2023-12-11T17:45:27.055 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:27.052+0000 7f05dd748700 1 -- 172.21.15.59:0/3686318034 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f05d8100530 msgr2=0x7f05d81009b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:27.056 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:27.052+0000 7f05dd748700 1 --2- 172.21.15.59:0/3686318034 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f05d8100530 0x7f05d81009b0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f05c00099c0 tx=0x7f05c0009cd0 comp rx=0 tx=0).stop 2023-12-11T17:45:27.056 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:27.053+0000 7f05dd748700 1 -- 172.21.15.59:0/3686318034 shutdown_connections 2023-12-11T17:45:27.056 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:27.053+0000 7f05dd748700 1 --2- 172.21.15.59:0/3686318034 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f05d8100ef0 0x7f05d810d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:27.056 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:27.053+0000 7f05dd748700 1 --2- 172.21.15.59:0/3686318034 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f05d8106560 0x7f05d8106940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:27.056 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:27.053+0000 7f05dd748700 1 --2- 172.21.15.59:0/3686318034 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f05d8100530 0x7f05d81009b0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:27.056 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:27.053+0000 7f05dd748700 1 -- 172.21.15.59:0/3686318034 >> 172.21.15.59:0/3686318034 conn(0x7f05d8076570 msgr2=0x7f05d8076980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:45:27.056 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:27.053+0000 7f05dd748700 1 -- 172.21.15.59:0/3686318034 shutdown_connections 2023-12-11T17:45:27.056 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:27.053+0000 7f05dd748700 1 -- 172.21.15.59:0/3686318034 wait complete. 2023-12-11T17:45:27.056 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:27.053+0000 7f05dd748700 1 Processor -- start 2023-12-11T17:45:27.058 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:27.053+0000 7f05dd748700 1 -- start start 2023-12-11T17:45:27.058 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:27.054+0000 7f05dd748700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f05d8100530 0x7f05d8198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:27.058 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:27.054+0000 7f05dd748700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f05d8100ef0 0x7f05d8198cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:27.058 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:27.054+0000 7f05dd748700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f05d8106560 0x7f05d81a3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:27.058 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:27.054+0000 7f05dd748700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05d8199520 con 0x7f05d8100530 2023-12-11T17:45:27.058 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:27.054+0000 7f05dd748700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05d81a3d50 con 0x7f05d8106560 2023-12-11T17:45:27.058 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:27.054+0000 7f05dd748700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05d81a3e90 con 0x7f05d8100ef0 2023-12-11T17:45:27.058 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:27.054+0000 7f05d6ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f05d8100530 0x7f05d8198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:27.058 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:27.054+0000 7f05d6ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f05d8100530 0x7f05d8198790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33850/0 (socket says 172.21.15.59:33850) 2023-12-11T17:45:27.058 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:27.054+0000 7f05d6ffd700 1 -- 172.21.15.59:0/2288723527 learned_addr learned my addr 172.21.15.59:0/2288723527 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:45:27.058 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:27.054+0000 7f05ce7fc700 1 --2- 172.21.15.59:0/2288723527 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f05d8100ef0 0x7f05d8198cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:27.059 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:27.054+0000 7f05d77fe700 1 --2- 172.21.15.59:0/2288723527 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f05d8106560 0x7f05d81a3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:27.059 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:27.054+0000 7f05d6ffd700 1 -- 172.21.15.59:0/2288723527 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f05d8100ef0 msgr2=0x7f05d8198cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:27.059 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:27.054+0000 7f05d6ffd700 1 --2- 172.21.15.59:0/2288723527 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f05d8100ef0 0x7f05d8198cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:27.059 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:27.054+0000 7f05d6ffd700 1 -- 172.21.15.59:0/2288723527 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f05d8106560 msgr2=0x7f05d81a3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:27.059 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:27.054+0000 7f05d6ffd700 1 --2- 172.21.15.59:0/2288723527 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f05d8106560 0x7f05d81a3810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:27.059 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:27.054+0000 7f05d6ffd700 1 -- 172.21.15.59:0/2288723527 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05d0009c40 con 0x7f05d8100530 2023-12-11T17:45:27.059 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:27.054+0000 7f05ce7fc700 1 --2- 172.21.15.59:0/2288723527 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f05d8100ef0 0x7f05d8198cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T17:45:27.059 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:27.055+0000 7f05d6ffd700 1 --2- 172.21.15.59:0/2288723527 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f05d8100530 0x7f05d8198790 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f05c800e9d0 tx=0x7f05c800ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:27.059 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:27.055+0000 7f05d4ff9700 1 -- 172.21.15.59:0/2288723527 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05c8009790 con 0x7f05d8100530 2023-12-11T17:45:27.059 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:27.055+0000 7f05d4ff9700 1 -- 172.21.15.59:0/2288723527 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f05c800cbd0 con 0x7f05d8100530 2023-12-11T17:45:27.059 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:27.055+0000 7f05dd748700 1 -- 172.21.15.59:0/2288723527 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05c00095f0 con 0x7f05d8100530 2023-12-11T17:45:27.059 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:27.055+0000 7f05d4ff9700 1 -- 172.21.15.59:0/2288723527 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05c8018680 con 0x7f05d8100530 2023-12-11T17:45:27.059 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:27.055+0000 7f05dd748700 1 -- 172.21.15.59:0/2288723527 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05d81a4480 con 0x7f05d8100530 2023-12-11T17:45:27.060 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:27.056+0000 7f05dd748700 1 -- 172.21.15.59:0/2288723527 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05d8108a10 con 0x7f05d8100530 2023-12-11T17:45:27.060 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:27.058+0000 7f05d4ff9700 1 -- 172.21.15.59:0/2288723527 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f05c800cd40 con 0x7f05d8100530 2023-12-11T17:45:27.060 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:27.058+0000 7f05d4ff9700 1 --2- 172.21.15.59:0/2288723527 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f05c40a1590 0x7f05c40a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:27.061 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:27.059+0000 7f05ce7fc700 1 --2- 172.21.15.59:0/2288723527 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f05c40a1590 0x7f05c40a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:27.061 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:27.059+0000 7f05d4ff9700 1 -- 172.21.15.59:0/2288723527 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f05c8014070 con 0x7f05d8100530 2023-12-11T17:45:27.061 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:27.059+0000 7f05ce7fc700 1 --2- 172.21.15.59:0/2288723527 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f05c40a1590 0x7f05c40a3a50 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f05c0009fd0 tx=0x7f05c0005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:27.064 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:27.062+0000 7f05d4ff9700 1 -- 172.21.15.59:0/2288723527 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f05c8096850 con 0x7f05d8100530 2023-12-11T17:45:27.162 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:27.160+0000 7fe297c8e700 1 -- 172.21.15.59:0/3014499236 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe290107620 msgr2=0x7fe290107a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:27.163 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:27.160+0000 7fe297c8e700 1 --2- 172.21.15.59:0/3014499236 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe290107620 0x7fe290107a00 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fe28c0099c0 tx=0x7fe28c009cd0 comp rx=0 tx=0).stop 2023-12-11T17:45:27.163 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:27.160+0000 7fe297c8e700 1 -- 172.21.15.59:0/3014499236 shutdown_connections 2023-12-11T17:45:27.163 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:27.160+0000 7fe297c8e700 1 --2- 172.21.15.59:0/3014499236 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe290069930 0x7fe29010e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:27.163 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:27.160+0000 7fe297c8e700 1 --2- 172.21.15.59:0/3014499236 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe290107620 0x7fe290107a00 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:27.163 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:27.160+0000 7fe297c8e700 1 --2- 172.21.15.59:0/3014499236 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe290068f70 0x7fe2900693f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:27.163 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:27.160+0000 7fe297c8e700 1 -- 172.21.15.59:0/3014499236 >> 172.21.15.59:0/3014499236 conn(0x7fe290076570 msgr2=0x7fe290076980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:45:27.163 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:27.160+0000 7fe297c8e700 1 -- 172.21.15.59:0/3014499236 shutdown_connections 2023-12-11T17:45:27.163 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:27.160+0000 7fe297c8e700 1 -- 172.21.15.59:0/3014499236 wait complete. 2023-12-11T17:45:27.165 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:27.161+0000 7fe297c8e700 1 Processor -- start 2023-12-11T17:45:27.165 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:27.161+0000 7fe297c8e700 1 -- start start 2023-12-11T17:45:27.165 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:27.161+0000 7fe297c8e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe290068f70 0x7fe29019d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:27.166 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:27.161+0000 7fe297c8e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe290069930 0x7fe29019dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:27.166 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:27.161+0000 7fe297c8e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe290107620 0x7fe290197840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:27.166 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:27.161+0000 7fe297c8e700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe290197eb0 con 0x7fe290069930 2023-12-11T17:45:27.166 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:27.161+0000 7fe297c8e700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe290198020 con 0x7fe290068f70 2023-12-11T17:45:27.166 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:27.161+0000 7fe297c8e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2901981e0 con 0x7fe290107620 2023-12-11T17:45:27.166 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:27.161+0000 7fe295229700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe290069930 0x7fe29019dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:27.166 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:27.161+0000 7fe295229700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe290069930 0x7fe29019dc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33866/0 (socket says 172.21.15.59:33866) 2023-12-11T17:45:27.166 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:27.161+0000 7fe295229700 1 -- 172.21.15.59:0/2497100349 learned_addr learned my addr 172.21.15.59:0/2497100349 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:45:27.166 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:27.161+0000 7fe295a2a700 1 --2- 172.21.15.59:0/2497100349 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe290068f70 0x7fe29019d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:27.166 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:27.161+0000 7fe29622b700 1 --2- 172.21.15.59:0/2497100349 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe290107620 0x7fe290197840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:27.167 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:27.162+0000 7fe295229700 1 -- 172.21.15.59:0/2497100349 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe290107620 msgr2=0x7fe290197840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:27.167 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:27.162+0000 7fe295229700 1 --2- 172.21.15.59:0/2497100349 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe290107620 0x7fe290197840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:27.167 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:27.162+0000 7fe295229700 1 -- 172.21.15.59:0/2497100349 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe290068f70 msgr2=0x7fe29019d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:27.167 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:27.162+0000 7fe295229700 1 --2- 172.21.15.59:0/2497100349 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe290068f70 0x7fe29019d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:27.167 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:27.162+0000 7fe295229700 1 -- 172.21.15.59:0/2497100349 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe28c0095f0 con 0x7fe290069930 2023-12-11T17:45:27.167 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:27.162+0000 7fe295a2a700 1 --2- 172.21.15.59:0/2497100349 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe290068f70 0x7fe29019d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T17:45:27.167 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:27.162+0000 7fe295229700 1 --2- 172.21.15.59:0/2497100349 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe290069930 0x7fe29019dc40 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fe28000d950 tx=0x7fe28000dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:27.167 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:27.162+0000 7fe286ffd700 1 -- 172.21.15.59:0/2497100349 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe280009b40 con 0x7fe290069930 2023-12-11T17:45:27.167 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:27.162+0000 7fe286ffd700 1 -- 172.21.15.59:0/2497100349 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe280009ca0 con 0x7fe290069930 2023-12-11T17:45:27.167 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:27.162+0000 7fe297c8e700 1 -- 172.21.15.59:0/2497100349 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2901984c0 con 0x7fe290069930 2023-12-11T17:45:27.167 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:27.162+0000 7fe286ffd700 1 -- 172.21.15.59:0/2497100349 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe28000f620 con 0x7fe290069930 2023-12-11T17:45:27.168 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:27.162+0000 7fe297c8e700 1 -- 172.21.15.59:0/2497100349 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2901a46f0 con 0x7fe290069930 2023-12-11T17:45:27.168 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:27.163+0000 7fe297c8e700 1 -- 172.21.15.59:0/2497100349 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe290109b00 con 0x7fe290069930 2023-12-11T17:45:27.168 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:27.165+0000 7fe286ffd700 1 -- 172.21.15.59:0/2497100349 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7fe280010460 con 0x7fe290069930 2023-12-11T17:45:27.168 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:27.165+0000 7fe286ffd700 1 --2- 172.21.15.59:0/2497100349 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fe27c0a1590 0x7fe27c0a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:27.168 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:27.165+0000 7fe295a2a700 1 --2- 172.21.15.59:0/2497100349 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fe27c0a1590 0x7fe27c0a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:27.168 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:27.165+0000 7fe286ffd700 1 -- 172.21.15.59:0/2497100349 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fe2800cd5d0 con 0x7fe290069930 2023-12-11T17:45:27.169 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:27.166+0000 7fe295a2a700 1 --2- 172.21.15.59:0/2497100349 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fe27c0a1590 0x7fe27c0a3a50 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe28c0036c0 tx=0x7fe28c01a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:27.171 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:27.169+0000 7fe286ffd700 1 -- 172.21.15.59:0/2497100349 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fe2800967d0 con 0x7fe290069930 2023-12-11T17:45:27.273 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:27.270+0000 7f05dd748700 1 -- 172.21.15.59:0/2288723527 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f05d8101bd0 con 0x7f05d8100530 2023-12-11T17:45:27.273 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:27.271+0000 7f05d4ff9700 1 -- 172.21.15.59:0/2288723527 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f05c8026020 con 0x7f05d8100530 2023-12-11T17:45:27.274 INFO:teuthology.orchestra.run.smithi059.stdout:214748364841 2023-12-11T17:45:27.277 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:27.274+0000 7f05dd748700 1 -- 172.21.15.59:0/2288723527 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f05c40a1590 msgr2=0x7f05c40a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:27.277 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:27.274+0000 7f05dd748700 1 --2- 172.21.15.59:0/2288723527 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f05c40a1590 0x7f05c40a3a50 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f05c0009fd0 tx=0x7f05c0005fd0 comp rx=0 tx=0).stop 2023-12-11T17:45:27.277 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:27.274+0000 7f05dd748700 1 -- 172.21.15.59:0/2288723527 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f05d8100530 msgr2=0x7f05d8198790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:27.277 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:27.274+0000 7f05dd748700 1 --2- 172.21.15.59:0/2288723527 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f05d8100530 0x7f05d8198790 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f05c800e9d0 tx=0x7f05c800ece0 comp rx=0 tx=0).stop 2023-12-11T17:45:27.277 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:27.274+0000 7f05dd748700 1 -- 172.21.15.59:0/2288723527 shutdown_connections 2023-12-11T17:45:27.277 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:27.274+0000 7f05dd748700 1 --2- 172.21.15.59:0/2288723527 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f05c40a1590 0x7f05c40a3a50 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:27.277 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:27.274+0000 7f05dd748700 1 --2- 172.21.15.59:0/2288723527 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f05d8106560 0x7f05d81a3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:27.277 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:27.274+0000 7f05dd748700 1 --2- 172.21.15.59:0/2288723527 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f05d8100530 0x7f05d8198790 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:27.277 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:27.274+0000 7f05dd748700 1 --2- 172.21.15.59:0/2288723527 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f05d8100ef0 0x7f05d8198cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:27.277 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:27.274+0000 7f05dd748700 1 -- 172.21.15.59:0/2288723527 >> 172.21.15.59:0/2288723527 conn(0x7f05d8076570 msgr2=0x7f05d80ff280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:45:27.277 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:27.274+0000 7f05dd748700 1 -- 172.21.15.59:0/2288723527 shutdown_connections 2023-12-11T17:45:27.278 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:27.274+0000 7f05dd748700 1 -- 172.21.15.59:0/2288723527 wait complete. 2023-12-11T17:45:27.317 INFO:tasks.cephadm.ceph_manager.ceph:need seq 158913789999 got 158913790001 for osd.5 2023-12-11T17:45:27.317 DEBUG:teuthology.parallel:result is None 2023-12-11T17:45:27.386 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:27.383+0000 7fe297c8e700 1 -- 172.21.15.59:0/2497100349 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fe290102cc0 con 0x7fe290069930 2023-12-11T17:45:27.386 INFO:teuthology.orchestra.run.smithi059.stdout:55834574915 2023-12-11T17:45:27.387 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:27.384+0000 7fe286ffd700 1 -- 172.21.15.59:0/2497100349 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe280020090 con 0x7fe290069930 2023-12-11T17:45:27.388 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:27.386+0000 7fe297c8e700 1 -- 172.21.15.59:0/2497100349 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fe27c0a1590 msgr2=0x7fe27c0a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:27.388 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:27.386+0000 7fe297c8e700 1 --2- 172.21.15.59:0/2497100349 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fe27c0a1590 0x7fe27c0a3a50 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe28c0036c0 tx=0x7fe28c01a040 comp rx=0 tx=0).stop 2023-12-11T17:45:27.388 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:27.386+0000 7fe297c8e700 1 -- 172.21.15.59:0/2497100349 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe290069930 msgr2=0x7fe29019dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:27.389 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:27.386+0000 7fe297c8e700 1 --2- 172.21.15.59:0/2497100349 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe290069930 0x7fe29019dc40 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fe28000d950 tx=0x7fe28000dd10 comp rx=0 tx=0).stop 2023-12-11T17:45:27.389 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:27.387+0000 7fe297c8e700 1 -- 172.21.15.59:0/2497100349 shutdown_connections 2023-12-11T17:45:27.390 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:27.387+0000 7fe297c8e700 1 --2- 172.21.15.59:0/2497100349 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fe27c0a1590 0x7fe27c0a3a50 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:27.390 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:27.387+0000 7fe297c8e700 1 --2- 172.21.15.59:0/2497100349 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe290107620 0x7fe290197840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:27.390 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:27.387+0000 7fe297c8e700 1 --2- 172.21.15.59:0/2497100349 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe290068f70 0x7fe29019d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:27.390 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:27.387+0000 7fe297c8e700 1 --2- 172.21.15.59:0/2497100349 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe290069930 0x7fe29019dc40 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:27.390 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:27.387+0000 7fe297c8e700 1 -- 172.21.15.59:0/2497100349 >> 172.21.15.59:0/2497100349 conn(0x7fe290076570 msgr2=0x7fe2900ff2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:45:27.390 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:27.387+0000 7fe297c8e700 1 -- 172.21.15.59:0/2497100349 shutdown_connections 2023-12-11T17:45:27.390 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:27.387+0000 7fe297c8e700 1 -- 172.21.15.59:0/2497100349 wait complete. 2023-12-11T17:45:27.760 INFO:tasks.cephadm.ceph_manager.ceph:need seq 214748364839 got 214748364841 for osd.7 2023-12-11T17:45:27.761 DEBUG:teuthology.parallel:result is None 2023-12-11T17:45:28.068 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:45:27 smithi059 ceph-mon[98847]: pgmap v294: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 299 B/s wr, 5 op/s; 155 B/s, 0 objects/s recovering 2023-12-11T17:45:28.068 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:45:27 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/2288723527' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-12-11T17:45:28.068 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:45:27 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/2497100349' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-12-11T17:45:28.069 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574913 got 55834574915 for osd.1 2023-12-11T17:45:28.069 DEBUG:teuthology.parallel:result is None 2023-12-11T17:45:28.069 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2023-12-11T17:45:28.069 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell --fsid d0a07eaa-984b-11ee-95a3-87774f69a715 -- ceph pg dump --format=json 2023-12-11T17:45:28.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:45:27 smithi115 ceph-mon[102748]: pgmap v294: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 299 B/s wr, 5 op/s; 155 B/s, 0 objects/s recovering 2023-12-11T17:45:28.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:45:27 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/2288723527' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-12-11T17:45:28.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:45:27 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/2497100349' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-12-11T17:45:28.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:45:27 smithi181 ceph-mon[103627]: pgmap v294: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 299 B/s wr, 5 op/s; 155 B/s, 0 objects/s recovering 2023-12-11T17:45:28.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:45:27 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/2288723527' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-12-11T17:45:28.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:45:27 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/2497100349' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-12-11T17:45:28.841 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/mon.a/config 2023-12-11T17:45:30.066 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:30.063+0000 7f3d6c112700 1 -- 172.21.15.59:0/1305498067 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3d64100ef0 msgr2=0x7f3d6410d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:30.066 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:30.063+0000 7f3d6c112700 1 --2- 172.21.15.59:0/1305498067 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3d64100ef0 0x7f3d6410d330 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c00b0d0 tx=0x7f3d5c00b3e0 comp rx=0 tx=0).stop 2023-12-11T17:45:30.066 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:30.063+0000 7f3d6c112700 1 -- 172.21.15.59:0/1305498067 shutdown_connections 2023-12-11T17:45:30.066 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:30.063+0000 7f3d6c112700 1 --2- 172.21.15.59:0/1305498067 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3d64100ef0 0x7f3d6410d330 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:30.067 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:30.063+0000 7f3d6c112700 1 --2- 172.21.15.59:0/1305498067 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3d64106560 0x7f3d64106940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:30.067 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:30.063+0000 7f3d6c112700 1 --2- 172.21.15.59:0/1305498067 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d64100530 0x7f3d641009b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:30.067 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:30.063+0000 7f3d6c112700 1 -- 172.21.15.59:0/1305498067 >> 172.21.15.59:0/1305498067 conn(0x7f3d64076570 msgr2=0x7f3d64076980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:45:30.067 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:30.063+0000 7f3d6c112700 1 -- 172.21.15.59:0/1305498067 shutdown_connections 2023-12-11T17:45:30.067 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:30.064+0000 7f3d6c112700 1 -- 172.21.15.59:0/1305498067 wait complete. 2023-12-11T17:45:30.069 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:30.064+0000 7f3d6c112700 1 Processor -- start 2023-12-11T17:45:30.069 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:30.064+0000 7f3d6c112700 1 -- start start 2023-12-11T17:45:30.069 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:30.064+0000 7f3d6c112700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3d64100530 0x7f3d641989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:30.069 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:30.064+0000 7f3d6c112700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d64100ef0 0x7f3d64198f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:30.069 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:30.064+0000 7f3d6c112700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3d64106560 0x7f3d64199480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:30.069 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:30.064+0000 7f3d6c112700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d641a3cd0 con 0x7f3d64100530 2023-12-11T17:45:30.069 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:30.064+0000 7f3d6c112700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d641a3e40 con 0x7f3d64106560 2023-12-11T17:45:30.069 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:30.064+0000 7f3d6c112700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d641a4000 con 0x7f3d64100ef0 2023-12-11T17:45:30.069 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:30.065+0000 7f3d69eae700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3d64100530 0x7f3d641989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:30.069 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:30.065+0000 7f3d69eae700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3d64100530 0x7f3d641989e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33878/0 (socket says 172.21.15.59:33878) 2023-12-11T17:45:30.069 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:30.065+0000 7f3d69eae700 1 -- 172.21.15.59:0/491493580 learned_addr learned my addr 172.21.15.59:0/491493580 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:45:30.070 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:30.065+0000 7f3d696ad700 1 --2- 172.21.15.59:0/491493580 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d64100ef0 0x7f3d64198f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:30.070 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:30.065+0000 7f3d6a6af700 1 --2- 172.21.15.59:0/491493580 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3d64106560 0x7f3d64199480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:30.070 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:30.065+0000 7f3d69eae700 1 -- 172.21.15.59:0/491493580 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d64100ef0 msgr2=0x7f3d64198f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:30.070 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:30.065+0000 7f3d69eae700 1 --2- 172.21.15.59:0/491493580 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d64100ef0 0x7f3d64198f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:30.070 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:30.065+0000 7f3d69eae700 1 -- 172.21.15.59:0/491493580 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3d64106560 msgr2=0x7f3d64199480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:30.070 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:30.065+0000 7f3d69eae700 1 --2- 172.21.15.59:0/491493580 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3d64106560 0x7f3d64199480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:30.070 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:30.065+0000 7f3d69eae700 1 -- 172.21.15.59:0/491493580 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d5c009d00 con 0x7f3d64100530 2023-12-11T17:45:30.070 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:30.065+0000 7f3d69eae700 1 --2- 172.21.15.59:0/491493580 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3d64100530 0x7f3d641989e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f3d6000e9a0 tx=0x7f3d6000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:30.070 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:30.065+0000 7f3d5affd700 1 -- 172.21.15.59:0/491493580 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d60009790 con 0x7f3d64100530 2023-12-11T17:45:30.070 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:30.065+0000 7f3d6c112700 1 -- 172.21.15.59:0/491493580 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d641a42e0 con 0x7f3d64100530 2023-12-11T17:45:30.070 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:30.065+0000 7f3d5affd700 1 -- 172.21.15.59:0/491493580 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3d6000cbd0 con 0x7f3d64100530 2023-12-11T17:45:30.070 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:30.065+0000 7f3d5affd700 1 -- 172.21.15.59:0/491493580 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d60018680 con 0x7f3d64100530 2023-12-11T17:45:30.070 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:30.065+0000 7f3d6c112700 1 -- 172.21.15.59:0/491493580 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d641a47e0 con 0x7f3d64100530 2023-12-11T17:45:30.070 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:30.066+0000 7f3d6c112700 1 -- 172.21.15.59:0/491493580 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d64108a10 con 0x7f3d64100530 2023-12-11T17:45:30.071 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:30.068+0000 7f3d5affd700 1 -- 172.21.15.59:0/491493580 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f3d6000cd40 con 0x7f3d64100530 2023-12-11T17:45:30.071 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:30.068+0000 7f3d5affd700 1 --2- 172.21.15.59:0/491493580 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f3d500a1510 0x7f3d500a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:30.071 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:30.068+0000 7f3d696ad700 1 --2- 172.21.15.59:0/491493580 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f3d500a1510 0x7f3d500a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:30.071 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:30.068+0000 7f3d5affd700 1 -- 172.21.15.59:0/491493580 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f3d60014070 con 0x7f3d64100530 2023-12-11T17:45:30.071 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:30.069+0000 7f3d696ad700 1 --2- 172.21.15.59:0/491493580 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f3d500a1510 0x7f3d500a39d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f3d64069710 tx=0x7f3d5400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:30.074 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:30.072+0000 7f3d5affd700 1 -- 172.21.15.59:0/491493580 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f3d60096870 con 0x7f3d64100530 2023-12-11T17:45:30.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:45:29 smithi115 ceph-mon[102748]: pgmap v295: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 6 op/s; 132 B/s, 0 objects/s recovering 2023-12-11T17:45:30.291 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:30.289+0000 7f3d6c112700 1 -- 172.21.15.59:0/491493580 --> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3d64069530 con 0x7f3d500a1510 2023-12-11T17:45:30.308 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:30.304+0000 7f3d5affd700 1 -- 172.21.15.59:0/491493580 <== mgr.14150 v2:172.21.15.59:6800/3627728353 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+426606 (secure 0 0 0) 0x7f3d64069530 con 0x7f3d500a1510 2023-12-11T17:45:30.314 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-12-11T17:45:30.317 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:30.308+0000 7f3d6c112700 1 -- 172.21.15.59:0/491493580 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f3d500a1510 msgr2=0x7f3d500a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:30.318 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:30.308+0000 7f3d6c112700 1 --2- 172.21.15.59:0/491493580 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f3d500a1510 0x7f3d500a39d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f3d64069710 tx=0x7f3d5400b410 comp rx=0 tx=0).stop 2023-12-11T17:45:30.318 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:30.308+0000 7f3d6c112700 1 -- 172.21.15.59:0/491493580 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3d64100530 msgr2=0x7f3d641989e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:30.318 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:30.308+0000 7f3d6c112700 1 --2- 172.21.15.59:0/491493580 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3d64100530 0x7f3d641989e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f3d6000e9a0 tx=0x7f3d6000ecb0 comp rx=0 tx=0).stop 2023-12-11T17:45:30.318 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:30.308+0000 7f3d6c112700 1 -- 172.21.15.59:0/491493580 shutdown_connections 2023-12-11T17:45:30.318 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:30.308+0000 7f3d6c112700 1 --2- 172.21.15.59:0/491493580 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f3d500a1510 0x7f3d500a39d0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:30.318 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:30.308+0000 7f3d6c112700 1 --2- 172.21.15.59:0/491493580 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3d64106560 0x7f3d64199480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:30.318 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:30.308+0000 7f3d6c112700 1 --2- 172.21.15.59:0/491493580 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3d64100530 0x7f3d641989e0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:30.318 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:30.308+0000 7f3d6c112700 1 --2- 172.21.15.59:0/491493580 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d64100ef0 0x7f3d64198f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:30.318 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:30.308+0000 7f3d6c112700 1 -- 172.21.15.59:0/491493580 >> 172.21.15.59:0/491493580 conn(0x7f3d64076570 msgr2=0x7f3d640ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:45:30.318 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:30.308+0000 7f3d6c112700 1 -- 172.21.15.59:0/491493580 shutdown_connections 2023-12-11T17:45:30.318 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:30.308+0000 7f3d6c112700 1 -- 172.21.15.59:0/491493580 wait complete. 2023-12-11T17:45:30.318 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2023-12-11T17:45:30.338 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:45:29 smithi059 ceph-mon[98847]: pgmap v295: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 6 op/s; 132 B/s, 0 objects/s recovering 2023-12-11T17:45:30.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:45:29 smithi181 ceph-mon[103627]: pgmap v295: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 6 op/s; 132 B/s, 0 objects/s recovering 2023-12-11T17:45:30.804 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":296,"stamp":"2023-12-11T17:45:30.130732+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655998,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":395,"num_read_kb":321,"num_write":342,"num_write_kb":3054,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1182329,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":278,"ondisk_log_size":278,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":336020,"kb_used_data":11756,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124605804,"statfs":{"total":1151940427776,"available":1151596343296,"internally_reserved":0,"allocated":12038144,"data_stored":7745912,"data_compressed":319479,"data_compressed_allocated":430080,"data_compressed_original":2359296,"omap_allocated":19142,"internal_metadata":331855162},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":57,"num_read_kb":30,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1011,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003546"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.880811+0000","last_change":"2023-12-11T17:44:20.624256+0000","last_active":"2023-12-11T17:45:14.880811+0000","last_peered":"2023-12-11T17:45:14.880811+0000","last_clean":"2023-12-11T17:45:14.880811+0000","last_became_active":"2023-12-11T17:44:20.623947+0000","last_became_peered":"2023-12-11T17:44:20.623947+0000","last_unstale":"2023-12-11T17:45:14.880811+0000","last_undegraded":"2023-12-11T17:45:14.880811+0000","last_fullsized":"2023-12-11T17:45:14.880811+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T18:12:43.669849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.874575+0000","last_change":"2023-12-11T17:44:20.620606+0000","last_active":"2023-12-11T17:45:14.874575+0000","last_peered":"2023-12-11T17:45:14.874575+0000","last_clean":"2023-12-11T17:45:14.874575+0000","last_became_active":"2023-12-11T17:44:20.620265+0000","last_became_peered":"2023-12-11T17:44:20.620265+0000","last_unstale":"2023-12-11T17:45:14.874575+0000","last_undegraded":"2023-12-11T17:45:14.874575+0000","last_fullsized":"2023-12-11T17:45:14.874575+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T19:53:35.349842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.874216+0000","last_change":"2023-12-11T17:44:20.620702+0000","last_active":"2023-12-11T17:45:14.874216+0000","last_peered":"2023-12-11T17:45:14.874216+0000","last_clean":"2023-12-11T17:45:14.874216+0000","last_became_active":"2023-12-11T17:44:20.620401+0000","last_became_peered":"2023-12-11T17:44:20.620401+0000","last_unstale":"2023-12-11T17:45:14.874216+0000","last_undegraded":"2023-12-11T17:45:14.874216+0000","last_fullsized":"2023-12-11T17:45:14.874216+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T21:14:21.156270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.876828+0000","last_change":"2023-12-11T17:44:20.614263+0000","last_active":"2023-12-11T17:45:14.876828+0000","last_peered":"2023-12-11T17:45:14.876828+0000","last_clean":"2023-12-11T17:45:14.876828+0000","last_became_active":"2023-12-11T17:44:20.613585+0000","last_became_peered":"2023-12-11T17:44:20.613585+0000","last_unstale":"2023-12-11T17:45:14.876828+0000","last_undegraded":"2023-12-11T17:45:14.876828+0000","last_fullsized":"2023-12-11T17:45:14.876828+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T23:03:26.514011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.873561+0000","last_change":"2023-12-11T17:44:20.615553+0000","last_active":"2023-12-11T17:45:14.873561+0000","last_peered":"2023-12-11T17:45:14.873561+0000","last_clean":"2023-12-11T17:45:14.873561+0000","last_became_active":"2023-12-11T17:44:20.615272+0000","last_became_peered":"2023-12-11T17:44:20.615272+0000","last_unstale":"2023-12-11T17:45:14.873561+0000","last_undegraded":"2023-12-11T17:45:14.873561+0000","last_fullsized":"2023-12-11T17:45:14.873561+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T20:37:02.178411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T17:45:15.882116+0000","last_change":"2023-12-11T17:45:15.882116+0000","last_active":"2023-12-11T17:45:15.882116+0000","last_peered":"2023-12-11T17:45:15.882116+0000","last_clean":"2023-12-11T17:45:15.882116+0000","last_became_active":"2023-12-11T17:45:15.881729+0000","last_became_peered":"2023-12-11T17:45:15.881729+0000","last_unstale":"2023-12-11T17:45:15.882116+0000","last_undegraded":"2023-12-11T17:45:15.882116+0000","last_fullsized":"2023-12-11T17:45:15.882116+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T05:39:09.190471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.873661+0000","last_change":"2023-12-11T17:44:20.623828+0000","last_active":"2023-12-11T17:45:14.873661+0000","last_peered":"2023-12-11T17:45:14.873661+0000","last_clean":"2023-12-11T17:45:14.873661+0000","last_became_active":"2023-12-11T17:44:20.623510+0000","last_became_peered":"2023-12-11T17:44:20.623510+0000","last_unstale":"2023-12-11T17:45:14.873661+0000","last_undegraded":"2023-12-11T17:45:14.873661+0000","last_fullsized":"2023-12-11T17:45:14.873661+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T22:04:32.298900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.880981+0000","last_change":"2023-12-11T17:44:20.620831+0000","last_active":"2023-12-11T17:45:14.880981+0000","last_peered":"2023-12-11T17:45:14.880981+0000","last_clean":"2023-12-11T17:45:14.880981+0000","last_became_active":"2023-12-11T17:44:20.620481+0000","last_became_peered":"2023-12-11T17:44:20.620481+0000","last_unstale":"2023-12-11T17:45:14.880981+0000","last_undegraded":"2023-12-11T17:45:14.880981+0000","last_fullsized":"2023-12-11T17:45:14.880981+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T22:14:28.838192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.880343+0000","last_change":"2023-12-11T17:44:20.619515+0000","last_active":"2023-12-11T17:45:14.880343+0000","last_peered":"2023-12-11T17:45:14.880343+0000","last_clean":"2023-12-11T17:45:14.880343+0000","last_became_active":"2023-12-11T17:44:20.619219+0000","last_became_peered":"2023-12-11T17:44:20.619219+0000","last_unstale":"2023-12-11T17:45:14.880343+0000","last_undegraded":"2023-12-11T17:45:14.880343+0000","last_fullsized":"2023-12-11T17:45:14.880343+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T23:08:55.112752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T17:45:15.885567+0000","last_change":"2023-12-11T17:45:15.885567+0000","last_active":"2023-12-11T17:45:15.885567+0000","last_peered":"2023-12-11T17:45:15.885567+0000","last_clean":"2023-12-11T17:45:15.885567+0000","last_became_active":"2023-12-11T17:45:15.885207+0000","last_became_peered":"2023-12-11T17:45:15.885207+0000","last_unstale":"2023-12-11T17:45:15.885567+0000","last_undegraded":"2023-12-11T17:45:15.885567+0000","last_fullsized":"2023-12-11T17:45:15.885567+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T01:24:25.763022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.874115+0000","last_change":"2023-12-11T17:44:20.621198+0000","last_active":"2023-12-11T17:45:14.874115+0000","last_peered":"2023-12-11T17:45:14.874115+0000","last_clean":"2023-12-11T17:45:14.874115+0000","last_became_active":"2023-12-11T17:44:20.620693+0000","last_became_peered":"2023-12-11T17:44:20.620693+0000","last_unstale":"2023-12-11T17:45:14.874115+0000","last_undegraded":"2023-12-11T17:45:14.874115+0000","last_fullsized":"2023-12-11T17:45:14.874115+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T22:10:23.396031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.876754+0000","last_change":"2023-12-11T17:44:20.614457+0000","last_active":"2023-12-11T17:45:14.876754+0000","last_peered":"2023-12-11T17:45:14.876754+0000","last_clean":"2023-12-11T17:45:14.876754+0000","last_became_active":"2023-12-11T17:44:20.613984+0000","last_became_peered":"2023-12-11T17:44:20.613984+0000","last_unstale":"2023-12-11T17:45:14.876754+0000","last_undegraded":"2023-12-11T17:45:14.876754+0000","last_fullsized":"2023-12-11T17:45:14.876754+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T00:50:01.101398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.880158+0000","last_change":"2023-12-11T17:44:20.619874+0000","last_active":"2023-12-11T17:45:14.880158+0000","last_peered":"2023-12-11T17:45:14.880158+0000","last_clean":"2023-12-11T17:45:14.880158+0000","last_became_active":"2023-12-11T17:44:20.618537+0000","last_became_peered":"2023-12-11T17:44:20.618537+0000","last_unstale":"2023-12-11T17:45:14.880158+0000","last_undegraded":"2023-12-11T17:45:14.880158+0000","last_fullsized":"2023-12-11T17:45:14.880158+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T18:42:43.587245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.873528+0000","last_change":"2023-12-11T17:44:20.616306+0000","last_active":"2023-12-11T17:45:14.873528+0000","last_peered":"2023-12-11T17:45:14.873528+0000","last_clean":"2023-12-11T17:45:14.873528+0000","last_became_active":"2023-12-11T17:44:20.615930+0000","last_became_peered":"2023-12-11T17:44:20.615930+0000","last_unstale":"2023-12-11T17:45:14.873528+0000","last_undegraded":"2023-12-11T17:45:14.873528+0000","last_fullsized":"2023-12-11T17:45:14.873528+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T05:40:24.495052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.873498+0000","last_change":"2023-12-11T17:44:20.624339+0000","last_active":"2023-12-11T17:45:14.873498+0000","last_peered":"2023-12-11T17:45:14.873498+0000","last_clean":"2023-12-11T17:45:14.873498+0000","last_became_active":"2023-12-11T17:44:20.624045+0000","last_became_peered":"2023-12-11T17:44:20.624045+0000","last_unstale":"2023-12-11T17:45:14.873498+0000","last_undegraded":"2023-12-11T17:45:14.873498+0000","last_fullsized":"2023-12-11T17:45:14.873498+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T01:32:30.792940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.879866+0000","last_change":"2023-12-11T17:44:20.620750+0000","last_active":"2023-12-11T17:45:14.879866+0000","last_peered":"2023-12-11T17:45:14.879866+0000","last_clean":"2023-12-11T17:45:14.879866+0000","last_became_active":"2023-12-11T17:44:20.620464+0000","last_became_peered":"2023-12-11T17:44:20.620464+0000","last_unstale":"2023-12-11T17:45:14.879866+0000","last_undegraded":"2023-12-11T17:45:14.879866+0000","last_fullsized":"2023-12-11T17:45:14.879866+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T05:43:33.680889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.880834+0000","last_change":"2023-12-11T17:44:20.624391+0000","last_active":"2023-12-11T17:45:14.880834+0000","last_peered":"2023-12-11T17:45:14.880834+0000","last_clean":"2023-12-11T17:45:14.880834+0000","last_became_active":"2023-12-11T17:44:20.624217+0000","last_became_peered":"2023-12-11T17:44:20.624217+0000","last_unstale":"2023-12-11T17:45:14.880834+0000","last_undegraded":"2023-12-11T17:45:14.880834+0000","last_fullsized":"2023-12-11T17:45:14.880834+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T01:06:41.660726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.877604+0000","last_change":"2023-12-11T17:44:20.612429+0000","last_active":"2023-12-11T17:45:14.877604+0000","last_peered":"2023-12-11T17:45:14.877604+0000","last_clean":"2023-12-11T17:45:14.877604+0000","last_became_active":"2023-12-11T17:44:20.612082+0000","last_became_peered":"2023-12-11T17:44:20.612082+0000","last_unstale":"2023-12-11T17:45:14.877604+0000","last_undegraded":"2023-12-11T17:45:14.877604+0000","last_fullsized":"2023-12-11T17:45:14.877604+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T02:34:42.364876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.877789+0000","last_change":"2023-12-11T17:44:20.614907+0000","last_active":"2023-12-11T17:45:14.877789+0000","last_peered":"2023-12-11T17:45:14.877789+0000","last_clean":"2023-12-11T17:45:14.877789+0000","last_became_active":"2023-12-11T17:44:20.614611+0000","last_became_peered":"2023-12-11T17:44:20.614611+0000","last_unstale":"2023-12-11T17:45:14.877789+0000","last_undegraded":"2023-12-11T17:45:14.877789+0000","last_fullsized":"2023-12-11T17:45:14.877789+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T05:29:50.095253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.877113+0000","last_change":"2023-12-11T17:44:24.690974+0000","last_active":"2023-12-11T17:45:14.877113+0000","last_peered":"2023-12-11T17:45:14.877113+0000","last_clean":"2023-12-11T17:45:14.877113+0000","last_became_active":"2023-12-11T17:44:24.690579+0000","last_became_peered":"2023-12-11T17:44:24.690579+0000","last_unstale":"2023-12-11T17:45:14.877113+0000","last_undegraded":"2023-12-11T17:45:14.877113+0000","last_fullsized":"2023-12-11T17:45:14.877113+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_clean_scrub_stamp":"2023-12-11T17:44:23.663161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:31:59.830486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'9","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T17:45:15.895326+0000","last_change":"2023-12-11T17:45:15.895326+0000","last_active":"2023-12-11T17:45:15.895326+0000","last_peered":"2023-12-11T17:45:15.895326+0000","last_clean":"2023-12-11T17:45:15.895326+0000","last_became_active":"2023-12-11T17:45:15.884451+0000","last_became_peered":"2023-12-11T17:45:15.884451+0000","last_unstale":"2023-12-11T17:45:15.895326+0000","last_undegraded":"2023-12-11T17:45:15.895326+0000","last_fullsized":"2023-12-11T17:45:15.895326+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_clean_scrub_stamp":"2023-12-11T17:44:18.636443+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T04:07:28.420942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,8],"acting":[9,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.881499+0000","last_change":"2023-12-11T17:44:20.621928+0000","last_active":"2023-12-11T17:45:14.881499+0000","last_peered":"2023-12-11T17:45:14.881499+0000","last_clean":"2023-12-11T17:45:14.881499+0000","last_became_active":"2023-12-11T17:44:20.621575+0000","last_became_peered":"2023-12-11T17:44:20.621575+0000","last_unstale":"2023-12-11T17:45:14.881499+0000","last_undegraded":"2023-12-11T17:45:14.881499+0000","last_fullsized":"2023-12-11T17:45:14.881499+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T17:57:44.860200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.876251+0000","last_change":"2023-12-11T17:44:24.704170+0000","last_active":"2023-12-11T17:45:14.876251+0000","last_peered":"2023-12-11T17:45:14.876251+0000","last_clean":"2023-12-11T17:45:14.876251+0000","last_became_active":"2023-12-11T17:44:24.703800+0000","last_became_peered":"2023-12-11T17:44:24.703800+0000","last_unstale":"2023-12-11T17:45:14.876251+0000","last_undegraded":"2023-12-11T17:45:14.876251+0000","last_fullsized":"2023-12-11T17:45:14.876251+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_clean_scrub_stamp":"2023-12-11T17:44:23.663161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T01:26:29.578910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.881742+0000","last_change":"2023-12-11T17:44:19.659660+0000","last_active":"2023-12-11T17:45:14.881742+0000","last_peered":"2023-12-11T17:45:14.881742+0000","last_clean":"2023-12-11T17:45:14.881742+0000","last_became_active":"2023-12-11T17:44:19.659283+0000","last_became_peered":"2023-12-11T17:44:19.659283+0000","last_unstale":"2023-12-11T17:45:14.881742+0000","last_undegraded":"2023-12-11T17:45:14.881742+0000","last_fullsized":"2023-12-11T17:45:14.881742+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_clean_scrub_stamp":"2023-12-11T17:44:18.636443+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T22:29:43.427794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.875448+0000","last_change":"2023-12-11T17:44:20.624962+0000","last_active":"2023-12-11T17:45:14.875448+0000","last_peered":"2023-12-11T17:45:14.875448+0000","last_clean":"2023-12-11T17:45:14.875448+0000","last_became_active":"2023-12-11T17:44:20.624640+0000","last_became_peered":"2023-12-11T17:44:20.624640+0000","last_unstale":"2023-12-11T17:45:14.875448+0000","last_undegraded":"2023-12-11T17:45:14.875448+0000","last_fullsized":"2023-12-11T17:45:14.875448+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T23:37:43.252062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.876519+0000","last_change":"2023-12-11T17:44:24.698866+0000","last_active":"2023-12-11T17:45:14.876519+0000","last_peered":"2023-12-11T17:45:14.876519+0000","last_clean":"2023-12-11T17:45:14.876519+0000","last_became_active":"2023-12-11T17:44:24.698481+0000","last_became_peered":"2023-12-11T17:44:24.698481+0000","last_unstale":"2023-12-11T17:45:14.876519+0000","last_undegraded":"2023-12-11T17:45:14.876519+0000","last_fullsized":"2023-12-11T17:45:14.876519+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_clean_scrub_stamp":"2023-12-11T17:44:23.663161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T20:50:19.571616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.873946+0000","last_change":"2023-12-11T17:44:19.662637+0000","last_active":"2023-12-11T17:45:14.873946+0000","last_peered":"2023-12-11T17:45:14.873946+0000","last_clean":"2023-12-11T17:45:14.873946+0000","last_became_active":"2023-12-11T17:44:19.662355+0000","last_became_peered":"2023-12-11T17:44:19.662355+0000","last_unstale":"2023-12-11T17:45:14.873946+0000","last_undegraded":"2023-12-11T17:45:14.873946+0000","last_fullsized":"2023-12-11T17:45:14.873946+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_clean_scrub_stamp":"2023-12-11T17:44:18.636443+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T17:56:37.436037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.879884+0000","last_change":"2023-12-11T17:44:20.615119+0000","last_active":"2023-12-11T17:45:14.879884+0000","last_peered":"2023-12-11T17:45:14.879884+0000","last_clean":"2023-12-11T17:45:14.879884+0000","last_became_active":"2023-12-11T17:44:20.614775+0000","last_became_peered":"2023-12-11T17:44:20.614775+0000","last_unstale":"2023-12-11T17:45:14.879884+0000","last_undegraded":"2023-12-11T17:45:14.879884+0000","last_fullsized":"2023-12-11T17:45:14.879884+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T20:45:47.162890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.876324+0000","last_change":"2023-12-11T17:44:24.707718+0000","last_active":"2023-12-11T17:45:14.876324+0000","last_peered":"2023-12-11T17:45:14.876324+0000","last_clean":"2023-12-11T17:45:14.876324+0000","last_became_active":"2023-12-11T17:44:24.707404+0000","last_became_peered":"2023-12-11T17:44:24.707404+0000","last_unstale":"2023-12-11T17:45:14.876324+0000","last_undegraded":"2023-12-11T17:45:14.876324+0000","last_fullsized":"2023-12-11T17:45:14.876324+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_clean_scrub_stamp":"2023-12-11T17:44:23.663161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T22:53:53.918649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"81'2","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T17:45:15.891766+0000","last_change":"2023-12-11T17:45:15.891299+0000","last_active":"2023-12-11T17:45:15.891766+0000","last_peered":"2023-12-11T17:45:15.891766+0000","last_clean":"2023-12-11T17:45:15.891766+0000","last_became_active":"2023-12-11T17:45:15.884838+0000","last_became_peered":"2023-12-11T17:45:15.884838+0000","last_unstale":"2023-12-11T17:45:15.891766+0000","last_undegraded":"2023-12-11T17:45:15.891766+0000","last_fullsized":"2023-12-11T17:45:15.891766+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_clean_scrub_stamp":"2023-12-11T17:44:18.636443+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T19:05:15.500425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.877473+0000","last_change":"2023-12-11T17:44:20.615894+0000","last_active":"2023-12-11T17:45:14.877473+0000","last_peered":"2023-12-11T17:45:14.877473+0000","last_clean":"2023-12-11T17:45:14.877473+0000","last_became_active":"2023-12-11T17:44:20.615582+0000","last_became_peered":"2023-12-11T17:44:20.615582+0000","last_unstale":"2023-12-11T17:45:14.877473+0000","last_undegraded":"2023-12-11T17:45:14.877473+0000","last_fullsized":"2023-12-11T17:45:14.877473+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T19:04:50.725088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.877639+0000","last_change":"2023-12-11T17:44:24.691459+0000","last_active":"2023-12-11T17:45:14.877639+0000","last_peered":"2023-12-11T17:45:14.877639+0000","last_clean":"2023-12-11T17:45:14.877639+0000","last_became_active":"2023-12-11T17:44:24.690301+0000","last_became_peered":"2023-12-11T17:44:24.690301+0000","last_unstale":"2023-12-11T17:45:14.877639+0000","last_undegraded":"2023-12-11T17:45:14.877639+0000","last_fullsized":"2023-12-11T17:45:14.877639+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_clean_scrub_stamp":"2023-12-11T17:44:23.663161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T17:58:04.685785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"86'6","reported_seq":83,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T17:45:27.906349+0000","last_change":"2023-12-11T17:45:15.894177+0000","last_active":"2023-12-11T17:45:27.906349+0000","last_peered":"2023-12-11T17:45:27.906349+0000","last_clean":"2023-12-11T17:45:27.906349+0000","last_became_active":"2023-12-11T17:45:15.887160+0000","last_became_peered":"2023-12-11T17:45:15.887160+0000","last_unstale":"2023-12-11T17:45:27.906349+0000","last_undegraded":"2023-12-11T17:45:27.906349+0000","last_fullsized":"2023-12-11T17:45:27.906349+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_clean_scrub_stamp":"2023-12-11T17:44:18.636443+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T19:45:46.073604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1011,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":37,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1011,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.873547+0000","last_change":"2023-12-11T17:44:20.608280+0000","last_active":"2023-12-11T17:45:14.873547+0000","last_peered":"2023-12-11T17:45:14.873547+0000","last_clean":"2023-12-11T17:45:14.873547+0000","last_became_active":"2023-12-11T17:44:20.607980+0000","last_became_peered":"2023-12-11T17:44:20.607980+0000","last_unstale":"2023-12-11T17:45:14.873547+0000","last_undegraded":"2023-12-11T17:45:14.873547+0000","last_fullsized":"2023-12-11T17:45:14.873547+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T04:45:56.108916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.877919+0000","last_change":"2023-12-11T17:44:24.703629+0000","last_active":"2023-12-11T17:45:14.877919+0000","last_peered":"2023-12-11T17:45:14.877919+0000","last_clean":"2023-12-11T17:45:14.877919+0000","last_became_active":"2023-12-11T17:44:24.703219+0000","last_became_peered":"2023-12-11T17:44:24.703219+0000","last_unstale":"2023-12-11T17:45:14.877919+0000","last_undegraded":"2023-12-11T17:45:14.877919+0000","last_fullsized":"2023-12-11T17:45:14.877919+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_clean_scrub_stamp":"2023-12-11T17:44:23.663161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T19:57:44.998039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.879362+0000","last_change":"2023-12-11T17:44:19.659311+0000","last_active":"2023-12-11T17:45:14.879362+0000","last_peered":"2023-12-11T17:45:14.879362+0000","last_clean":"2023-12-11T17:45:14.879362+0000","last_became_active":"2023-12-11T17:44:19.658994+0000","last_became_peered":"2023-12-11T17:44:19.658994+0000","last_unstale":"2023-12-11T17:45:14.879362+0000","last_undegraded":"2023-12-11T17:45:14.879362+0000","last_fullsized":"2023-12-11T17:45:14.879362+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_clean_scrub_stamp":"2023-12-11T17:44:18.636443+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T17:46:35.701179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.878600+0000","last_change":"2023-12-11T17:44:20.615684+0000","last_active":"2023-12-11T17:45:14.878600+0000","last_peered":"2023-12-11T17:45:14.878600+0000","last_clean":"2023-12-11T17:45:14.878600+0000","last_became_active":"2023-12-11T17:44:20.615373+0000","last_became_peered":"2023-12-11T17:44:20.615373+0000","last_unstale":"2023-12-11T17:45:14.878600+0000","last_undegraded":"2023-12-11T17:45:14.878600+0000","last_fullsized":"2023-12-11T17:45:14.878600+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T19:45:59.980253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.880529+0000","last_change":"2023-12-11T17:44:24.699085+0000","last_active":"2023-12-11T17:45:14.880529+0000","last_peered":"2023-12-11T17:45:14.880529+0000","last_clean":"2023-12-11T17:45:14.880529+0000","last_became_active":"2023-12-11T17:44:24.698717+0000","last_became_peered":"2023-12-11T17:44:24.698717+0000","last_unstale":"2023-12-11T17:45:14.880529+0000","last_undegraded":"2023-12-11T17:45:14.880529+0000","last_fullsized":"2023-12-11T17:45:14.880529+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_clean_scrub_stamp":"2023-12-11T17:44:23.663161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T05:00:52.341938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.880487+0000","last_change":"2023-12-11T17:44:19.660392+0000","last_active":"2023-12-11T17:45:14.880487+0000","last_peered":"2023-12-11T17:45:14.880487+0000","last_clean":"2023-12-11T17:45:14.880487+0000","last_became_active":"2023-12-11T17:44:19.660099+0000","last_became_peered":"2023-12-11T17:44:19.660099+0000","last_unstale":"2023-12-11T17:45:14.880487+0000","last_undegraded":"2023-12-11T17:45:14.880487+0000","last_fullsized":"2023-12-11T17:45:14.880487+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_clean_scrub_stamp":"2023-12-11T17:44:18.636443+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T00:38:37.279230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.881168+0000","last_change":"2023-12-11T17:44:20.615629+0000","last_active":"2023-12-11T17:45:14.881168+0000","last_peered":"2023-12-11T17:45:14.881168+0000","last_clean":"2023-12-11T17:45:14.881168+0000","last_became_active":"2023-12-11T17:44:20.615193+0000","last_became_peered":"2023-12-11T17:44:20.615193+0000","last_unstale":"2023-12-11T17:45:14.881168+0000","last_undegraded":"2023-12-11T17:45:14.881168+0000","last_fullsized":"2023-12-11T17:45:14.881168+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:09:56.237298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.874251+0000","last_change":"2023-12-11T17:44:24.705845+0000","last_active":"2023-12-11T17:45:14.874251+0000","last_peered":"2023-12-11T17:45:14.874251+0000","last_clean":"2023-12-11T17:45:14.874251+0000","last_became_active":"2023-12-11T17:44:24.705434+0000","last_became_peered":"2023-12-11T17:44:24.705434+0000","last_unstale":"2023-12-11T17:45:14.874251+0000","last_undegraded":"2023-12-11T17:45:14.874251+0000","last_fullsized":"2023-12-11T17:45:14.874251+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_clean_scrub_stamp":"2023-12-11T17:44:23.663161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T02:37:25.663753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.877124+0000","last_change":"2023-12-11T17:44:19.661317+0000","last_active":"2023-12-11T17:45:14.877124+0000","last_peered":"2023-12-11T17:45:14.877124+0000","last_clean":"2023-12-11T17:45:14.877124+0000","last_became_active":"2023-12-11T17:44:19.660993+0000","last_became_peered":"2023-12-11T17:44:19.660993+0000","last_unstale":"2023-12-11T17:45:14.877124+0000","last_undegraded":"2023-12-11T17:45:14.877124+0000","last_fullsized":"2023-12-11T17:45:14.877124+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_clean_scrub_stamp":"2023-12-11T17:44:18.636443+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T23:00:07.345964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.876765+0000","last_change":"2023-12-11T17:44:20.623681+0000","last_active":"2023-12-11T17:45:14.876765+0000","last_peered":"2023-12-11T17:45:14.876765+0000","last_clean":"2023-12-11T17:45:14.876765+0000","last_became_active":"2023-12-11T17:44:20.623288+0000","last_became_peered":"2023-12-11T17:44:20.623288+0000","last_unstale":"2023-12-11T17:45:14.876765+0000","last_undegraded":"2023-12-11T17:45:14.876765+0000","last_fullsized":"2023-12-11T17:45:14.876765+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T23:47:55.348480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.881054+0000","last_change":"2023-12-11T17:44:24.687613+0000","last_active":"2023-12-11T17:45:14.881054+0000","last_peered":"2023-12-11T17:45:14.881054+0000","last_clean":"2023-12-11T17:45:14.881054+0000","last_became_active":"2023-12-11T17:44:24.687240+0000","last_became_peered":"2023-12-11T17:44:24.687240+0000","last_unstale":"2023-12-11T17:45:14.881054+0000","last_undegraded":"2023-12-11T17:45:14.881054+0000","last_fullsized":"2023-12-11T17:45:14.881054+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_clean_scrub_stamp":"2023-12-11T17:44:23.663161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T20:54:25.409631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"86'3","reported_seq":62,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T17:45:25.902251+0000","last_change":"2023-12-11T17:44:19.653571+0000","last_active":"2023-12-11T17:45:25.902251+0000","last_peered":"2023-12-11T17:45:25.902251+0000","last_clean":"2023-12-11T17:45:25.902251+0000","last_became_active":"2023-12-11T17:44:19.653035+0000","last_became_peered":"2023-12-11T17:44:19.653035+0000","last_unstale":"2023-12-11T17:45:25.902251+0000","last_undegraded":"2023-12-11T17:45:25.902251+0000","last_fullsized":"2023-12-11T17:45:25.902251+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_clean_scrub_stamp":"2023-12-11T17:44:18.636443+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T02:29:38.310132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":40,"num_read_kb":40,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.880174+0000","last_change":"2023-12-11T17:44:20.623777+0000","last_active":"2023-12-11T17:45:14.880174+0000","last_peered":"2023-12-11T17:45:14.880174+0000","last_clean":"2023-12-11T17:45:14.880174+0000","last_became_active":"2023-12-11T17:44:20.623449+0000","last_became_peered":"2023-12-11T17:44:20.623449+0000","last_unstale":"2023-12-11T17:45:14.880174+0000","last_undegraded":"2023-12-11T17:45:14.880174+0000","last_fullsized":"2023-12-11T17:45:14.880174+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T05:23:03.493677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.876453+0000","last_change":"2023-12-11T17:44:24.693893+0000","last_active":"2023-12-11T17:45:14.876453+0000","last_peered":"2023-12-11T17:45:14.876453+0000","last_clean":"2023-12-11T17:45:14.876453+0000","last_became_active":"2023-12-11T17:44:24.693441+0000","last_became_peered":"2023-12-11T17:44:24.693441+0000","last_unstale":"2023-12-11T17:45:14.876453+0000","last_undegraded":"2023-12-11T17:45:14.876453+0000","last_fullsized":"2023-12-11T17:45:14.876453+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_clean_scrub_stamp":"2023-12-11T17:44:23.663161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T23:25:06.457176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.874018+0000","last_change":"2023-12-11T17:44:19.662988+0000","last_active":"2023-12-11T17:45:14.874018+0000","last_peered":"2023-12-11T17:45:14.874018+0000","last_clean":"2023-12-11T17:45:14.874018+0000","last_became_active":"2023-12-11T17:44:19.662660+0000","last_became_peered":"2023-12-11T17:44:19.662660+0000","last_unstale":"2023-12-11T17:45:14.874018+0000","last_undegraded":"2023-12-11T17:45:14.874018+0000","last_fullsized":"2023-12-11T17:45:14.874018+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_clean_scrub_stamp":"2023-12-11T17:44:18.636443+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T19:08:50.703161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T17:45:15.887273+0000","last_change":"2023-12-11T17:45:15.887273+0000","last_active":"2023-12-11T17:45:15.887273+0000","last_peered":"2023-12-11T17:45:15.887273+0000","last_clean":"2023-12-11T17:45:15.887273+0000","last_became_active":"2023-12-11T17:45:15.886905+0000","last_became_peered":"2023-12-11T17:45:15.886905+0000","last_unstale":"2023-12-11T17:45:15.887273+0000","last_undegraded":"2023-12-11T17:45:15.887273+0000","last_fullsized":"2023-12-11T17:45:15.887273+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T01:56:43.076856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,5],"acting":[0,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.876666+0000","last_change":"2023-12-11T17:44:24.698715+0000","last_active":"2023-12-11T17:45:14.876666+0000","last_peered":"2023-12-11T17:45:14.876666+0000","last_clean":"2023-12-11T17:45:14.876666+0000","last_became_active":"2023-12-11T17:44:24.698359+0000","last_became_peered":"2023-12-11T17:44:24.698359+0000","last_unstale":"2023-12-11T17:45:14.876666+0000","last_undegraded":"2023-12-11T17:45:14.876666+0000","last_fullsized":"2023-12-11T17:45:14.876666+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_clean_scrub_stamp":"2023-12-11T17:44:23.663161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T01:07:23.870922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.877378+0000","last_change":"2023-12-11T17:44:19.655914+0000","last_active":"2023-12-11T17:45:14.877378+0000","last_peered":"2023-12-11T17:45:14.877378+0000","last_clean":"2023-12-11T17:45:14.877378+0000","last_became_active":"2023-12-11T17:44:19.655593+0000","last_became_peered":"2023-12-11T17:44:19.655593+0000","last_unstale":"2023-12-11T17:45:14.877378+0000","last_undegraded":"2023-12-11T17:45:14.877378+0000","last_fullsized":"2023-12-11T17:45:14.877378+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_clean_scrub_stamp":"2023-12-11T17:44:18.636443+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T01:54:50.246100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.880594+0000","last_change":"2023-12-11T17:44:20.618469+0000","last_active":"2023-12-11T17:45:14.880594+0000","last_peered":"2023-12-11T17:45:14.880594+0000","last_clean":"2023-12-11T17:45:14.880594+0000","last_became_active":"2023-12-11T17:44:20.618186+0000","last_became_peered":"2023-12-11T17:44:20.618186+0000","last_unstale":"2023-12-11T17:45:14.880594+0000","last_undegraded":"2023-12-11T17:45:14.880594+0000","last_fullsized":"2023-12-11T17:45:14.880594+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T20:53:22.405100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.876665+0000","last_change":"2023-12-11T17:44:24.697048+0000","last_active":"2023-12-11T17:45:14.876665+0000","last_peered":"2023-12-11T17:45:14.876665+0000","last_clean":"2023-12-11T17:45:14.876665+0000","last_became_active":"2023-12-11T17:44:24.696660+0000","last_became_peered":"2023-12-11T17:44:24.696660+0000","last_unstale":"2023-12-11T17:45:14.876665+0000","last_undegraded":"2023-12-11T17:45:14.876665+0000","last_fullsized":"2023-12-11T17:45:14.876665+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_clean_scrub_stamp":"2023-12-11T17:44:23.663161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T21:50:51.018386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"87'33","reported_seq":140,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T17:45:26.903700+0000","last_change":"2023-12-11T17:44:19.653528+0000","last_active":"2023-12-11T17:45:26.903700+0000","last_peered":"2023-12-11T17:45:26.903700+0000","last_clean":"2023-12-11T17:45:26.903700+0000","last_became_active":"2023-12-11T17:44:19.653215+0000","last_became_peered":"2023-12-11T17:44:19.653215+0000","last_unstale":"2023-12-11T17:45:26.903700+0000","last_undegraded":"2023-12-11T17:45:26.903700+0000","last_fullsized":"2023-12-11T17:45:26.903700+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_clean_scrub_stamp":"2023-12-11T17:44:18.636443+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T21:19:12.024008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60996,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":88,"num_read_kb":103,"num_write":34,"num_write_kb":75,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.873815+0000","last_change":"2023-12-11T17:44:20.614085+0000","last_active":"2023-12-11T17:45:14.873815+0000","last_peered":"2023-12-11T17:45:14.873815+0000","last_clean":"2023-12-11T17:45:14.873815+0000","last_became_active":"2023-12-11T17:44:20.613768+0000","last_became_peered":"2023-12-11T17:44:20.613768+0000","last_unstale":"2023-12-11T17:45:14.873815+0000","last_undegraded":"2023-12-11T17:45:14.873815+0000","last_fullsized":"2023-12-11T17:45:14.873815+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T23:42:52.133207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.879671+0000","last_change":"2023-12-11T17:44:24.702039+0000","last_active":"2023-12-11T17:45:14.879671+0000","last_peered":"2023-12-11T17:45:14.879671+0000","last_clean":"2023-12-11T17:45:14.879671+0000","last_became_active":"2023-12-11T17:44:24.701539+0000","last_became_peered":"2023-12-11T17:44:24.701539+0000","last_unstale":"2023-12-11T17:45:14.879671+0000","last_undegraded":"2023-12-11T17:45:14.879671+0000","last_fullsized":"2023-12-11T17:45:14.879671+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_clean_scrub_stamp":"2023-12-11T17:44:23.663161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T02:23:55.393595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.878779+0000","last_change":"2023-12-11T17:44:19.662118+0000","last_active":"2023-12-11T17:45:14.878779+0000","last_peered":"2023-12-11T17:45:14.878779+0000","last_clean":"2023-12-11T17:45:14.878779+0000","last_became_active":"2023-12-11T17:44:19.661796+0000","last_became_peered":"2023-12-11T17:44:19.661796+0000","last_unstale":"2023-12-11T17:45:14.878779+0000","last_undegraded":"2023-12-11T17:45:14.878779+0000","last_fullsized":"2023-12-11T17:45:14.878779+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_clean_scrub_stamp":"2023-12-11T17:44:18.636443+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T21:08:02.842544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.879934+0000","last_change":"2023-12-11T17:44:20.615403+0000","last_active":"2023-12-11T17:45:14.879934+0000","last_peered":"2023-12-11T17:45:14.879934+0000","last_clean":"2023-12-11T17:45:14.879934+0000","last_became_active":"2023-12-11T17:44:20.615106+0000","last_became_peered":"2023-12-11T17:44:20.615106+0000","last_unstale":"2023-12-11T17:45:14.879934+0000","last_undegraded":"2023-12-11T17:45:14.879934+0000","last_fullsized":"2023-12-11T17:45:14.879934+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T02:10:24.047720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.880650+0000","last_change":"2023-12-11T17:44:24.699737+0000","last_active":"2023-12-11T17:45:14.880650+0000","last_peered":"2023-12-11T17:45:14.880650+0000","last_clean":"2023-12-11T17:45:14.880650+0000","last_became_active":"2023-12-11T17:44:24.699345+0000","last_became_peered":"2023-12-11T17:44:24.699345+0000","last_unstale":"2023-12-11T17:45:14.880650+0000","last_undegraded":"2023-12-11T17:45:14.880650+0000","last_fullsized":"2023-12-11T17:45:14.880650+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_clean_scrub_stamp":"2023-12-11T17:44:23.663161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T01:30:25.471636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.878104+0000","last_change":"2023-12-11T17:44:19.661325+0000","last_active":"2023-12-11T17:45:14.878104+0000","last_peered":"2023-12-11T17:45:14.878104+0000","last_clean":"2023-12-11T17:45:14.878104+0000","last_became_active":"2023-12-11T17:44:19.661012+0000","last_became_peered":"2023-12-11T17:44:19.661012+0000","last_unstale":"2023-12-11T17:45:14.878104+0000","last_undegraded":"2023-12-11T17:45:14.878104+0000","last_fullsized":"2023-12-11T17:45:14.878104+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_clean_scrub_stamp":"2023-12-11T17:44:18.636443+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T19:33:08.258019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.873803+0000","last_change":"2023-12-11T17:44:20.614795+0000","last_active":"2023-12-11T17:45:14.873803+0000","last_peered":"2023-12-11T17:45:14.873803+0000","last_clean":"2023-12-11T17:45:14.873803+0000","last_became_active":"2023-12-11T17:44:20.613826+0000","last_became_peered":"2023-12-11T17:44:20.613826+0000","last_unstale":"2023-12-11T17:45:14.873803+0000","last_undegraded":"2023-12-11T17:45:14.873803+0000","last_fullsized":"2023-12-11T17:45:14.873803+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T21:32:19.029040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.877354+0000","last_change":"2023-12-11T17:44:24.701569+0000","last_active":"2023-12-11T17:45:14.877354+0000","last_peered":"2023-12-11T17:45:14.877354+0000","last_clean":"2023-12-11T17:45:14.877354+0000","last_became_active":"2023-12-11T17:44:24.701173+0000","last_became_peered":"2023-12-11T17:44:24.701173+0000","last_unstale":"2023-12-11T17:45:14.877354+0000","last_undegraded":"2023-12-11T17:45:14.877354+0000","last_fullsized":"2023-12-11T17:45:14.877354+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_clean_scrub_stamp":"2023-12-11T17:44:23.663161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T23:02:45.843827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"86'4","reported_seq":64,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T17:45:23.838749+0000","last_change":"2023-12-11T17:44:19.659643+0000","last_active":"2023-12-11T17:45:23.838749+0000","last_peered":"2023-12-11T17:45:23.838749+0000","last_clean":"2023-12-11T17:45:23.838749+0000","last_became_active":"2023-12-11T17:44:19.659320+0000","last_became_peered":"2023-12-11T17:44:19.659320+0000","last_unstale":"2023-12-11T17:45:23.838749+0000","last_undegraded":"2023-12-11T17:45:23.838749+0000","last_fullsized":"2023-12-11T17:45:23.838749+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_clean_scrub_stamp":"2023-12-11T17:44:18.636443+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T20:02:03.633439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":977,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":42,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.873555+0000","last_change":"2023-12-11T17:44:20.619445+0000","last_active":"2023-12-11T17:45:14.873555+0000","last_peered":"2023-12-11T17:45:14.873555+0000","last_clean":"2023-12-11T17:45:14.873555+0000","last_became_active":"2023-12-11T17:44:20.619094+0000","last_became_peered":"2023-12-11T17:44:20.619094+0000","last_unstale":"2023-12-11T17:45:14.873555+0000","last_undegraded":"2023-12-11T17:45:14.873555+0000","last_fullsized":"2023-12-11T17:45:14.873555+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T04:03:21.917498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.880483+0000","last_change":"2023-12-11T17:44:24.701821+0000","last_active":"2023-12-11T17:45:14.880483+0000","last_peered":"2023-12-11T17:45:14.880483+0000","last_clean":"2023-12-11T17:45:14.880483+0000","last_became_active":"2023-12-11T17:44:24.700590+0000","last_became_peered":"2023-12-11T17:44:24.700590+0000","last_unstale":"2023-12-11T17:45:14.880483+0000","last_undegraded":"2023-12-11T17:45:14.880483+0000","last_fullsized":"2023-12-11T17:45:14.880483+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_clean_scrub_stamp":"2023-12-11T17:44:23.663161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T19:58:28.422648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.873469+0000","last_change":"2023-12-11T17:44:19.662540+0000","last_active":"2023-12-11T17:45:14.873469+0000","last_peered":"2023-12-11T17:45:14.873469+0000","last_clean":"2023-12-11T17:45:14.873469+0000","last_became_active":"2023-12-11T17:44:19.662231+0000","last_became_peered":"2023-12-11T17:44:19.662231+0000","last_unstale":"2023-12-11T17:45:14.873469+0000","last_undegraded":"2023-12-11T17:45:14.873469+0000","last_fullsized":"2023-12-11T17:45:14.873469+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_clean_scrub_stamp":"2023-12-11T17:44:18.636443+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T21:56:45.241138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.878470+0000","last_change":"2023-12-11T17:44:20.620457+0000","last_active":"2023-12-11T17:45:14.878470+0000","last_peered":"2023-12-11T17:45:14.878470+0000","last_clean":"2023-12-11T17:45:14.878470+0000","last_became_active":"2023-12-11T17:44:20.619949+0000","last_became_peered":"2023-12-11T17:44:20.619949+0000","last_unstale":"2023-12-11T17:45:14.878470+0000","last_undegraded":"2023-12-11T17:45:14.878470+0000","last_fullsized":"2023-12-11T17:45:14.878470+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T22:59:14.328817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.878371+0000","last_change":"2023-12-11T17:44:24.691573+0000","last_active":"2023-12-11T17:45:14.878371+0000","last_peered":"2023-12-11T17:45:14.878371+0000","last_clean":"2023-12-11T17:45:14.878371+0000","last_became_active":"2023-12-11T17:44:24.691182+0000","last_became_peered":"2023-12-11T17:44:24.691182+0000","last_unstale":"2023-12-11T17:45:14.878371+0000","last_undegraded":"2023-12-11T17:45:14.878371+0000","last_fullsized":"2023-12-11T17:45:14.878371+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_clean_scrub_stamp":"2023-12-11T17:44:23.663161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T02:18:36.582108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.874171+0000","last_change":"2023-12-11T17:44:19.653470+0000","last_active":"2023-12-11T17:45:14.874171+0000","last_peered":"2023-12-11T17:45:14.874171+0000","last_clean":"2023-12-11T17:45:14.874171+0000","last_became_active":"2023-12-11T17:44:19.653148+0000","last_became_peered":"2023-12-11T17:44:19.653148+0000","last_unstale":"2023-12-11T17:45:14.874171+0000","last_undegraded":"2023-12-11T17:45:14.874171+0000","last_fullsized":"2023-12-11T17:45:14.874171+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_clean_scrub_stamp":"2023-12-11T17:44:18.636443+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T05:26:12.017814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.880579+0000","last_change":"2023-12-11T17:44:20.617019+0000","last_active":"2023-12-11T17:45:14.880579+0000","last_peered":"2023-12-11T17:45:14.880579+0000","last_clean":"2023-12-11T17:45:14.880579+0000","last_became_active":"2023-12-11T17:44:20.616770+0000","last_became_peered":"2023-12-11T17:44:20.616770+0000","last_unstale":"2023-12-11T17:45:14.880579+0000","last_undegraded":"2023-12-11T17:45:14.880579+0000","last_fullsized":"2023-12-11T17:45:14.880579+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T18:32:15.050179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.873531+0000","last_change":"2023-12-11T17:44:24.694843+0000","last_active":"2023-12-11T17:45:14.873531+0000","last_peered":"2023-12-11T17:45:14.873531+0000","last_clean":"2023-12-11T17:45:14.873531+0000","last_became_active":"2023-12-11T17:44:24.694423+0000","last_became_peered":"2023-12-11T17:44:24.694423+0000","last_unstale":"2023-12-11T17:45:14.873531+0000","last_undegraded":"2023-12-11T17:45:14.873531+0000","last_fullsized":"2023-12-11T17:45:14.873531+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_clean_scrub_stamp":"2023-12-11T17:44:23.663161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T18:42:35.281782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.877196+0000","last_change":"2023-12-11T17:44:19.655676+0000","last_active":"2023-12-11T17:45:14.877196+0000","last_peered":"2023-12-11T17:45:14.877196+0000","last_clean":"2023-12-11T17:45:14.877196+0000","last_became_active":"2023-12-11T17:44:19.655331+0000","last_became_peered":"2023-12-11T17:44:19.655331+0000","last_unstale":"2023-12-11T17:45:14.877196+0000","last_undegraded":"2023-12-11T17:45:14.877196+0000","last_fullsized":"2023-12-11T17:45:14.877196+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_clean_scrub_stamp":"2023-12-11T17:44:18.636443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T00:01:27.963540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.879898+0000","last_change":"2023-12-11T17:44:20.617396+0000","last_active":"2023-12-11T17:45:14.879898+0000","last_peered":"2023-12-11T17:45:14.879898+0000","last_clean":"2023-12-11T17:45:14.879898+0000","last_became_active":"2023-12-11T17:44:20.617086+0000","last_became_peered":"2023-12-11T17:44:20.617086+0000","last_unstale":"2023-12-11T17:45:14.879898+0000","last_undegraded":"2023-12-11T17:45:14.879898+0000","last_fullsized":"2023-12-11T17:45:14.879898+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T02:31:55.763125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.879744+0000","last_change":"2023-12-11T17:44:24.703283+0000","last_active":"2023-12-11T17:45:14.879744+0000","last_peered":"2023-12-11T17:45:14.879744+0000","last_clean":"2023-12-11T17:45:14.879744+0000","last_became_active":"2023-12-11T17:44:24.701703+0000","last_became_peered":"2023-12-11T17:44:24.701703+0000","last_unstale":"2023-12-11T17:45:14.879744+0000","last_undegraded":"2023-12-11T17:45:14.879744+0000","last_fullsized":"2023-12-11T17:45:14.879744+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_clean_scrub_stamp":"2023-12-11T17:44:23.663161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T18:52:50.570036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.873678+0000","last_change":"2023-12-11T17:44:19.662851+0000","last_active":"2023-12-11T17:45:14.873678+0000","last_peered":"2023-12-11T17:45:14.873678+0000","last_clean":"2023-12-11T17:45:14.873678+0000","last_became_active":"2023-12-11T17:44:19.662548+0000","last_became_peered":"2023-12-11T17:44:19.662548+0000","last_unstale":"2023-12-11T17:45:14.873678+0000","last_undegraded":"2023-12-11T17:45:14.873678+0000","last_fullsized":"2023-12-11T17:45:14.873678+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_clean_scrub_stamp":"2023-12-11T17:44:18.636443+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T02:34:14.914687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.878774+0000","last_change":"2023-12-11T17:44:20.612750+0000","last_active":"2023-12-11T17:45:14.878774+0000","last_peered":"2023-12-11T17:45:14.878774+0000","last_clean":"2023-12-11T17:45:14.878774+0000","last_became_active":"2023-12-11T17:44:20.612479+0000","last_became_peered":"2023-12-11T17:44:20.612479+0000","last_unstale":"2023-12-11T17:45:14.878774+0000","last_undegraded":"2023-12-11T17:45:14.878774+0000","last_fullsized":"2023-12-11T17:45:14.878774+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:45:03.217844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.880124+0000","last_change":"2023-12-11T17:44:24.703050+0000","last_active":"2023-12-11T17:45:14.880124+0000","last_peered":"2023-12-11T17:45:14.880124+0000","last_clean":"2023-12-11T17:45:14.880124+0000","last_became_active":"2023-12-11T17:44:24.701408+0000","last_became_peered":"2023-12-11T17:44:24.701408+0000","last_unstale":"2023-12-11T17:45:14.880124+0000","last_undegraded":"2023-12-11T17:45:14.880124+0000","last_fullsized":"2023-12-11T17:45:14.880124+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_clean_scrub_stamp":"2023-12-11T17:44:23.663161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T01:02:44.726831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.878644+0000","last_change":"2023-12-11T17:44:20.616004+0000","last_active":"2023-12-11T17:45:14.878644+0000","last_peered":"2023-12-11T17:45:14.878644+0000","last_clean":"2023-12-11T17:45:14.878644+0000","last_became_active":"2023-12-11T17:44:20.614985+0000","last_became_peered":"2023-12-11T17:44:20.614985+0000","last_unstale":"2023-12-11T17:45:14.878644+0000","last_undegraded":"2023-12-11T17:45:14.878644+0000","last_fullsized":"2023-12-11T17:45:14.878644+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T05:31:13.414818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'4","reported_seq":62,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T17:45:24.839438+0000","last_change":"2023-12-11T17:44:19.663092+0000","last_active":"2023-12-11T17:45:24.839438+0000","last_peered":"2023-12-11T17:45:24.839438+0000","last_clean":"2023-12-11T17:45:24.839438+0000","last_became_active":"2023-12-11T17:44:19.662728+0000","last_became_peered":"2023-12-11T17:44:19.662728+0000","last_unstale":"2023-12-11T17:45:24.839438+0000","last_undegraded":"2023-12-11T17:45:24.839438+0000","last_fullsized":"2023-12-11T17:45:24.839438+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_clean_scrub_stamp":"2023-12-11T17:44:18.636443+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T20:05:23.952470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":39,"num_read_kb":39,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"86'1","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.874001+0000","last_change":"2023-12-11T17:44:24.694194+0000","last_active":"2023-12-11T17:45:14.874001+0000","last_peered":"2023-12-11T17:45:14.874001+0000","last_clean":"2023-12-11T17:45:14.874001+0000","last_became_active":"2023-12-11T17:44:24.693860+0000","last_became_peered":"2023-12-11T17:44:24.693860+0000","last_unstale":"2023-12-11T17:45:14.874001+0000","last_undegraded":"2023-12-11T17:45:14.874001+0000","last_fullsized":"2023-12-11T17:45:14.874001+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_clean_scrub_stamp":"2023-12-11T17:44:23.663161+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T01:16:04.877531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T17:45:15.889584+0000","last_change":"2023-12-11T17:45:15.887755+0000","last_active":"2023-12-11T17:45:15.889584+0000","last_peered":"2023-12-11T17:45:15.889584+0000","last_clean":"2023-12-11T17:45:15.889584+0000","last_became_active":"2023-12-11T17:45:15.887541+0000","last_became_peered":"2023-12-11T17:45:15.887541+0000","last_unstale":"2023-12-11T17:45:15.889584+0000","last_undegraded":"2023-12-11T17:45:15.889584+0000","last_fullsized":"2023-12-11T17:45:15.889584+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T02:57:39.431558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.876825+0000","last_change":"2023-12-11T17:44:19.651980+0000","last_active":"2023-12-11T17:45:14.876825+0000","last_peered":"2023-12-11T17:45:14.876825+0000","last_clean":"2023-12-11T17:45:14.876825+0000","last_became_active":"2023-12-11T17:44:19.651659+0000","last_became_peered":"2023-12-11T17:44:19.651659+0000","last_unstale":"2023-12-11T17:45:14.876825+0000","last_undegraded":"2023-12-11T17:45:14.876825+0000","last_fullsized":"2023-12-11T17:45:14.876825+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_clean_scrub_stamp":"2023-12-11T17:44:18.636443+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T19:29:21.453758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":322,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.873442+0000","last_change":"2023-12-11T17:43:21.021919+0000","last_active":"2023-12-11T17:45:14.873442+0000","last_peered":"2023-12-11T17:45:14.873442+0000","last_clean":"2023-12-11T17:45:14.873442+0000","last_became_active":"2023-12-11T17:43:21.021489+0000","last_became_peered":"2023-12-11T17:43:21.021489+0000","last_unstale":"2023-12-11T17:45:14.873442+0000","last_undegraded":"2023-12-11T17:45:14.873442+0000","last_fullsized":"2023-12-11T17:45:14.873442+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:40:27.026768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:40:27.026768+0000","last_clean_scrub_stamp":"2023-12-11T17:40:27.026768+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T22:30:14.730794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.879358+0000","last_change":"2023-12-11T17:44:24.701996+0000","last_active":"2023-12-11T17:45:14.879358+0000","last_peered":"2023-12-11T17:45:14.879358+0000","last_clean":"2023-12-11T17:45:14.879358+0000","last_became_active":"2023-12-11T17:44:24.701574+0000","last_became_peered":"2023-12-11T17:44:24.701574+0000","last_unstale":"2023-12-11T17:45:14.879358+0000","last_undegraded":"2023-12-11T17:45:14.879358+0000","last_fullsized":"2023-12-11T17:45:14.879358+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_clean_scrub_stamp":"2023-12-11T17:44:23.663161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T19:49:01.458432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.880311+0000","last_change":"2023-12-11T17:44:20.614765+0000","last_active":"2023-12-11T17:45:14.880311+0000","last_peered":"2023-12-11T17:45:14.880311+0000","last_clean":"2023-12-11T17:45:14.880311+0000","last_became_active":"2023-12-11T17:44:20.614462+0000","last_became_peered":"2023-12-11T17:44:20.614462+0000","last_unstale":"2023-12-11T17:45:14.880311+0000","last_undegraded":"2023-12-11T17:45:14.880311+0000","last_fullsized":"2023-12-11T17:45:14.880311+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:08:20.784124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.873636+0000","last_change":"2023-12-11T17:44:19.661528+0000","last_active":"2023-12-11T17:45:14.873636+0000","last_peered":"2023-12-11T17:45:14.873636+0000","last_clean":"2023-12-11T17:45:14.873636+0000","last_became_active":"2023-12-11T17:44:19.661208+0000","last_became_peered":"2023-12-11T17:44:19.661208+0000","last_unstale":"2023-12-11T17:45:14.873636+0000","last_undegraded":"2023-12-11T17:45:14.873636+0000","last_fullsized":"2023-12-11T17:45:14.873636+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_clean_scrub_stamp":"2023-12-11T17:44:18.636443+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T18:31:23.295193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.880386+0000","last_change":"2023-12-11T17:44:24.688218+0000","last_active":"2023-12-11T17:45:14.880386+0000","last_peered":"2023-12-11T17:45:14.880386+0000","last_clean":"2023-12-11T17:45:14.880386+0000","last_became_active":"2023-12-11T17:44:24.687809+0000","last_became_peered":"2023-12-11T17:44:24.687809+0000","last_unstale":"2023-12-11T17:45:14.880386+0000","last_undegraded":"2023-12-11T17:45:14.880386+0000","last_fullsized":"2023-12-11T17:45:14.880386+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_clean_scrub_stamp":"2023-12-11T17:44:23.663161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T02:12:50.975614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.873899+0000","last_change":"2023-12-11T17:44:19.657797+0000","last_active":"2023-12-11T17:45:14.873899+0000","last_peered":"2023-12-11T17:45:14.873899+0000","last_clean":"2023-12-11T17:45:14.873899+0000","last_became_active":"2023-12-11T17:44:19.657483+0000","last_became_peered":"2023-12-11T17:44:19.657483+0000","last_unstale":"2023-12-11T17:45:14.873899+0000","last_undegraded":"2023-12-11T17:45:14.873899+0000","last_fullsized":"2023-12-11T17:45:14.873899+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_clean_scrub_stamp":"2023-12-11T17:44:18.636443+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T22:43:35.785943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.879897+0000","last_change":"2023-12-11T17:44:20.610002+0000","last_active":"2023-12-11T17:45:14.879897+0000","last_peered":"2023-12-11T17:45:14.879897+0000","last_clean":"2023-12-11T17:45:14.879897+0000","last_became_active":"2023-12-11T17:44:20.609579+0000","last_became_peered":"2023-12-11T17:44:20.609579+0000","last_unstale":"2023-12-11T17:45:14.879897+0000","last_undegraded":"2023-12-11T17:45:14.879897+0000","last_fullsized":"2023-12-11T17:45:14.879897+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:18:25.670403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.881080+0000","last_change":"2023-12-11T17:44:24.694707+0000","last_active":"2023-12-11T17:45:14.881080+0000","last_peered":"2023-12-11T17:45:14.881080+0000","last_clean":"2023-12-11T17:45:14.881080+0000","last_became_active":"2023-12-11T17:44:24.694275+0000","last_became_peered":"2023-12-11T17:44:24.694275+0000","last_unstale":"2023-12-11T17:45:14.881080+0000","last_undegraded":"2023-12-11T17:45:14.881080+0000","last_fullsized":"2023-12-11T17:45:14.881080+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_clean_scrub_stamp":"2023-12-11T17:44:23.663161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T18:28:12.922991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"86'6","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.877680+0000","last_change":"2023-12-11T17:44:19.658825+0000","last_active":"2023-12-11T17:45:14.877680+0000","last_peered":"2023-12-11T17:45:14.877680+0000","last_clean":"2023-12-11T17:45:14.877680+0000","last_became_active":"2023-12-11T17:44:19.658425+0000","last_became_peered":"2023-12-11T17:44:19.658425+0000","last_unstale":"2023-12-11T17:45:14.877680+0000","last_undegraded":"2023-12-11T17:45:14.877680+0000","last_fullsized":"2023-12-11T17:45:14.877680+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_clean_scrub_stamp":"2023-12-11T17:44:18.636443+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T04:06:15.289454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.881577+0000","last_change":"2023-12-11T17:44:20.617557+0000","last_active":"2023-12-11T17:45:14.881577+0000","last_peered":"2023-12-11T17:45:14.881577+0000","last_clean":"2023-12-11T17:45:14.881577+0000","last_became_active":"2023-12-11T17:44:20.616879+0000","last_became_peered":"2023-12-11T17:44:20.616879+0000","last_unstale":"2023-12-11T17:45:14.881577+0000","last_undegraded":"2023-12-11T17:45:14.881577+0000","last_fullsized":"2023-12-11T17:45:14.881577+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T19:34:04.954862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.876786+0000","last_change":"2023-12-11T17:44:24.691471+0000","last_active":"2023-12-11T17:45:14.876786+0000","last_peered":"2023-12-11T17:45:14.876786+0000","last_clean":"2023-12-11T17:45:14.876786+0000","last_became_active":"2023-12-11T17:44:24.690390+0000","last_became_peered":"2023-12-11T17:44:24.690390+0000","last_unstale":"2023-12-11T17:45:14.876786+0000","last_undegraded":"2023-12-11T17:45:14.876786+0000","last_fullsized":"2023-12-11T17:45:14.876786+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_clean_scrub_stamp":"2023-12-11T17:44:23.663161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T00:26:20.141717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.877069+0000","last_change":"2023-12-11T17:44:19.661004+0000","last_active":"2023-12-11T17:45:14.877069+0000","last_peered":"2023-12-11T17:45:14.877069+0000","last_clean":"2023-12-11T17:45:14.877069+0000","last_became_active":"2023-12-11T17:44:19.660691+0000","last_became_peered":"2023-12-11T17:44:19.660691+0000","last_unstale":"2023-12-11T17:45:14.877069+0000","last_undegraded":"2023-12-11T17:45:14.877069+0000","last_fullsized":"2023-12-11T17:45:14.877069+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_clean_scrub_stamp":"2023-12-11T17:44:18.636443+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T17:54:53.956473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.874627+0000","last_change":"2023-12-11T17:44:20.621437+0000","last_active":"2023-12-11T17:45:14.874627+0000","last_peered":"2023-12-11T17:45:14.874627+0000","last_clean":"2023-12-11T17:45:14.874627+0000","last_became_active":"2023-12-11T17:44:20.621113+0000","last_became_peered":"2023-12-11T17:44:20.621113+0000","last_unstale":"2023-12-11T17:45:14.874627+0000","last_undegraded":"2023-12-11T17:45:14.874627+0000","last_fullsized":"2023-12-11T17:45:14.874627+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T21:05:50.097605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.877109+0000","last_change":"2023-12-11T17:44:24.697432+0000","last_active":"2023-12-11T17:45:14.877109+0000","last_peered":"2023-12-11T17:45:14.877109+0000","last_clean":"2023-12-11T17:45:14.877109+0000","last_became_active":"2023-12-11T17:44:24.696924+0000","last_became_peered":"2023-12-11T17:44:24.696924+0000","last_unstale":"2023-12-11T17:45:14.877109+0000","last_undegraded":"2023-12-11T17:45:14.877109+0000","last_fullsized":"2023-12-11T17:45:14.877109+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_clean_scrub_stamp":"2023-12-11T17:44:23.663161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T21:00:01.967894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.879985+0000","last_change":"2023-12-11T17:44:19.665080+0000","last_active":"2023-12-11T17:45:14.879985+0000","last_peered":"2023-12-11T17:45:14.879985+0000","last_clean":"2023-12-11T17:45:14.879985+0000","last_became_active":"2023-12-11T17:44:19.664862+0000","last_became_peered":"2023-12-11T17:44:19.664862+0000","last_unstale":"2023-12-11T17:45:14.879985+0000","last_undegraded":"2023-12-11T17:45:14.879985+0000","last_fullsized":"2023-12-11T17:45:14.879985+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_clean_scrub_stamp":"2023-12-11T17:44:18.636443+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T20:22:05.909035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.873446+0000","last_change":"2023-12-11T17:44:20.614140+0000","last_active":"2023-12-11T17:45:14.873446+0000","last_peered":"2023-12-11T17:45:14.873446+0000","last_clean":"2023-12-11T17:45:14.873446+0000","last_became_active":"2023-12-11T17:44:20.613808+0000","last_became_peered":"2023-12-11T17:44:20.613808+0000","last_unstale":"2023-12-11T17:45:14.873446+0000","last_undegraded":"2023-12-11T17:45:14.873446+0000","last_fullsized":"2023-12-11T17:45:14.873446+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T20:55:57.572836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.877546+0000","last_change":"2023-12-11T17:44:24.697388+0000","last_active":"2023-12-11T17:45:14.877546+0000","last_peered":"2023-12-11T17:45:14.877546+0000","last_clean":"2023-12-11T17:45:14.877546+0000","last_became_active":"2023-12-11T17:44:24.696964+0000","last_became_peered":"2023-12-11T17:44:24.696964+0000","last_unstale":"2023-12-11T17:45:14.877546+0000","last_undegraded":"2023-12-11T17:45:14.877546+0000","last_fullsized":"2023-12-11T17:45:14.877546+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_clean_scrub_stamp":"2023-12-11T17:44:23.663161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T18:08:04.162075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.880959+0000","last_change":"2023-12-11T17:44:19.660327+0000","last_active":"2023-12-11T17:45:14.880959+0000","last_peered":"2023-12-11T17:45:14.880959+0000","last_clean":"2023-12-11T17:45:14.880959+0000","last_became_active":"2023-12-11T17:44:19.659999+0000","last_became_peered":"2023-12-11T17:44:19.659999+0000","last_unstale":"2023-12-11T17:45:14.880959+0000","last_undegraded":"2023-12-11T17:45:14.880959+0000","last_fullsized":"2023-12-11T17:45:14.880959+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_clean_scrub_stamp":"2023-12-11T17:44:18.636443+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T23:51:24.580854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.879951+0000","last_change":"2023-12-11T17:44:20.624126+0000","last_active":"2023-12-11T17:45:14.879951+0000","last_peered":"2023-12-11T17:45:14.879951+0000","last_clean":"2023-12-11T17:45:14.879951+0000","last_became_active":"2023-12-11T17:44:20.623836+0000","last_became_peered":"2023-12-11T17:44:20.623836+0000","last_unstale":"2023-12-11T17:45:14.879951+0000","last_undegraded":"2023-12-11T17:45:14.879951+0000","last_fullsized":"2023-12-11T17:45:14.879951+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T21:43:39.123299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"86'1","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.876615+0000","last_change":"2023-12-11T17:44:24.698994+0000","last_active":"2023-12-11T17:45:14.876615+0000","last_peered":"2023-12-11T17:45:14.876615+0000","last_clean":"2023-12-11T17:45:14.876615+0000","last_became_active":"2023-12-11T17:44:24.698583+0000","last_became_peered":"2023-12-11T17:44:24.698583+0000","last_unstale":"2023-12-11T17:45:14.876615+0000","last_undegraded":"2023-12-11T17:45:14.876615+0000","last_fullsized":"2023-12-11T17:45:14.876615+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_clean_scrub_stamp":"2023-12-11T17:44:23.663161+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T00:03:20.178331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"86'1","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.877402+0000","last_change":"2023-12-11T17:44:19.662505+0000","last_active":"2023-12-11T17:45:14.877402+0000","last_peered":"2023-12-11T17:45:14.877402+0000","last_clean":"2023-12-11T17:45:14.877402+0000","last_became_active":"2023-12-11T17:44:19.661478+0000","last_became_peered":"2023-12-11T17:44:19.661478+0000","last_unstale":"2023-12-11T17:45:14.877402+0000","last_undegraded":"2023-12-11T17:45:14.877402+0000","last_fullsized":"2023-12-11T17:45:14.877402+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_clean_scrub_stamp":"2023-12-11T17:44:18.636443+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T23:28:09.345693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.881400+0000","last_change":"2023-12-11T17:44:20.615026+0000","last_active":"2023-12-11T17:45:14.881400+0000","last_peered":"2023-12-11T17:45:14.881400+0000","last_clean":"2023-12-11T17:45:14.881400+0000","last_became_active":"2023-12-11T17:44:20.614669+0000","last_became_peered":"2023-12-11T17:44:20.614669+0000","last_unstale":"2023-12-11T17:45:14.881400+0000","last_undegraded":"2023-12-11T17:45:14.881400+0000","last_fullsized":"2023-12-11T17:45:14.881400+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T20:52:20.455326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.877196+0000","last_change":"2023-12-11T17:44:24.694681+0000","last_active":"2023-12-11T17:45:14.877196+0000","last_peered":"2023-12-11T17:45:14.877196+0000","last_clean":"2023-12-11T17:45:14.877196+0000","last_became_active":"2023-12-11T17:44:24.694172+0000","last_became_peered":"2023-12-11T17:44:24.694172+0000","last_unstale":"2023-12-11T17:45:14.877196+0000","last_undegraded":"2023-12-11T17:45:14.877196+0000","last_fullsized":"2023-12-11T17:45:14.877196+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_clean_scrub_stamp":"2023-12-11T17:44:23.663161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T20:54:46.367735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":38,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T17:45:16.146416+0000","last_change":"2023-12-11T17:45:16.146416+0000","last_active":"2023-12-11T17:45:16.146416+0000","last_peered":"2023-12-11T17:45:16.146416+0000","last_clean":"2023-12-11T17:45:16.146416+0000","last_became_active":"2023-12-11T17:45:16.141794+0000","last_became_peered":"2023-12-11T17:45:16.141794+0000","last_unstale":"2023-12-11T17:45:16.146416+0000","last_undegraded":"2023-12-11T17:45:16.146416+0000","last_fullsized":"2023-12-11T17:45:16.146416+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_clean_scrub_stamp":"2023-12-11T17:44:18.636443+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:05:27.729213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.873702+0000","last_change":"2023-12-11T17:44:20.621879+0000","last_active":"2023-12-11T17:45:14.873702+0000","last_peered":"2023-12-11T17:45:14.873702+0000","last_clean":"2023-12-11T17:45:14.873702+0000","last_became_active":"2023-12-11T17:44:20.621430+0000","last_became_peered":"2023-12-11T17:44:20.621430+0000","last_unstale":"2023-12-11T17:45:14.873702+0000","last_undegraded":"2023-12-11T17:45:14.873702+0000","last_fullsized":"2023-12-11T17:45:14.873702+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T18:08:46.359629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.880424+0000","last_change":"2023-12-11T17:44:24.700599+0000","last_active":"2023-12-11T17:45:14.880424+0000","last_peered":"2023-12-11T17:45:14.880424+0000","last_clean":"2023-12-11T17:45:14.880424+0000","last_became_active":"2023-12-11T17:44:24.700121+0000","last_became_peered":"2023-12-11T17:44:24.700121+0000","last_unstale":"2023-12-11T17:45:14.880424+0000","last_undegraded":"2023-12-11T17:45:14.880424+0000","last_fullsized":"2023-12-11T17:45:14.880424+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_clean_scrub_stamp":"2023-12-11T17:44:23.663161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T01:08:18.000642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.878068+0000","last_change":"2023-12-11T17:44:19.662414+0000","last_active":"2023-12-11T17:45:14.878068+0000","last_peered":"2023-12-11T17:45:14.878068+0000","last_clean":"2023-12-11T17:45:14.878068+0000","last_became_active":"2023-12-11T17:44:19.661753+0000","last_became_peered":"2023-12-11T17:44:19.661753+0000","last_unstale":"2023-12-11T17:45:14.878068+0000","last_undegraded":"2023-12-11T17:45:14.878068+0000","last_fullsized":"2023-12-11T17:45:14.878068+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_clean_scrub_stamp":"2023-12-11T17:44:18.636443+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T00:49:11.713347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.874224+0000","last_change":"2023-12-11T17:44:20.620973+0000","last_active":"2023-12-11T17:45:14.874224+0000","last_peered":"2023-12-11T17:45:14.874224+0000","last_clean":"2023-12-11T17:45:14.874224+0000","last_became_active":"2023-12-11T17:44:20.620284+0000","last_became_peered":"2023-12-11T17:44:20.620284+0000","last_unstale":"2023-12-11T17:45:14.874224+0000","last_undegraded":"2023-12-11T17:45:14.874224+0000","last_fullsized":"2023-12-11T17:45:14.874224+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:00:52.529062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.873585+0000","last_change":"2023-12-11T17:44:24.696888+0000","last_active":"2023-12-11T17:45:14.873585+0000","last_peered":"2023-12-11T17:45:14.873585+0000","last_clean":"2023-12-11T17:45:14.873585+0000","last_became_active":"2023-12-11T17:44:24.696475+0000","last_became_peered":"2023-12-11T17:44:24.696475+0000","last_unstale":"2023-12-11T17:45:14.873585+0000","last_undegraded":"2023-12-11T17:45:14.873585+0000","last_fullsized":"2023-12-11T17:45:14.873585+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_clean_scrub_stamp":"2023-12-11T17:44:23.663161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T21:42:17.287557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"81'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.877574+0000","last_change":"2023-12-11T17:44:19.659094+0000","last_active":"2023-12-11T17:45:14.877574+0000","last_peered":"2023-12-11T17:45:14.877574+0000","last_clean":"2023-12-11T17:45:14.877574+0000","last_became_active":"2023-12-11T17:44:19.658794+0000","last_became_peered":"2023-12-11T17:44:19.658794+0000","last_unstale":"2023-12-11T17:45:14.877574+0000","last_undegraded":"2023-12-11T17:45:14.877574+0000","last_fullsized":"2023-12-11T17:45:14.877574+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_clean_scrub_stamp":"2023-12-11T17:44:18.636443+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T01:39:41.151915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.873936+0000","last_change":"2023-12-11T17:44:20.623342+0000","last_active":"2023-12-11T17:45:14.873936+0000","last_peered":"2023-12-11T17:45:14.873936+0000","last_clean":"2023-12-11T17:45:14.873936+0000","last_became_active":"2023-12-11T17:44:20.622957+0000","last_became_peered":"2023-12-11T17:44:20.622957+0000","last_unstale":"2023-12-11T17:45:14.873936+0000","last_undegraded":"2023-12-11T17:45:14.873936+0000","last_fullsized":"2023-12-11T17:45:14.873936+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:25:00.910645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.876776+0000","last_change":"2023-12-11T17:44:24.690746+0000","last_active":"2023-12-11T17:45:14.876776+0000","last_peered":"2023-12-11T17:45:14.876776+0000","last_clean":"2023-12-11T17:45:14.876776+0000","last_became_active":"2023-12-11T17:44:24.690290+0000","last_became_peered":"2023-12-11T17:44:24.690290+0000","last_unstale":"2023-12-11T17:45:14.876776+0000","last_undegraded":"2023-12-11T17:45:14.876776+0000","last_fullsized":"2023-12-11T17:45:14.876776+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_clean_scrub_stamp":"2023-12-11T17:44:23.663161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T01:26:14.409832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"81'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.880570+0000","last_change":"2023-12-11T17:44:19.655775+0000","last_active":"2023-12-11T17:45:14.880570+0000","last_peered":"2023-12-11T17:45:14.880570+0000","last_clean":"2023-12-11T17:45:14.880570+0000","last_became_active":"2023-12-11T17:44:19.654360+0000","last_became_peered":"2023-12-11T17:44:19.654360+0000","last_unstale":"2023-12-11T17:45:14.880570+0000","last_undegraded":"2023-12-11T17:45:14.880570+0000","last_fullsized":"2023-12-11T17:45:14.880570+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_clean_scrub_stamp":"2023-12-11T17:44:18.636443+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T00:07:08.678389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.874458+0000","last_change":"2023-12-11T17:44:20.614898+0000","last_active":"2023-12-11T17:45:14.874458+0000","last_peered":"2023-12-11T17:45:14.874458+0000","last_clean":"2023-12-11T17:45:14.874458+0000","last_became_active":"2023-12-11T17:44:20.613972+0000","last_became_peered":"2023-12-11T17:44:20.613972+0000","last_unstale":"2023-12-11T17:45:14.874458+0000","last_undegraded":"2023-12-11T17:45:14.874458+0000","last_fullsized":"2023-12-11T17:45:14.874458+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T04:28:51.576955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.880057+0000","last_change":"2023-12-11T17:44:20.617502+0000","last_active":"2023-12-11T17:45:14.880057+0000","last_peered":"2023-12-11T17:45:14.880057+0000","last_clean":"2023-12-11T17:45:14.880057+0000","last_became_active":"2023-12-11T17:44:20.617252+0000","last_became_peered":"2023-12-11T17:44:20.617252+0000","last_unstale":"2023-12-11T17:45:14.880057+0000","last_undegraded":"2023-12-11T17:45:14.880057+0000","last_fullsized":"2023-12-11T17:45:14.880057+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T19:23:37.133765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.877027+0000","last_change":"2023-12-11T17:44:20.619325+0000","last_active":"2023-12-11T17:45:14.877027+0000","last_peered":"2023-12-11T17:45:14.877027+0000","last_clean":"2023-12-11T17:45:14.877027+0000","last_became_active":"2023-12-11T17:44:20.619016+0000","last_became_peered":"2023-12-11T17:44:20.619016+0000","last_unstale":"2023-12-11T17:45:14.877027+0000","last_undegraded":"2023-12-11T17:45:14.877027+0000","last_fullsized":"2023-12-11T17:45:14.877027+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T20:21:24.877379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.876297+0000","last_change":"2023-12-11T17:44:20.624432+0000","last_active":"2023-12-11T17:45:14.876297+0000","last_peered":"2023-12-11T17:45:14.876297+0000","last_clean":"2023-12-11T17:45:14.876297+0000","last_became_active":"2023-12-11T17:44:20.624148+0000","last_became_peered":"2023-12-11T17:44:20.624148+0000","last_unstale":"2023-12-11T17:45:14.876297+0000","last_undegraded":"2023-12-11T17:45:14.876297+0000","last_fullsized":"2023-12-11T17:45:14.876297+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T19:15:46.054968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.873629+0000","last_change":"2023-12-11T17:44:20.624134+0000","last_active":"2023-12-11T17:45:14.873629+0000","last_peered":"2023-12-11T17:45:14.873629+0000","last_clean":"2023-12-11T17:45:14.873629+0000","last_became_active":"2023-12-11T17:44:20.623858+0000","last_became_peered":"2023-12-11T17:44:20.623858+0000","last_unstale":"2023-12-11T17:45:14.873629+0000","last_undegraded":"2023-12-11T17:45:14.873629+0000","last_fullsized":"2023-12-11T17:45:14.873629+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:24:48.946619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.880004+0000","last_change":"2023-12-11T17:44:20.620407+0000","last_active":"2023-12-11T17:45:14.880004+0000","last_peered":"2023-12-11T17:45:14.880004+0000","last_clean":"2023-12-11T17:45:14.880004+0000","last_became_active":"2023-12-11T17:44:20.620106+0000","last_became_peered":"2023-12-11T17:44:20.620106+0000","last_unstale":"2023-12-11T17:45:14.880004+0000","last_undegraded":"2023-12-11T17:45:14.880004+0000","last_fullsized":"2023-12-11T17:45:14.880004+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T22:18:36.780037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.880745+0000","last_change":"2023-12-11T17:44:20.618426+0000","last_active":"2023-12-11T17:45:14.880745+0000","last_peered":"2023-12-11T17:45:14.880745+0000","last_clean":"2023-12-11T17:45:14.880745+0000","last_became_active":"2023-12-11T17:44:20.618136+0000","last_became_peered":"2023-12-11T17:44:20.618136+0000","last_unstale":"2023-12-11T17:45:14.880745+0000","last_undegraded":"2023-12-11T17:45:14.880745+0000","last_fullsized":"2023-12-11T17:45:14.880745+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T05:02:04.096866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.877765+0000","last_change":"2023-12-11T17:44:20.619609+0000","last_active":"2023-12-11T17:45:14.877765+0000","last_peered":"2023-12-11T17:45:14.877765+0000","last_clean":"2023-12-11T17:45:14.877765+0000","last_became_active":"2023-12-11T17:44:20.619345+0000","last_became_peered":"2023-12-11T17:44:20.619345+0000","last_unstale":"2023-12-11T17:45:14.877765+0000","last_undegraded":"2023-12-11T17:45:14.877765+0000","last_fullsized":"2023-12-11T17:45:14.877765+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T19:56:00.839819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.881657+0000","last_change":"2023-12-11T17:44:20.614115+0000","last_active":"2023-12-11T17:45:14.881657+0000","last_peered":"2023-12-11T17:45:14.881657+0000","last_clean":"2023-12-11T17:45:14.881657+0000","last_became_active":"2023-12-11T17:44:20.613800+0000","last_became_peered":"2023-12-11T17:44:20.613800+0000","last_unstale":"2023-12-11T17:45:14.881657+0000","last_undegraded":"2023-12-11T17:45:14.881657+0000","last_fullsized":"2023-12-11T17:45:14.881657+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T20:33:37.170611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T17:45:15.891039+0000","last_change":"2023-12-11T17:45:15.890531+0000","last_active":"2023-12-11T17:45:15.891039+0000","last_peered":"2023-12-11T17:45:15.891039+0000","last_clean":"2023-12-11T17:45:15.891039+0000","last_became_active":"2023-12-11T17:45:15.890139+0000","last_became_peered":"2023-12-11T17:45:15.890139+0000","last_unstale":"2023-12-11T17:45:15.891039+0000","last_undegraded":"2023-12-11T17:45:15.891039+0000","last_fullsized":"2023-12-11T17:45:15.891039+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T19:13:27.577867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,8],"acting":[6,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.880882+0000","last_change":"2023-12-11T17:44:20.617195+0000","last_active":"2023-12-11T17:45:14.880882+0000","last_peered":"2023-12-11T17:45:14.880882+0000","last_clean":"2023-12-11T17:45:14.880882+0000","last_became_active":"2023-12-11T17:44:20.616529+0000","last_became_peered":"2023-12-11T17:44:20.616529+0000","last_unstale":"2023-12-11T17:45:14.880882+0000","last_undegraded":"2023-12-11T17:45:14.880882+0000","last_fullsized":"2023-12-11T17:45:14.880882+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T23:53:50.558483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.880580+0000","last_change":"2023-12-11T17:44:20.618290+0000","last_active":"2023-12-11T17:45:14.880580+0000","last_peered":"2023-12-11T17:45:14.880580+0000","last_clean":"2023-12-11T17:45:14.880580+0000","last_became_active":"2023-12-11T17:44:20.617982+0000","last_became_peered":"2023-12-11T17:44:20.617982+0000","last_unstale":"2023-12-11T17:45:14.880580+0000","last_undegraded":"2023-12-11T17:45:14.880580+0000","last_fullsized":"2023-12-11T17:45:14.880580+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T21:58:14.444462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.880250+0000","last_change":"2023-12-11T17:44:20.619958+0000","last_active":"2023-12-11T17:45:14.880250+0000","last_peered":"2023-12-11T17:45:14.880250+0000","last_clean":"2023-12-11T17:45:14.880250+0000","last_became_active":"2023-12-11T17:44:20.619504+0000","last_became_peered":"2023-12-11T17:44:20.619504+0000","last_unstale":"2023-12-11T17:45:14.880250+0000","last_undegraded":"2023-12-11T17:45:14.880250+0000","last_fullsized":"2023-12-11T17:45:14.880250+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T19:36:41.193000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.880874+0000","last_change":"2023-12-11T17:44:20.612953+0000","last_active":"2023-12-11T17:45:14.880874+0000","last_peered":"2023-12-11T17:45:14.880874+0000","last_clean":"2023-12-11T17:45:14.880874+0000","last_became_active":"2023-12-11T17:44:20.612618+0000","last_became_peered":"2023-12-11T17:44:20.612618+0000","last_unstale":"2023-12-11T17:45:14.880874+0000","last_undegraded":"2023-12-11T17:45:14.880874+0000","last_fullsized":"2023-12-11T17:45:14.880874+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T04:25:39.528227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":3542208,"data_compressed":211312,"data_compressed_allocated":270336,"data_compressed_original":3506176,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65384,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":259,"num_read_kb":202,"num_write":159,"num_write_kb":114,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1593,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":196152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":77,"seq":330712481818,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":962560,"data_stored":483492,"data_compressed":17682,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 11 17:44:38 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.502},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.67600000000000005}]},{"osd":1,"last update":"Mon Dec 11 17:44:38 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.63400000000000001}]},{"osd":2,"last update":"Mon Dec 11 17:44:38 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.54200000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48599999999999999}]},{"osd":4,"last update":"Mon Dec 11 17:44:38 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.71899999999999997}]},{"osd":5,"last update":"Mon Dec 11 17:44:38 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.66000000000000003}]},{"osd":6,"last update":"Mon Dec 11 17:44:38 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.448,"5min":1.448,"15min":1.448},"last":0.80100000000000005}]},{"osd":7,"last update":"Mon Dec 11 17:44:38 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.874},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.385,"5min":1.385,"15min":1.385},"last":0.81799999999999995}]},{"osd":8,"last update":"Mon Dec 11 17:45:21 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.74099999999999999}]},{"osd":9,"last update":"Mon Dec 11 17:44:38 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.5089999999999999,"5min":1.5089999999999999,"15min":1.5089999999999999},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.83099999999999996}]},{"osd":10,"last update":"Mon Dec 11 17:44:38 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.4870000000000001,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.5309999999999999,"5min":1.5309999999999999,"15min":1.5309999999999999},"last":0.92600000000000005}]}]},{"osd":10,"up_from":70,"seq":300647710748,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28096,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717056,"statfs":{"total":95995035648,"available":95966265344,"internally_reserved":0,"allocated":1101824,"data_stored":1131938,"data_compressed":53447,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 11 17:45:21 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.42699999999999999}]},{"osd":1,"last update":"Mon Dec 11 17:45:21 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.50700000000000001,"15min":0.505},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.52300000000000002,"15min":0.51600000000000001},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.315}]},{"osd":2,"last update":"Mon Dec 11 17:45:21 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.63700000000000001,"15min":0.63600000000000001},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.81499999999999995,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.749},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.65200000000000002,"15min":0.64900000000000002},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.147,"5min":1.147,"15min":1.147},"last":0.629}]},{"osd":3,"last update":"Mon Dec 11 17:45:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.626,"15min":0.63700000000000001},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.76100000000000001,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.69399999999999995,"15min":0.70199999999999996},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.40899999999999997}]},{"osd":4,"last update":"Mon Dec 11 17:45:21 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.60999999999999999,"15min":0.60599999999999998},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.57499999999999996,"15min":0.57599999999999996},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.47399999999999998}]},{"osd":5,"last update":"Mon Dec 11 17:45:21 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.71299999999999997,"15min":0.71099999999999997},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.74399999999999999,"15min":0.746},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.72599999999999998}]},{"osd":6,"last update":"Mon Dec 11 17:45:21 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.71799999999999997,"15min":0.72899999999999998},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.57799999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56799999999999995}]},{"osd":8,"last update":"Mon Dec 11 17:45:21 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.83999999999999997,"15min":0.83399999999999996},"min":{"1min":0.66300000000000003,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.417,"5min":1.417,"15min":1.417},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.82299999999999995,"15min":0.82899999999999996},"min":{"1min":0.58499999999999996,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.452,"5min":1.452,"15min":1.452},"last":0.67700000000000005}]},{"osd":9,"last update":"Mon Dec 11 17:45:21 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.83999999999999997,"15min":0.84599999999999997},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.069,"5min":1.202,"15min":1.202},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.81999999999999995,"15min":0.83799999999999997},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.997,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.54400000000000004}]},{"osd":11,"last update":"Mon Dec 11 17:44:40 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.71199999999999997}]}]},{"osd":9,"up_from":65,"seq":279172874274,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":983040,"data_stored":487948,"data_compressed":17682,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 11 17:45:01 2023","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.46800000000000003,"15min":0.48799999999999999},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.51300000000000001,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.47699999999999998,"15min":0.49099999999999999},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.60799999999999998,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.28399999999999997}]},{"osd":1,"last update":"Mon Dec 11 17:45:01 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.64100000000000001,"15min":0.65100000000000002},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.86299999999999999,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.501},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.57499999999999996,"15min":0.57799999999999996},"min":{"1min":0.44,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.749,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.67600000000000005}]},{"osd":2,"last update":"Mon Dec 11 17:45:01 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.67900000000000005,"15min":0.69499999999999995},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.73899999999999999,"5min":1.046,"15min":1.046},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.65300000000000002,"15min":0.65900000000000003},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.76900000000000002}]},{"osd":3,"last update":"Mon Dec 11 17:45:01 2023","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.55700000000000005,"15min":0.57599999999999996},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.629,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.622,"15min":0.64800000000000002},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.81299999999999994,"5min":1.046,"15min":1.046},"last":0.42899999999999999}]},{"osd":4,"last update":"Mon Dec 11 17:45:01 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.66900000000000004,"15min":0.67900000000000005},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.83599999999999997,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.67800000000000005,"15min":0.68200000000000005},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.90100000000000002,"5min":1.157,"15min":1.157},"last":0.64800000000000002}]},{"osd":5,"last update":"Mon Dec 11 17:45:01 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.71599999999999997,"15min":0.72999999999999998},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.82299999999999995,"5min":1.204,"15min":1.204},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.79200000000000004,"15min":0.81699999999999995},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.88200000000000001,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.70399999999999996}]},{"osd":6,"last update":"Mon Dec 11 17:43:57 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.5289999999999999,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.44800000000000001}]},{"osd":7,"last update":"Mon Dec 11 17:45:01 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.76500000000000001,"15min":0.77900000000000003},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.0029999999999999,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.81599999999999995,"15min":0.83899999999999997},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.95499999999999996,"5min":1.224,"15min":1.224},"last":0.55900000000000005}]},{"osd":8,"last update":"Mon Dec 11 17:45:01 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.78000000000000003,"15min":0.80200000000000005},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.92300000000000004,"5min":1.131,"15min":1.131},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.80300000000000005,"15min":0.81899999999999995},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.1100000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.60399999999999998}]},{"osd":10,"last update":"Mon Dec 11 17:45:23 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.73999999999999999,"15min":0.754},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.98399999999999999,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.78700000000000003,"15min":0.80500000000000005},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.98599999999999999,"5min":1.252,"15min":1.252},"last":0.69399999999999995}]},{"osd":11,"last update":"Mon Dec 11 17:45:25 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.66000000000000003}]}]},{"osd":8,"up_from":57,"seq":244813135911,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":978944,"data_stored":483772,"data_compressed":17682,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 11 17:44:37 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.54300000000000004,"15min":0.53600000000000003},"min":{"1min":0.36499999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.56200000000000006,"15min":0.56299999999999994},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.78000000000000003,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.73499999999999999}]},{"osd":1,"last update":"Mon Dec 11 17:44:37 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.57799999999999996,"15min":0.57599999999999996},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.94999999999999996,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.62,"5min":0.59699999999999998,"15min":0.59299999999999997},"min":{"1min":0.42799999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.86199999999999999}]},{"osd":2,"last update":"Mon Dec 11 17:44:37 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.437,"15min":0.42999999999999999},"min":{"1min":0.32000000000000001,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.47899999999999998,"15min":0.48199999999999998},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.68899999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.58199999999999996}]},{"osd":3,"last update":"Mon Dec 11 17:44:37 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.59599999999999997,"15min":0.59999999999999998},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.73099999999999998,"5min":1.077,"15min":1.077},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.57899999999999996,"15min":0.57599999999999996},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.80200000000000005,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.69699999999999995}]},{"osd":4,"last update":"Mon Dec 11 17:44:37 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.628,"15min":0.625},"min":{"1min":0.47299999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.87,"5min":1.141,"15min":1.141},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.621,"15min":0.61099999999999999},"min":{"1min":0.53000000000000003,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.0069999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.84599999999999997}]},{"osd":5,"last update":"Mon Dec 11 17:44:37 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.56299999999999994,"15min":0.56499999999999995},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.72799999999999998,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.59199999999999997,"15min":0.59799999999999998},"min":{"1min":0.35599999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.76300000000000001,"5min":1.482,"15min":1.482},"last":0.63900000000000001}]},{"osd":6,"last update":"Mon Dec 11 17:44:37 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.64300000000000002,"15min":0.63400000000000001},"min":{"1min":0.45000000000000001,"5min":0.37,"15min":0.37},"max":{"1min":0.84599999999999997,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.68500000000000005,"15min":0.68300000000000005},"min":{"1min":0.53600000000000003,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.87,"5min":1.008,"15min":1.008},"last":0.80200000000000005}]},{"osd":7,"last update":"Mon Dec 11 17:44:37 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.68000000000000005,"15min":0.67300000000000004},"min":{"1min":0.48699999999999999,"5min":0.432,"15min":0.432},"max":{"1min":0.93300000000000005,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.73099999999999998,"15min":0.72499999999999998},"min":{"1min":0.55400000000000005,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.546,"5min":1.546,"15min":1.546},"last":0.878}]},{"osd":9,"last update":"Mon Dec 11 17:45:00 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.68300000000000005,"15min":0.67100000000000004},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.92600000000000005},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.69899999999999995,"15min":0.68200000000000005},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.94399999999999995}]},{"osd":10,"last update":"Mon Dec 11 17:45:20 2023","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.73299999999999998,"15min":0.70099999999999996},"min":{"1min":0.58299999999999996,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.4890000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.96399999999999997},{"interface":"front","average":{"1min":0.92900000000000005,"5min":0.73199999999999998,"15min":0.69899999999999995},"min":{"1min":0.67700000000000005,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":1.02}]},{"osd":11,"last update":"Mon Dec 11 17:45:20 2023","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":1.002},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.97899999999999998}]}]},{"osd":7,"up_from":50,"seq":214748364842,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1024000,"data_stored":1079486,"data_compressed":53447,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 11 17:44:13 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.54700000000000004,"15min":0.54300000000000004},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.44},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.55200000000000005,"15min":0.54800000000000004},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.42399999999999999}]},{"osd":1,"last update":"Mon Dec 11 17:45:14 2023","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.40600000000000003,"15min":0.40899999999999997},"min":{"1min":0.317,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.51700000000000002,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.373},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.44900000000000001,"15min":0.44800000000000001},"min":{"1min":0.30299999999999999,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.40699999999999997}]},{"osd":2,"last update":"Mon Dec 11 17:45:14 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.57699999999999996,"15min":0.57799999999999996},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.59699999999999998,"15min":0.59299999999999997},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.81499999999999995,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.58799999999999997}]},{"osd":3,"last update":"Mon Dec 11 17:45:14 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.58199999999999996,"15min":0.56799999999999995},"min":{"1min":0.40200000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.625,"5min":0.61099999999999999,"15min":0.60399999999999998},"min":{"1min":0.441,"5min":0.377,"15min":0.377},"max":{"1min":0.80300000000000005,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.53000000000000003}]},{"osd":4,"last update":"Mon Dec 11 17:45:14 2023","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.52700000000000002,"15min":0.52900000000000003},"min":{"1min":0.34000000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.64600000000000002,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.432,"5min":0.47499999999999998,"15min":0.48699999999999999},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.60399999999999998,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.38800000000000001}]},{"osd":5,"last update":"Mon Dec 11 17:45:14 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.622,"15min":0.624},"min":{"1min":0.48399999999999999,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.77500000000000002,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.496,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.85799999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.83299999999999996}]},{"osd":6,"last update":"Mon Dec 11 17:45:14 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.54500000000000004,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.872,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.63600000000000001,"15min":0.63700000000000001},"min":{"1min":0.41199999999999998,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.45800000000000002}]},{"osd":8,"last update":"Mon Dec 11 17:44:37 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.68300000000000005,"15min":0.69199999999999995},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.88200000000000001,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.70299999999999996,"15min":0.70899999999999996},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.92500000000000004,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.79400000000000004}]},{"osd":9,"last update":"Mon Dec 11 17:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.71899999999999997,"15min":0.72099999999999997},"min":{"1min":0.46899999999999997,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.90700000000000003,"5min":1.056,"15min":1.056},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.745,"15min":0.747},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":0.90700000000000003,"5min":1.109,"15min":1.109},"last":0.68899999999999995}]},{"osd":10,"last update":"Mon Dec 11 17:45:19 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.70099999999999996,"15min":0.69699999999999995},"min":{"1min":0.51900000000000002,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.93799999999999994,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.69499999999999995,"15min":0.69299999999999995},"min":{"1min":0.60499999999999998,"5min":0.432,"15min":0.432},"max":{"1min":0.80900000000000005,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.70799999999999996}]},{"osd":11,"last update":"Mon Dec 11 17:45:21 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.80800000000000005}]}]},{"osd":6,"up_from":45,"seq":193273528367,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":958464,"data_stored":479028,"data_compressed":17682,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 11 17:44:54 2023","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.40400000000000003,"15min":0.38800000000000001},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.433,"5min":0.41899999999999998,"15min":0.40899999999999997},"min":{"1min":0.29699999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.38}]},{"osd":1,"last update":"Mon Dec 11 17:44:54 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.55700000000000005,"15min":0.54300000000000004},"min":{"1min":0.41699999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.56899999999999995,"15min":0.55100000000000005},"min":{"1min":0.42799999999999999,"5min":0.375,"15min":0.375},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.53200000000000003}]},{"osd":2,"last update":"Mon Dec 11 17:44:54 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.58399999999999996,"15min":0.56000000000000005},"min":{"1min":0.44500000000000001,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.83599999999999997,"5min":1.107,"15min":1.107},"last":0.433},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.58899999999999997,"15min":0.56000000000000005},"min":{"1min":0.34799999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.83199999999999996,"5min":1.236,"15min":1.236},"last":0.44900000000000001}]},{"osd":3,"last update":"Mon Dec 11 17:44:54 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.503,"15min":0.48499999999999999},"min":{"1min":0.32800000000000001,"5min":0.317,"15min":0.317},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.50900000000000001,"15min":0.48999999999999999},"min":{"1min":0.375,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.89200000000000002,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.48699999999999999}]},{"osd":4,"last update":"Mon Dec 11 17:44:54 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.62,"15min":0.60599999999999998},"min":{"1min":0.443,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.64600000000000002,"15min":0.63200000000000001},"min":{"1min":0.46300000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.96499999999999997,"5min":1.083,"15min":1.083},"last":0.59499999999999997}]},{"osd":5,"last update":"Mon Dec 11 17:44:54 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.64400000000000002,"15min":0.63},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.85799999999999998,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.67100000000000004,"15min":0.65300000000000002},"min":{"1min":0.56999999999999995,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.83299999999999996,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.63}]},{"osd":7,"last update":"Mon Dec 11 17:45:13 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.70799999999999996,"15min":0.67600000000000005},"min":{"1min":0.52800000000000002,"5min":0.442,"15min":0.442},"max":{"1min":2.4359999999999999,"5min":2.4359999999999999,"15min":2.4359999999999999},"last":0.58499999999999996},{"interface":"front","average":{"1min":1.21,"5min":0.81200000000000006,"15min":0.74099999999999999},"min":{"1min":0.50600000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":7.9029999999999996,"5min":7.9029999999999996,"15min":7.9029999999999996},"last":0.65500000000000003}]},{"osd":8,"last update":"Mon Dec 11 17:44:40 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.752,"15min":0.751},"min":{"1min":0.54900000000000004,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.002,"5min":1.157,"15min":1.157},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.69899999999999995,"15min":0.69699999999999995},"min":{"1min":0.495,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.97399999999999998,"5min":1.028,"15min":1.028},"last":0.64100000000000001}]},{"osd":9,"last update":"Mon Dec 11 17:44:57 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.74099999999999999,"15min":0.752},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.92400000000000004,"5min":1.371,"15min":1.371},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.76000000000000001,"15min":0.77400000000000002},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.92700000000000005,"5min":1.133,"15min":1.133},"last":0.67800000000000005}]},{"osd":10,"last update":"Mon Dec 11 17:45:21 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.84399999999999997,"15min":0.83599999999999997},"min":{"1min":0.60099999999999998,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":2.4900000000000002,"5min":2.4900000000000002,"15min":2.4900000000000002},"last":0.66500000000000004},{"interface":"front","average":{"1min":1.222,"5min":0.93000000000000005,"15min":0.88200000000000001},"min":{"1min":0.626,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":7.931,"5min":7.931,"15min":7.931},"last":0.70599999999999996}]},{"osd":11,"last update":"Mon Dec 11 17:45:23 2023","interfaces":[{"interface":"back","average":{"1min":1.228,"5min":1.228,"15min":1.228},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":7.9610000000000003,"5min":7.9610000000000003,"15min":7.9610000000000003},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":2.7280000000000002,"5min":2.7280000000000002,"15min":2.7280000000000002},"last":0.68899999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574916,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1032192,"data_stored":540945,"data_compressed":17682,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 11 17:45:15 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.53400000000000003,"15min":0.49299999999999999},"min":{"1min":0.33600000000000002,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":4.3520000000000003,"5min":4.3520000000000003,"15min":4.3520000000000003},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.57199999999999995,"15min":0.52600000000000002},"min":{"1min":0.35799999999999998,"5min":0.308,"15min":0.308},"max":{"1min":4.3739999999999997,"5min":4.3739999999999997,"15min":4.3739999999999997},"last":0.45200000000000001}]},{"osd":2,"last update":"Mon Dec 11 17:44:30 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.54800000000000004,"15min":0.51800000000000002},"min":{"1min":0.34799999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":1.4690000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.499},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.57099999999999995,"15min":0.54200000000000004},"min":{"1min":0.41699999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.4530000000000001,"5min":1.4530000000000001,"15min":1.4530000000000001},"last":0.54800000000000004}]},{"osd":3,"last update":"Mon Dec 11 17:44:56 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.52600000000000002,"15min":0.51800000000000002},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.83799999999999997,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.54600000000000004,"15min":0.51900000000000002},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.81699999999999995,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.60199999999999998}]},{"osd":4,"last update":"Mon Dec 11 17:45:15 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.45100000000000001,"15min":0.41099999999999998},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.83499999999999996,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.45900000000000002,"15min":0.40600000000000003},"min":{"1min":0.26900000000000002,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.80300000000000005,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.40100000000000002}]},{"osd":5,"last update":"Mon Dec 11 17:44:36 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.59899999999999998,"15min":0.56299999999999994},"min":{"1min":0.41699999999999998,"5min":0.37,"15min":0.37},"max":{"1min":1.5109999999999999,"5min":1.5109999999999999,"15min":1.5109999999999999},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.60699999999999998,"15min":0.57599999999999996},"min":{"1min":0.53900000000000003,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.411,"5min":1.411,"15min":1.411},"last":0.48199999999999998}]},{"osd":6,"last update":"Mon Dec 11 17:44:56 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.59799999999999998,"15min":0.58799999999999997},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.81100000000000005,"5min":1.014,"15min":1.014},"last":1.454},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.622,"15min":0.60999999999999999},"min":{"1min":0.40699999999999997,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.89600000000000002,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.56200000000000006}]},{"osd":7,"last update":"Mon Dec 11 17:45:15 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.59699999999999998,"15min":0.58399999999999996},"min":{"1min":0.30399999999999999,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.97699999999999998,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.59599999999999997,"15min":0.59099999999999997},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.0409999999999999,"5min":1.379,"15min":1.379},"last":0.69099999999999995}]},{"osd":8,"last update":"Mon Dec 11 17:44:36 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.71699999999999997,"15min":0.70699999999999996},"min":{"1min":0.48699999999999999,"5min":0.433,"15min":0.433},"max":{"1min":1.5349999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.755,"5min":0.69399999999999995,"15min":0.68400000000000005},"min":{"1min":0.48999999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.4259999999999999,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":0.67700000000000005}]},{"osd":9,"last update":"Mon Dec 11 17:45:01 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.754,"15min":0.73099999999999998},"min":{"1min":0.53300000000000003,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":4.4690000000000003,"5min":4.4690000000000003,"15min":4.4690000000000003},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.73599999999999999,"15min":0.73199999999999998},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.97799999999999998,"5min":1.038,"15min":1.038},"last":0.83199999999999996}]},{"osd":10,"last update":"Mon Dec 11 17:45:24 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.70199999999999996,"15min":0.69399999999999995},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.92700000000000005,"5min":1.006,"15min":1.006},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.74399999999999999,"15min":0.73799999999999999},"min":{"1min":0.47399999999999998,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.57499999999999996}]},{"osd":11,"last update":"Mon Dec 11 17:45:20 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.65900000000000003}]}]},{"osd":0,"up_from":8,"seq":34359738440,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":983040,"data_stored":480416,"data_compressed":17682,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Dec 11 17:45:14 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.54500000000000004,"15min":0.51400000000000001},"min":{"1min":0.38600000000000001,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":2.327,"5min":2.327,"15min":2.327},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.55600000000000005,"15min":0.501},"min":{"1min":0.34699999999999998,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.4890000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.49099999999999999}]},{"osd":2,"last update":"Mon Dec 11 17:44:38 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.56799999999999995,"15min":0.51500000000000001},"min":{"1min":0.38200000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":4.1029999999999998,"5min":4.1029999999999998,"15min":4.1029999999999998},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.55100000000000005,"15min":0.51100000000000001},"min":{"1min":0.33400000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.746,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.56499999999999995}]},{"osd":3,"last update":"Mon Dec 11 17:45:24 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":3.3029999999999999,"5min":3.3029999999999999,"15min":3.3029999999999999},"last":0.308},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.34000000000000002}]},{"osd":4,"last update":"Mon Dec 11 17:45:24 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.55900000000000005,"15min":0.52700000000000002},"min":{"1min":0.434,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.28,"5min":1.28,"15min":1.28},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.58899999999999997,"15min":0.55200000000000005},"min":{"1min":0.41999999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":2.5019999999999998,"5min":2.5019999999999998,"15min":2.5019999999999998},"last":0.57699999999999996}]},{"osd":5,"last update":"Mon Dec 11 17:44:38 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.60599999999999998,"15min":0.58799999999999997},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":2.399,"5min":2.399,"15min":2.399},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.61299999999999999,"15min":0.59599999999999997},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.82699999999999996,"5min":0.995,"15min":0.995},"last":0.52600000000000002}]},{"osd":6,"last update":"Mon Dec 11 17:44:57 2023","interfaces":[{"interface":"back","average":{"1min":1.3999999999999999,"5min":0.68999999999999995,"15min":0.56000000000000005},"min":{"1min":0.32900000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":16.738,"5min":16.738,"15min":16.738},"last":0.39700000000000002},{"interface":"front","average":{"1min":1.3029999999999999,"5min":0.74199999999999999,"15min":0.63400000000000001},"min":{"1min":0.379,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":12.920999999999999,"5min":12.920999999999999,"15min":12.920999999999999},"last":0.442}]},{"osd":7,"last update":"Mon Dec 11 17:45:24 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.499},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.58799999999999997}]},{"osd":8,"last update":"Mon Dec 11 17:44:38 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.70299999999999996,"15min":0.68999999999999995},"min":{"1min":0.43099999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":3.3450000000000002,"5min":3.3450000000000002,"15min":3.3450000000000002},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.76100000000000001,"15min":0.76900000000000002},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.054,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.65000000000000002}]},{"osd":9,"last update":"Mon Dec 11 17:45:00 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.65700000000000003,"15min":0.67200000000000004},"min":{"1min":0.39500000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.84999999999999998,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.76100000000000001,"15min":0.76500000000000001},"min":{"1min":0.49099999999999999,"5min":0.443,"15min":0.443},"max":{"1min":1.421,"5min":1.421,"15min":1.421},"last":0.61899999999999999}]},{"osd":10,"last update":"Mon Dec 11 17:45:24 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.80000000000000004,"15min":0.79800000000000004},"min":{"1min":0.59599999999999997,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.012,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.71799999999999997,"15min":0.70999999999999996},"min":{"1min":0.60999999999999999,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.3300000000000001,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.63900000000000001}]},{"osd":11,"last update":"Mon Dec 11 17:44:42 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":2.456,"5min":2.456,"15min":2.456},"last":0.63},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.107,"5min":1.107,"15min":1.107},"last":0.68700000000000006}]}]},{"osd":2,"up_from":18,"seq":77309411392,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1040384,"data_stored":540662,"data_compressed":17682,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 11 17:44:34 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.52900000000000003,"15min":0.502},"min":{"1min":0.32100000000000001,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.84899999999999998,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.51800000000000002,"15min":0.50800000000000001},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.79200000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.41099999999999998}]},{"osd":1,"last update":"Mon Dec 11 17:44:34 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.54100000000000004,"15min":0.51900000000000002},"min":{"1min":0.38900000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.37},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.56999999999999995,"15min":0.54100000000000004},"min":{"1min":0.35399999999999998,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.38900000000000001}]},{"osd":3,"last update":"Mon Dec 11 17:45:01 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.57299999999999995,"15min":0.54200000000000004},"min":{"1min":0.40100000000000002,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.88800000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.55000000000000004,"15min":0.53000000000000003},"min":{"1min":0.34399999999999997,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.34899999999999998}]},{"osd":4,"last update":"Mon Dec 11 17:45:22 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.47199999999999998}]},{"osd":5,"last update":"Mon Dec 11 17:44:34 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.46800000000000003,"15min":0.44600000000000001},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.46000000000000002,"15min":0.441},"min":{"1min":0.28299999999999997,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.54500000000000004}]},{"osd":6,"last update":"Mon Dec 11 17:44:52 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.66600000000000004,"15min":0.64000000000000001},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.626},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.626,"15min":0.61099999999999999},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.56799999999999995}]},{"osd":7,"last update":"Mon Dec 11 17:45:24 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.60799999999999998}]},{"osd":8,"last update":"Mon Dec 11 17:44:40 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.58199999999999996,"15min":0.57499999999999996},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.442},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.64600000000000002,"15min":0.64100000000000001},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.61599999999999999}]},{"osd":9,"last update":"Mon Dec 11 17:45:01 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.71899999999999997,"15min":0.70699999999999996},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.754,"15min":0.74399999999999999},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.64300000000000002}]},{"osd":10,"last update":"Mon Dec 11 17:45:22 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.71099999999999997,"15min":0.69199999999999995},"min":{"1min":0.66700000000000004,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.76000000000000001,"15min":0.73699999999999999},"min":{"1min":0.70299999999999996,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.68300000000000005}]},{"osd":11,"last update":"Mon Dec 11 17:45:22 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.877,"5min":0.877,"15min":0.877},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.70799999999999996}]}]},{"osd":3,"up_from":25,"seq":107374182459,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":966656,"data_stored":478891,"data_compressed":17682,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 11 17:44:57 2023","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.38600000000000001,"15min":0.35799999999999998},"min":{"1min":0.26200000000000001,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":0.56599999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.433,"15min":0.42099999999999999},"min":{"1min":0.30299999999999999,"5min":0.251,"15min":0.251},"max":{"1min":0.623,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.44700000000000001}]},{"osd":1,"last update":"Mon Dec 11 17:44:57 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.58399999999999996,"15min":0.56000000000000005},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.88400000000000001,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.58099999999999996,"15min":0.56699999999999995},"min":{"1min":0.36399999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.89000000000000001,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.61799999999999999}]},{"osd":2,"last update":"Mon Dec 11 17:44:57 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.60799999999999998,"15min":0.60399999999999998},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.77900000000000003,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.60599999999999998,"15min":0.60199999999999998},"min":{"1min":0.39000000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.92700000000000005,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.66200000000000003}]},{"osd":4,"last update":"Mon Dec 11 17:45:21 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.60499999999999998,"15min":0.57199999999999995},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.3580000000000001,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.56200000000000006,"15min":0.55400000000000005},"min":{"1min":0.45000000000000001,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":0.54300000000000004}]},{"osd":5,"last update":"Mon Dec 11 17:44:35 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.61299999999999999,"15min":0.58899999999999997},"min":{"1min":0.432,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.64100000000000001,"15min":0.61799999999999999},"min":{"1min":0.41099999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.93100000000000005,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.78400000000000003}]},{"osd":6,"last update":"Mon Dec 11 17:44:57 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.48499999999999999,"15min":0.46800000000000003},"min":{"1min":0.33900000000000002,"5min":0.316,"15min":0.316},"max":{"1min":0.65000000000000002,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.59599999999999997,"15min":0.55900000000000005},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.88100000000000001,"5min":1.163,"15min":1.163},"last":0.67600000000000005}]},{"osd":7,"last update":"Mon Dec 11 17:45:15 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68100000000000005,"15min":0.65300000000000002},"min":{"1min":0.46300000000000002,"5min":0.432,"15min":0.432},"max":{"1min":0.997,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.69799999999999995,"15min":0.67400000000000004},"min":{"1min":0.51700000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.304,"5min":1.304,"15min":1.304},"last":0.74099999999999999}]},{"osd":8,"last update":"Mon Dec 11 17:43:38 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.66900000000000004}]},{"osd":9,"last update":"Mon Dec 11 17:44:57 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.78300000000000003,"15min":0.80700000000000005},"min":{"1min":0.41199999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.91200000000000003,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.69899999999999995,"15min":0.72199999999999998},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.751,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.60499999999999998}]},{"osd":10,"last update":"Mon Dec 11 17:45:21 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.81499999999999995,"15min":0.83499999999999996},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.0940000000000001,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.80800000000000005,"15min":0.82399999999999995},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.3320000000000001,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.80700000000000005}]},{"osd":11,"last update":"Mon Dec 11 17:45:21 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.81299999999999994}]}]},{"osd":4,"up_from":32,"seq":137438953527,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":970752,"data_stored":484067,"data_compressed":17682,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 11 17:44:15 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.52800000000000002,"15min":0.51600000000000001},"min":{"1min":0.36899999999999999,"5min":0.31,"15min":0.31},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.51800000000000002,"15min":0.50700000000000001},"min":{"1min":0.35599999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.34399999999999997}]},{"osd":1,"last update":"Mon Dec 11 17:45:18 2023","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.39700000000000002,"15min":0.38700000000000001},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.57399999999999995,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.28599999999999998},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.40100000000000002,"15min":0.377},"min":{"1min":0.26600000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.54900000000000004,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.315}]},{"osd":2,"last update":"Mon Dec 11 17:45:18 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.58799999999999997,"15min":0.56000000000000005},"min":{"1min":0.442,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.91900000000000004,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.58299999999999996,"15min":0.57899999999999996},"min":{"1min":0.40100000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.81000000000000005,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.376}]},{"osd":3,"last update":"Mon Dec 11 17:45:18 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.60299999999999998,"15min":0.58499999999999996},"min":{"1min":0.42799999999999999,"5min":0.372,"15min":0.372},"max":{"1min":0.76600000000000001,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.60399999999999998,"15min":0.58999999999999997},"min":{"1min":0.36199999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.79800000000000004,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.49399999999999999}]},{"osd":5,"last update":"Mon Dec 11 17:44:36 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.59499999999999997,"15min":0.58799999999999997},"min":{"1min":0.39500000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.442},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.57899999999999996,"15min":0.57399999999999995},"min":{"1min":0.38,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.82699999999999996,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.42599999999999999}]},{"osd":6,"last update":"Mon Dec 11 17:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63400000000000001,"15min":0.62},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.877,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.65200000000000002,"15min":0.63800000000000001},"min":{"1min":0.436,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.83999999999999997,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.51600000000000001}]},{"osd":7,"last update":"Mon Dec 11 17:45:18 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.53200000000000003,"15min":0.53600000000000003},"min":{"1min":0.29899999999999999,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.745,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.55500000000000005,"15min":0.53900000000000003},"min":{"1min":0.39800000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.47199999999999998}]},{"osd":8,"last update":"Mon Dec 11 17:44:39 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.68400000000000005,"15min":0.67800000000000005},"min":{"1min":0.46200000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.66100000000000003,"15min":0.65500000000000003},"min":{"1min":0.44700000000000001,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.88300000000000001,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.53100000000000003}]},{"osd":9,"last update":"Mon Dec 11 17:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.69799999999999995,"15min":0.69599999999999995},"min":{"1min":0.52200000000000002,"5min":0.502,"15min":0.502},"max":{"1min":0.90900000000000003,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.745,"5min":0.75700000000000001,"15min":0.75900000000000001},"min":{"1min":0.54200000000000004,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.97799999999999998,"5min":1.026,"15min":1.026},"last":0.52200000000000002}]},{"osd":10,"last update":"Mon Dec 11 17:45:18 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.71599999999999997,"15min":0.71999999999999997},"min":{"1min":0.51500000000000001,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.86299999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.752,"15min":0.76200000000000001},"min":{"1min":0.50800000000000001,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.95099999999999996,"5min":0.997,"15min":0.997},"last":0.55500000000000005}]},{"osd":11,"last update":"Mon Dec 11 17:45:24 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.61699999999999999}]}]},{"osd":5,"up_from":37,"seq":158913790003,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1036288,"data_stored":1075267,"data_compressed":53447,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 11 17:44:31 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.501,"15min":0.497},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.66000000000000003,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.53000000000000003,"15min":0.52200000000000002},"min":{"1min":0.36599999999999999,"5min":0.314,"15min":0.314},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.57499999999999996}]},{"osd":1,"last update":"Mon Dec 11 17:44:31 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53200000000000003,"15min":0.52300000000000002},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.79600000000000004,"5min":0.872,"15min":0.872},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.54100000000000004,"15min":0.53300000000000003},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.71799999999999997,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.78900000000000003}]},{"osd":2,"last update":"Mon Dec 11 17:44:31 2023","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.375,"15min":0.36399999999999999},"min":{"1min":0.28599999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.59899999999999998,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.377,"5min":0.40600000000000003,"15min":0.40400000000000003},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.57199999999999995,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.61099999999999999}]},{"osd":3,"last update":"Mon Dec 11 17:44:31 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.54700000000000004,"15min":0.53300000000000003},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.67000000000000004,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.56899999999999995,"15min":0.55800000000000005},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.78700000000000003,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.68700000000000006}]},{"osd":4,"last update":"Mon Dec 11 17:44:31 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.53600000000000003,"15min":0.52600000000000002},"min":{"1min":0.39200000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.71199999999999997,"5min":0.752,"15min":0.752},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58299999999999996,"15min":0.57199999999999995},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.88700000000000001,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.86399999999999999}]},{"osd":6,"last update":"Mon Dec 11 17:44:57 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.63200000000000001,"15min":0.61799999999999999},"min":{"1min":0.38,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.87,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68000000000000005,"15min":0.66800000000000004},"min":{"1min":0.56000000000000005,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.871,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.80800000000000005}]},{"osd":7,"last update":"Mon Dec 11 17:45:15 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.64100000000000001,"15min":0.63300000000000001},"min":{"1min":0.46300000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.61899999999999999,"15min":0.61299999999999999},"min":{"1min":0.47599999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.83999999999999997,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.876}]},{"osd":8,"last update":"Mon Dec 11 17:44:36 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.59199999999999997,"15min":0.60299999999999998},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.79500000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.628,"5min":0.64300000000000002,"15min":0.64500000000000002},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.80200000000000005,"5min":1.012,"15min":1.012},"last":0.83599999999999997}]},{"osd":9,"last update":"Mon Dec 11 17:45:00 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.69999999999999996,"15min":0.70399999999999996},"min":{"1min":0.56200000000000006,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.83399999999999996,"5min":1.246,"15min":1.246},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.75600000000000001,"15min":0.75800000000000001},"min":{"1min":0.52000000000000002,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.95999999999999996,"5min":1.218,"15min":1.218},"last":0.75800000000000001}]},{"osd":10,"last update":"Mon Dec 11 17:45:17 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.67200000000000004,"15min":0.66000000000000003},"min":{"1min":0.50800000000000001,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.748,"5min":0.69299999999999995,"15min":0.68400000000000005},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.89700000000000002,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.90900000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34126,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34126,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35765,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35765,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35765,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35765,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":62007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":61724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":62963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-11T17:45:30.812 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell --fsid d0a07eaa-984b-11ee-95a3-87774f69a715 -- ceph pg dump --format=json 2023-12-11T17:45:31.470 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/mon.a/config 2023-12-11T17:45:31.645 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:45:31 smithi181 ceph-mon[103627]: pgmap v296: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s; 84 B/s, 0 objects/s recovering 2023-12-11T17:45:31.645 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:45:31 smithi181 ceph-mon[103627]: from='client.15066 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-11T17:45:31.645 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:45:31 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:45:31.645 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:45:31 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:45:31.645 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:45:31 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:45:31.646 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:45:31 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:45:31.646 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:45:31 smithi181 ceph-mon[103627]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:45:31.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:45:31 smithi059 ceph-mon[98847]: pgmap v296: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s; 84 B/s, 0 objects/s recovering 2023-12-11T17:45:31.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:45:31 smithi059 ceph-mon[98847]: from='client.15066 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-11T17:45:31.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:45:31 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:45:31.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:45:31 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:45:31.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:45:31 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:45:31.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:45:31 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:45:31.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:45:31 smithi059 ceph-mon[98847]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:45:31.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:45:31 smithi115 ceph-mon[102748]: pgmap v296: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s; 84 B/s, 0 objects/s recovering 2023-12-11T17:45:31.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:45:31 smithi115 ceph-mon[102748]: from='client.15066 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-11T17:45:31.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:45:31 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:45:31.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:45:31 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:45:31.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:45:31 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:45:31.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:45:31 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:45:31.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:45:31 smithi115 ceph-mon[102748]: from='mgr.14150 172.21.15.59:0/3939738105' entity='mgr.x' 2023-12-11T17:45:32.703 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:32.700+0000 7fc71b147700 1 -- 172.21.15.59:0/2279518190 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc714100ef0 msgr2=0x7fc71410d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:32.703 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:32.700+0000 7fc71b147700 1 --2- 172.21.15.59:0/2279518190 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc714100ef0 0x7fc71410d330 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fc70c00b0d0 tx=0x7fc70c00b3e0 comp rx=0 tx=0).stop 2023-12-11T17:45:32.703 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:32.700+0000 7fc71b147700 1 -- 172.21.15.59:0/2279518190 shutdown_connections 2023-12-11T17:45:32.703 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:32.700+0000 7fc71b147700 1 --2- 172.21.15.59:0/2279518190 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc714100ef0 0x7fc71410d330 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:32.704 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:32.700+0000 7fc71b147700 1 --2- 172.21.15.59:0/2279518190 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc714106560 0x7fc714106940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:32.704 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:32.700+0000 7fc71b147700 1 --2- 172.21.15.59:0/2279518190 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc714100530 0x7fc7141009b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:32.704 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:32.700+0000 7fc71b147700 1 -- 172.21.15.59:0/2279518190 >> 172.21.15.59:0/2279518190 conn(0x7fc714076570 msgr2=0x7fc714076980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:45:32.704 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:32.700+0000 7fc71b147700 1 -- 172.21.15.59:0/2279518190 shutdown_connections 2023-12-11T17:45:32.704 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:32.701+0000 7fc71b147700 1 -- 172.21.15.59:0/2279518190 wait complete. 2023-12-11T17:45:32.706 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:32.701+0000 7fc71b147700 1 Processor -- start 2023-12-11T17:45:32.706 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:32.701+0000 7fc71b147700 1 -- start start 2023-12-11T17:45:32.706 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:32.701+0000 7fc71b147700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc714100530 0x7fc71419c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:32.706 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:32.701+0000 7fc71b147700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc714100ef0 0x7fc71419cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:32.706 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:32.701+0000 7fc71b147700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc714106560 0x7fc7141967d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:32.706 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:32.701+0000 7fc71b147700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc714196e40 con 0x7fc714100ef0 2023-12-11T17:45:32.706 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:32.701+0000 7fc71b147700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc714196fb0 con 0x7fc714106560 2023-12-11T17:45:32.706 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:32.701+0000 7fc71b147700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc714197170 con 0x7fc714100530 2023-12-11T17:45:32.706 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:32.702+0000 7fc713fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc714100ef0 0x7fc71419cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:32.706 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:32.702+0000 7fc713fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc714100ef0 0x7fc71419cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33892/0 (socket says 172.21.15.59:33892) 2023-12-11T17:45:32.706 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:32.702+0000 7fc713fff700 1 -- 172.21.15.59:0/1399201117 learned_addr learned my addr 172.21.15.59:0/1399201117 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:45:32.707 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:32.702+0000 7fc7196e4700 1 --2- 172.21.15.59:0/1399201117 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc714106560 0x7fc7141967d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:32.707 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:32.702+0000 7fc718ee3700 1 --2- 172.21.15.59:0/1399201117 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc714100530 0x7fc71419c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:32.707 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:32.702+0000 7fc713fff700 1 -- 172.21.15.59:0/1399201117 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc714100530 msgr2=0x7fc71419c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:32.707 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:32.702+0000 7fc713fff700 1 --2- 172.21.15.59:0/1399201117 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc714100530 0x7fc71419c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:32.707 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:32.702+0000 7fc713fff700 1 -- 172.21.15.59:0/1399201117 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc714106560 msgr2=0x7fc7141967d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:32.707 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:32.702+0000 7fc713fff700 1 --2- 172.21.15.59:0/1399201117 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc714106560 0x7fc7141967d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:32.707 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:32.702+0000 7fc713fff700 1 -- 172.21.15.59:0/1399201117 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc70c009d00 con 0x7fc714100ef0 2023-12-11T17:45:32.707 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:32.702+0000 7fc718ee3700 1 --2- 172.21.15.59:0/1399201117 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc714100530 0x7fc71419c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T17:45:32.707 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:32.702+0000 7fc713fff700 1 --2- 172.21.15.59:0/1399201117 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc714100ef0 0x7fc71419cb80 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fc70000d950 tx=0x7fc70000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:32.707 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:32.702+0000 7fc711ffb700 1 -- 172.21.15.59:0/1399201117 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc700009940 con 0x7fc714100ef0 2023-12-11T17:45:32.707 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:32.702+0000 7fc711ffb700 1 -- 172.21.15.59:0/1399201117 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc700010460 con 0x7fc714100ef0 2023-12-11T17:45:32.707 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:32.702+0000 7fc71b147700 1 -- 172.21.15.59:0/1399201117 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc714197450 con 0x7fc714100ef0 2023-12-11T17:45:32.707 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:32.702+0000 7fc711ffb700 1 -- 172.21.15.59:0/1399201117 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc70000f620 con 0x7fc714100ef0 2023-12-11T17:45:32.707 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:32.702+0000 7fc71b147700 1 -- 172.21.15.59:0/1399201117 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7141a3630 con 0x7fc714100ef0 2023-12-11T17:45:32.708 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:32.703+0000 7fc71b147700 1 -- 172.21.15.59:0/1399201117 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc714108a10 con 0x7fc714100ef0 2023-12-11T17:45:32.708 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:32.704+0000 7fc711ffb700 1 -- 172.21.15.59:0/1399201117 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7fc7000108d0 con 0x7fc714100ef0 2023-12-11T17:45:32.708 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:32.705+0000 7fc711ffb700 1 --2- 172.21.15.59:0/1399201117 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fc7040a1510 0x7fc7040a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:32.708 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:32.705+0000 7fc718ee3700 1 --2- 172.21.15.59:0/1399201117 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fc7040a1510 0x7fc7040a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:32.708 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:32.705+0000 7fc711ffb700 1 -- 172.21.15.59:0/1399201117 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fc7000cd060 con 0x7fc714100ef0 2023-12-11T17:45:32.709 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:32.706+0000 7fc718ee3700 1 --2- 172.21.15.59:0/1399201117 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fc7040a1510 0x7fc7040a39d0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fc708005950 tx=0x7fc7080058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:32.712 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:32.710+0000 7fc711ffb700 1 -- 172.21.15.59:0/1399201117 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fc7001002f0 con 0x7fc714100ef0 2023-12-11T17:45:32.907 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:32.904+0000 7fc71b147700 1 -- 172.21.15.59:0/1399201117 --> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc714197c40 con 0x7fc7040a1510 2023-12-11T17:45:32.922 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:32.919+0000 7fc711ffb700 1 -- 172.21.15.59:0/1399201117 <== mgr.14150 v2:172.21.15.59:6800/3627728353 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+426621 (secure 0 0 0) 0x7fc714197c40 con 0x7fc7040a1510 2023-12-11T17:45:32.927 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-12-11T17:45:32.931 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:32.923+0000 7fc71b147700 1 -- 172.21.15.59:0/1399201117 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fc7040a1510 msgr2=0x7fc7040a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:32.931 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:32.923+0000 7fc71b147700 1 --2- 172.21.15.59:0/1399201117 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fc7040a1510 0x7fc7040a39d0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fc708005950 tx=0x7fc7080058e0 comp rx=0 tx=0).stop 2023-12-11T17:45:32.931 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:32.923+0000 7fc71b147700 1 -- 172.21.15.59:0/1399201117 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc714100ef0 msgr2=0x7fc71419cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:32.931 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:32.923+0000 7fc71b147700 1 --2- 172.21.15.59:0/1399201117 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc714100ef0 0x7fc71419cb80 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fc70000d950 tx=0x7fc70000dc60 comp rx=0 tx=0).stop 2023-12-11T17:45:32.931 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:32.923+0000 7fc71b147700 1 -- 172.21.15.59:0/1399201117 shutdown_connections 2023-12-11T17:45:32.931 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:32.923+0000 7fc71b147700 1 --2- 172.21.15.59:0/1399201117 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fc7040a1510 0x7fc7040a39d0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:32.932 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:32.923+0000 7fc71b147700 1 --2- 172.21.15.59:0/1399201117 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc714106560 0x7fc7141967d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:32.932 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:32.923+0000 7fc71b147700 1 --2- 172.21.15.59:0/1399201117 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc714100530 0x7fc71419c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:32.932 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:32.923+0000 7fc71b147700 1 --2- 172.21.15.59:0/1399201117 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc714100ef0 0x7fc71419cb80 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:32.932 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:32.923+0000 7fc71b147700 1 -- 172.21.15.59:0/1399201117 >> 172.21.15.59:0/1399201117 conn(0x7fc714076570 msgr2=0x7fc7140ff230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:45:32.932 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:32.923+0000 7fc71b147700 1 -- 172.21.15.59:0/1399201117 shutdown_connections 2023-12-11T17:45:32.932 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:32.923+0000 7fc71b147700 1 -- 172.21.15.59:0/1399201117 wait complete. 2023-12-11T17:45:32.932 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2023-12-11T17:45:33.334 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":297,"stamp":"2023-12-11T17:45:32.131318+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655998,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":405,"num_read_kb":326,"num_write":342,"num_write_kb":3054,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1182329,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":278,"ondisk_log_size":278,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":336020,"kb_used_data":11756,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124605804,"statfs":{"total":1151940427776,"available":1151596343296,"internally_reserved":0,"allocated":12038144,"data_stored":7745912,"data_compressed":319479,"data_compressed_allocated":430080,"data_compressed_original":2359296,"omap_allocated":19142,"internal_metadata":331855162},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":70,"num_read_kb":35,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003564"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.880811+0000","last_change":"2023-12-11T17:44:20.624256+0000","last_active":"2023-12-11T17:45:14.880811+0000","last_peered":"2023-12-11T17:45:14.880811+0000","last_clean":"2023-12-11T17:45:14.880811+0000","last_became_active":"2023-12-11T17:44:20.623947+0000","last_became_peered":"2023-12-11T17:44:20.623947+0000","last_unstale":"2023-12-11T17:45:14.880811+0000","last_undegraded":"2023-12-11T17:45:14.880811+0000","last_fullsized":"2023-12-11T17:45:14.880811+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T18:12:43.669849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.874575+0000","last_change":"2023-12-11T17:44:20.620606+0000","last_active":"2023-12-11T17:45:14.874575+0000","last_peered":"2023-12-11T17:45:14.874575+0000","last_clean":"2023-12-11T17:45:14.874575+0000","last_became_active":"2023-12-11T17:44:20.620265+0000","last_became_peered":"2023-12-11T17:44:20.620265+0000","last_unstale":"2023-12-11T17:45:14.874575+0000","last_undegraded":"2023-12-11T17:45:14.874575+0000","last_fullsized":"2023-12-11T17:45:14.874575+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T19:53:35.349842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.874216+0000","last_change":"2023-12-11T17:44:20.620702+0000","last_active":"2023-12-11T17:45:14.874216+0000","last_peered":"2023-12-11T17:45:14.874216+0000","last_clean":"2023-12-11T17:45:14.874216+0000","last_became_active":"2023-12-11T17:44:20.620401+0000","last_became_peered":"2023-12-11T17:44:20.620401+0000","last_unstale":"2023-12-11T17:45:14.874216+0000","last_undegraded":"2023-12-11T17:45:14.874216+0000","last_fullsized":"2023-12-11T17:45:14.874216+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T21:14:21.156270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.876828+0000","last_change":"2023-12-11T17:44:20.614263+0000","last_active":"2023-12-11T17:45:14.876828+0000","last_peered":"2023-12-11T17:45:14.876828+0000","last_clean":"2023-12-11T17:45:14.876828+0000","last_became_active":"2023-12-11T17:44:20.613585+0000","last_became_peered":"2023-12-11T17:44:20.613585+0000","last_unstale":"2023-12-11T17:45:14.876828+0000","last_undegraded":"2023-12-11T17:45:14.876828+0000","last_fullsized":"2023-12-11T17:45:14.876828+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T23:03:26.514011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.873561+0000","last_change":"2023-12-11T17:44:20.615553+0000","last_active":"2023-12-11T17:45:14.873561+0000","last_peered":"2023-12-11T17:45:14.873561+0000","last_clean":"2023-12-11T17:45:14.873561+0000","last_became_active":"2023-12-11T17:44:20.615272+0000","last_became_peered":"2023-12-11T17:44:20.615272+0000","last_unstale":"2023-12-11T17:45:14.873561+0000","last_undegraded":"2023-12-11T17:45:14.873561+0000","last_fullsized":"2023-12-11T17:45:14.873561+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T20:37:02.178411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T17:45:15.882116+0000","last_change":"2023-12-11T17:45:15.882116+0000","last_active":"2023-12-11T17:45:15.882116+0000","last_peered":"2023-12-11T17:45:15.882116+0000","last_clean":"2023-12-11T17:45:15.882116+0000","last_became_active":"2023-12-11T17:45:15.881729+0000","last_became_peered":"2023-12-11T17:45:15.881729+0000","last_unstale":"2023-12-11T17:45:15.882116+0000","last_undegraded":"2023-12-11T17:45:15.882116+0000","last_fullsized":"2023-12-11T17:45:15.882116+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T05:39:09.190471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.873661+0000","last_change":"2023-12-11T17:44:20.623828+0000","last_active":"2023-12-11T17:45:14.873661+0000","last_peered":"2023-12-11T17:45:14.873661+0000","last_clean":"2023-12-11T17:45:14.873661+0000","last_became_active":"2023-12-11T17:44:20.623510+0000","last_became_peered":"2023-12-11T17:44:20.623510+0000","last_unstale":"2023-12-11T17:45:14.873661+0000","last_undegraded":"2023-12-11T17:45:14.873661+0000","last_fullsized":"2023-12-11T17:45:14.873661+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T22:04:32.298900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.880981+0000","last_change":"2023-12-11T17:44:20.620831+0000","last_active":"2023-12-11T17:45:14.880981+0000","last_peered":"2023-12-11T17:45:14.880981+0000","last_clean":"2023-12-11T17:45:14.880981+0000","last_became_active":"2023-12-11T17:44:20.620481+0000","last_became_peered":"2023-12-11T17:44:20.620481+0000","last_unstale":"2023-12-11T17:45:14.880981+0000","last_undegraded":"2023-12-11T17:45:14.880981+0000","last_fullsized":"2023-12-11T17:45:14.880981+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T22:14:28.838192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.880343+0000","last_change":"2023-12-11T17:44:20.619515+0000","last_active":"2023-12-11T17:45:14.880343+0000","last_peered":"2023-12-11T17:45:14.880343+0000","last_clean":"2023-12-11T17:45:14.880343+0000","last_became_active":"2023-12-11T17:44:20.619219+0000","last_became_peered":"2023-12-11T17:44:20.619219+0000","last_unstale":"2023-12-11T17:45:14.880343+0000","last_undegraded":"2023-12-11T17:45:14.880343+0000","last_fullsized":"2023-12-11T17:45:14.880343+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T23:08:55.112752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T17:45:15.885567+0000","last_change":"2023-12-11T17:45:15.885567+0000","last_active":"2023-12-11T17:45:15.885567+0000","last_peered":"2023-12-11T17:45:15.885567+0000","last_clean":"2023-12-11T17:45:15.885567+0000","last_became_active":"2023-12-11T17:45:15.885207+0000","last_became_peered":"2023-12-11T17:45:15.885207+0000","last_unstale":"2023-12-11T17:45:15.885567+0000","last_undegraded":"2023-12-11T17:45:15.885567+0000","last_fullsized":"2023-12-11T17:45:15.885567+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T01:24:25.763022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.874115+0000","last_change":"2023-12-11T17:44:20.621198+0000","last_active":"2023-12-11T17:45:14.874115+0000","last_peered":"2023-12-11T17:45:14.874115+0000","last_clean":"2023-12-11T17:45:14.874115+0000","last_became_active":"2023-12-11T17:44:20.620693+0000","last_became_peered":"2023-12-11T17:44:20.620693+0000","last_unstale":"2023-12-11T17:45:14.874115+0000","last_undegraded":"2023-12-11T17:45:14.874115+0000","last_fullsized":"2023-12-11T17:45:14.874115+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T22:10:23.396031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.876754+0000","last_change":"2023-12-11T17:44:20.614457+0000","last_active":"2023-12-11T17:45:14.876754+0000","last_peered":"2023-12-11T17:45:14.876754+0000","last_clean":"2023-12-11T17:45:14.876754+0000","last_became_active":"2023-12-11T17:44:20.613984+0000","last_became_peered":"2023-12-11T17:44:20.613984+0000","last_unstale":"2023-12-11T17:45:14.876754+0000","last_undegraded":"2023-12-11T17:45:14.876754+0000","last_fullsized":"2023-12-11T17:45:14.876754+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T00:50:01.101398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.880158+0000","last_change":"2023-12-11T17:44:20.619874+0000","last_active":"2023-12-11T17:45:14.880158+0000","last_peered":"2023-12-11T17:45:14.880158+0000","last_clean":"2023-12-11T17:45:14.880158+0000","last_became_active":"2023-12-11T17:44:20.618537+0000","last_became_peered":"2023-12-11T17:44:20.618537+0000","last_unstale":"2023-12-11T17:45:14.880158+0000","last_undegraded":"2023-12-11T17:45:14.880158+0000","last_fullsized":"2023-12-11T17:45:14.880158+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T18:42:43.587245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.873528+0000","last_change":"2023-12-11T17:44:20.616306+0000","last_active":"2023-12-11T17:45:14.873528+0000","last_peered":"2023-12-11T17:45:14.873528+0000","last_clean":"2023-12-11T17:45:14.873528+0000","last_became_active":"2023-12-11T17:44:20.615930+0000","last_became_peered":"2023-12-11T17:44:20.615930+0000","last_unstale":"2023-12-11T17:45:14.873528+0000","last_undegraded":"2023-12-11T17:45:14.873528+0000","last_fullsized":"2023-12-11T17:45:14.873528+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T05:40:24.495052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.873498+0000","last_change":"2023-12-11T17:44:20.624339+0000","last_active":"2023-12-11T17:45:14.873498+0000","last_peered":"2023-12-11T17:45:14.873498+0000","last_clean":"2023-12-11T17:45:14.873498+0000","last_became_active":"2023-12-11T17:44:20.624045+0000","last_became_peered":"2023-12-11T17:44:20.624045+0000","last_unstale":"2023-12-11T17:45:14.873498+0000","last_undegraded":"2023-12-11T17:45:14.873498+0000","last_fullsized":"2023-12-11T17:45:14.873498+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T01:32:30.792940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.879866+0000","last_change":"2023-12-11T17:44:20.620750+0000","last_active":"2023-12-11T17:45:14.879866+0000","last_peered":"2023-12-11T17:45:14.879866+0000","last_clean":"2023-12-11T17:45:14.879866+0000","last_became_active":"2023-12-11T17:44:20.620464+0000","last_became_peered":"2023-12-11T17:44:20.620464+0000","last_unstale":"2023-12-11T17:45:14.879866+0000","last_undegraded":"2023-12-11T17:45:14.879866+0000","last_fullsized":"2023-12-11T17:45:14.879866+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T05:43:33.680889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.880834+0000","last_change":"2023-12-11T17:44:20.624391+0000","last_active":"2023-12-11T17:45:14.880834+0000","last_peered":"2023-12-11T17:45:14.880834+0000","last_clean":"2023-12-11T17:45:14.880834+0000","last_became_active":"2023-12-11T17:44:20.624217+0000","last_became_peered":"2023-12-11T17:44:20.624217+0000","last_unstale":"2023-12-11T17:45:14.880834+0000","last_undegraded":"2023-12-11T17:45:14.880834+0000","last_fullsized":"2023-12-11T17:45:14.880834+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T01:06:41.660726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.877604+0000","last_change":"2023-12-11T17:44:20.612429+0000","last_active":"2023-12-11T17:45:14.877604+0000","last_peered":"2023-12-11T17:45:14.877604+0000","last_clean":"2023-12-11T17:45:14.877604+0000","last_became_active":"2023-12-11T17:44:20.612082+0000","last_became_peered":"2023-12-11T17:44:20.612082+0000","last_unstale":"2023-12-11T17:45:14.877604+0000","last_undegraded":"2023-12-11T17:45:14.877604+0000","last_fullsized":"2023-12-11T17:45:14.877604+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T02:34:42.364876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.877789+0000","last_change":"2023-12-11T17:44:20.614907+0000","last_active":"2023-12-11T17:45:14.877789+0000","last_peered":"2023-12-11T17:45:14.877789+0000","last_clean":"2023-12-11T17:45:14.877789+0000","last_became_active":"2023-12-11T17:44:20.614611+0000","last_became_peered":"2023-12-11T17:44:20.614611+0000","last_unstale":"2023-12-11T17:45:14.877789+0000","last_undegraded":"2023-12-11T17:45:14.877789+0000","last_fullsized":"2023-12-11T17:45:14.877789+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T05:29:50.095253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.877113+0000","last_change":"2023-12-11T17:44:24.690974+0000","last_active":"2023-12-11T17:45:14.877113+0000","last_peered":"2023-12-11T17:45:14.877113+0000","last_clean":"2023-12-11T17:45:14.877113+0000","last_became_active":"2023-12-11T17:44:24.690579+0000","last_became_peered":"2023-12-11T17:44:24.690579+0000","last_unstale":"2023-12-11T17:45:14.877113+0000","last_undegraded":"2023-12-11T17:45:14.877113+0000","last_fullsized":"2023-12-11T17:45:14.877113+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_clean_scrub_stamp":"2023-12-11T17:44:23.663161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:31:59.830486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'9","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T17:45:15.895326+0000","last_change":"2023-12-11T17:45:15.895326+0000","last_active":"2023-12-11T17:45:15.895326+0000","last_peered":"2023-12-11T17:45:15.895326+0000","last_clean":"2023-12-11T17:45:15.895326+0000","last_became_active":"2023-12-11T17:45:15.884451+0000","last_became_peered":"2023-12-11T17:45:15.884451+0000","last_unstale":"2023-12-11T17:45:15.895326+0000","last_undegraded":"2023-12-11T17:45:15.895326+0000","last_fullsized":"2023-12-11T17:45:15.895326+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_clean_scrub_stamp":"2023-12-11T17:44:18.636443+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T04:07:28.420942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,8],"acting":[9,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.881499+0000","last_change":"2023-12-11T17:44:20.621928+0000","last_active":"2023-12-11T17:45:14.881499+0000","last_peered":"2023-12-11T17:45:14.881499+0000","last_clean":"2023-12-11T17:45:14.881499+0000","last_became_active":"2023-12-11T17:44:20.621575+0000","last_became_peered":"2023-12-11T17:44:20.621575+0000","last_unstale":"2023-12-11T17:45:14.881499+0000","last_undegraded":"2023-12-11T17:45:14.881499+0000","last_fullsized":"2023-12-11T17:45:14.881499+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T17:57:44.860200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.876251+0000","last_change":"2023-12-11T17:44:24.704170+0000","last_active":"2023-12-11T17:45:14.876251+0000","last_peered":"2023-12-11T17:45:14.876251+0000","last_clean":"2023-12-11T17:45:14.876251+0000","last_became_active":"2023-12-11T17:44:24.703800+0000","last_became_peered":"2023-12-11T17:44:24.703800+0000","last_unstale":"2023-12-11T17:45:14.876251+0000","last_undegraded":"2023-12-11T17:45:14.876251+0000","last_fullsized":"2023-12-11T17:45:14.876251+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_clean_scrub_stamp":"2023-12-11T17:44:23.663161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T01:26:29.578910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.881742+0000","last_change":"2023-12-11T17:44:19.659660+0000","last_active":"2023-12-11T17:45:14.881742+0000","last_peered":"2023-12-11T17:45:14.881742+0000","last_clean":"2023-12-11T17:45:14.881742+0000","last_became_active":"2023-12-11T17:44:19.659283+0000","last_became_peered":"2023-12-11T17:44:19.659283+0000","last_unstale":"2023-12-11T17:45:14.881742+0000","last_undegraded":"2023-12-11T17:45:14.881742+0000","last_fullsized":"2023-12-11T17:45:14.881742+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_clean_scrub_stamp":"2023-12-11T17:44:18.636443+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T22:29:43.427794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.875448+0000","last_change":"2023-12-11T17:44:20.624962+0000","last_active":"2023-12-11T17:45:14.875448+0000","last_peered":"2023-12-11T17:45:14.875448+0000","last_clean":"2023-12-11T17:45:14.875448+0000","last_became_active":"2023-12-11T17:44:20.624640+0000","last_became_peered":"2023-12-11T17:44:20.624640+0000","last_unstale":"2023-12-11T17:45:14.875448+0000","last_undegraded":"2023-12-11T17:45:14.875448+0000","last_fullsized":"2023-12-11T17:45:14.875448+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T23:37:43.252062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.876519+0000","last_change":"2023-12-11T17:44:24.698866+0000","last_active":"2023-12-11T17:45:14.876519+0000","last_peered":"2023-12-11T17:45:14.876519+0000","last_clean":"2023-12-11T17:45:14.876519+0000","last_became_active":"2023-12-11T17:44:24.698481+0000","last_became_peered":"2023-12-11T17:44:24.698481+0000","last_unstale":"2023-12-11T17:45:14.876519+0000","last_undegraded":"2023-12-11T17:45:14.876519+0000","last_fullsized":"2023-12-11T17:45:14.876519+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_clean_scrub_stamp":"2023-12-11T17:44:23.663161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T20:50:19.571616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.873946+0000","last_change":"2023-12-11T17:44:19.662637+0000","last_active":"2023-12-11T17:45:14.873946+0000","last_peered":"2023-12-11T17:45:14.873946+0000","last_clean":"2023-12-11T17:45:14.873946+0000","last_became_active":"2023-12-11T17:44:19.662355+0000","last_became_peered":"2023-12-11T17:44:19.662355+0000","last_unstale":"2023-12-11T17:45:14.873946+0000","last_undegraded":"2023-12-11T17:45:14.873946+0000","last_fullsized":"2023-12-11T17:45:14.873946+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_clean_scrub_stamp":"2023-12-11T17:44:18.636443+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T17:56:37.436037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.879884+0000","last_change":"2023-12-11T17:44:20.615119+0000","last_active":"2023-12-11T17:45:14.879884+0000","last_peered":"2023-12-11T17:45:14.879884+0000","last_clean":"2023-12-11T17:45:14.879884+0000","last_became_active":"2023-12-11T17:44:20.614775+0000","last_became_peered":"2023-12-11T17:44:20.614775+0000","last_unstale":"2023-12-11T17:45:14.879884+0000","last_undegraded":"2023-12-11T17:45:14.879884+0000","last_fullsized":"2023-12-11T17:45:14.879884+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T20:45:47.162890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.876324+0000","last_change":"2023-12-11T17:44:24.707718+0000","last_active":"2023-12-11T17:45:14.876324+0000","last_peered":"2023-12-11T17:45:14.876324+0000","last_clean":"2023-12-11T17:45:14.876324+0000","last_became_active":"2023-12-11T17:44:24.707404+0000","last_became_peered":"2023-12-11T17:44:24.707404+0000","last_unstale":"2023-12-11T17:45:14.876324+0000","last_undegraded":"2023-12-11T17:45:14.876324+0000","last_fullsized":"2023-12-11T17:45:14.876324+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_clean_scrub_stamp":"2023-12-11T17:44:23.663161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T22:53:53.918649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"81'2","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T17:45:15.891766+0000","last_change":"2023-12-11T17:45:15.891299+0000","last_active":"2023-12-11T17:45:15.891766+0000","last_peered":"2023-12-11T17:45:15.891766+0000","last_clean":"2023-12-11T17:45:15.891766+0000","last_became_active":"2023-12-11T17:45:15.884838+0000","last_became_peered":"2023-12-11T17:45:15.884838+0000","last_unstale":"2023-12-11T17:45:15.891766+0000","last_undegraded":"2023-12-11T17:45:15.891766+0000","last_fullsized":"2023-12-11T17:45:15.891766+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_clean_scrub_stamp":"2023-12-11T17:44:18.636443+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T19:05:15.500425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.877473+0000","last_change":"2023-12-11T17:44:20.615894+0000","last_active":"2023-12-11T17:45:14.877473+0000","last_peered":"2023-12-11T17:45:14.877473+0000","last_clean":"2023-12-11T17:45:14.877473+0000","last_became_active":"2023-12-11T17:44:20.615582+0000","last_became_peered":"2023-12-11T17:44:20.615582+0000","last_unstale":"2023-12-11T17:45:14.877473+0000","last_undegraded":"2023-12-11T17:45:14.877473+0000","last_fullsized":"2023-12-11T17:45:14.877473+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T19:04:50.725088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.877639+0000","last_change":"2023-12-11T17:44:24.691459+0000","last_active":"2023-12-11T17:45:14.877639+0000","last_peered":"2023-12-11T17:45:14.877639+0000","last_clean":"2023-12-11T17:45:14.877639+0000","last_became_active":"2023-12-11T17:44:24.690301+0000","last_became_peered":"2023-12-11T17:44:24.690301+0000","last_unstale":"2023-12-11T17:45:14.877639+0000","last_undegraded":"2023-12-11T17:45:14.877639+0000","last_fullsized":"2023-12-11T17:45:14.877639+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_clean_scrub_stamp":"2023-12-11T17:44:23.663161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T17:58:04.685785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"86'6","reported_seq":83,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T17:45:27.906349+0000","last_change":"2023-12-11T17:45:15.894177+0000","last_active":"2023-12-11T17:45:27.906349+0000","last_peered":"2023-12-11T17:45:27.906349+0000","last_clean":"2023-12-11T17:45:27.906349+0000","last_became_active":"2023-12-11T17:45:15.887160+0000","last_became_peered":"2023-12-11T17:45:15.887160+0000","last_unstale":"2023-12-11T17:45:27.906349+0000","last_undegraded":"2023-12-11T17:45:27.906349+0000","last_fullsized":"2023-12-11T17:45:27.906349+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_clean_scrub_stamp":"2023-12-11T17:44:18.636443+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T19:45:46.073604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1011,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":37,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1011,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.873547+0000","last_change":"2023-12-11T17:44:20.608280+0000","last_active":"2023-12-11T17:45:14.873547+0000","last_peered":"2023-12-11T17:45:14.873547+0000","last_clean":"2023-12-11T17:45:14.873547+0000","last_became_active":"2023-12-11T17:44:20.607980+0000","last_became_peered":"2023-12-11T17:44:20.607980+0000","last_unstale":"2023-12-11T17:45:14.873547+0000","last_undegraded":"2023-12-11T17:45:14.873547+0000","last_fullsized":"2023-12-11T17:45:14.873547+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T04:45:56.108916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.877919+0000","last_change":"2023-12-11T17:44:24.703629+0000","last_active":"2023-12-11T17:45:14.877919+0000","last_peered":"2023-12-11T17:45:14.877919+0000","last_clean":"2023-12-11T17:45:14.877919+0000","last_became_active":"2023-12-11T17:44:24.703219+0000","last_became_peered":"2023-12-11T17:44:24.703219+0000","last_unstale":"2023-12-11T17:45:14.877919+0000","last_undegraded":"2023-12-11T17:45:14.877919+0000","last_fullsized":"2023-12-11T17:45:14.877919+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_clean_scrub_stamp":"2023-12-11T17:44:23.663161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T19:57:44.998039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.879362+0000","last_change":"2023-12-11T17:44:19.659311+0000","last_active":"2023-12-11T17:45:14.879362+0000","last_peered":"2023-12-11T17:45:14.879362+0000","last_clean":"2023-12-11T17:45:14.879362+0000","last_became_active":"2023-12-11T17:44:19.658994+0000","last_became_peered":"2023-12-11T17:44:19.658994+0000","last_unstale":"2023-12-11T17:45:14.879362+0000","last_undegraded":"2023-12-11T17:45:14.879362+0000","last_fullsized":"2023-12-11T17:45:14.879362+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_clean_scrub_stamp":"2023-12-11T17:44:18.636443+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T17:46:35.701179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.878600+0000","last_change":"2023-12-11T17:44:20.615684+0000","last_active":"2023-12-11T17:45:14.878600+0000","last_peered":"2023-12-11T17:45:14.878600+0000","last_clean":"2023-12-11T17:45:14.878600+0000","last_became_active":"2023-12-11T17:44:20.615373+0000","last_became_peered":"2023-12-11T17:44:20.615373+0000","last_unstale":"2023-12-11T17:45:14.878600+0000","last_undegraded":"2023-12-11T17:45:14.878600+0000","last_fullsized":"2023-12-11T17:45:14.878600+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T19:45:59.980253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.880529+0000","last_change":"2023-12-11T17:44:24.699085+0000","last_active":"2023-12-11T17:45:14.880529+0000","last_peered":"2023-12-11T17:45:14.880529+0000","last_clean":"2023-12-11T17:45:14.880529+0000","last_became_active":"2023-12-11T17:44:24.698717+0000","last_became_peered":"2023-12-11T17:44:24.698717+0000","last_unstale":"2023-12-11T17:45:14.880529+0000","last_undegraded":"2023-12-11T17:45:14.880529+0000","last_fullsized":"2023-12-11T17:45:14.880529+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_clean_scrub_stamp":"2023-12-11T17:44:23.663161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T05:00:52.341938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.880487+0000","last_change":"2023-12-11T17:44:19.660392+0000","last_active":"2023-12-11T17:45:14.880487+0000","last_peered":"2023-12-11T17:45:14.880487+0000","last_clean":"2023-12-11T17:45:14.880487+0000","last_became_active":"2023-12-11T17:44:19.660099+0000","last_became_peered":"2023-12-11T17:44:19.660099+0000","last_unstale":"2023-12-11T17:45:14.880487+0000","last_undegraded":"2023-12-11T17:45:14.880487+0000","last_fullsized":"2023-12-11T17:45:14.880487+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_clean_scrub_stamp":"2023-12-11T17:44:18.636443+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T00:38:37.279230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.881168+0000","last_change":"2023-12-11T17:44:20.615629+0000","last_active":"2023-12-11T17:45:14.881168+0000","last_peered":"2023-12-11T17:45:14.881168+0000","last_clean":"2023-12-11T17:45:14.881168+0000","last_became_active":"2023-12-11T17:44:20.615193+0000","last_became_peered":"2023-12-11T17:44:20.615193+0000","last_unstale":"2023-12-11T17:45:14.881168+0000","last_undegraded":"2023-12-11T17:45:14.881168+0000","last_fullsized":"2023-12-11T17:45:14.881168+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:09:56.237298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.874251+0000","last_change":"2023-12-11T17:44:24.705845+0000","last_active":"2023-12-11T17:45:14.874251+0000","last_peered":"2023-12-11T17:45:14.874251+0000","last_clean":"2023-12-11T17:45:14.874251+0000","last_became_active":"2023-12-11T17:44:24.705434+0000","last_became_peered":"2023-12-11T17:44:24.705434+0000","last_unstale":"2023-12-11T17:45:14.874251+0000","last_undegraded":"2023-12-11T17:45:14.874251+0000","last_fullsized":"2023-12-11T17:45:14.874251+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_clean_scrub_stamp":"2023-12-11T17:44:23.663161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T02:37:25.663753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.877124+0000","last_change":"2023-12-11T17:44:19.661317+0000","last_active":"2023-12-11T17:45:14.877124+0000","last_peered":"2023-12-11T17:45:14.877124+0000","last_clean":"2023-12-11T17:45:14.877124+0000","last_became_active":"2023-12-11T17:44:19.660993+0000","last_became_peered":"2023-12-11T17:44:19.660993+0000","last_unstale":"2023-12-11T17:45:14.877124+0000","last_undegraded":"2023-12-11T17:45:14.877124+0000","last_fullsized":"2023-12-11T17:45:14.877124+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_clean_scrub_stamp":"2023-12-11T17:44:18.636443+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T23:00:07.345964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.876765+0000","last_change":"2023-12-11T17:44:20.623681+0000","last_active":"2023-12-11T17:45:14.876765+0000","last_peered":"2023-12-11T17:45:14.876765+0000","last_clean":"2023-12-11T17:45:14.876765+0000","last_became_active":"2023-12-11T17:44:20.623288+0000","last_became_peered":"2023-12-11T17:44:20.623288+0000","last_unstale":"2023-12-11T17:45:14.876765+0000","last_undegraded":"2023-12-11T17:45:14.876765+0000","last_fullsized":"2023-12-11T17:45:14.876765+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T23:47:55.348480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.881054+0000","last_change":"2023-12-11T17:44:24.687613+0000","last_active":"2023-12-11T17:45:14.881054+0000","last_peered":"2023-12-11T17:45:14.881054+0000","last_clean":"2023-12-11T17:45:14.881054+0000","last_became_active":"2023-12-11T17:44:24.687240+0000","last_became_peered":"2023-12-11T17:44:24.687240+0000","last_unstale":"2023-12-11T17:45:14.881054+0000","last_undegraded":"2023-12-11T17:45:14.881054+0000","last_fullsized":"2023-12-11T17:45:14.881054+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_clean_scrub_stamp":"2023-12-11T17:44:23.663161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T20:54:25.409631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"86'3","reported_seq":62,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T17:45:25.902251+0000","last_change":"2023-12-11T17:44:19.653571+0000","last_active":"2023-12-11T17:45:25.902251+0000","last_peered":"2023-12-11T17:45:25.902251+0000","last_clean":"2023-12-11T17:45:25.902251+0000","last_became_active":"2023-12-11T17:44:19.653035+0000","last_became_peered":"2023-12-11T17:44:19.653035+0000","last_unstale":"2023-12-11T17:45:25.902251+0000","last_undegraded":"2023-12-11T17:45:25.902251+0000","last_fullsized":"2023-12-11T17:45:25.902251+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_clean_scrub_stamp":"2023-12-11T17:44:18.636443+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T02:29:38.310132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":40,"num_read_kb":40,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.880174+0000","last_change":"2023-12-11T17:44:20.623777+0000","last_active":"2023-12-11T17:45:14.880174+0000","last_peered":"2023-12-11T17:45:14.880174+0000","last_clean":"2023-12-11T17:45:14.880174+0000","last_became_active":"2023-12-11T17:44:20.623449+0000","last_became_peered":"2023-12-11T17:44:20.623449+0000","last_unstale":"2023-12-11T17:45:14.880174+0000","last_undegraded":"2023-12-11T17:45:14.880174+0000","last_fullsized":"2023-12-11T17:45:14.880174+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T05:23:03.493677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.876453+0000","last_change":"2023-12-11T17:44:24.693893+0000","last_active":"2023-12-11T17:45:14.876453+0000","last_peered":"2023-12-11T17:45:14.876453+0000","last_clean":"2023-12-11T17:45:14.876453+0000","last_became_active":"2023-12-11T17:44:24.693441+0000","last_became_peered":"2023-12-11T17:44:24.693441+0000","last_unstale":"2023-12-11T17:45:14.876453+0000","last_undegraded":"2023-12-11T17:45:14.876453+0000","last_fullsized":"2023-12-11T17:45:14.876453+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_clean_scrub_stamp":"2023-12-11T17:44:23.663161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T23:25:06.457176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.874018+0000","last_change":"2023-12-11T17:44:19.662988+0000","last_active":"2023-12-11T17:45:14.874018+0000","last_peered":"2023-12-11T17:45:14.874018+0000","last_clean":"2023-12-11T17:45:14.874018+0000","last_became_active":"2023-12-11T17:44:19.662660+0000","last_became_peered":"2023-12-11T17:44:19.662660+0000","last_unstale":"2023-12-11T17:45:14.874018+0000","last_undegraded":"2023-12-11T17:45:14.874018+0000","last_fullsized":"2023-12-11T17:45:14.874018+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_clean_scrub_stamp":"2023-12-11T17:44:18.636443+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T19:08:50.703161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T17:45:15.887273+0000","last_change":"2023-12-11T17:45:15.887273+0000","last_active":"2023-12-11T17:45:15.887273+0000","last_peered":"2023-12-11T17:45:15.887273+0000","last_clean":"2023-12-11T17:45:15.887273+0000","last_became_active":"2023-12-11T17:45:15.886905+0000","last_became_peered":"2023-12-11T17:45:15.886905+0000","last_unstale":"2023-12-11T17:45:15.887273+0000","last_undegraded":"2023-12-11T17:45:15.887273+0000","last_fullsized":"2023-12-11T17:45:15.887273+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T01:56:43.076856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,5],"acting":[0,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.876666+0000","last_change":"2023-12-11T17:44:24.698715+0000","last_active":"2023-12-11T17:45:14.876666+0000","last_peered":"2023-12-11T17:45:14.876666+0000","last_clean":"2023-12-11T17:45:14.876666+0000","last_became_active":"2023-12-11T17:44:24.698359+0000","last_became_peered":"2023-12-11T17:44:24.698359+0000","last_unstale":"2023-12-11T17:45:14.876666+0000","last_undegraded":"2023-12-11T17:45:14.876666+0000","last_fullsized":"2023-12-11T17:45:14.876666+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_clean_scrub_stamp":"2023-12-11T17:44:23.663161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T01:07:23.870922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.877378+0000","last_change":"2023-12-11T17:44:19.655914+0000","last_active":"2023-12-11T17:45:14.877378+0000","last_peered":"2023-12-11T17:45:14.877378+0000","last_clean":"2023-12-11T17:45:14.877378+0000","last_became_active":"2023-12-11T17:44:19.655593+0000","last_became_peered":"2023-12-11T17:44:19.655593+0000","last_unstale":"2023-12-11T17:45:14.877378+0000","last_undegraded":"2023-12-11T17:45:14.877378+0000","last_fullsized":"2023-12-11T17:45:14.877378+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_clean_scrub_stamp":"2023-12-11T17:44:18.636443+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T01:54:50.246100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.880594+0000","last_change":"2023-12-11T17:44:20.618469+0000","last_active":"2023-12-11T17:45:14.880594+0000","last_peered":"2023-12-11T17:45:14.880594+0000","last_clean":"2023-12-11T17:45:14.880594+0000","last_became_active":"2023-12-11T17:44:20.618186+0000","last_became_peered":"2023-12-11T17:44:20.618186+0000","last_unstale":"2023-12-11T17:45:14.880594+0000","last_undegraded":"2023-12-11T17:45:14.880594+0000","last_fullsized":"2023-12-11T17:45:14.880594+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T20:53:22.405100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.876665+0000","last_change":"2023-12-11T17:44:24.697048+0000","last_active":"2023-12-11T17:45:14.876665+0000","last_peered":"2023-12-11T17:45:14.876665+0000","last_clean":"2023-12-11T17:45:14.876665+0000","last_became_active":"2023-12-11T17:44:24.696660+0000","last_became_peered":"2023-12-11T17:44:24.696660+0000","last_unstale":"2023-12-11T17:45:14.876665+0000","last_undegraded":"2023-12-11T17:45:14.876665+0000","last_fullsized":"2023-12-11T17:45:14.876665+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_clean_scrub_stamp":"2023-12-11T17:44:23.663161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T21:50:51.018386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"87'33","reported_seq":140,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T17:45:26.903700+0000","last_change":"2023-12-11T17:44:19.653528+0000","last_active":"2023-12-11T17:45:26.903700+0000","last_peered":"2023-12-11T17:45:26.903700+0000","last_clean":"2023-12-11T17:45:26.903700+0000","last_became_active":"2023-12-11T17:44:19.653215+0000","last_became_peered":"2023-12-11T17:44:19.653215+0000","last_unstale":"2023-12-11T17:45:26.903700+0000","last_undegraded":"2023-12-11T17:45:26.903700+0000","last_fullsized":"2023-12-11T17:45:26.903700+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_clean_scrub_stamp":"2023-12-11T17:44:18.636443+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T21:19:12.024008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60996,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":88,"num_read_kb":103,"num_write":34,"num_write_kb":75,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.873815+0000","last_change":"2023-12-11T17:44:20.614085+0000","last_active":"2023-12-11T17:45:14.873815+0000","last_peered":"2023-12-11T17:45:14.873815+0000","last_clean":"2023-12-11T17:45:14.873815+0000","last_became_active":"2023-12-11T17:44:20.613768+0000","last_became_peered":"2023-12-11T17:44:20.613768+0000","last_unstale":"2023-12-11T17:45:14.873815+0000","last_undegraded":"2023-12-11T17:45:14.873815+0000","last_fullsized":"2023-12-11T17:45:14.873815+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T23:42:52.133207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.879671+0000","last_change":"2023-12-11T17:44:24.702039+0000","last_active":"2023-12-11T17:45:14.879671+0000","last_peered":"2023-12-11T17:45:14.879671+0000","last_clean":"2023-12-11T17:45:14.879671+0000","last_became_active":"2023-12-11T17:44:24.701539+0000","last_became_peered":"2023-12-11T17:44:24.701539+0000","last_unstale":"2023-12-11T17:45:14.879671+0000","last_undegraded":"2023-12-11T17:45:14.879671+0000","last_fullsized":"2023-12-11T17:45:14.879671+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_clean_scrub_stamp":"2023-12-11T17:44:23.663161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T02:23:55.393595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.878779+0000","last_change":"2023-12-11T17:44:19.662118+0000","last_active":"2023-12-11T17:45:14.878779+0000","last_peered":"2023-12-11T17:45:14.878779+0000","last_clean":"2023-12-11T17:45:14.878779+0000","last_became_active":"2023-12-11T17:44:19.661796+0000","last_became_peered":"2023-12-11T17:44:19.661796+0000","last_unstale":"2023-12-11T17:45:14.878779+0000","last_undegraded":"2023-12-11T17:45:14.878779+0000","last_fullsized":"2023-12-11T17:45:14.878779+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_clean_scrub_stamp":"2023-12-11T17:44:18.636443+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T21:08:02.842544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.879934+0000","last_change":"2023-12-11T17:44:20.615403+0000","last_active":"2023-12-11T17:45:14.879934+0000","last_peered":"2023-12-11T17:45:14.879934+0000","last_clean":"2023-12-11T17:45:14.879934+0000","last_became_active":"2023-12-11T17:44:20.615106+0000","last_became_peered":"2023-12-11T17:44:20.615106+0000","last_unstale":"2023-12-11T17:45:14.879934+0000","last_undegraded":"2023-12-11T17:45:14.879934+0000","last_fullsized":"2023-12-11T17:45:14.879934+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T02:10:24.047720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.880650+0000","last_change":"2023-12-11T17:44:24.699737+0000","last_active":"2023-12-11T17:45:14.880650+0000","last_peered":"2023-12-11T17:45:14.880650+0000","last_clean":"2023-12-11T17:45:14.880650+0000","last_became_active":"2023-12-11T17:44:24.699345+0000","last_became_peered":"2023-12-11T17:44:24.699345+0000","last_unstale":"2023-12-11T17:45:14.880650+0000","last_undegraded":"2023-12-11T17:45:14.880650+0000","last_fullsized":"2023-12-11T17:45:14.880650+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_clean_scrub_stamp":"2023-12-11T17:44:23.663161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T01:30:25.471636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.878104+0000","last_change":"2023-12-11T17:44:19.661325+0000","last_active":"2023-12-11T17:45:14.878104+0000","last_peered":"2023-12-11T17:45:14.878104+0000","last_clean":"2023-12-11T17:45:14.878104+0000","last_became_active":"2023-12-11T17:44:19.661012+0000","last_became_peered":"2023-12-11T17:44:19.661012+0000","last_unstale":"2023-12-11T17:45:14.878104+0000","last_undegraded":"2023-12-11T17:45:14.878104+0000","last_fullsized":"2023-12-11T17:45:14.878104+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_clean_scrub_stamp":"2023-12-11T17:44:18.636443+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T19:33:08.258019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.873803+0000","last_change":"2023-12-11T17:44:20.614795+0000","last_active":"2023-12-11T17:45:14.873803+0000","last_peered":"2023-12-11T17:45:14.873803+0000","last_clean":"2023-12-11T17:45:14.873803+0000","last_became_active":"2023-12-11T17:44:20.613826+0000","last_became_peered":"2023-12-11T17:44:20.613826+0000","last_unstale":"2023-12-11T17:45:14.873803+0000","last_undegraded":"2023-12-11T17:45:14.873803+0000","last_fullsized":"2023-12-11T17:45:14.873803+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T21:32:19.029040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.877354+0000","last_change":"2023-12-11T17:44:24.701569+0000","last_active":"2023-12-11T17:45:14.877354+0000","last_peered":"2023-12-11T17:45:14.877354+0000","last_clean":"2023-12-11T17:45:14.877354+0000","last_became_active":"2023-12-11T17:44:24.701173+0000","last_became_peered":"2023-12-11T17:44:24.701173+0000","last_unstale":"2023-12-11T17:45:14.877354+0000","last_undegraded":"2023-12-11T17:45:14.877354+0000","last_fullsized":"2023-12-11T17:45:14.877354+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_clean_scrub_stamp":"2023-12-11T17:44:23.663161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T23:02:45.843827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"86'4","reported_seq":69,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T17:45:28.845717+0000","last_change":"2023-12-11T17:44:19.659643+0000","last_active":"2023-12-11T17:45:28.845717+0000","last_peered":"2023-12-11T17:45:28.845717+0000","last_clean":"2023-12-11T17:45:28.845717+0000","last_became_active":"2023-12-11T17:44:19.659320+0000","last_became_peered":"2023-12-11T17:44:19.659320+0000","last_unstale":"2023-12-11T17:45:28.845717+0000","last_undegraded":"2023-12-11T17:45:28.845717+0000","last_fullsized":"2023-12-11T17:45:28.845717+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_clean_scrub_stamp":"2023-12-11T17:44:18.636443+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T20:02:03.633439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":977,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":47,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.873555+0000","last_change":"2023-12-11T17:44:20.619445+0000","last_active":"2023-12-11T17:45:14.873555+0000","last_peered":"2023-12-11T17:45:14.873555+0000","last_clean":"2023-12-11T17:45:14.873555+0000","last_became_active":"2023-12-11T17:44:20.619094+0000","last_became_peered":"2023-12-11T17:44:20.619094+0000","last_unstale":"2023-12-11T17:45:14.873555+0000","last_undegraded":"2023-12-11T17:45:14.873555+0000","last_fullsized":"2023-12-11T17:45:14.873555+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T04:03:21.917498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.880483+0000","last_change":"2023-12-11T17:44:24.701821+0000","last_active":"2023-12-11T17:45:14.880483+0000","last_peered":"2023-12-11T17:45:14.880483+0000","last_clean":"2023-12-11T17:45:14.880483+0000","last_became_active":"2023-12-11T17:44:24.700590+0000","last_became_peered":"2023-12-11T17:44:24.700590+0000","last_unstale":"2023-12-11T17:45:14.880483+0000","last_undegraded":"2023-12-11T17:45:14.880483+0000","last_fullsized":"2023-12-11T17:45:14.880483+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_clean_scrub_stamp":"2023-12-11T17:44:23.663161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T19:58:28.422648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.873469+0000","last_change":"2023-12-11T17:44:19.662540+0000","last_active":"2023-12-11T17:45:14.873469+0000","last_peered":"2023-12-11T17:45:14.873469+0000","last_clean":"2023-12-11T17:45:14.873469+0000","last_became_active":"2023-12-11T17:44:19.662231+0000","last_became_peered":"2023-12-11T17:44:19.662231+0000","last_unstale":"2023-12-11T17:45:14.873469+0000","last_undegraded":"2023-12-11T17:45:14.873469+0000","last_fullsized":"2023-12-11T17:45:14.873469+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_clean_scrub_stamp":"2023-12-11T17:44:18.636443+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T21:56:45.241138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.878470+0000","last_change":"2023-12-11T17:44:20.620457+0000","last_active":"2023-12-11T17:45:14.878470+0000","last_peered":"2023-12-11T17:45:14.878470+0000","last_clean":"2023-12-11T17:45:14.878470+0000","last_became_active":"2023-12-11T17:44:20.619949+0000","last_became_peered":"2023-12-11T17:44:20.619949+0000","last_unstale":"2023-12-11T17:45:14.878470+0000","last_undegraded":"2023-12-11T17:45:14.878470+0000","last_fullsized":"2023-12-11T17:45:14.878470+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T22:59:14.328817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.878371+0000","last_change":"2023-12-11T17:44:24.691573+0000","last_active":"2023-12-11T17:45:14.878371+0000","last_peered":"2023-12-11T17:45:14.878371+0000","last_clean":"2023-12-11T17:45:14.878371+0000","last_became_active":"2023-12-11T17:44:24.691182+0000","last_became_peered":"2023-12-11T17:44:24.691182+0000","last_unstale":"2023-12-11T17:45:14.878371+0000","last_undegraded":"2023-12-11T17:45:14.878371+0000","last_fullsized":"2023-12-11T17:45:14.878371+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_clean_scrub_stamp":"2023-12-11T17:44:23.663161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T02:18:36.582108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.874171+0000","last_change":"2023-12-11T17:44:19.653470+0000","last_active":"2023-12-11T17:45:14.874171+0000","last_peered":"2023-12-11T17:45:14.874171+0000","last_clean":"2023-12-11T17:45:14.874171+0000","last_became_active":"2023-12-11T17:44:19.653148+0000","last_became_peered":"2023-12-11T17:44:19.653148+0000","last_unstale":"2023-12-11T17:45:14.874171+0000","last_undegraded":"2023-12-11T17:45:14.874171+0000","last_fullsized":"2023-12-11T17:45:14.874171+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_clean_scrub_stamp":"2023-12-11T17:44:18.636443+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T05:26:12.017814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.880579+0000","last_change":"2023-12-11T17:44:20.617019+0000","last_active":"2023-12-11T17:45:14.880579+0000","last_peered":"2023-12-11T17:45:14.880579+0000","last_clean":"2023-12-11T17:45:14.880579+0000","last_became_active":"2023-12-11T17:44:20.616770+0000","last_became_peered":"2023-12-11T17:44:20.616770+0000","last_unstale":"2023-12-11T17:45:14.880579+0000","last_undegraded":"2023-12-11T17:45:14.880579+0000","last_fullsized":"2023-12-11T17:45:14.880579+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T18:32:15.050179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.873531+0000","last_change":"2023-12-11T17:44:24.694843+0000","last_active":"2023-12-11T17:45:14.873531+0000","last_peered":"2023-12-11T17:45:14.873531+0000","last_clean":"2023-12-11T17:45:14.873531+0000","last_became_active":"2023-12-11T17:44:24.694423+0000","last_became_peered":"2023-12-11T17:44:24.694423+0000","last_unstale":"2023-12-11T17:45:14.873531+0000","last_undegraded":"2023-12-11T17:45:14.873531+0000","last_fullsized":"2023-12-11T17:45:14.873531+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_clean_scrub_stamp":"2023-12-11T17:44:23.663161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T18:42:35.281782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.877196+0000","last_change":"2023-12-11T17:44:19.655676+0000","last_active":"2023-12-11T17:45:14.877196+0000","last_peered":"2023-12-11T17:45:14.877196+0000","last_clean":"2023-12-11T17:45:14.877196+0000","last_became_active":"2023-12-11T17:44:19.655331+0000","last_became_peered":"2023-12-11T17:44:19.655331+0000","last_unstale":"2023-12-11T17:45:14.877196+0000","last_undegraded":"2023-12-11T17:45:14.877196+0000","last_fullsized":"2023-12-11T17:45:14.877196+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_clean_scrub_stamp":"2023-12-11T17:44:18.636443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T00:01:27.963540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.879898+0000","last_change":"2023-12-11T17:44:20.617396+0000","last_active":"2023-12-11T17:45:14.879898+0000","last_peered":"2023-12-11T17:45:14.879898+0000","last_clean":"2023-12-11T17:45:14.879898+0000","last_became_active":"2023-12-11T17:44:20.617086+0000","last_became_peered":"2023-12-11T17:44:20.617086+0000","last_unstale":"2023-12-11T17:45:14.879898+0000","last_undegraded":"2023-12-11T17:45:14.879898+0000","last_fullsized":"2023-12-11T17:45:14.879898+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T02:31:55.763125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.879744+0000","last_change":"2023-12-11T17:44:24.703283+0000","last_active":"2023-12-11T17:45:14.879744+0000","last_peered":"2023-12-11T17:45:14.879744+0000","last_clean":"2023-12-11T17:45:14.879744+0000","last_became_active":"2023-12-11T17:44:24.701703+0000","last_became_peered":"2023-12-11T17:44:24.701703+0000","last_unstale":"2023-12-11T17:45:14.879744+0000","last_undegraded":"2023-12-11T17:45:14.879744+0000","last_fullsized":"2023-12-11T17:45:14.879744+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_clean_scrub_stamp":"2023-12-11T17:44:23.663161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T18:52:50.570036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.873678+0000","last_change":"2023-12-11T17:44:19.662851+0000","last_active":"2023-12-11T17:45:14.873678+0000","last_peered":"2023-12-11T17:45:14.873678+0000","last_clean":"2023-12-11T17:45:14.873678+0000","last_became_active":"2023-12-11T17:44:19.662548+0000","last_became_peered":"2023-12-11T17:44:19.662548+0000","last_unstale":"2023-12-11T17:45:14.873678+0000","last_undegraded":"2023-12-11T17:45:14.873678+0000","last_fullsized":"2023-12-11T17:45:14.873678+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_clean_scrub_stamp":"2023-12-11T17:44:18.636443+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T02:34:14.914687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.878774+0000","last_change":"2023-12-11T17:44:20.612750+0000","last_active":"2023-12-11T17:45:14.878774+0000","last_peered":"2023-12-11T17:45:14.878774+0000","last_clean":"2023-12-11T17:45:14.878774+0000","last_became_active":"2023-12-11T17:44:20.612479+0000","last_became_peered":"2023-12-11T17:44:20.612479+0000","last_unstale":"2023-12-11T17:45:14.878774+0000","last_undegraded":"2023-12-11T17:45:14.878774+0000","last_fullsized":"2023-12-11T17:45:14.878774+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:45:03.217844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.880124+0000","last_change":"2023-12-11T17:44:24.703050+0000","last_active":"2023-12-11T17:45:14.880124+0000","last_peered":"2023-12-11T17:45:14.880124+0000","last_clean":"2023-12-11T17:45:14.880124+0000","last_became_active":"2023-12-11T17:44:24.701408+0000","last_became_peered":"2023-12-11T17:44:24.701408+0000","last_unstale":"2023-12-11T17:45:14.880124+0000","last_undegraded":"2023-12-11T17:45:14.880124+0000","last_fullsized":"2023-12-11T17:45:14.880124+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_clean_scrub_stamp":"2023-12-11T17:44:23.663161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T01:02:44.726831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.878644+0000","last_change":"2023-12-11T17:44:20.616004+0000","last_active":"2023-12-11T17:45:14.878644+0000","last_peered":"2023-12-11T17:45:14.878644+0000","last_clean":"2023-12-11T17:45:14.878644+0000","last_became_active":"2023-12-11T17:44:20.614985+0000","last_became_peered":"2023-12-11T17:44:20.614985+0000","last_unstale":"2023-12-11T17:45:14.878644+0000","last_undegraded":"2023-12-11T17:45:14.878644+0000","last_fullsized":"2023-12-11T17:45:14.878644+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T05:31:13.414818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'4","reported_seq":67,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T17:45:29.846444+0000","last_change":"2023-12-11T17:44:19.663092+0000","last_active":"2023-12-11T17:45:29.846444+0000","last_peered":"2023-12-11T17:45:29.846444+0000","last_clean":"2023-12-11T17:45:29.846444+0000","last_became_active":"2023-12-11T17:44:19.662728+0000","last_became_peered":"2023-12-11T17:44:19.662728+0000","last_unstale":"2023-12-11T17:45:29.846444+0000","last_undegraded":"2023-12-11T17:45:29.846444+0000","last_fullsized":"2023-12-11T17:45:29.846444+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_clean_scrub_stamp":"2023-12-11T17:44:18.636443+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T20:05:23.952470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":44,"num_read_kb":44,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"86'1","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.874001+0000","last_change":"2023-12-11T17:44:24.694194+0000","last_active":"2023-12-11T17:45:14.874001+0000","last_peered":"2023-12-11T17:45:14.874001+0000","last_clean":"2023-12-11T17:45:14.874001+0000","last_became_active":"2023-12-11T17:44:24.693860+0000","last_became_peered":"2023-12-11T17:44:24.693860+0000","last_unstale":"2023-12-11T17:45:14.874001+0000","last_undegraded":"2023-12-11T17:45:14.874001+0000","last_fullsized":"2023-12-11T17:45:14.874001+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_clean_scrub_stamp":"2023-12-11T17:44:23.663161+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T01:16:04.877531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T17:45:15.889584+0000","last_change":"2023-12-11T17:45:15.887755+0000","last_active":"2023-12-11T17:45:15.889584+0000","last_peered":"2023-12-11T17:45:15.889584+0000","last_clean":"2023-12-11T17:45:15.889584+0000","last_became_active":"2023-12-11T17:45:15.887541+0000","last_became_peered":"2023-12-11T17:45:15.887541+0000","last_unstale":"2023-12-11T17:45:15.889584+0000","last_undegraded":"2023-12-11T17:45:15.889584+0000","last_fullsized":"2023-12-11T17:45:15.889584+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T02:57:39.431558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.876825+0000","last_change":"2023-12-11T17:44:19.651980+0000","last_active":"2023-12-11T17:45:14.876825+0000","last_peered":"2023-12-11T17:45:14.876825+0000","last_clean":"2023-12-11T17:45:14.876825+0000","last_became_active":"2023-12-11T17:44:19.651659+0000","last_became_peered":"2023-12-11T17:44:19.651659+0000","last_unstale":"2023-12-11T17:45:14.876825+0000","last_undegraded":"2023-12-11T17:45:14.876825+0000","last_fullsized":"2023-12-11T17:45:14.876825+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_clean_scrub_stamp":"2023-12-11T17:44:18.636443+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T19:29:21.453758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":322,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.873442+0000","last_change":"2023-12-11T17:43:21.021919+0000","last_active":"2023-12-11T17:45:14.873442+0000","last_peered":"2023-12-11T17:45:14.873442+0000","last_clean":"2023-12-11T17:45:14.873442+0000","last_became_active":"2023-12-11T17:43:21.021489+0000","last_became_peered":"2023-12-11T17:43:21.021489+0000","last_unstale":"2023-12-11T17:45:14.873442+0000","last_undegraded":"2023-12-11T17:45:14.873442+0000","last_fullsized":"2023-12-11T17:45:14.873442+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:40:27.026768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:40:27.026768+0000","last_clean_scrub_stamp":"2023-12-11T17:40:27.026768+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T22:30:14.730794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.879358+0000","last_change":"2023-12-11T17:44:24.701996+0000","last_active":"2023-12-11T17:45:14.879358+0000","last_peered":"2023-12-11T17:45:14.879358+0000","last_clean":"2023-12-11T17:45:14.879358+0000","last_became_active":"2023-12-11T17:44:24.701574+0000","last_became_peered":"2023-12-11T17:44:24.701574+0000","last_unstale":"2023-12-11T17:45:14.879358+0000","last_undegraded":"2023-12-11T17:45:14.879358+0000","last_fullsized":"2023-12-11T17:45:14.879358+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_clean_scrub_stamp":"2023-12-11T17:44:23.663161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T19:49:01.458432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.880311+0000","last_change":"2023-12-11T17:44:20.614765+0000","last_active":"2023-12-11T17:45:14.880311+0000","last_peered":"2023-12-11T17:45:14.880311+0000","last_clean":"2023-12-11T17:45:14.880311+0000","last_became_active":"2023-12-11T17:44:20.614462+0000","last_became_peered":"2023-12-11T17:44:20.614462+0000","last_unstale":"2023-12-11T17:45:14.880311+0000","last_undegraded":"2023-12-11T17:45:14.880311+0000","last_fullsized":"2023-12-11T17:45:14.880311+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:08:20.784124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.873636+0000","last_change":"2023-12-11T17:44:19.661528+0000","last_active":"2023-12-11T17:45:14.873636+0000","last_peered":"2023-12-11T17:45:14.873636+0000","last_clean":"2023-12-11T17:45:14.873636+0000","last_became_active":"2023-12-11T17:44:19.661208+0000","last_became_peered":"2023-12-11T17:44:19.661208+0000","last_unstale":"2023-12-11T17:45:14.873636+0000","last_undegraded":"2023-12-11T17:45:14.873636+0000","last_fullsized":"2023-12-11T17:45:14.873636+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_clean_scrub_stamp":"2023-12-11T17:44:18.636443+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T18:31:23.295193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.880386+0000","last_change":"2023-12-11T17:44:24.688218+0000","last_active":"2023-12-11T17:45:14.880386+0000","last_peered":"2023-12-11T17:45:14.880386+0000","last_clean":"2023-12-11T17:45:14.880386+0000","last_became_active":"2023-12-11T17:44:24.687809+0000","last_became_peered":"2023-12-11T17:44:24.687809+0000","last_unstale":"2023-12-11T17:45:14.880386+0000","last_undegraded":"2023-12-11T17:45:14.880386+0000","last_fullsized":"2023-12-11T17:45:14.880386+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_clean_scrub_stamp":"2023-12-11T17:44:23.663161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T02:12:50.975614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.873899+0000","last_change":"2023-12-11T17:44:19.657797+0000","last_active":"2023-12-11T17:45:14.873899+0000","last_peered":"2023-12-11T17:45:14.873899+0000","last_clean":"2023-12-11T17:45:14.873899+0000","last_became_active":"2023-12-11T17:44:19.657483+0000","last_became_peered":"2023-12-11T17:44:19.657483+0000","last_unstale":"2023-12-11T17:45:14.873899+0000","last_undegraded":"2023-12-11T17:45:14.873899+0000","last_fullsized":"2023-12-11T17:45:14.873899+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_clean_scrub_stamp":"2023-12-11T17:44:18.636443+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T22:43:35.785943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.879897+0000","last_change":"2023-12-11T17:44:20.610002+0000","last_active":"2023-12-11T17:45:14.879897+0000","last_peered":"2023-12-11T17:45:14.879897+0000","last_clean":"2023-12-11T17:45:14.879897+0000","last_became_active":"2023-12-11T17:44:20.609579+0000","last_became_peered":"2023-12-11T17:44:20.609579+0000","last_unstale":"2023-12-11T17:45:14.879897+0000","last_undegraded":"2023-12-11T17:45:14.879897+0000","last_fullsized":"2023-12-11T17:45:14.879897+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:18:25.670403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.881080+0000","last_change":"2023-12-11T17:44:24.694707+0000","last_active":"2023-12-11T17:45:14.881080+0000","last_peered":"2023-12-11T17:45:14.881080+0000","last_clean":"2023-12-11T17:45:14.881080+0000","last_became_active":"2023-12-11T17:44:24.694275+0000","last_became_peered":"2023-12-11T17:44:24.694275+0000","last_unstale":"2023-12-11T17:45:14.881080+0000","last_undegraded":"2023-12-11T17:45:14.881080+0000","last_fullsized":"2023-12-11T17:45:14.881080+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_clean_scrub_stamp":"2023-12-11T17:44:23.663161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T18:28:12.922991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"86'6","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.877680+0000","last_change":"2023-12-11T17:44:19.658825+0000","last_active":"2023-12-11T17:45:14.877680+0000","last_peered":"2023-12-11T17:45:14.877680+0000","last_clean":"2023-12-11T17:45:14.877680+0000","last_became_active":"2023-12-11T17:44:19.658425+0000","last_became_peered":"2023-12-11T17:44:19.658425+0000","last_unstale":"2023-12-11T17:45:14.877680+0000","last_undegraded":"2023-12-11T17:45:14.877680+0000","last_fullsized":"2023-12-11T17:45:14.877680+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_clean_scrub_stamp":"2023-12-11T17:44:18.636443+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T04:06:15.289454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.881577+0000","last_change":"2023-12-11T17:44:20.617557+0000","last_active":"2023-12-11T17:45:14.881577+0000","last_peered":"2023-12-11T17:45:14.881577+0000","last_clean":"2023-12-11T17:45:14.881577+0000","last_became_active":"2023-12-11T17:44:20.616879+0000","last_became_peered":"2023-12-11T17:44:20.616879+0000","last_unstale":"2023-12-11T17:45:14.881577+0000","last_undegraded":"2023-12-11T17:45:14.881577+0000","last_fullsized":"2023-12-11T17:45:14.881577+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T19:34:04.954862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.876786+0000","last_change":"2023-12-11T17:44:24.691471+0000","last_active":"2023-12-11T17:45:14.876786+0000","last_peered":"2023-12-11T17:45:14.876786+0000","last_clean":"2023-12-11T17:45:14.876786+0000","last_became_active":"2023-12-11T17:44:24.690390+0000","last_became_peered":"2023-12-11T17:44:24.690390+0000","last_unstale":"2023-12-11T17:45:14.876786+0000","last_undegraded":"2023-12-11T17:45:14.876786+0000","last_fullsized":"2023-12-11T17:45:14.876786+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_clean_scrub_stamp":"2023-12-11T17:44:23.663161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T00:26:20.141717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.877069+0000","last_change":"2023-12-11T17:44:19.661004+0000","last_active":"2023-12-11T17:45:14.877069+0000","last_peered":"2023-12-11T17:45:14.877069+0000","last_clean":"2023-12-11T17:45:14.877069+0000","last_became_active":"2023-12-11T17:44:19.660691+0000","last_became_peered":"2023-12-11T17:44:19.660691+0000","last_unstale":"2023-12-11T17:45:14.877069+0000","last_undegraded":"2023-12-11T17:45:14.877069+0000","last_fullsized":"2023-12-11T17:45:14.877069+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_clean_scrub_stamp":"2023-12-11T17:44:18.636443+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T17:54:53.956473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.874627+0000","last_change":"2023-12-11T17:44:20.621437+0000","last_active":"2023-12-11T17:45:14.874627+0000","last_peered":"2023-12-11T17:45:14.874627+0000","last_clean":"2023-12-11T17:45:14.874627+0000","last_became_active":"2023-12-11T17:44:20.621113+0000","last_became_peered":"2023-12-11T17:44:20.621113+0000","last_unstale":"2023-12-11T17:45:14.874627+0000","last_undegraded":"2023-12-11T17:45:14.874627+0000","last_fullsized":"2023-12-11T17:45:14.874627+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T21:05:50.097605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.877109+0000","last_change":"2023-12-11T17:44:24.697432+0000","last_active":"2023-12-11T17:45:14.877109+0000","last_peered":"2023-12-11T17:45:14.877109+0000","last_clean":"2023-12-11T17:45:14.877109+0000","last_became_active":"2023-12-11T17:44:24.696924+0000","last_became_peered":"2023-12-11T17:44:24.696924+0000","last_unstale":"2023-12-11T17:45:14.877109+0000","last_undegraded":"2023-12-11T17:45:14.877109+0000","last_fullsized":"2023-12-11T17:45:14.877109+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_clean_scrub_stamp":"2023-12-11T17:44:23.663161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T21:00:01.967894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.879985+0000","last_change":"2023-12-11T17:44:19.665080+0000","last_active":"2023-12-11T17:45:14.879985+0000","last_peered":"2023-12-11T17:45:14.879985+0000","last_clean":"2023-12-11T17:45:14.879985+0000","last_became_active":"2023-12-11T17:44:19.664862+0000","last_became_peered":"2023-12-11T17:44:19.664862+0000","last_unstale":"2023-12-11T17:45:14.879985+0000","last_undegraded":"2023-12-11T17:45:14.879985+0000","last_fullsized":"2023-12-11T17:45:14.879985+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_clean_scrub_stamp":"2023-12-11T17:44:18.636443+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T20:22:05.909035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.873446+0000","last_change":"2023-12-11T17:44:20.614140+0000","last_active":"2023-12-11T17:45:14.873446+0000","last_peered":"2023-12-11T17:45:14.873446+0000","last_clean":"2023-12-11T17:45:14.873446+0000","last_became_active":"2023-12-11T17:44:20.613808+0000","last_became_peered":"2023-12-11T17:44:20.613808+0000","last_unstale":"2023-12-11T17:45:14.873446+0000","last_undegraded":"2023-12-11T17:45:14.873446+0000","last_fullsized":"2023-12-11T17:45:14.873446+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T20:55:57.572836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.877546+0000","last_change":"2023-12-11T17:44:24.697388+0000","last_active":"2023-12-11T17:45:14.877546+0000","last_peered":"2023-12-11T17:45:14.877546+0000","last_clean":"2023-12-11T17:45:14.877546+0000","last_became_active":"2023-12-11T17:44:24.696964+0000","last_became_peered":"2023-12-11T17:44:24.696964+0000","last_unstale":"2023-12-11T17:45:14.877546+0000","last_undegraded":"2023-12-11T17:45:14.877546+0000","last_fullsized":"2023-12-11T17:45:14.877546+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_clean_scrub_stamp":"2023-12-11T17:44:23.663161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T18:08:04.162075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.880959+0000","last_change":"2023-12-11T17:44:19.660327+0000","last_active":"2023-12-11T17:45:14.880959+0000","last_peered":"2023-12-11T17:45:14.880959+0000","last_clean":"2023-12-11T17:45:14.880959+0000","last_became_active":"2023-12-11T17:44:19.659999+0000","last_became_peered":"2023-12-11T17:44:19.659999+0000","last_unstale":"2023-12-11T17:45:14.880959+0000","last_undegraded":"2023-12-11T17:45:14.880959+0000","last_fullsized":"2023-12-11T17:45:14.880959+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_clean_scrub_stamp":"2023-12-11T17:44:18.636443+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T23:51:24.580854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.879951+0000","last_change":"2023-12-11T17:44:20.624126+0000","last_active":"2023-12-11T17:45:14.879951+0000","last_peered":"2023-12-11T17:45:14.879951+0000","last_clean":"2023-12-11T17:45:14.879951+0000","last_became_active":"2023-12-11T17:44:20.623836+0000","last_became_peered":"2023-12-11T17:44:20.623836+0000","last_unstale":"2023-12-11T17:45:14.879951+0000","last_undegraded":"2023-12-11T17:45:14.879951+0000","last_fullsized":"2023-12-11T17:45:14.879951+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T21:43:39.123299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"86'1","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.876615+0000","last_change":"2023-12-11T17:44:24.698994+0000","last_active":"2023-12-11T17:45:14.876615+0000","last_peered":"2023-12-11T17:45:14.876615+0000","last_clean":"2023-12-11T17:45:14.876615+0000","last_became_active":"2023-12-11T17:44:24.698583+0000","last_became_peered":"2023-12-11T17:44:24.698583+0000","last_unstale":"2023-12-11T17:45:14.876615+0000","last_undegraded":"2023-12-11T17:45:14.876615+0000","last_fullsized":"2023-12-11T17:45:14.876615+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_clean_scrub_stamp":"2023-12-11T17:44:23.663161+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T00:03:20.178331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"86'1","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.877402+0000","last_change":"2023-12-11T17:44:19.662505+0000","last_active":"2023-12-11T17:45:14.877402+0000","last_peered":"2023-12-11T17:45:14.877402+0000","last_clean":"2023-12-11T17:45:14.877402+0000","last_became_active":"2023-12-11T17:44:19.661478+0000","last_became_peered":"2023-12-11T17:44:19.661478+0000","last_unstale":"2023-12-11T17:45:14.877402+0000","last_undegraded":"2023-12-11T17:45:14.877402+0000","last_fullsized":"2023-12-11T17:45:14.877402+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_clean_scrub_stamp":"2023-12-11T17:44:18.636443+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T23:28:09.345693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.881400+0000","last_change":"2023-12-11T17:44:20.615026+0000","last_active":"2023-12-11T17:45:14.881400+0000","last_peered":"2023-12-11T17:45:14.881400+0000","last_clean":"2023-12-11T17:45:14.881400+0000","last_became_active":"2023-12-11T17:44:20.614669+0000","last_became_peered":"2023-12-11T17:44:20.614669+0000","last_unstale":"2023-12-11T17:45:14.881400+0000","last_undegraded":"2023-12-11T17:45:14.881400+0000","last_fullsized":"2023-12-11T17:45:14.881400+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T20:52:20.455326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.877196+0000","last_change":"2023-12-11T17:44:24.694681+0000","last_active":"2023-12-11T17:45:14.877196+0000","last_peered":"2023-12-11T17:45:14.877196+0000","last_clean":"2023-12-11T17:45:14.877196+0000","last_became_active":"2023-12-11T17:44:24.694172+0000","last_became_peered":"2023-12-11T17:44:24.694172+0000","last_unstale":"2023-12-11T17:45:14.877196+0000","last_undegraded":"2023-12-11T17:45:14.877196+0000","last_fullsized":"2023-12-11T17:45:14.877196+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_clean_scrub_stamp":"2023-12-11T17:44:23.663161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T20:54:46.367735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":38,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T17:45:16.146416+0000","last_change":"2023-12-11T17:45:16.146416+0000","last_active":"2023-12-11T17:45:16.146416+0000","last_peered":"2023-12-11T17:45:16.146416+0000","last_clean":"2023-12-11T17:45:16.146416+0000","last_became_active":"2023-12-11T17:45:16.141794+0000","last_became_peered":"2023-12-11T17:45:16.141794+0000","last_unstale":"2023-12-11T17:45:16.146416+0000","last_undegraded":"2023-12-11T17:45:16.146416+0000","last_fullsized":"2023-12-11T17:45:16.146416+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_clean_scrub_stamp":"2023-12-11T17:44:18.636443+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:05:27.729213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.873702+0000","last_change":"2023-12-11T17:44:20.621879+0000","last_active":"2023-12-11T17:45:14.873702+0000","last_peered":"2023-12-11T17:45:14.873702+0000","last_clean":"2023-12-11T17:45:14.873702+0000","last_became_active":"2023-12-11T17:44:20.621430+0000","last_became_peered":"2023-12-11T17:44:20.621430+0000","last_unstale":"2023-12-11T17:45:14.873702+0000","last_undegraded":"2023-12-11T17:45:14.873702+0000","last_fullsized":"2023-12-11T17:45:14.873702+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T18:08:46.359629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.880424+0000","last_change":"2023-12-11T17:44:24.700599+0000","last_active":"2023-12-11T17:45:14.880424+0000","last_peered":"2023-12-11T17:45:14.880424+0000","last_clean":"2023-12-11T17:45:14.880424+0000","last_became_active":"2023-12-11T17:44:24.700121+0000","last_became_peered":"2023-12-11T17:44:24.700121+0000","last_unstale":"2023-12-11T17:45:14.880424+0000","last_undegraded":"2023-12-11T17:45:14.880424+0000","last_fullsized":"2023-12-11T17:45:14.880424+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_clean_scrub_stamp":"2023-12-11T17:44:23.663161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T01:08:18.000642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.878068+0000","last_change":"2023-12-11T17:44:19.662414+0000","last_active":"2023-12-11T17:45:14.878068+0000","last_peered":"2023-12-11T17:45:14.878068+0000","last_clean":"2023-12-11T17:45:14.878068+0000","last_became_active":"2023-12-11T17:44:19.661753+0000","last_became_peered":"2023-12-11T17:44:19.661753+0000","last_unstale":"2023-12-11T17:45:14.878068+0000","last_undegraded":"2023-12-11T17:45:14.878068+0000","last_fullsized":"2023-12-11T17:45:14.878068+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_clean_scrub_stamp":"2023-12-11T17:44:18.636443+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T00:49:11.713347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.874224+0000","last_change":"2023-12-11T17:44:20.620973+0000","last_active":"2023-12-11T17:45:14.874224+0000","last_peered":"2023-12-11T17:45:14.874224+0000","last_clean":"2023-12-11T17:45:14.874224+0000","last_became_active":"2023-12-11T17:44:20.620284+0000","last_became_peered":"2023-12-11T17:44:20.620284+0000","last_unstale":"2023-12-11T17:45:14.874224+0000","last_undegraded":"2023-12-11T17:45:14.874224+0000","last_fullsized":"2023-12-11T17:45:14.874224+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:00:52.529062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.873585+0000","last_change":"2023-12-11T17:44:24.696888+0000","last_active":"2023-12-11T17:45:14.873585+0000","last_peered":"2023-12-11T17:45:14.873585+0000","last_clean":"2023-12-11T17:45:14.873585+0000","last_became_active":"2023-12-11T17:44:24.696475+0000","last_became_peered":"2023-12-11T17:44:24.696475+0000","last_unstale":"2023-12-11T17:45:14.873585+0000","last_undegraded":"2023-12-11T17:45:14.873585+0000","last_fullsized":"2023-12-11T17:45:14.873585+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_clean_scrub_stamp":"2023-12-11T17:44:23.663161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T21:42:17.287557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"81'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.877574+0000","last_change":"2023-12-11T17:44:19.659094+0000","last_active":"2023-12-11T17:45:14.877574+0000","last_peered":"2023-12-11T17:45:14.877574+0000","last_clean":"2023-12-11T17:45:14.877574+0000","last_became_active":"2023-12-11T17:44:19.658794+0000","last_became_peered":"2023-12-11T17:44:19.658794+0000","last_unstale":"2023-12-11T17:45:14.877574+0000","last_undegraded":"2023-12-11T17:45:14.877574+0000","last_fullsized":"2023-12-11T17:45:14.877574+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_clean_scrub_stamp":"2023-12-11T17:44:18.636443+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T01:39:41.151915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.873936+0000","last_change":"2023-12-11T17:44:20.623342+0000","last_active":"2023-12-11T17:45:14.873936+0000","last_peered":"2023-12-11T17:45:14.873936+0000","last_clean":"2023-12-11T17:45:14.873936+0000","last_became_active":"2023-12-11T17:44:20.622957+0000","last_became_peered":"2023-12-11T17:44:20.622957+0000","last_unstale":"2023-12-11T17:45:14.873936+0000","last_undegraded":"2023-12-11T17:45:14.873936+0000","last_fullsized":"2023-12-11T17:45:14.873936+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:25:00.910645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.876776+0000","last_change":"2023-12-11T17:44:24.690746+0000","last_active":"2023-12-11T17:45:14.876776+0000","last_peered":"2023-12-11T17:45:14.876776+0000","last_clean":"2023-12-11T17:45:14.876776+0000","last_became_active":"2023-12-11T17:44:24.690290+0000","last_became_peered":"2023-12-11T17:44:24.690290+0000","last_unstale":"2023-12-11T17:45:14.876776+0000","last_undegraded":"2023-12-11T17:45:14.876776+0000","last_fullsized":"2023-12-11T17:45:14.876776+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:23.663161+0000","last_clean_scrub_stamp":"2023-12-11T17:44:23.663161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T01:26:14.409832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"81'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.880570+0000","last_change":"2023-12-11T17:44:19.655775+0000","last_active":"2023-12-11T17:45:14.880570+0000","last_peered":"2023-12-11T17:45:14.880570+0000","last_clean":"2023-12-11T17:45:14.880570+0000","last_became_active":"2023-12-11T17:44:19.654360+0000","last_became_peered":"2023-12-11T17:44:19.654360+0000","last_unstale":"2023-12-11T17:45:14.880570+0000","last_undegraded":"2023-12-11T17:45:14.880570+0000","last_fullsized":"2023-12-11T17:45:14.880570+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:18.636443+0000","last_clean_scrub_stamp":"2023-12-11T17:44:18.636443+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T00:07:08.678389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.874458+0000","last_change":"2023-12-11T17:44:20.614898+0000","last_active":"2023-12-11T17:45:14.874458+0000","last_peered":"2023-12-11T17:45:14.874458+0000","last_clean":"2023-12-11T17:45:14.874458+0000","last_became_active":"2023-12-11T17:44:20.613972+0000","last_became_peered":"2023-12-11T17:44:20.613972+0000","last_unstale":"2023-12-11T17:45:14.874458+0000","last_undegraded":"2023-12-11T17:45:14.874458+0000","last_fullsized":"2023-12-11T17:45:14.874458+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T04:28:51.576955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.880057+0000","last_change":"2023-12-11T17:44:20.617502+0000","last_active":"2023-12-11T17:45:14.880057+0000","last_peered":"2023-12-11T17:45:14.880057+0000","last_clean":"2023-12-11T17:45:14.880057+0000","last_became_active":"2023-12-11T17:44:20.617252+0000","last_became_peered":"2023-12-11T17:44:20.617252+0000","last_unstale":"2023-12-11T17:45:14.880057+0000","last_undegraded":"2023-12-11T17:45:14.880057+0000","last_fullsized":"2023-12-11T17:45:14.880057+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T19:23:37.133765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.877027+0000","last_change":"2023-12-11T17:44:20.619325+0000","last_active":"2023-12-11T17:45:14.877027+0000","last_peered":"2023-12-11T17:45:14.877027+0000","last_clean":"2023-12-11T17:45:14.877027+0000","last_became_active":"2023-12-11T17:44:20.619016+0000","last_became_peered":"2023-12-11T17:44:20.619016+0000","last_unstale":"2023-12-11T17:45:14.877027+0000","last_undegraded":"2023-12-11T17:45:14.877027+0000","last_fullsized":"2023-12-11T17:45:14.877027+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T20:21:24.877379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.876297+0000","last_change":"2023-12-11T17:44:20.624432+0000","last_active":"2023-12-11T17:45:14.876297+0000","last_peered":"2023-12-11T17:45:14.876297+0000","last_clean":"2023-12-11T17:45:14.876297+0000","last_became_active":"2023-12-11T17:44:20.624148+0000","last_became_peered":"2023-12-11T17:44:20.624148+0000","last_unstale":"2023-12-11T17:45:14.876297+0000","last_undegraded":"2023-12-11T17:45:14.876297+0000","last_fullsized":"2023-12-11T17:45:14.876297+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T19:15:46.054968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.873629+0000","last_change":"2023-12-11T17:44:20.624134+0000","last_active":"2023-12-11T17:45:14.873629+0000","last_peered":"2023-12-11T17:45:14.873629+0000","last_clean":"2023-12-11T17:45:14.873629+0000","last_became_active":"2023-12-11T17:44:20.623858+0000","last_became_peered":"2023-12-11T17:44:20.623858+0000","last_unstale":"2023-12-11T17:45:14.873629+0000","last_undegraded":"2023-12-11T17:45:14.873629+0000","last_fullsized":"2023-12-11T17:45:14.873629+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T03:24:48.946619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.880004+0000","last_change":"2023-12-11T17:44:20.620407+0000","last_active":"2023-12-11T17:45:14.880004+0000","last_peered":"2023-12-11T17:45:14.880004+0000","last_clean":"2023-12-11T17:45:14.880004+0000","last_became_active":"2023-12-11T17:44:20.620106+0000","last_became_peered":"2023-12-11T17:44:20.620106+0000","last_unstale":"2023-12-11T17:45:14.880004+0000","last_undegraded":"2023-12-11T17:45:14.880004+0000","last_fullsized":"2023-12-11T17:45:14.880004+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T22:18:36.780037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.880745+0000","last_change":"2023-12-11T17:44:20.618426+0000","last_active":"2023-12-11T17:45:14.880745+0000","last_peered":"2023-12-11T17:45:14.880745+0000","last_clean":"2023-12-11T17:45:14.880745+0000","last_became_active":"2023-12-11T17:44:20.618136+0000","last_became_peered":"2023-12-11T17:44:20.618136+0000","last_unstale":"2023-12-11T17:45:14.880745+0000","last_undegraded":"2023-12-11T17:45:14.880745+0000","last_fullsized":"2023-12-11T17:45:14.880745+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T05:02:04.096866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.877765+0000","last_change":"2023-12-11T17:44:20.619609+0000","last_active":"2023-12-11T17:45:14.877765+0000","last_peered":"2023-12-11T17:45:14.877765+0000","last_clean":"2023-12-11T17:45:14.877765+0000","last_became_active":"2023-12-11T17:44:20.619345+0000","last_became_peered":"2023-12-11T17:44:20.619345+0000","last_unstale":"2023-12-11T17:45:14.877765+0000","last_undegraded":"2023-12-11T17:45:14.877765+0000","last_fullsized":"2023-12-11T17:45:14.877765+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T19:56:00.839819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.881657+0000","last_change":"2023-12-11T17:44:20.614115+0000","last_active":"2023-12-11T17:45:14.881657+0000","last_peered":"2023-12-11T17:45:14.881657+0000","last_clean":"2023-12-11T17:45:14.881657+0000","last_became_active":"2023-12-11T17:44:20.613800+0000","last_became_peered":"2023-12-11T17:44:20.613800+0000","last_unstale":"2023-12-11T17:45:14.881657+0000","last_undegraded":"2023-12-11T17:45:14.881657+0000","last_fullsized":"2023-12-11T17:45:14.881657+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T20:33:37.170611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-11T17:45:15.891039+0000","last_change":"2023-12-11T17:45:15.890531+0000","last_active":"2023-12-11T17:45:15.891039+0000","last_peered":"2023-12-11T17:45:15.891039+0000","last_clean":"2023-12-11T17:45:15.891039+0000","last_became_active":"2023-12-11T17:45:15.890139+0000","last_became_peered":"2023-12-11T17:45:15.890139+0000","last_unstale":"2023-12-11T17:45:15.891039+0000","last_undegraded":"2023-12-11T17:45:15.891039+0000","last_fullsized":"2023-12-11T17:45:15.891039+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T19:13:27.577867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,8],"acting":[6,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.880882+0000","last_change":"2023-12-11T17:44:20.617195+0000","last_active":"2023-12-11T17:45:14.880882+0000","last_peered":"2023-12-11T17:45:14.880882+0000","last_clean":"2023-12-11T17:45:14.880882+0000","last_became_active":"2023-12-11T17:44:20.616529+0000","last_became_peered":"2023-12-11T17:44:20.616529+0000","last_unstale":"2023-12-11T17:45:14.880882+0000","last_undegraded":"2023-12-11T17:45:14.880882+0000","last_fullsized":"2023-12-11T17:45:14.880882+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T23:53:50.558483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.880580+0000","last_change":"2023-12-11T17:44:20.618290+0000","last_active":"2023-12-11T17:45:14.880580+0000","last_peered":"2023-12-11T17:45:14.880580+0000","last_clean":"2023-12-11T17:45:14.880580+0000","last_became_active":"2023-12-11T17:44:20.617982+0000","last_became_peered":"2023-12-11T17:44:20.617982+0000","last_unstale":"2023-12-11T17:45:14.880580+0000","last_undegraded":"2023-12-11T17:45:14.880580+0000","last_fullsized":"2023-12-11T17:45:14.880580+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T21:58:14.444462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.880250+0000","last_change":"2023-12-11T17:44:20.619958+0000","last_active":"2023-12-11T17:45:14.880250+0000","last_peered":"2023-12-11T17:45:14.880250+0000","last_clean":"2023-12-11T17:45:14.880250+0000","last_became_active":"2023-12-11T17:44:20.619504+0000","last_became_peered":"2023-12-11T17:44:20.619504+0000","last_unstale":"2023-12-11T17:45:14.880250+0000","last_undegraded":"2023-12-11T17:45:14.880250+0000","last_fullsized":"2023-12-11T17:45:14.880250+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-12T19:36:41.193000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-11T17:45:14.880874+0000","last_change":"2023-12-11T17:44:20.612953+0000","last_active":"2023-12-11T17:45:14.880874+0000","last_peered":"2023-12-11T17:45:14.880874+0000","last_clean":"2023-12-11T17:45:14.880874+0000","last_became_active":"2023-12-11T17:44:20.612618+0000","last_became_peered":"2023-12-11T17:44:20.612618+0000","last_unstale":"2023-12-11T17:45:14.880874+0000","last_undegraded":"2023-12-11T17:45:14.880874+0000","last_fullsized":"2023-12-11T17:45:14.880874+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-11T17:44:19.637731+0000","last_clean_scrub_stamp":"2023-12-11T17:44:19.637731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T04:25:39.528227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":3542208,"data_compressed":211312,"data_compressed_allocated":270336,"data_compressed_original":3506176,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65384,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":269,"num_read_kb":207,"num_write":159,"num_write_kb":114,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1593,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":196152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":77,"seq":330712481818,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":962560,"data_stored":483492,"data_compressed":17682,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 11 17:44:38 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.502},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.67600000000000005}]},{"osd":1,"last update":"Mon Dec 11 17:44:38 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.63400000000000001}]},{"osd":2,"last update":"Mon Dec 11 17:44:38 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.54200000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48599999999999999}]},{"osd":4,"last update":"Mon Dec 11 17:44:38 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.71899999999999997}]},{"osd":5,"last update":"Mon Dec 11 17:44:38 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.66000000000000003}]},{"osd":6,"last update":"Mon Dec 11 17:44:38 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.448,"5min":1.448,"15min":1.448},"last":0.80100000000000005}]},{"osd":7,"last update":"Mon Dec 11 17:44:38 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.874},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.385,"5min":1.385,"15min":1.385},"last":0.81799999999999995}]},{"osd":8,"last update":"Mon Dec 11 17:45:21 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.74099999999999999}]},{"osd":9,"last update":"Mon Dec 11 17:44:38 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.5089999999999999,"5min":1.5089999999999999,"15min":1.5089999999999999},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.83099999999999996}]},{"osd":10,"last update":"Mon Dec 11 17:44:38 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.4870000000000001,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.5309999999999999,"5min":1.5309999999999999,"15min":1.5309999999999999},"last":0.92600000000000005}]}]},{"osd":10,"up_from":70,"seq":300647710748,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28096,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717056,"statfs":{"total":95995035648,"available":95966265344,"internally_reserved":0,"allocated":1101824,"data_stored":1131938,"data_compressed":53447,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 11 17:45:21 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.42699999999999999}]},{"osd":1,"last update":"Mon Dec 11 17:45:21 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.50700000000000001,"15min":0.505},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.52300000000000002,"15min":0.51600000000000001},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.315}]},{"osd":2,"last update":"Mon Dec 11 17:45:21 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.63700000000000001,"15min":0.63600000000000001},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.81499999999999995,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.749},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.65200000000000002,"15min":0.64900000000000002},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.147,"5min":1.147,"15min":1.147},"last":0.629}]},{"osd":3,"last update":"Mon Dec 11 17:45:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.626,"15min":0.63700000000000001},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.76100000000000001,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.69399999999999995,"15min":0.70199999999999996},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.40899999999999997}]},{"osd":4,"last update":"Mon Dec 11 17:45:21 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.60999999999999999,"15min":0.60599999999999998},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.57499999999999996,"15min":0.57599999999999996},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.47399999999999998}]},{"osd":5,"last update":"Mon Dec 11 17:45:21 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.71299999999999997,"15min":0.71099999999999997},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.74399999999999999,"15min":0.746},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.72599999999999998}]},{"osd":6,"last update":"Mon Dec 11 17:45:21 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.71799999999999997,"15min":0.72899999999999998},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.57799999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56799999999999995}]},{"osd":8,"last update":"Mon Dec 11 17:45:21 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.83999999999999997,"15min":0.83399999999999996},"min":{"1min":0.66300000000000003,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.417,"5min":1.417,"15min":1.417},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.82299999999999995,"15min":0.82899999999999996},"min":{"1min":0.58499999999999996,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.452,"5min":1.452,"15min":1.452},"last":0.67700000000000005}]},{"osd":9,"last update":"Mon Dec 11 17:45:21 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.83999999999999997,"15min":0.84599999999999997},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.069,"5min":1.202,"15min":1.202},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.81999999999999995,"15min":0.83799999999999997},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.997,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.54400000000000004}]},{"osd":11,"last update":"Mon Dec 11 17:44:40 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.71199999999999997}]}]},{"osd":9,"up_from":65,"seq":279172874275,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":983040,"data_stored":487948,"data_compressed":17682,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 11 17:45:01 2023","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.46800000000000003,"15min":0.48799999999999999},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.51300000000000001,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.47699999999999998,"15min":0.49099999999999999},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.60799999999999998,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.32100000000000001}]},{"osd":1,"last update":"Mon Dec 11 17:45:01 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.64100000000000001,"15min":0.65100000000000002},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.86299999999999999,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.57499999999999996,"15min":0.57799999999999996},"min":{"1min":0.44,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.749,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.78800000000000003}]},{"osd":2,"last update":"Mon Dec 11 17:45:01 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.67900000000000005,"15min":0.69499999999999995},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.73899999999999999,"5min":1.046,"15min":1.046},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.65300000000000002,"15min":0.65900000000000003},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.70799999999999996}]},{"osd":3,"last update":"Mon Dec 11 17:45:01 2023","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.55700000000000005,"15min":0.57599999999999996},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.629,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.622,"15min":0.64800000000000002},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.81299999999999994,"5min":1.046,"15min":1.046},"last":0.35999999999999999}]},{"osd":4,"last update":"Mon Dec 11 17:45:01 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.66900000000000004,"15min":0.67900000000000005},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.83599999999999997,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.67800000000000005,"15min":0.68200000000000005},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.90100000000000002,"5min":1.157,"15min":1.157},"last":0.67900000000000005}]},{"osd":5,"last update":"Mon Dec 11 17:45:01 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.71599999999999997,"15min":0.72999999999999998},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.82299999999999995,"5min":1.204,"15min":1.204},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.79200000000000004,"15min":0.81699999999999995},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.88200000000000001,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.69899999999999995}]},{"osd":6,"last update":"Mon Dec 11 17:43:57 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.5289999999999999,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.44800000000000001}]},{"osd":7,"last update":"Mon Dec 11 17:45:01 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.76500000000000001,"15min":0.77900000000000003},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.0029999999999999,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.81599999999999995,"15min":0.83899999999999997},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.95499999999999996,"5min":1.224,"15min":1.224},"last":0.65600000000000003}]},{"osd":8,"last update":"Mon Dec 11 17:45:01 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.78000000000000003,"15min":0.80200000000000005},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.92300000000000004,"5min":1.131,"15min":1.131},"last":0.749},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.80300000000000005,"15min":0.81899999999999995},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.1100000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.64100000000000001}]},{"osd":10,"last update":"Mon Dec 11 17:45:23 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.73999999999999999,"15min":0.754},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.98399999999999999,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.78700000000000003,"15min":0.80500000000000005},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.98599999999999999,"5min":1.252,"15min":1.252},"last":0.76500000000000001}]},{"osd":11,"last update":"Mon Dec 11 17:45:25 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.627}]}]},{"osd":8,"up_from":57,"seq":244813135911,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":978944,"data_stored":483772,"data_compressed":17682,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 11 17:44:37 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.54300000000000004,"15min":0.53600000000000003},"min":{"1min":0.36499999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.56200000000000006,"15min":0.56299999999999994},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.78000000000000003,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.73499999999999999}]},{"osd":1,"last update":"Mon Dec 11 17:44:37 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.57799999999999996,"15min":0.57599999999999996},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.94999999999999996,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.62,"5min":0.59699999999999998,"15min":0.59299999999999997},"min":{"1min":0.42799999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.86199999999999999}]},{"osd":2,"last update":"Mon Dec 11 17:44:37 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.437,"15min":0.42999999999999999},"min":{"1min":0.32000000000000001,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.47899999999999998,"15min":0.48199999999999998},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.68899999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.58199999999999996}]},{"osd":3,"last update":"Mon Dec 11 17:44:37 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.59599999999999997,"15min":0.59999999999999998},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.73099999999999998,"5min":1.077,"15min":1.077},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.57899999999999996,"15min":0.57599999999999996},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.80200000000000005,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.69699999999999995}]},{"osd":4,"last update":"Mon Dec 11 17:44:37 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.628,"15min":0.625},"min":{"1min":0.47299999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.87,"5min":1.141,"15min":1.141},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.621,"15min":0.61099999999999999},"min":{"1min":0.53000000000000003,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.0069999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.84599999999999997}]},{"osd":5,"last update":"Mon Dec 11 17:44:37 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.56299999999999994,"15min":0.56499999999999995},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.72799999999999998,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.59199999999999997,"15min":0.59799999999999998},"min":{"1min":0.35599999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.76300000000000001,"5min":1.482,"15min":1.482},"last":0.63900000000000001}]},{"osd":6,"last update":"Mon Dec 11 17:44:37 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.64300000000000002,"15min":0.63400000000000001},"min":{"1min":0.45000000000000001,"5min":0.37,"15min":0.37},"max":{"1min":0.84599999999999997,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.68500000000000005,"15min":0.68300000000000005},"min":{"1min":0.53600000000000003,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.87,"5min":1.008,"15min":1.008},"last":0.80200000000000005}]},{"osd":7,"last update":"Mon Dec 11 17:44:37 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.68000000000000005,"15min":0.67300000000000004},"min":{"1min":0.48699999999999999,"5min":0.432,"15min":0.432},"max":{"1min":0.93300000000000005,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.73099999999999998,"15min":0.72499999999999998},"min":{"1min":0.55400000000000005,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.546,"5min":1.546,"15min":1.546},"last":0.878}]},{"osd":9,"last update":"Mon Dec 11 17:45:00 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.68300000000000005,"15min":0.67100000000000004},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.92600000000000005},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.69899999999999995,"15min":0.68200000000000005},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.94399999999999995}]},{"osd":10,"last update":"Mon Dec 11 17:45:20 2023","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.73299999999999998,"15min":0.70099999999999996},"min":{"1min":0.58299999999999996,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.4890000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.96399999999999997},{"interface":"front","average":{"1min":0.92900000000000005,"5min":0.73199999999999998,"15min":0.69899999999999995},"min":{"1min":0.67700000000000005,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":1.02}]},{"osd":11,"last update":"Mon Dec 11 17:45:20 2023","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":1.002},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.97899999999999998}]}]},{"osd":7,"up_from":50,"seq":214748364842,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1024000,"data_stored":1079486,"data_compressed":53447,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 11 17:44:13 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.54700000000000004,"15min":0.54300000000000004},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.44},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.55200000000000005,"15min":0.54800000000000004},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.42399999999999999}]},{"osd":1,"last update":"Mon Dec 11 17:45:14 2023","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.40600000000000003,"15min":0.40899999999999997},"min":{"1min":0.317,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.51700000000000002,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.373},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.44900000000000001,"15min":0.44800000000000001},"min":{"1min":0.30299999999999999,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.40699999999999997}]},{"osd":2,"last update":"Mon Dec 11 17:45:14 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.57699999999999996,"15min":0.57799999999999996},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.59699999999999998,"15min":0.59299999999999997},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.81499999999999995,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.58799999999999997}]},{"osd":3,"last update":"Mon Dec 11 17:45:14 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.58199999999999996,"15min":0.56799999999999995},"min":{"1min":0.40200000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.625,"5min":0.61099999999999999,"15min":0.60399999999999998},"min":{"1min":0.441,"5min":0.377,"15min":0.377},"max":{"1min":0.80300000000000005,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.53000000000000003}]},{"osd":4,"last update":"Mon Dec 11 17:45:14 2023","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.52700000000000002,"15min":0.52900000000000003},"min":{"1min":0.34000000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.64600000000000002,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.432,"5min":0.47499999999999998,"15min":0.48699999999999999},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.60399999999999998,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.38800000000000001}]},{"osd":5,"last update":"Mon Dec 11 17:45:14 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.622,"15min":0.624},"min":{"1min":0.48399999999999999,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.77500000000000002,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.496,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.85799999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.83299999999999996}]},{"osd":6,"last update":"Mon Dec 11 17:45:14 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.54500000000000004,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.872,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.63600000000000001,"15min":0.63700000000000001},"min":{"1min":0.41199999999999998,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.45800000000000002}]},{"osd":8,"last update":"Mon Dec 11 17:44:37 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.68300000000000005,"15min":0.69199999999999995},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.88200000000000001,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.70299999999999996,"15min":0.70899999999999996},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.92500000000000004,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.79400000000000004}]},{"osd":9,"last update":"Mon Dec 11 17:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.71899999999999997,"15min":0.72099999999999997},"min":{"1min":0.46899999999999997,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.90700000000000003,"5min":1.056,"15min":1.056},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.745,"15min":0.747},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":0.90700000000000003,"5min":1.109,"15min":1.109},"last":0.68899999999999995}]},{"osd":10,"last update":"Mon Dec 11 17:45:19 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.70099999999999996,"15min":0.69699999999999995},"min":{"1min":0.51900000000000002,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.93799999999999994,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.69499999999999995,"15min":0.69299999999999995},"min":{"1min":0.60499999999999998,"5min":0.432,"15min":0.432},"max":{"1min":0.80900000000000005,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.70799999999999996}]},{"osd":11,"last update":"Mon Dec 11 17:45:21 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.80800000000000005}]}]},{"osd":6,"up_from":45,"seq":193273528368,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":958464,"data_stored":479028,"data_compressed":17682,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 11 17:44:54 2023","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.40400000000000003,"15min":0.38800000000000001},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.29399999999999998},{"interface":"front","average":{"1min":0.433,"5min":0.41899999999999998,"15min":0.40899999999999997},"min":{"1min":0.29699999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.315}]},{"osd":1,"last update":"Mon Dec 11 17:44:54 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.55700000000000005,"15min":0.54300000000000004},"min":{"1min":0.41699999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.56899999999999995,"15min":0.55100000000000005},"min":{"1min":0.42799999999999999,"5min":0.375,"15min":0.375},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.48299999999999998}]},{"osd":2,"last update":"Mon Dec 11 17:44:54 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.58399999999999996,"15min":0.56000000000000005},"min":{"1min":0.44500000000000001,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.83599999999999997,"5min":1.107,"15min":1.107},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.58899999999999997,"15min":0.56000000000000005},"min":{"1min":0.34799999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.83199999999999996,"5min":1.236,"15min":1.236},"last":0.59899999999999998}]},{"osd":3,"last update":"Mon Dec 11 17:44:54 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.503,"15min":0.48499999999999999},"min":{"1min":0.32800000000000001,"5min":0.317,"15min":0.317},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.50900000000000001,"15min":0.48999999999999999},"min":{"1min":0.375,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.89200000000000002,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.40899999999999997}]},{"osd":4,"last update":"Mon Dec 11 17:44:54 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.62,"15min":0.60599999999999998},"min":{"1min":0.443,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.64600000000000002,"15min":0.63200000000000001},"min":{"1min":0.46300000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.96499999999999997,"5min":1.083,"15min":1.083},"last":0.439}]},{"osd":5,"last update":"Mon Dec 11 17:44:54 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.64400000000000002,"15min":0.63},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.85799999999999998,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.67100000000000004,"15min":0.65300000000000002},"min":{"1min":0.56999999999999995,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.83299999999999996,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.79500000000000004}]},{"osd":7,"last update":"Mon Dec 11 17:45:13 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.70799999999999996,"15min":0.67600000000000005},"min":{"1min":0.52800000000000002,"5min":0.442,"15min":0.442},"max":{"1min":2.4359999999999999,"5min":2.4359999999999999,"15min":2.4359999999999999},"last":0.54600000000000004},{"interface":"front","average":{"1min":1.21,"5min":0.81200000000000006,"15min":0.74099999999999999},"min":{"1min":0.50600000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":7.9029999999999996,"5min":7.9029999999999996,"15min":7.9029999999999996},"last":0.72799999999999998}]},{"osd":8,"last update":"Mon Dec 11 17:44:40 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.752,"15min":0.751},"min":{"1min":0.54900000000000004,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.002,"5min":1.157,"15min":1.157},"last":0.628},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.69899999999999995,"15min":0.69699999999999995},"min":{"1min":0.495,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.97399999999999998,"5min":1.028,"15min":1.028},"last":0.70499999999999996}]},{"osd":9,"last update":"Mon Dec 11 17:44:57 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.74099999999999999,"15min":0.752},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.92400000000000004,"5min":1.371,"15min":1.371},"last":0.496},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.76000000000000001,"15min":0.77400000000000002},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.92700000000000005,"5min":1.133,"15min":1.133},"last":0.45700000000000002}]},{"osd":10,"last update":"Mon Dec 11 17:45:21 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.84399999999999997,"15min":0.83599999999999997},"min":{"1min":0.60099999999999998,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":2.4900000000000002,"5min":2.4900000000000002,"15min":2.4900000000000002},"last":0.61699999999999999},{"interface":"front","average":{"1min":1.222,"5min":0.93000000000000005,"15min":0.88200000000000001},"min":{"1min":0.626,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":7.931,"5min":7.931,"15min":7.931},"last":0.68999999999999995}]},{"osd":11,"last update":"Mon Dec 11 17:45:23 2023","interfaces":[{"interface":"back","average":{"1min":1.228,"5min":1.228,"15min":1.228},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":7.9610000000000003,"5min":7.9610000000000003,"15min":7.9610000000000003},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":2.7280000000000002,"5min":2.7280000000000002,"15min":2.7280000000000002},"last":0.77700000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574917,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1032192,"data_stored":540945,"data_compressed":17682,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 11 17:45:15 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.53400000000000003,"15min":0.49299999999999999},"min":{"1min":0.33600000000000002,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":4.3520000000000003,"5min":4.3520000000000003,"15min":4.3520000000000003},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.57199999999999995,"15min":0.52600000000000002},"min":{"1min":0.35799999999999998,"5min":0.308,"15min":0.308},"max":{"1min":4.3739999999999997,"5min":4.3739999999999997,"15min":4.3739999999999997},"last":0.48599999999999999}]},{"osd":2,"last update":"Mon Dec 11 17:44:30 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.54800000000000004,"15min":0.51800000000000002},"min":{"1min":0.34799999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":1.4690000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.57099999999999995,"15min":0.54200000000000004},"min":{"1min":0.41699999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.4530000000000001,"5min":1.4530000000000001,"15min":1.4530000000000001},"last":0.82999999999999996}]},{"osd":3,"last update":"Mon Dec 11 17:44:56 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.52600000000000002,"15min":0.51800000000000002},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.83799999999999997,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.54600000000000004,"15min":0.51900000000000002},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.81699999999999995,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.60299999999999998}]},{"osd":4,"last update":"Mon Dec 11 17:45:15 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.45100000000000001,"15min":0.41099999999999998},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.83499999999999996,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.45900000000000002,"15min":0.40600000000000003},"min":{"1min":0.26900000000000002,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.80300000000000005,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.64100000000000001}]},{"osd":5,"last update":"Mon Dec 11 17:44:36 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.59899999999999998,"15min":0.56299999999999994},"min":{"1min":0.41699999999999998,"5min":0.37,"15min":0.37},"max":{"1min":1.5109999999999999,"5min":1.5109999999999999,"15min":1.5109999999999999},"last":0.623},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.60699999999999998,"15min":0.57599999999999996},"min":{"1min":0.53900000000000003,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.411,"5min":1.411,"15min":1.411},"last":0.78500000000000003}]},{"osd":6,"last update":"Mon Dec 11 17:44:56 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.59799999999999998,"15min":0.58799999999999997},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.81100000000000005,"5min":1.014,"15min":1.014},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.622,"15min":0.60999999999999999},"min":{"1min":0.40699999999999997,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.89600000000000002,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.75600000000000001}]},{"osd":7,"last update":"Mon Dec 11 17:45:15 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.59699999999999998,"15min":0.58399999999999996},"min":{"1min":0.30399999999999999,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.97699999999999998,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.59599999999999997,"15min":0.59099999999999997},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.0409999999999999,"5min":1.379,"15min":1.379},"last":0.73599999999999999}]},{"osd":8,"last update":"Mon Dec 11 17:44:36 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.71699999999999997,"15min":0.70699999999999996},"min":{"1min":0.48699999999999999,"5min":0.433,"15min":0.433},"max":{"1min":1.5349999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.755,"5min":0.69399999999999995,"15min":0.68400000000000005},"min":{"1min":0.48999999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.4259999999999999,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":0.80300000000000005}]},{"osd":9,"last update":"Mon Dec 11 17:45:01 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.754,"15min":0.73099999999999998},"min":{"1min":0.53300000000000003,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":4.4690000000000003,"5min":4.4690000000000003,"15min":4.4690000000000003},"last":0.872},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.73599999999999999,"15min":0.73199999999999998},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.97799999999999998,"5min":1.038,"15min":1.038},"last":0.81599999999999995}]},{"osd":10,"last update":"Mon Dec 11 17:45:24 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.70199999999999996,"15min":0.69399999999999995},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.92700000000000005,"5min":1.006,"15min":1.006},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.74399999999999999,"15min":0.73799999999999999},"min":{"1min":0.47399999999999998,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.90200000000000002}]},{"osd":11,"last update":"Mon Dec 11 17:45:20 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.92700000000000005}]}]},{"osd":0,"up_from":8,"seq":34359738441,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":983040,"data_stored":480416,"data_compressed":17682,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Dec 11 17:45:14 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.54500000000000004,"15min":0.51400000000000001},"min":{"1min":0.38600000000000001,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":2.327,"5min":2.327,"15min":2.327},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.55600000000000005,"15min":0.501},"min":{"1min":0.34699999999999998,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.4890000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.47899999999999998}]},{"osd":2,"last update":"Mon Dec 11 17:44:38 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.56799999999999995,"15min":0.51500000000000001},"min":{"1min":0.38200000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":4.1029999999999998,"5min":4.1029999999999998,"15min":4.1029999999999998},"last":0.627},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.55100000000000005,"15min":0.51100000000000001},"min":{"1min":0.33400000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.746,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.66900000000000004}]},{"osd":3,"last update":"Mon Dec 11 17:45:24 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":3.3029999999999999,"5min":3.3029999999999999,"15min":3.3029999999999999},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.435}]},{"osd":4,"last update":"Mon Dec 11 17:45:24 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.55900000000000005,"15min":0.52700000000000002},"min":{"1min":0.434,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.28,"5min":1.28,"15min":1.28},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.58899999999999997,"15min":0.55200000000000005},"min":{"1min":0.41999999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":2.5019999999999998,"5min":2.5019999999999998,"15min":2.5019999999999998},"last":0.41299999999999998}]},{"osd":5,"last update":"Mon Dec 11 17:44:38 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.60599999999999998,"15min":0.58799999999999997},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":2.399,"5min":2.399,"15min":2.399},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.61299999999999999,"15min":0.59599999999999997},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.82699999999999996,"5min":0.995,"15min":0.995},"last":0.70799999999999996}]},{"osd":6,"last update":"Mon Dec 11 17:44:57 2023","interfaces":[{"interface":"back","average":{"1min":1.3999999999999999,"5min":0.68999999999999995,"15min":0.56000000000000005},"min":{"1min":0.32900000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":16.738,"5min":16.738,"15min":16.738},"last":0.61799999999999999},{"interface":"front","average":{"1min":1.3029999999999999,"5min":0.74199999999999999,"15min":0.63400000000000001},"min":{"1min":0.379,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":12.920999999999999,"5min":12.920999999999999,"15min":12.920999999999999},"last":0.60299999999999998}]},{"osd":7,"last update":"Mon Dec 11 17:45:24 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.63900000000000001}]},{"osd":8,"last update":"Mon Dec 11 17:44:38 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.70299999999999996,"15min":0.68999999999999995},"min":{"1min":0.43099999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":3.3450000000000002,"5min":3.3450000000000002,"15min":3.3450000000000002},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.76100000000000001,"15min":0.76900000000000002},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.054,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.72499999999999998}]},{"osd":9,"last update":"Mon Dec 11 17:45:00 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.65700000000000003,"15min":0.67200000000000004},"min":{"1min":0.39500000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.84999999999999998,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.76100000000000001,"15min":0.76500000000000001},"min":{"1min":0.49099999999999999,"5min":0.443,"15min":0.443},"max":{"1min":1.421,"5min":1.421,"15min":1.421},"last":0.51300000000000001}]},{"osd":10,"last update":"Mon Dec 11 17:45:24 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.80000000000000004,"15min":0.79800000000000004},"min":{"1min":0.59599999999999997,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.012,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.71799999999999997,"15min":0.70999999999999996},"min":{"1min":0.60999999999999999,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.3300000000000001,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.45400000000000001}]},{"osd":11,"last update":"Mon Dec 11 17:44:42 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":2.456,"5min":2.456,"15min":2.456},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.107,"5min":1.107,"15min":1.107},"last":0.69099999999999995}]}]},{"osd":2,"up_from":18,"seq":77309411393,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1040384,"data_stored":540662,"data_compressed":17682,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 11 17:44:34 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.52900000000000003,"15min":0.502},"min":{"1min":0.32100000000000001,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.84899999999999998,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.51800000000000002,"15min":0.50800000000000001},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.79200000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.42499999999999999}]},{"osd":1,"last update":"Mon Dec 11 17:44:34 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.54100000000000004,"15min":0.51900000000000002},"min":{"1min":0.38900000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.56999999999999995,"15min":0.54100000000000004},"min":{"1min":0.35399999999999998,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.58899999999999997}]},{"osd":3,"last update":"Mon Dec 11 17:45:01 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.57299999999999995,"15min":0.54200000000000004},"min":{"1min":0.40100000000000002,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.88800000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.55000000000000004,"15min":0.53000000000000003},"min":{"1min":0.34399999999999997,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.55900000000000005}]},{"osd":4,"last update":"Mon Dec 11 17:45:22 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.623}]},{"osd":5,"last update":"Mon Dec 11 17:44:34 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.46800000000000003,"15min":0.44600000000000001},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.46000000000000002,"15min":0.441},"min":{"1min":0.28299999999999997,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.66300000000000003}]},{"osd":6,"last update":"Mon Dec 11 17:44:52 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.66600000000000004,"15min":0.64000000000000001},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.626,"15min":0.61099999999999999},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.745}]},{"osd":7,"last update":"Mon Dec 11 17:45:24 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.86499999999999999}]},{"osd":8,"last update":"Mon Dec 11 17:44:40 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.58199999999999996,"15min":0.57499999999999996},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.64600000000000002,"15min":0.64100000000000001},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.94599999999999995}]},{"osd":9,"last update":"Mon Dec 11 17:45:01 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.71899999999999997,"15min":0.70699999999999996},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.873},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.754,"15min":0.74399999999999999},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.88700000000000001}]},{"osd":10,"last update":"Mon Dec 11 17:45:22 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.71099999999999997,"15min":0.69199999999999995},"min":{"1min":0.66700000000000004,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.76000000000000001,"15min":0.73699999999999999},"min":{"1min":0.70299999999999996,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.91300000000000003}]},{"osd":11,"last update":"Mon Dec 11 17:45:22 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.93200000000000005},{"interface":"front","average":{"1min":0.877,"5min":0.877,"15min":0.877},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.97199999999999998}]}]},{"osd":3,"up_from":25,"seq":107374182459,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":966656,"data_stored":478891,"data_compressed":17682,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 11 17:44:57 2023","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.38600000000000001,"15min":0.35799999999999998},"min":{"1min":0.26200000000000001,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":0.56599999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.433,"15min":0.42099999999999999},"min":{"1min":0.30299999999999999,"5min":0.251,"15min":0.251},"max":{"1min":0.623,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.44700000000000001}]},{"osd":1,"last update":"Mon Dec 11 17:44:57 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.58399999999999996,"15min":0.56000000000000005},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.88400000000000001,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.58099999999999996,"15min":0.56699999999999995},"min":{"1min":0.36399999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.89000000000000001,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.61799999999999999}]},{"osd":2,"last update":"Mon Dec 11 17:44:57 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.60799999999999998,"15min":0.60399999999999998},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.77900000000000003,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.60599999999999998,"15min":0.60199999999999998},"min":{"1min":0.39000000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.92700000000000005,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.66200000000000003}]},{"osd":4,"last update":"Mon Dec 11 17:45:21 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.60499999999999998,"15min":0.57199999999999995},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.3580000000000001,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.56200000000000006,"15min":0.55400000000000005},"min":{"1min":0.45000000000000001,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":0.54300000000000004}]},{"osd":5,"last update":"Mon Dec 11 17:44:35 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.61299999999999999,"15min":0.58899999999999997},"min":{"1min":0.432,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.64100000000000001,"15min":0.61799999999999999},"min":{"1min":0.41099999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.93100000000000005,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.78400000000000003}]},{"osd":6,"last update":"Mon Dec 11 17:44:57 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.48499999999999999,"15min":0.46800000000000003},"min":{"1min":0.33900000000000002,"5min":0.316,"15min":0.316},"max":{"1min":0.65000000000000002,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.59599999999999997,"15min":0.55900000000000005},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.88100000000000001,"5min":1.163,"15min":1.163},"last":0.67600000000000005}]},{"osd":7,"last update":"Mon Dec 11 17:45:15 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68100000000000005,"15min":0.65300000000000002},"min":{"1min":0.46300000000000002,"5min":0.432,"15min":0.432},"max":{"1min":0.997,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.69799999999999995,"15min":0.67400000000000004},"min":{"1min":0.51700000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.304,"5min":1.304,"15min":1.304},"last":0.74099999999999999}]},{"osd":8,"last update":"Mon Dec 11 17:43:38 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.66900000000000004}]},{"osd":9,"last update":"Mon Dec 11 17:44:57 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.78300000000000003,"15min":0.80700000000000005},"min":{"1min":0.41199999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.91200000000000003,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.69899999999999995,"15min":0.72199999999999998},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.751,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.60499999999999998}]},{"osd":10,"last update":"Mon Dec 11 17:45:21 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.81499999999999995,"15min":0.83499999999999996},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.0940000000000001,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.80800000000000005,"15min":0.82399999999999995},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.3320000000000001,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.80700000000000005}]},{"osd":11,"last update":"Mon Dec 11 17:45:21 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.81299999999999994}]}]},{"osd":4,"up_from":32,"seq":137438953528,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":970752,"data_stored":484067,"data_compressed":17682,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 11 17:44:15 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.52800000000000002,"15min":0.51600000000000001},"min":{"1min":0.36899999999999999,"5min":0.31,"15min":0.31},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.51800000000000002,"15min":0.50700000000000001},"min":{"1min":0.35599999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.34399999999999997}]},{"osd":1,"last update":"Mon Dec 11 17:45:18 2023","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.39700000000000002,"15min":0.38700000000000001},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.57399999999999995,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.40100000000000002,"15min":0.377},"min":{"1min":0.26600000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.54900000000000004,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.38800000000000001}]},{"osd":2,"last update":"Mon Dec 11 17:45:18 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.58799999999999997,"15min":0.56000000000000005},"min":{"1min":0.442,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.91900000000000004,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.58299999999999996,"15min":0.57899999999999996},"min":{"1min":0.40100000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.81000000000000005,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.54000000000000004}]},{"osd":3,"last update":"Mon Dec 11 17:45:18 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.60299999999999998,"15min":0.58499999999999996},"min":{"1min":0.42799999999999999,"5min":0.372,"15min":0.372},"max":{"1min":0.76600000000000001,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.60399999999999998,"15min":0.58999999999999997},"min":{"1min":0.36199999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.79800000000000004,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.63600000000000001}]},{"osd":5,"last update":"Mon Dec 11 17:44:36 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.59499999999999997,"15min":0.58799999999999997},"min":{"1min":0.39500000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.504},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.57899999999999996,"15min":0.57399999999999995},"min":{"1min":0.38,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.82699999999999996,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.52100000000000002}]},{"osd":6,"last update":"Mon Dec 11 17:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63400000000000001,"15min":0.62},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.877,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.65200000000000002,"15min":0.63800000000000001},"min":{"1min":0.436,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.83999999999999997,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.58899999999999997}]},{"osd":7,"last update":"Mon Dec 11 17:45:18 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.53200000000000003,"15min":0.53600000000000003},"min":{"1min":0.29899999999999999,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.745,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.55500000000000005,"15min":0.53900000000000003},"min":{"1min":0.39800000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.57299999999999995}]},{"osd":8,"last update":"Mon Dec 11 17:44:39 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.68400000000000005,"15min":0.67800000000000005},"min":{"1min":0.46200000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.66100000000000003,"15min":0.65500000000000003},"min":{"1min":0.44700000000000001,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.88300000000000001,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.67000000000000004}]},{"osd":9,"last update":"Mon Dec 11 17:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.69799999999999995,"15min":0.69599999999999995},"min":{"1min":0.52200000000000002,"5min":0.502,"15min":0.502},"max":{"1min":0.90900000000000003,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.745,"5min":0.75700000000000001,"15min":0.75900000000000001},"min":{"1min":0.54200000000000004,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.97799999999999998,"5min":1.026,"15min":1.026},"last":0.74399999999999999}]},{"osd":10,"last update":"Mon Dec 11 17:45:18 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.71599999999999997,"15min":0.71999999999999997},"min":{"1min":0.51500000000000001,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.86299999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.624},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.752,"15min":0.76200000000000001},"min":{"1min":0.50800000000000001,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.95099999999999996,"5min":0.997,"15min":0.997},"last":0.77500000000000002}]},{"osd":11,"last update":"Mon Dec 11 17:45:24 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.75900000000000001}]}]},{"osd":5,"up_from":37,"seq":158913790003,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1036288,"data_stored":1075267,"data_compressed":53447,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Dec 11 17:44:31 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.501,"15min":0.497},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.66000000000000003,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.53000000000000003,"15min":0.52200000000000002},"min":{"1min":0.36599999999999999,"5min":0.314,"15min":0.314},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.57499999999999996}]},{"osd":1,"last update":"Mon Dec 11 17:44:31 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53200000000000003,"15min":0.52300000000000002},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.79600000000000004,"5min":0.872,"15min":0.872},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.54100000000000004,"15min":0.53300000000000003},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.71799999999999997,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.78900000000000003}]},{"osd":2,"last update":"Mon Dec 11 17:44:31 2023","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.375,"15min":0.36399999999999999},"min":{"1min":0.28599999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.59899999999999998,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.377,"5min":0.40600000000000003,"15min":0.40400000000000003},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.57199999999999995,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.61099999999999999}]},{"osd":3,"last update":"Mon Dec 11 17:44:31 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.54700000000000004,"15min":0.53300000000000003},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.67000000000000004,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.56899999999999995,"15min":0.55800000000000005},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.78700000000000003,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.68700000000000006}]},{"osd":4,"last update":"Mon Dec 11 17:44:31 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.53600000000000003,"15min":0.52600000000000002},"min":{"1min":0.39200000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.71199999999999997,"5min":0.752,"15min":0.752},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58299999999999996,"15min":0.57199999999999995},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.88700000000000001,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.86399999999999999}]},{"osd":6,"last update":"Mon Dec 11 17:44:57 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.63200000000000001,"15min":0.61799999999999999},"min":{"1min":0.38,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.87,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68000000000000005,"15min":0.66800000000000004},"min":{"1min":0.56000000000000005,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.871,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.80800000000000005}]},{"osd":7,"last update":"Mon Dec 11 17:45:15 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.64100000000000001,"15min":0.63300000000000001},"min":{"1min":0.46300000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.61899999999999999,"15min":0.61299999999999999},"min":{"1min":0.47599999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.83999999999999997,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.876}]},{"osd":8,"last update":"Mon Dec 11 17:44:36 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.59199999999999997,"15min":0.60299999999999998},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.79500000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.628,"5min":0.64300000000000002,"15min":0.64500000000000002},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.80200000000000005,"5min":1.012,"15min":1.012},"last":0.83599999999999997}]},{"osd":9,"last update":"Mon Dec 11 17:45:00 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.69999999999999996,"15min":0.70399999999999996},"min":{"1min":0.56200000000000006,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.83399999999999996,"5min":1.246,"15min":1.246},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.75600000000000001,"15min":0.75800000000000001},"min":{"1min":0.52000000000000002,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.95999999999999996,"5min":1.218,"15min":1.218},"last":0.75800000000000001}]},{"osd":10,"last update":"Mon Dec 11 17:45:17 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.67200000000000004,"15min":0.66000000000000003},"min":{"1min":0.50800000000000001,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.748,"5min":0.69299999999999995,"15min":0.68400000000000005},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.89700000000000002,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.90900000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34126,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34126,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35765,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35765,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35765,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35765,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":62007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":61724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":62963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-11T17:45:33.342 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2023-12-11T17:45:33.342 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-12-11T17:45:33.342 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2023-12-11T17:45:33.343 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell --fsid d0a07eaa-984b-11ee-95a3-87774f69a715 -- ceph health --format=json 2023-12-11T17:45:33.645 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:45:33 smithi181 ceph-mon[103627]: pgmap v297: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-12-11T17:45:33.645 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:45:33 smithi181 ceph-mon[103627]: from='client.15072 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-11T17:45:33.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:45:33 smithi059 ceph-mon[98847]: pgmap v297: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-12-11T17:45:33.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:45:33 smithi059 ceph-mon[98847]: from='client.15072 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-11T17:45:33.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:45:33 smithi115 ceph-mon[102748]: pgmap v297: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-12-11T17:45:33.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:45:33 smithi115 ceph-mon[102748]: from='client.15072 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-11T17:45:33.948 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/mon.a/config 2023-12-11T17:45:35.252 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:35.247+0000 7f469e924700 1 -- 172.21.15.59:0/3342616366 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4698069930 msgr2=0x7f469810e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:35.252 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:35.247+0000 7f469e924700 1 --2- 172.21.15.59:0/3342616366 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4698069930 0x7f469810e420 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f469000b0d0 tx=0x7f469000b3e0 comp rx=0 tx=0).stop 2023-12-11T17:45:35.252 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:35.247+0000 7f469e924700 1 -- 172.21.15.59:0/3342616366 shutdown_connections 2023-12-11T17:45:35.252 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:35.247+0000 7f469e924700 1 --2- 172.21.15.59:0/3342616366 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4698069930 0x7f469810e420 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:35.252 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:35.247+0000 7f469e924700 1 --2- 172.21.15.59:0/3342616366 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4698107620 0x7f4698107a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:35.253 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:35.247+0000 7f469e924700 1 --2- 172.21.15.59:0/3342616366 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4698068f70 0x7f46980693f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:35.253 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:35.247+0000 7f469e924700 1 -- 172.21.15.59:0/3342616366 >> 172.21.15.59:0/3342616366 conn(0x7f4698076570 msgr2=0x7f4698076980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:45:35.253 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:35.247+0000 7f469e924700 1 -- 172.21.15.59:0/3342616366 shutdown_connections 2023-12-11T17:45:35.253 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:35.247+0000 7f469e924700 1 -- 172.21.15.59:0/3342616366 wait complete. 2023-12-11T17:45:35.253 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:35.248+0000 7f469e924700 1 Processor -- start 2023-12-11T17:45:35.253 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:35.248+0000 7f469e924700 1 -- start start 2023-12-11T17:45:35.253 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:35.248+0000 7f469e924700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4698068f70 0x7f4698100600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:35.253 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:35.248+0000 7f469e924700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4698069930 0x7f4698100b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:35.253 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:35.248+0000 7f469e924700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4698107620 0x7f4698101080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:35.253 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:35.248+0000 7f469e924700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46981a5ed0 con 0x7f4698069930 2023-12-11T17:45:35.253 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:35.248+0000 7f469e924700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46981a6040 con 0x7f4698107620 2023-12-11T17:45:35.253 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:35.248+0000 7f469e924700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46981a6200 con 0x7f4698068f70 2023-12-11T17:45:35.253 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:35.249+0000 7f46977fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4698069930 0x7f4698100b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:35.253 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:35.249+0000 7f46977fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4698069930 0x7f4698100b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33906/0 (socket says 172.21.15.59:33906) 2023-12-11T17:45:35.254 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:35.249+0000 7f46977fe700 1 -- 172.21.15.59:0/4115832496 learned_addr learned my addr 172.21.15.59:0/4115832496 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:45:35.254 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:35.249+0000 7f4697fff700 1 --2- 172.21.15.59:0/4115832496 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4698068f70 0x7f4698100600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:35.254 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:35.249+0000 7f469cec1700 1 --2- 172.21.15.59:0/4115832496 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4698107620 0x7f4698101080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:35.254 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:35.249+0000 7f46977fe700 1 -- 172.21.15.59:0/4115832496 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4698068f70 msgr2=0x7f4698100600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:35.254 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:35.249+0000 7f46977fe700 1 --2- 172.21.15.59:0/4115832496 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4698068f70 0x7f4698100600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:35.254 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:35.249+0000 7f46977fe700 1 -- 172.21.15.59:0/4115832496 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4698107620 msgr2=0x7f4698101080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:35.254 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:35.249+0000 7f46977fe700 1 --2- 172.21.15.59:0/4115832496 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4698107620 0x7f4698101080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:35.254 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:35.249+0000 7f46977fe700 1 -- 172.21.15.59:0/4115832496 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4690009d00 con 0x7f4698069930 2023-12-11T17:45:35.254 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:35.249+0000 7f46977fe700 1 --2- 172.21.15.59:0/4115832496 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4698069930 0x7f4698100b40 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f468c00d980 tx=0x7f468c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:35.254 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:35.249+0000 7f46957fa700 1 -- 172.21.15.59:0/4115832496 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f468c004220 con 0x7f4698069930 2023-12-11T17:45:35.254 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:35.249+0000 7f469e924700 1 -- 172.21.15.59:0/4115832496 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46981a64e0 con 0x7f4698069930 2023-12-11T17:45:35.254 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:35.249+0000 7f46957fa700 1 -- 172.21.15.59:0/4115832496 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f468c007640 con 0x7f4698069930 2023-12-11T17:45:35.254 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:35.249+0000 7f46957fa700 1 -- 172.21.15.59:0/4115832496 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f468c005610 con 0x7f4698069930 2023-12-11T17:45:35.254 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:35.249+0000 7f469e924700 1 -- 172.21.15.59:0/4115832496 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46981a6a30 con 0x7f4698069930 2023-12-11T17:45:35.255 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:35.250+0000 7f469e924700 1 -- 172.21.15.59:0/4115832496 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4698109a20 con 0x7f4698069930 2023-12-11T17:45:35.255 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:35.251+0000 7f46957fa700 1 -- 172.21.15.59:0/4115832496 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f468c004750 con 0x7f4698069930 2023-12-11T17:45:35.256 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:35.252+0000 7f46957fa700 1 --2- 172.21.15.59:0/4115832496 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f46800a1510 0x7f46800a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:35.256 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:35.252+0000 7f4697fff700 1 --2- 172.21.15.59:0/4115832496 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f46800a1510 0x7f46800a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:35.256 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:35.252+0000 7f46957fa700 1 -- 172.21.15.59:0/4115832496 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f468c0cd4c0 con 0x7f4698069930 2023-12-11T17:45:35.256 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:35.252+0000 7f4697fff700 1 --2- 172.21.15.59:0/4115832496 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f46800a1510 0x7f46800a39d0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f4688005950 tx=0x7f468800a300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:35.258 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:35.255+0000 7f46957fa700 1 -- 172.21.15.59:0/4115832496 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f468c0966c0 con 0x7f4698069930 2023-12-11T17:45:35.502 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:35.499+0000 7f469e924700 1 -- 172.21.15.59:0/4115832496 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f4698101a90 con 0x7f4698069930 2023-12-11T17:45:35.502 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:35.499+0000 7f46957fa700 1 -- 172.21.15.59:0/4115832496 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f468c099ce0 con 0x7f4698069930 2023-12-11T17:45:35.503 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-12-11T17:45:35.503 INFO:teuthology.orchestra.run.smithi059.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-12-11T17:45:35.504 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:35.501+0000 7f469e924700 1 -- 172.21.15.59:0/4115832496 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f46800a1510 msgr2=0x7f46800a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:35.504 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:35.501+0000 7f469e924700 1 --2- 172.21.15.59:0/4115832496 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f46800a1510 0x7f46800a39d0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f4688005950 tx=0x7f468800a300 comp rx=0 tx=0).stop 2023-12-11T17:45:35.504 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:35.501+0000 7f469e924700 1 -- 172.21.15.59:0/4115832496 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4698069930 msgr2=0x7f4698100b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:35.504 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:35.501+0000 7f469e924700 1 --2- 172.21.15.59:0/4115832496 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4698069930 0x7f4698100b40 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f468c00d980 tx=0x7f468c00dc90 comp rx=0 tx=0).stop 2023-12-11T17:45:35.504 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:35.502+0000 7f469e924700 1 -- 172.21.15.59:0/4115832496 shutdown_connections 2023-12-11T17:45:35.504 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:35.502+0000 7f469e924700 1 --2- 172.21.15.59:0/4115832496 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f46800a1510 0x7f46800a39d0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:35.504 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:35.502+0000 7f469e924700 1 --2- 172.21.15.59:0/4115832496 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4698107620 0x7f4698101080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:35.504 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:35.502+0000 7f469e924700 1 --2- 172.21.15.59:0/4115832496 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4698068f70 0x7f4698100600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:35.505 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:35.502+0000 7f469e924700 1 --2- 172.21.15.59:0/4115832496 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4698069930 0x7f4698100b40 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:35.505 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:35.502+0000 7f469e924700 1 -- 172.21.15.59:0/4115832496 >> 172.21.15.59:0/4115832496 conn(0x7f4698076570 msgr2=0x7f46980ff4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:45:35.505 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:35.502+0000 7f469e924700 1 -- 172.21.15.59:0/4115832496 shutdown_connections 2023-12-11T17:45:35.505 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:35.502+0000 7f469e924700 1 -- 172.21.15.59:0/4115832496 wait complete. 2023-12-11T17:45:35.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:45:35 smithi115 ceph-mon[102748]: pgmap v298: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-12-11T17:45:35.891 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:45:35 smithi059 ceph-mon[98847]: pgmap v298: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-12-11T17:45:35.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:45:35 smithi181 ceph-mon[103627]: pgmap v298: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-12-11T17:45:35.914 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2023-12-11T17:45:35.915 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-11T17:45:36.046 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.042+0000 7f03f22f3700 1 -- 172.21.15.59:0/1474603570 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f03ec101f10 msgr2=0x7f03ec1087c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:36.047 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.042+0000 7f03f22f3700 1 --2- 172.21.15.59:0/1474603570 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f03ec101f10 0x7f03ec1087c0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f03e400b0d0 tx=0x7f03e400b3e0 comp rx=0 tx=0).stop 2023-12-11T17:45:36.047 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.042+0000 7f03f22f3700 1 -- 172.21.15.59:0/1474603570 shutdown_connections 2023-12-11T17:45:36.047 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.042+0000 7f03f22f3700 1 --2- 172.21.15.59:0/1474603570 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f03ec101f10 0x7f03ec1087c0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:36.047 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.042+0000 7f03f22f3700 1 --2- 172.21.15.59:0/1474603570 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f03ec100380 0x7f03ec1007a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:36.047 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.042+0000 7f03f22f3700 1 --2- 172.21.15.59:0/1474603570 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f03ec101570 0x7f03ec1019d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:36.047 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.042+0000 7f03f22f3700 1 -- 172.21.15.59:0/1474603570 >> 172.21.15.59:0/1474603570 conn(0x7f03ec0fb8f0 msgr2=0x7f03ec0fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:45:36.047 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.042+0000 7f03f22f3700 1 -- 172.21.15.59:0/1474603570 shutdown_connections 2023-12-11T17:45:36.047 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.042+0000 7f03f22f3700 1 -- 172.21.15.59:0/1474603570 wait complete. 2023-12-11T17:45:36.047 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.043+0000 7f03f22f3700 1 Processor -- start 2023-12-11T17:45:36.047 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.043+0000 7f03f22f3700 1 -- start start 2023-12-11T17:45:36.047 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.043+0000 7f03f22f3700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f03ec100380 0x7f03ec197540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:36.048 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.043+0000 7f03f22f3700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f03ec101570 0x7f03ec197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:36.048 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.043+0000 7f03f22f3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f03ec101f10 0x7f03ec19eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:36.048 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.043+0000 7f03f22f3700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03ec19f070 con 0x7f03ec100380 2023-12-11T17:45:36.048 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.043+0000 7f03f22f3700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03ec19f1b0 con 0x7f03ec101570 2023-12-11T17:45:36.048 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.043+0000 7f03f22f3700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03ec19f340 con 0x7f03ec101f10 2023-12-11T17:45:36.048 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.043+0000 7f03ebfff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f03ec100380 0x7f03ec197540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:36.048 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.043+0000 7f03ebfff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f03ec100380 0x7f03ec197540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33914/0 (socket says 172.21.15.59:33914) 2023-12-11T17:45:36.048 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.043+0000 7f03ebfff700 1 -- 172.21.15.59:0/4134060425 learned_addr learned my addr 172.21.15.59:0/4134060425 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:45:36.048 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.043+0000 7f03eb7fe700 1 --2- 172.21.15.59:0/4134060425 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f03ec101570 0x7f03ec197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:36.048 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.044+0000 7f03f0890700 1 --2- 172.21.15.59:0/4134060425 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f03ec101f10 0x7f03ec19eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:36.048 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.044+0000 7f03ebfff700 1 -- 172.21.15.59:0/4134060425 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f03ec101f10 msgr2=0x7f03ec19eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:36.048 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.044+0000 7f03ebfff700 1 --2- 172.21.15.59:0/4134060425 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f03ec101f10 0x7f03ec19eaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:36.048 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.044+0000 7f03ebfff700 1 -- 172.21.15.59:0/4134060425 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f03ec101570 msgr2=0x7f03ec197a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:36.048 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.044+0000 7f03ebfff700 1 --2- 172.21.15.59:0/4134060425 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f03ec101570 0x7f03ec197a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:36.049 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.044+0000 7f03ebfff700 1 -- 172.21.15.59:0/4134060425 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03e4009d00 con 0x7f03ec100380 2023-12-11T17:45:36.049 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.044+0000 7f03ebfff700 1 --2- 172.21.15.59:0/4134060425 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f03ec100380 0x7f03ec197540 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f03dc00d980 tx=0x7f03dc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:36.049 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.044+0000 7f03e97fa700 1 -- 172.21.15.59:0/4134060425 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03dc004020 con 0x7f03ec100380 2023-12-11T17:45:36.049 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.044+0000 7f03f22f3700 1 -- 172.21.15.59:0/4134060425 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03ec19f620 con 0x7f03ec100380 2023-12-11T17:45:36.049 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.044+0000 7f03e97fa700 1 -- 172.21.15.59:0/4134060425 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f03dc009730 con 0x7f03ec100380 2023-12-11T17:45:36.049 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.044+0000 7f03e97fa700 1 -- 172.21.15.59:0/4134060425 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03dc003a30 con 0x7f03ec100380 2023-12-11T17:45:36.049 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.044+0000 7f03f22f3700 1 -- 172.21.15.59:0/4134060425 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03ec19fb30 con 0x7f03ec100380 2023-12-11T17:45:36.050 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.045+0000 7f03f22f3700 1 -- 172.21.15.59:0/4134060425 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03ec068230 con 0x7f03ec100380 2023-12-11T17:45:36.050 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.047+0000 7f03e97fa700 1 -- 172.21.15.59:0/4134060425 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f03dc0098a0 con 0x7f03ec100380 2023-12-11T17:45:36.050 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.047+0000 7f03e97fa700 1 --2- 172.21.15.59:0/4134060425 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f03d80a1510 0x7f03d80a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:36.050 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.047+0000 7f03e97fa700 1 -- 172.21.15.59:0/4134060425 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f03dc0cd260 con 0x7f03ec100380 2023-12-11T17:45:36.050 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.048+0000 7f03eb7fe700 1 --2- 172.21.15.59:0/4134060425 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f03d80a1510 0x7f03d80a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:36.050 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.048+0000 7f03eb7fe700 1 --2- 172.21.15.59:0/4134060425 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f03d80a1510 0x7f03d80a39d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f03d4005950 tx=0x7f03d400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:36.053 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.051+0000 7f03e97fa700 1 -- 172.21.15.59:0/4134060425 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f03dc096400 con 0x7f03ec100380 2023-12-11T17:45:36.259 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.256+0000 7f03f22f3700 1 -- 172.21.15.59:0/4134060425 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f03ec10bbe0 con 0x7f03ec100380 2023-12-11T17:45:36.260 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.258+0000 7f03e97fa700 1 -- 172.21.15.59:0/4134060425 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+24459 (secure 0 0 0) 0x7f03dc099a20 con 0x7f03ec100380 2023-12-11T17:45:36.261 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-12-11T17:45:36.261 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":88,"fsid":"d0a07eaa-984b-11ee-95a3-87774f69a715","created":"2023-12-11T17:37:14.526539+0000","modified":"2023-12-11T17:45:15.870614+0000","last_up_change":"2023-12-11T17:43:38.535503+0000","last_in_change":"2023-12-11T17:43:21.014888+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-11T17:40:26.596029+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-11T17:44:18.586290+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-11T17:44:19.432952+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-11T17:44:23.097308+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3c0b3218-4b2b-40f8-8bef-ed8cd5633607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":1040759398},{"type":"v1","addr":"172.21.15.59:6803","nonce":1040759398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":1040759398},{"type":"v1","addr":"172.21.15.59:6805","nonce":1040759398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":1040759398},{"type":"v1","addr":"172.21.15.59:6809","nonce":1040759398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":1040759398},{"type":"v1","addr":"172.21.15.59:6807","nonce":1040759398}]},"public_addr":"172.21.15.59:6803/1040759398","cluster_addr":"172.21.15.59:6805/1040759398","heartbeat_back_addr":"172.21.15.59:6809/1040759398","heartbeat_front_addr":"172.21.15.59:6807/1040759398","state":["exists","up"]},{"osd":1,"uuid":"2ead7578-765b-4f5a-a72c-863731ed69e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":2776354245},{"type":"v1","addr":"172.21.15.115:6801","nonce":2776354245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":2776354245},{"type":"v1","addr":"172.21.15.115:6803","nonce":2776354245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":2776354245},{"type":"v1","addr":"172.21.15.115:6807","nonce":2776354245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":2776354245},{"type":"v1","addr":"172.21.15.115:6805","nonce":2776354245}]},"public_addr":"172.21.15.115:6801/2776354245","cluster_addr":"172.21.15.115:6803/2776354245","heartbeat_back_addr":"172.21.15.115:6807/2776354245","heartbeat_front_addr":"172.21.15.115:6805/2776354245","state":["exists","up"]},{"osd":2,"uuid":"8cb1d7ec-53cc-4672-b0b6-70d73797c44f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4287675481},{"type":"v1","addr":"172.21.15.181:6801","nonce":4287675481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4287675481},{"type":"v1","addr":"172.21.15.181:6803","nonce":4287675481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4287675481},{"type":"v1","addr":"172.21.15.181:6807","nonce":4287675481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4287675481},{"type":"v1","addr":"172.21.15.181:6805","nonce":4287675481}]},"public_addr":"172.21.15.181:6801/4287675481","cluster_addr":"172.21.15.181:6803/4287675481","heartbeat_back_addr":"172.21.15.181:6807/4287675481","heartbeat_front_addr":"172.21.15.181:6805/4287675481","state":["exists","up"]},{"osd":3,"uuid":"5d64d9af-831b-4d0a-8ace-a452169351d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":219909995},{"type":"v1","addr":"172.21.15.59:6811","nonce":219909995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":219909995},{"type":"v1","addr":"172.21.15.59:6813","nonce":219909995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":219909995},{"type":"v1","addr":"172.21.15.59:6817","nonce":219909995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":219909995},{"type":"v1","addr":"172.21.15.59:6815","nonce":219909995}]},"public_addr":"172.21.15.59:6811/219909995","cluster_addr":"172.21.15.59:6813/219909995","heartbeat_back_addr":"172.21.15.59:6817/219909995","heartbeat_front_addr":"172.21.15.59:6815/219909995","state":["exists","up"]},{"osd":4,"uuid":"ef8fad28-1db5-4cba-b5f2-10e54e8e1549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":1021473826},{"type":"v1","addr":"172.21.15.115:6809","nonce":1021473826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":1021473826},{"type":"v1","addr":"172.21.15.115:6811","nonce":1021473826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":1021473826},{"type":"v1","addr":"172.21.15.115:6815","nonce":1021473826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":1021473826},{"type":"v1","addr":"172.21.15.115:6813","nonce":1021473826}]},"public_addr":"172.21.15.115:6809/1021473826","cluster_addr":"172.21.15.115:6811/1021473826","heartbeat_back_addr":"172.21.15.115:6815/1021473826","heartbeat_front_addr":"172.21.15.115:6813/1021473826","state":["exists","up"]},{"osd":5,"uuid":"4233a512-9e8f-4db8-aed2-6c3af706e589","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":2405172521},{"type":"v1","addr":"172.21.15.181:6809","nonce":2405172521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":2405172521},{"type":"v1","addr":"172.21.15.181:6811","nonce":2405172521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2405172521},{"type":"v1","addr":"172.21.15.181:6815","nonce":2405172521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":2405172521},{"type":"v1","addr":"172.21.15.181:6813","nonce":2405172521}]},"public_addr":"172.21.15.181:6809/2405172521","cluster_addr":"172.21.15.181:6811/2405172521","heartbeat_back_addr":"172.21.15.181:6815/2405172521","heartbeat_front_addr":"172.21.15.181:6813/2405172521","state":["exists","up"]},{"osd":6,"uuid":"8faab332-016e-4c77-8370-8bef95fc6f34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":815374853},{"type":"v1","addr":"172.21.15.59:6819","nonce":815374853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":815374853},{"type":"v1","addr":"172.21.15.59:6821","nonce":815374853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":815374853},{"type":"v1","addr":"172.21.15.59:6825","nonce":815374853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":815374853},{"type":"v1","addr":"172.21.15.59:6823","nonce":815374853}]},"public_addr":"172.21.15.59:6819/815374853","cluster_addr":"172.21.15.59:6821/815374853","heartbeat_back_addr":"172.21.15.59:6825/815374853","heartbeat_front_addr":"172.21.15.59:6823/815374853","state":["exists","up"]},{"osd":7,"uuid":"f20f470a-0d40-4ed3-8e0e-fd58675cf143","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":1314971870},{"type":"v1","addr":"172.21.15.115:6817","nonce":1314971870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":1314971870},{"type":"v1","addr":"172.21.15.115:6819","nonce":1314971870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":1314971870},{"type":"v1","addr":"172.21.15.115:6823","nonce":1314971870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":1314971870},{"type":"v1","addr":"172.21.15.115:6821","nonce":1314971870}]},"public_addr":"172.21.15.115:6817/1314971870","cluster_addr":"172.21.15.115:6819/1314971870","heartbeat_back_addr":"172.21.15.115:6823/1314971870","heartbeat_front_addr":"172.21.15.115:6821/1314971870","state":["exists","up"]},{"osd":8,"uuid":"eb657d32-1a95-4fd7-a293-d0a970837fc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":3954509873},{"type":"v1","addr":"172.21.15.181:6817","nonce":3954509873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":3954509873},{"type":"v1","addr":"172.21.15.181:6819","nonce":3954509873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":3954509873},{"type":"v1","addr":"172.21.15.181:6823","nonce":3954509873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":3954509873},{"type":"v1","addr":"172.21.15.181:6821","nonce":3954509873}]},"public_addr":"172.21.15.181:6817/3954509873","cluster_addr":"172.21.15.181:6819/3954509873","heartbeat_back_addr":"172.21.15.181:6823/3954509873","heartbeat_front_addr":"172.21.15.181:6821/3954509873","state":["exists","up"]},{"osd":9,"uuid":"8f7717d4-b6f2-4ec1-9b58-62feb0c9a146","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":1163095677},{"type":"v1","addr":"172.21.15.59:6827","nonce":1163095677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":1163095677},{"type":"v1","addr":"172.21.15.59:6829","nonce":1163095677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":1163095677},{"type":"v1","addr":"172.21.15.59:6833","nonce":1163095677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":1163095677},{"type":"v1","addr":"172.21.15.59:6831","nonce":1163095677}]},"public_addr":"172.21.15.59:6827/1163095677","cluster_addr":"172.21.15.59:6829/1163095677","heartbeat_back_addr":"172.21.15.59:6833/1163095677","heartbeat_front_addr":"172.21.15.59:6831/1163095677","state":["exists","up"]},{"osd":10,"uuid":"b75c179a-875d-43bc-b4c1-e8eb80fb64fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":2199610269},{"type":"v1","addr":"172.21.15.115:6825","nonce":2199610269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":2199610269},{"type":"v1","addr":"172.21.15.115:6827","nonce":2199610269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":2199610269},{"type":"v1","addr":"172.21.15.115:6831","nonce":2199610269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":2199610269},{"type":"v1","addr":"172.21.15.115:6829","nonce":2199610269}]},"public_addr":"172.21.15.115:6825/2199610269","cluster_addr":"172.21.15.115:6827/2199610269","heartbeat_back_addr":"172.21.15.115:6831/2199610269","heartbeat_front_addr":"172.21.15.115:6829/2199610269","state":["exists","up"]},{"osd":11,"uuid":"5e141dfe-8a00-4bbc-8204-2dc9be24e18d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":226506989},{"type":"v1","addr":"172.21.15.181:6825","nonce":226506989}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":226506989},{"type":"v1","addr":"172.21.15.181:6827","nonce":226506989}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":226506989},{"type":"v1","addr":"172.21.15.181:6831","nonce":226506989}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":226506989},{"type":"v1","addr":"172.21.15.181:6829","nonce":226506989}]},"public_addr":"172.21.15.181:6825/226506989","cluster_addr":"172.21.15.181:6827/226506989","heartbeat_back_addr":"172.21.15.181:6831/226506989","heartbeat_front_addr":"172.21.15.181:6829/226506989","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:39:42.687084+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:40:03.903448+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:40:24.899908+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:40:46.600251+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:41:07.690037+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:41:28.488855+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:41:49.459274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:42:10.863881+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:42:32.248391+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:42:54.112265+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:43:15.634872+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:43:36.922534+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":7}]},{"pgid":"3.28","mappings":[{"from":5,"to":6},{"from":3,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.59:6800/1259916482":"2023-12-12T17:38:14.037922+0000","172.21.15.59:0/17457774":"2023-12-12T17:38:14.037922+0000","172.21.15.59:6801/1672530462":"2023-12-12T17:37:44.773925+0000","172.21.15.59:0/1053944547":"2023-12-12T17:38:14.037922+0000","172.21.15.59:6801/1259916482":"2023-12-12T17:38:14.037922+0000","172.21.15.59:6800/1672530462":"2023-12-12T17:37:44.773925+0000","172.21.15.59:0/3570767650":"2023-12-12T17:37:44.773925+0000","172.21.15.59:0/3853462899":"2023-12-12T17:37:44.773925+0000","172.21.15.59:0/2859415380":"2023-12-12T17:38:14.037922+0000","172.21.15.59:0/2192742819":"2023-12-12T17:37:44.773925+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-11T17:45:36.263 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.260+0000 7f03f22f3700 1 -- 172.21.15.59:0/4134060425 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f03d80a1510 msgr2=0x7f03d80a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:36.263 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.260+0000 7f03f22f3700 1 --2- 172.21.15.59:0/4134060425 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f03d80a1510 0x7f03d80a39d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f03d4005950 tx=0x7f03d400b410 comp rx=0 tx=0).stop 2023-12-11T17:45:36.263 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.260+0000 7f03f22f3700 1 -- 172.21.15.59:0/4134060425 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f03ec100380 msgr2=0x7f03ec197540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:36.263 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.260+0000 7f03f22f3700 1 --2- 172.21.15.59:0/4134060425 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f03ec100380 0x7f03ec197540 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f03dc00d980 tx=0x7f03dc00dc90 comp rx=0 tx=0).stop 2023-12-11T17:45:36.263 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.260+0000 7f03f22f3700 1 -- 172.21.15.59:0/4134060425 shutdown_connections 2023-12-11T17:45:36.263 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.260+0000 7f03f22f3700 1 --2- 172.21.15.59:0/4134060425 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f03d80a1510 0x7f03d80a39d0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:36.263 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.260+0000 7f03f22f3700 1 --2- 172.21.15.59:0/4134060425 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f03ec101f10 0x7f03ec19eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:36.263 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.260+0000 7f03f22f3700 1 --2- 172.21.15.59:0/4134060425 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f03ec100380 0x7f03ec197540 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:36.263 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.260+0000 7f03f22f3700 1 --2- 172.21.15.59:0/4134060425 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f03ec101570 0x7f03ec197a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:36.263 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.260+0000 7f03f22f3700 1 -- 172.21.15.59:0/4134060425 >> 172.21.15.59:0/4134060425 conn(0x7f03ec0fb8f0 msgr2=0x7f03ec068610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:45:36.263 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.260+0000 7f03f22f3700 1 -- 172.21.15.59:0/4134060425 shutdown_connections 2023-12-11T17:45:36.264 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.260+0000 7f03f22f3700 1 -- 172.21.15.59:0/4134060425 wait complete. 2023-12-11T17:45:36.276 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-11T17:40:26.596029+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-11T17:44:18.586290+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-11T17:44:19.432952+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-12-11T17:44:23.097308+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-12-11T17:45:36.276 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-11T17:45:36.408 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.403+0000 7ff64ec6e700 1 -- 172.21.15.59:0/4187579546 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff648102790 msgr2=0x7ff648102bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:36.408 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.403+0000 7ff64ec6e700 1 --2- 172.21.15.59:0/4187579546 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff648102790 0x7ff648102bf0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7ff6340099c0 tx=0x7ff634009cd0 comp rx=0 tx=0).stop 2023-12-11T17:45:36.408 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.403+0000 7ff64ec6e700 1 -- 172.21.15.59:0/4187579546 shutdown_connections 2023-12-11T17:45:36.408 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.403+0000 7ff64ec6e700 1 --2- 172.21.15.59:0/4187579546 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff648103130 0x7ff6481099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:36.408 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.403+0000 7ff64ec6e700 1 --2- 172.21.15.59:0/4187579546 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff6481015a0 0x7ff6481019c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:36.408 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.403+0000 7ff64ec6e700 1 --2- 172.21.15.59:0/4187579546 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff648102790 0x7ff648102bf0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:36.408 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.403+0000 7ff64ec6e700 1 -- 172.21.15.59:0/4187579546 >> 172.21.15.59:0/4187579546 conn(0x7ff6480fcb10 msgr2=0x7ff6480fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:45:36.408 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.403+0000 7ff64ec6e700 1 -- 172.21.15.59:0/4187579546 shutdown_connections 2023-12-11T17:45:36.408 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.403+0000 7ff64ec6e700 1 -- 172.21.15.59:0/4187579546 wait complete. 2023-12-11T17:45:36.408 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.404+0000 7ff64ec6e700 1 Processor -- start 2023-12-11T17:45:36.409 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.404+0000 7ff64ec6e700 1 -- start start 2023-12-11T17:45:36.409 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.404+0000 7ff64ec6e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff6481015a0 0x7ff648197510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:36.409 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.404+0000 7ff64ec6e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff648102790 0x7ff648197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:36.409 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.404+0000 7ff64ec6e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff648103130 0x7ff64819ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:36.409 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.404+0000 7ff64ec6e700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff64819f040 con 0x7ff648102790 2023-12-11T17:45:36.409 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.404+0000 7ff64ec6e700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff64819f180 con 0x7ff6481015a0 2023-12-11T17:45:36.409 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.404+0000 7ff64ec6e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff64819f310 con 0x7ff648103130 2023-12-11T17:45:36.409 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.404+0000 7ff647fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff648102790 0x7ff648197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:36.409 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.404+0000 7ff647fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff648102790 0x7ff648197a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33930/0 (socket says 172.21.15.59:33930) 2023-12-11T17:45:36.409 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.404+0000 7ff647fff700 1 -- 172.21.15.59:0/2159073834 learned_addr learned my addr 172.21.15.59:0/2159073834 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:45:36.409 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.404+0000 7ff64d20b700 1 --2- 172.21.15.59:0/2159073834 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff648103130 0x7ff64819ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:36.409 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.404+0000 7ff64ca0a700 1 --2- 172.21.15.59:0/2159073834 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff6481015a0 0x7ff648197510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:36.409 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.404+0000 7ff647fff700 1 -- 172.21.15.59:0/2159073834 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff648103130 msgr2=0x7ff64819ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:36.409 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.404+0000 7ff647fff700 1 --2- 172.21.15.59:0/2159073834 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff648103130 0x7ff64819ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:36.410 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.404+0000 7ff647fff700 1 -- 172.21.15.59:0/2159073834 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff6481015a0 msgr2=0x7ff648197510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:36.410 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.404+0000 7ff647fff700 1 --2- 172.21.15.59:0/2159073834 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff6481015a0 0x7ff648197510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:36.410 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.404+0000 7ff647fff700 1 -- 172.21.15.59:0/2159073834 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6340095f0 con 0x7ff648102790 2023-12-11T17:45:36.410 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.404+0000 7ff647fff700 1 --2- 172.21.15.59:0/2159073834 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff648102790 0x7ff648197a50 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7ff6340036f0 tx=0x7ff6340047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:36.410 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.405+0000 7ff645ffb700 1 -- 172.21.15.59:0/2159073834 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff63401d070 con 0x7ff648102790 2023-12-11T17:45:36.410 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.405+0000 7ff645ffb700 1 -- 172.21.15.59:0/2159073834 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff634003f90 con 0x7ff648102790 2023-12-11T17:45:36.410 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.405+0000 7ff645ffb700 1 -- 172.21.15.59:0/2159073834 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff634022c20 con 0x7ff648102790 2023-12-11T17:45:36.410 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.405+0000 7ff64ec6e700 1 -- 172.21.15.59:0/2159073834 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff64819f590 con 0x7ff648102790 2023-12-11T17:45:36.410 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.405+0000 7ff64ec6e700 1 -- 172.21.15.59:0/2159073834 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff64819fa40 con 0x7ff648102790 2023-12-11T17:45:36.411 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.406+0000 7ff64ec6e700 1 -- 172.21.15.59:0/2159073834 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff648108a70 con 0x7ff648102790 2023-12-11T17:45:36.411 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.407+0000 7ff645ffb700 1 -- 172.21.15.59:0/2159073834 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7ff634022470 con 0x7ff648102790 2023-12-11T17:45:36.411 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.407+0000 7ff645ffb700 1 --2- 172.21.15.59:0/2159073834 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7ff6380a1590 0x7ff6380a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:36.411 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.408+0000 7ff645ffb700 1 -- 172.21.15.59:0/2159073834 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8999+0+0 (secure 0 0 0) 0x7ff6340ce6d0 con 0x7ff648102790 2023-12-11T17:45:36.411 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.408+0000 7ff64ca0a700 1 --2- 172.21.15.59:0/2159073834 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7ff6380a1590 0x7ff6380a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:36.411 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.408+0000 7ff64ca0a700 1 --2- 172.21.15.59:0/2159073834 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7ff6380a1590 0x7ff6380a3a50 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff6481025f0 tx=0x7ff63c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:36.414 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.412+0000 7ff645ffb700 1 -- 172.21.15.59:0/2159073834 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7ff634097a20 con 0x7ff648102790 2023-12-11T17:45:36.618 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.616+0000 7ff64ec6e700 1 -- 172.21.15.59:0/2159073834 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff64810cdc0 con 0x7ff648102790 2023-12-11T17:45:36.618 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.616+0000 7ff645ffb700 1 -- 172.21.15.59:0/2159073834 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff634027020 con 0x7ff648102790 2023-12-11T17:45:36.619 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2023-12-11T17:45:36.621 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.618+0000 7ff64ec6e700 1 -- 172.21.15.59:0/2159073834 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7ff6380a1590 msgr2=0x7ff6380a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:36.621 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.618+0000 7ff64ec6e700 1 --2- 172.21.15.59:0/2159073834 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7ff6380a1590 0x7ff6380a3a50 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff6481025f0 tx=0x7ff63c00b410 comp rx=0 tx=0).stop 2023-12-11T17:45:36.621 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.618+0000 7ff64ec6e700 1 -- 172.21.15.59:0/2159073834 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff648102790 msgr2=0x7ff648197a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:36.621 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.618+0000 7ff64ec6e700 1 --2- 172.21.15.59:0/2159073834 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff648102790 0x7ff648197a50 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7ff6340036f0 tx=0x7ff6340047b0 comp rx=0 tx=0).stop 2023-12-11T17:45:36.621 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.618+0000 7ff64ec6e700 1 -- 172.21.15.59:0/2159073834 shutdown_connections 2023-12-11T17:45:36.621 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.618+0000 7ff64ec6e700 1 --2- 172.21.15.59:0/2159073834 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7ff6380a1590 0x7ff6380a3a50 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:36.622 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.618+0000 7ff64ec6e700 1 --2- 172.21.15.59:0/2159073834 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff648103130 0x7ff64819ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:36.622 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.618+0000 7ff64ec6e700 1 --2- 172.21.15.59:0/2159073834 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff6481015a0 0x7ff648197510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:36.622 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.618+0000 7ff64ec6e700 1 --2- 172.21.15.59:0/2159073834 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff648102790 0x7ff648197a50 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:36.622 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.618+0000 7ff64ec6e700 1 -- 172.21.15.59:0/2159073834 >> 172.21.15.59:0/2159073834 conn(0x7ff6480fcb10 msgr2=0x7ff648105bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:45:36.622 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.619+0000 7ff64ec6e700 1 -- 172.21.15.59:0/2159073834 shutdown_connections 2023-12-11T17:45:36.622 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.619+0000 7ff64ec6e700 1 -- 172.21.15.59:0/2159073834 wait complete. 2023-12-11T17:45:36.633 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:45:36 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/4115832496' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-12-11T17:45:36.633 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:45:36 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/4134060425' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-11T17:45:36.634 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-11T17:45:36.766 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.762+0000 7fcdf9096700 1 -- 172.21.15.59:0/85144641 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcdf40ff860 msgr2=0x7fcdf40ffc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:36.766 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.762+0000 7fcdf9096700 1 --2- 172.21.15.59:0/85144641 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcdf40ff860 0x7fcdf40ffc80 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fcde4009990 tx=0x7fcde4009ca0 comp rx=0 tx=0).stop 2023-12-11T17:45:36.766 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.763+0000 7fcdf9096700 1 -- 172.21.15.59:0/85144641 shutdown_connections 2023-12-11T17:45:36.766 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.763+0000 7fcdf9096700 1 --2- 172.21.15.59:0/85144641 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcdf4103270 0x7fcdf4109b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:36.766 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.763+0000 7fcdf9096700 1 --2- 172.21.15.59:0/85144641 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcdf40ff860 0x7fcdf40ffc80 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:36.766 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.763+0000 7fcdf9096700 1 --2- 172.21.15.59:0/85144641 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcdf41028d0 0x7fcdf4102d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:36.766 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.763+0000 7fcdf9096700 1 -- 172.21.15.59:0/85144641 >> 172.21.15.59:0/85144641 conn(0x7fcdf40fb170 msgr2=0x7fcdf40fd5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:45:36.766 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.763+0000 7fcdf9096700 1 -- 172.21.15.59:0/85144641 shutdown_connections 2023-12-11T17:45:36.766 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.763+0000 7fcdf9096700 1 -- 172.21.15.59:0/85144641 wait complete. 2023-12-11T17:45:36.766 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.763+0000 7fcdf9096700 1 Processor -- start 2023-12-11T17:45:36.767 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.763+0000 7fcdf9096700 1 -- start start 2023-12-11T17:45:36.769 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.764+0000 7fcdf9096700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcdf40ff860 0x7fcdf4197530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:36.769 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.764+0000 7fcdf9096700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcdf41028d0 0x7fcdf4197a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:36.769 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.764+0000 7fcdf9096700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcdf4103270 0x7fcdf419ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:36.769 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.764+0000 7fcdf9096700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf419f060 con 0x7fcdf4103270 2023-12-11T17:45:36.769 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.764+0000 7fcdf9096700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf419f1a0 con 0x7fcdf40ff860 2023-12-11T17:45:36.769 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.764+0000 7fcdf9096700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf419f330 con 0x7fcdf41028d0 2023-12-11T17:45:36.769 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.764+0000 7fcdf359e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcdf4103270 0x7fcdf419ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:36.769 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.764+0000 7fcdf359e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcdf4103270 0x7fcdf419ea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33946/0 (socket says 172.21.15.59:33946) 2023-12-11T17:45:36.769 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.764+0000 7fcdf359e700 1 -- 172.21.15.59:0/2877482521 learned_addr learned my addr 172.21.15.59:0/2877482521 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:45:36.769 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.764+0000 7fcdf2d9d700 1 --2- 172.21.15.59:0/2877482521 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcdf40ff860 0x7fcdf4197530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:36.769 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.764+0000 7fcdf259c700 1 --2- 172.21.15.59:0/2877482521 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcdf41028d0 0x7fcdf4197a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:36.769 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.764+0000 7fcdf359e700 1 -- 172.21.15.59:0/2877482521 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcdf41028d0 msgr2=0x7fcdf4197a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:36.769 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.764+0000 7fcdf359e700 1 --2- 172.21.15.59:0/2877482521 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcdf41028d0 0x7fcdf4197a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:36.770 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.764+0000 7fcdf359e700 1 -- 172.21.15.59:0/2877482521 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcdf40ff860 msgr2=0x7fcdf4197530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:36.770 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.764+0000 7fcdf359e700 1 --2- 172.21.15.59:0/2877482521 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcdf40ff860 0x7fcdf4197530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:36.770 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.764+0000 7fcdf359e700 1 -- 172.21.15.59:0/2877482521 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcde40095f0 con 0x7fcdf4103270 2023-12-11T17:45:36.770 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.764+0000 7fcdf2d9d700 1 --2- 172.21.15.59:0/2877482521 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcdf40ff860 0x7fcdf4197530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T17:45:36.770 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.764+0000 7fcdf359e700 1 --2- 172.21.15.59:0/2877482521 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcdf4103270 0x7fcdf419ea90 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fcdec007ee0 tx=0x7fcdec00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:36.770 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.765+0000 7fcddbfff700 1 -- 172.21.15.59:0/2877482521 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdec00dd20 con 0x7fcdf4103270 2023-12-11T17:45:36.770 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.765+0000 7fcddbfff700 1 -- 172.21.15.59:0/2877482521 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcdec00f040 con 0x7fcdf4103270 2023-12-11T17:45:36.770 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.765+0000 7fcddbfff700 1 -- 172.21.15.59:0/2877482521 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdec0048a0 con 0x7fcdf4103270 2023-12-11T17:45:36.770 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.765+0000 7fcdf9096700 1 -- 172.21.15.59:0/2877482521 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdf419f610 con 0x7fcdf4103270 2023-12-11T17:45:36.770 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.765+0000 7fcdf9096700 1 -- 172.21.15.59:0/2877482521 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdf419fad0 con 0x7fcdf4103270 2023-12-11T17:45:36.770 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.766+0000 7fcdf9096700 1 -- 172.21.15.59:0/2877482521 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdf4108bb0 con 0x7fcdf4103270 2023-12-11T17:45:36.771 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.767+0000 7fcddbfff700 1 -- 172.21.15.59:0/2877482521 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7fcdec003a00 con 0x7fcdf4103270 2023-12-11T17:45:36.771 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.767+0000 7fcddbfff700 1 --2- 172.21.15.59:0/2877482521 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fcddc0a1510 0x7fcddc0a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:36.771 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.767+0000 7fcddbfff700 1 -- 172.21.15.59:0/2877482521 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fcdec0cca10 con 0x7fcdf4103270 2023-12-11T17:45:36.771 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.767+0000 7fcdf2d9d700 1 --2- 172.21.15.59:0/2877482521 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fcddc0a1510 0x7fcddc0a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:36.771 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.768+0000 7fcdf2d9d700 1 --2- 172.21.15.59:0/2877482521 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fcddc0a1510 0x7fcddc0a39d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fcde400b5c0 tx=0x7fcde4003790 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:36.774 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.771+0000 7fcddbfff700 1 -- 172.21.15.59:0/2877482521 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fcdec04f2f0 con 0x7fcdf4103270 2023-12-11T17:45:36.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:45:36 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/4115832496' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-12-11T17:45:36.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:45:36 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/4134060425' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-11T17:45:36.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:45:36 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/4115832496' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-12-11T17:45:36.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:45:36 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/4134060425' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-11T17:45:36.978 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.975+0000 7fcdf9096700 1 -- 172.21.15.59:0/2877482521 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fcdf410ce90 con 0x7fcdf4103270 2023-12-11T17:45:36.978 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.976+0000 7fcddbfff700 1 -- 172.21.15.59:0/2877482521 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7fcdec095c10 con 0x7fcdf4103270 2023-12-11T17:45:36.978 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2023-12-11T17:45:36.981 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.978+0000 7fcdf9096700 1 -- 172.21.15.59:0/2877482521 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fcddc0a1510 msgr2=0x7fcddc0a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:36.981 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.978+0000 7fcdf9096700 1 --2- 172.21.15.59:0/2877482521 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fcddc0a1510 0x7fcddc0a39d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fcde400b5c0 tx=0x7fcde4003790 comp rx=0 tx=0).stop 2023-12-11T17:45:36.981 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.978+0000 7fcdf9096700 1 -- 172.21.15.59:0/2877482521 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcdf4103270 msgr2=0x7fcdf419ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:36.981 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.978+0000 7fcdf9096700 1 --2- 172.21.15.59:0/2877482521 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcdf4103270 0x7fcdf419ea90 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fcdec007ee0 tx=0x7fcdec00d3b0 comp rx=0 tx=0).stop 2023-12-11T17:45:36.981 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.978+0000 7fcdf9096700 1 -- 172.21.15.59:0/2877482521 shutdown_connections 2023-12-11T17:45:36.981 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.978+0000 7fcdf9096700 1 --2- 172.21.15.59:0/2877482521 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fcddc0a1510 0x7fcddc0a39d0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:36.981 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.978+0000 7fcdf9096700 1 --2- 172.21.15.59:0/2877482521 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcdf4103270 0x7fcdf419ea90 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:36.981 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.978+0000 7fcdf9096700 1 --2- 172.21.15.59:0/2877482521 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcdf40ff860 0x7fcdf4197530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:36.981 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.978+0000 7fcdf9096700 1 --2- 172.21.15.59:0/2877482521 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcdf41028d0 0x7fcdf4197a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:36.981 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.978+0000 7fcdf9096700 1 -- 172.21.15.59:0/2877482521 >> 172.21.15.59:0/2877482521 conn(0x7fcdf40fb170 msgr2=0x7fcdf4105cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:45:36.981 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.978+0000 7fcdf9096700 1 -- 172.21.15.59:0/2877482521 shutdown_connections 2023-12-11T17:45:36.981 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:36.978+0000 7fcdf9096700 1 -- 172.21.15.59:0/2877482521 wait complete. 2023-12-11T17:45:36.993 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-11T17:45:37.125 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.120+0000 7fdb4e2fb700 1 -- 172.21.15.59:0/3637246942 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb48100480 msgr2=0x7fdb481008a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:37.125 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.120+0000 7fdb4e2fb700 1 --2- 172.21.15.59:0/3637246942 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb48100480 0x7fdb481008a0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c0099c0 tx=0x7fdb3c009cd0 comp rx=0 tx=0).stop 2023-12-11T17:45:37.125 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.120+0000 7fdb4e2fb700 1 -- 172.21.15.59:0/3637246942 shutdown_connections 2023-12-11T17:45:37.125 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.120+0000 7fdb4e2fb700 1 --2- 172.21.15.59:0/3637246942 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdb48102010 0x7fdb481088c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:37.125 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.120+0000 7fdb4e2fb700 1 --2- 172.21.15.59:0/3637246942 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb48100480 0x7fdb481008a0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:37.125 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.120+0000 7fdb4e2fb700 1 --2- 172.21.15.59:0/3637246942 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdb48101670 0x7fdb48101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:37.125 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.120+0000 7fdb4e2fb700 1 -- 172.21.15.59:0/3637246942 >> 172.21.15.59:0/3637246942 conn(0x7fdb480fb9f0 msgr2=0x7fdb480fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:45:37.125 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.120+0000 7fdb4e2fb700 1 -- 172.21.15.59:0/3637246942 shutdown_connections 2023-12-11T17:45:37.125 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.120+0000 7fdb4e2fb700 1 -- 172.21.15.59:0/3637246942 wait complete. 2023-12-11T17:45:37.125 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.121+0000 7fdb4e2fb700 1 Processor -- start 2023-12-11T17:45:37.125 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.121+0000 7fdb4e2fb700 1 -- start start 2023-12-11T17:45:37.126 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.121+0000 7fdb4e2fb700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdb48100480 0x7fdb48196400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:37.126 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.121+0000 7fdb4e2fb700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb48101670 0x7fdb48196940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:37.126 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.121+0000 7fdb4e2fb700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdb48102010 0x7fdb4819d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:37.126 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.121+0000 7fdb4e2fb700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb4819df30 con 0x7fdb48101670 2023-12-11T17:45:37.126 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.121+0000 7fdb4e2fb700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb4819e070 con 0x7fdb48100480 2023-12-11T17:45:37.126 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.121+0000 7fdb4e2fb700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb4819e200 con 0x7fdb48102010 2023-12-11T17:45:37.126 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.121+0000 7fdb477fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb48101670 0x7fdb48196940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:37.126 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.121+0000 7fdb477fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb48101670 0x7fdb48196940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33958/0 (socket says 172.21.15.59:33958) 2023-12-11T17:45:37.126 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.121+0000 7fdb477fe700 1 -- 172.21.15.59:0/2027507048 learned_addr learned my addr 172.21.15.59:0/2027507048 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:45:37.126 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.121+0000 7fdb47fff700 1 --2- 172.21.15.59:0/2027507048 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdb48100480 0x7fdb48196400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:37.126 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.121+0000 7fdb4c898700 1 --2- 172.21.15.59:0/2027507048 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdb48102010 0x7fdb4819d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:37.126 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.121+0000 7fdb477fe700 1 -- 172.21.15.59:0/2027507048 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdb48102010 msgr2=0x7fdb4819d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:37.126 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.121+0000 7fdb477fe700 1 --2- 172.21.15.59:0/2027507048 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdb48102010 0x7fdb4819d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:37.127 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.121+0000 7fdb477fe700 1 -- 172.21.15.59:0/2027507048 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdb48100480 msgr2=0x7fdb48196400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:37.127 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.121+0000 7fdb477fe700 1 --2- 172.21.15.59:0/2027507048 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdb48100480 0x7fdb48196400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:37.127 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.121+0000 7fdb477fe700 1 -- 172.21.15.59:0/2027507048 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb3c0095f0 con 0x7fdb48101670 2023-12-11T17:45:37.127 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.121+0000 7fdb4c898700 1 --2- 172.21.15.59:0/2027507048 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdb48102010 0x7fdb4819d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-11T17:45:37.127 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.122+0000 7fdb477fe700 1 --2- 172.21.15.59:0/2027507048 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb48101670 0x7fdb48196940 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fdb3800c920 tx=0x7fdb3800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:37.127 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.122+0000 7fdb457fa700 1 -- 172.21.15.59:0/2027507048 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb38007b00 con 0x7fdb48101670 2023-12-11T17:45:37.127 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.122+0000 7fdb4e2fb700 1 -- 172.21.15.59:0/2027507048 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb48068730 con 0x7fdb48101670 2023-12-11T17:45:37.127 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.122+0000 7fdb457fa700 1 -- 172.21.15.59:0/2027507048 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdb3800ce70 con 0x7fdb48101670 2023-12-11T17:45:37.127 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.122+0000 7fdb457fa700 1 -- 172.21.15.59:0/2027507048 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb38005460 con 0x7fdb48101670 2023-12-11T17:45:37.127 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.122+0000 7fdb4e2fb700 1 -- 172.21.15.59:0/2027507048 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb48068c90 con 0x7fdb48101670 2023-12-11T17:45:37.128 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.123+0000 7fdb4e2fb700 1 -- 172.21.15.59:0/2027507048 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb48068310 con 0x7fdb48101670 2023-12-11T17:45:37.128 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.124+0000 7fdb457fa700 1 -- 172.21.15.59:0/2027507048 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7fdb380055c0 con 0x7fdb48101670 2023-12-11T17:45:37.128 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.124+0000 7fdb457fa700 1 --2- 172.21.15.59:0/2027507048 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fdb300a5920 0x7fdb300a7de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:37.128 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.125+0000 7fdb47fff700 1 --2- 172.21.15.59:0/2027507048 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fdb300a5920 0x7fdb300a7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:37.128 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.125+0000 7fdb457fa700 1 -- 172.21.15.59:0/2027507048 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fdb380cd1a0 con 0x7fdb48101670 2023-12-11T17:45:37.128 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.125+0000 7fdb47fff700 1 --2- 172.21.15.59:0/2027507048 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fdb300a5920 0x7fdb300a7de0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c003820 tx=0x7fdb3c005920 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:37.130 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.128+0000 7fdb457fa700 1 -- 172.21.15.59:0/2027507048 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fdb380963a0 con 0x7fdb48101670 2023-12-11T17:45:37.328 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.326+0000 7fdb4e2fb700 1 -- 172.21.15.59:0/2027507048 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdb4810be30 con 0x7fdb48101670 2023-12-11T17:45:37.329 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.326+0000 7fdb457fa700 1 -- 172.21.15.59:0/2027507048 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7fdb380cae60 con 0x7fdb48101670 2023-12-11T17:45:37.329 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2023-12-11T17:45:37.331 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.329+0000 7fdb4e2fb700 1 -- 172.21.15.59:0/2027507048 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fdb300a5920 msgr2=0x7fdb300a7de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:37.331 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.329+0000 7fdb4e2fb700 1 --2- 172.21.15.59:0/2027507048 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fdb300a5920 0x7fdb300a7de0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c003820 tx=0x7fdb3c005920 comp rx=0 tx=0).stop 2023-12-11T17:45:37.331 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.329+0000 7fdb4e2fb700 1 -- 172.21.15.59:0/2027507048 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb48101670 msgr2=0x7fdb48196940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:37.332 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.329+0000 7fdb4e2fb700 1 --2- 172.21.15.59:0/2027507048 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb48101670 0x7fdb48196940 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fdb3800c920 tx=0x7fdb3800cc30 comp rx=0 tx=0).stop 2023-12-11T17:45:37.332 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.329+0000 7fdb4e2fb700 1 -- 172.21.15.59:0/2027507048 shutdown_connections 2023-12-11T17:45:37.332 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.329+0000 7fdb4e2fb700 1 --2- 172.21.15.59:0/2027507048 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fdb300a5920 0x7fdb300a7de0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:37.332 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.329+0000 7fdb4e2fb700 1 --2- 172.21.15.59:0/2027507048 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdb48102010 0x7fdb4819d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:37.332 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.329+0000 7fdb4e2fb700 1 --2- 172.21.15.59:0/2027507048 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdb48100480 0x7fdb48196400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:37.332 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.329+0000 7fdb4e2fb700 1 --2- 172.21.15.59:0/2027507048 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb48101670 0x7fdb48196940 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:37.332 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.329+0000 7fdb4e2fb700 1 -- 172.21.15.59:0/2027507048 >> 172.21.15.59:0/2027507048 conn(0x7fdb480fb9f0 msgr2=0x7fdb48104a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:45:37.332 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.329+0000 7fdb4e2fb700 1 -- 172.21.15.59:0/2027507048 shutdown_connections 2023-12-11T17:45:37.332 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.329+0000 7fdb4e2fb700 1 -- 172.21.15.59:0/2027507048 wait complete. 2023-12-11T17:45:37.344 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-12-11T17:45:37.475 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.471+0000 7f523e6b9700 1 -- 172.21.15.59:0/877954348 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f52381004b0 msgr2=0x7f52381008d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:37.476 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.471+0000 7f523e6b9700 1 --2- 172.21.15.59:0/877954348 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f52381004b0 0x7f52381008d0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f52280099c0 tx=0x7f5228009cd0 comp rx=0 tx=0).stop 2023-12-11T17:45:37.476 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.471+0000 7f523e6b9700 1 -- 172.21.15.59:0/877954348 shutdown_connections 2023-12-11T17:45:37.476 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.471+0000 7f523e6b9700 1 --2- 172.21.15.59:0/877954348 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5238102040 0x7f52381088f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:37.476 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.471+0000 7f523e6b9700 1 --2- 172.21.15.59:0/877954348 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f52381004b0 0x7f52381008d0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:37.476 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.471+0000 7f523e6b9700 1 --2- 172.21.15.59:0/877954348 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f52381016a0 0x7f5238101b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:37.476 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.471+0000 7f523e6b9700 1 -- 172.21.15.59:0/877954348 >> 172.21.15.59:0/877954348 conn(0x7f52380fba20 msgr2=0x7f52380fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:45:37.476 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.471+0000 7f523e6b9700 1 -- 172.21.15.59:0/877954348 shutdown_connections 2023-12-11T17:45:37.476 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.471+0000 7f523e6b9700 1 -- 172.21.15.59:0/877954348 wait complete. 2023-12-11T17:45:37.476 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.471+0000 7f523e6b9700 1 Processor -- start 2023-12-11T17:45:37.476 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.471+0000 7f523e6b9700 1 -- start start 2023-12-11T17:45:37.476 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.472+0000 7f523e6b9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f52381004b0 0x7f523806a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:37.476 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.472+0000 7f523e6b9700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f52381016a0 0x7f523819c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:37.476 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.472+0000 7f523e6b9700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5238102040 0x7f523819e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:37.476 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.472+0000 7f523e6b9700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f523819f060 con 0x7f52381016a0 2023-12-11T17:45:37.477 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.472+0000 7f523e6b9700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f523819f1d0 con 0x7f5238102040 2023-12-11T17:45:37.477 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.472+0000 7f523e6b9700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f523819f390 con 0x7f52381004b0 2023-12-11T17:45:37.477 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.472+0000 7f522ffff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f52381016a0 0x7f523819c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:37.477 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.472+0000 7f522ffff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f52381016a0 0x7f523819c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33972/0 (socket says 172.21.15.59:33972) 2023-12-11T17:45:37.477 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.472+0000 7f522ffff700 1 -- 172.21.15.59:0/2657435861 learned_addr learned my addr 172.21.15.59:0/2657435861 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:45:37.477 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.472+0000 7f5237fff700 1 --2- 172.21.15.59:0/2657435861 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f52381004b0 0x7f523806a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:37.477 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.472+0000 7f523cc56700 1 --2- 172.21.15.59:0/2657435861 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5238102040 0x7f523819e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:37.477 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.472+0000 7f522ffff700 1 -- 172.21.15.59:0/2657435861 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f52381004b0 msgr2=0x7f523806a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:37.477 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.472+0000 7f522ffff700 1 --2- 172.21.15.59:0/2657435861 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f52381004b0 0x7f523806a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:37.477 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.472+0000 7f522ffff700 1 -- 172.21.15.59:0/2657435861 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5238102040 msgr2=0x7f523819e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:37.477 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.472+0000 7f522ffff700 1 --2- 172.21.15.59:0/2657435861 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5238102040 0x7f523819e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:37.477 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.472+0000 7f522ffff700 1 -- 172.21.15.59:0/2657435861 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52280095f0 con 0x7f52381016a0 2023-12-11T17:45:37.477 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.472+0000 7f523cc56700 1 --2- 172.21.15.59:0/2657435861 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5238102040 0x7f523819e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-11T17:45:37.477 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.472+0000 7f522ffff700 1 --2- 172.21.15.59:0/2657435861 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f52381016a0 0x7f523819c250 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f522000e9a0 tx=0x7f522000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:37.478 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.472+0000 7f5235ffb700 1 -- 172.21.15.59:0/2657435861 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5220009790 con 0x7f52381016a0 2023-12-11T17:45:37.478 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.472+0000 7f523e6b9700 1 -- 172.21.15.59:0/2657435861 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f523819f610 con 0x7f52381016a0 2023-12-11T17:45:37.478 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.472+0000 7f5235ffb700 1 -- 172.21.15.59:0/2657435861 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f522000cbd0 con 0x7f52381016a0 2023-12-11T17:45:37.478 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.472+0000 7f5235ffb700 1 -- 172.21.15.59:0/2657435861 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5220018680 con 0x7f52381016a0 2023-12-11T17:45:37.478 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.473+0000 7f523e6b9700 1 -- 172.21.15.59:0/2657435861 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f523819fb20 con 0x7f52381016a0 2023-12-11T17:45:37.479 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.473+0000 7f523e6b9700 1 -- 172.21.15.59:0/2657435861 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5238107920 con 0x7f52381016a0 2023-12-11T17:45:37.479 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.474+0000 7f5235ffb700 1 -- 172.21.15.59:0/2657435861 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f522000cd40 con 0x7f52381016a0 2023-12-11T17:45:37.479 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.475+0000 7f5235ffb700 1 --2- 172.21.15.59:0/2657435861 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f52240a1510 0x7f52240a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:37.479 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.475+0000 7f5237fff700 1 --2- 172.21.15.59:0/2657435861 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f52240a1510 0x7f52240a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:37.479 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.475+0000 7f5235ffb700 1 -- 172.21.15.59:0/2657435861 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f5220014070 con 0x7f52381016a0 2023-12-11T17:45:37.479 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.475+0000 7f5237fff700 1 --2- 172.21.15.59:0/2657435861 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f52240a1510 0x7f52240a39d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f5228003730 tx=0x7f5228000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:37.481 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.479+0000 7f5235ffb700 1 -- 172.21.15.59:0/2657435861 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f52200522f0 con 0x7f52381016a0 2023-12-11T17:45:37.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:45:37 smithi059 ceph-mon[98847]: pgmap v299: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-12-11T17:45:37.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:45:37 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/2159073834' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-11T17:45:37.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:45:37 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/2877482521' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-11T17:45:37.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:45:37 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/2027507048' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-11T17:45:37.686 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.684+0000 7f523e6b9700 1 -- 172.21.15.59:0/2657435861 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f523810bdd0 con 0x7f52381016a0 2023-12-11T17:45:37.687 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.684+0000 7f5235ffb700 1 -- 172.21.15.59:0/2657435861 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v88) v1 ==== 103+0+11 (secure 0 0 0) 0x7f5220096630 con 0x7f52381016a0 2023-12-11T17:45:37.687 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2023-12-11T17:45:37.689 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.686+0000 7f523e6b9700 1 -- 172.21.15.59:0/2657435861 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f52240a1510 msgr2=0x7f52240a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:37.689 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.686+0000 7f523e6b9700 1 --2- 172.21.15.59:0/2657435861 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f52240a1510 0x7f52240a39d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f5228003730 tx=0x7f5228000bc0 comp rx=0 tx=0).stop 2023-12-11T17:45:37.689 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.686+0000 7f523e6b9700 1 -- 172.21.15.59:0/2657435861 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f52381016a0 msgr2=0x7f523819c250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:37.689 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.686+0000 7f523e6b9700 1 --2- 172.21.15.59:0/2657435861 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f52381016a0 0x7f523819c250 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f522000e9a0 tx=0x7f522000ecb0 comp rx=0 tx=0).stop 2023-12-11T17:45:37.689 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.686+0000 7f523e6b9700 1 -- 172.21.15.59:0/2657435861 shutdown_connections 2023-12-11T17:45:37.689 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.686+0000 7f523e6b9700 1 --2- 172.21.15.59:0/2657435861 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f52240a1510 0x7f52240a39d0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:37.690 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.686+0000 7f523e6b9700 1 --2- 172.21.15.59:0/2657435861 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5238102040 0x7f523819e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:37.690 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.686+0000 7f523e6b9700 1 --2- 172.21.15.59:0/2657435861 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f52381004b0 0x7f523806a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:37.690 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.686+0000 7f523e6b9700 1 --2- 172.21.15.59:0/2657435861 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f52381016a0 0x7f523819c250 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:37.690 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.686+0000 7f523e6b9700 1 -- 172.21.15.59:0/2657435861 >> 172.21.15.59:0/2657435861 conn(0x7f52380fba20 msgr2=0x7f5238104a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:45:37.690 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.687+0000 7f523e6b9700 1 -- 172.21.15.59:0/2657435861 shutdown_connections 2023-12-11T17:45:37.690 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.687+0000 7f523e6b9700 1 -- 172.21.15.59:0/2657435861 wait complete. 2023-12-11T17:45:37.703 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-11T17:45:37.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:45:37 smithi115 ceph-mon[102748]: pgmap v299: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-12-11T17:45:37.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:45:37 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/2159073834' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-11T17:45:37.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:45:37 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/2877482521' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-11T17:45:37.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:45:37 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/2027507048' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-11T17:45:37.830 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.826+0000 7f2baecde700 1 -- 172.21.15.59:0/4151671537 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ba8103040 msgr2=0x7f2ba81098f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:37.830 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.826+0000 7f2baecde700 1 --2- 172.21.15.59:0/4151671537 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ba8103040 0x7f2ba81098f0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f2ba000b0d0 tx=0x7f2ba000b3e0 comp rx=0 tx=0).stop 2023-12-11T17:45:37.831 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.826+0000 7f2baecde700 1 -- 172.21.15.59:0/4151671537 shutdown_connections 2023-12-11T17:45:37.831 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.826+0000 7f2baecde700 1 --2- 172.21.15.59:0/4151671537 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ba8103040 0x7f2ba81098f0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:37.831 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.826+0000 7f2baecde700 1 --2- 172.21.15.59:0/4151671537 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2ba81014b0 0x7f2ba81018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:37.831 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.826+0000 7f2baecde700 1 --2- 172.21.15.59:0/4151671537 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ba81026a0 0x7f2ba8102b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:37.831 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.826+0000 7f2baecde700 1 -- 172.21.15.59:0/4151671537 >> 172.21.15.59:0/4151671537 conn(0x7f2ba80fca20 msgr2=0x7f2ba80fee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:45:37.831 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.826+0000 7f2baecde700 1 -- 172.21.15.59:0/4151671537 shutdown_connections 2023-12-11T17:45:37.831 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.826+0000 7f2baecde700 1 -- 172.21.15.59:0/4151671537 wait complete. 2023-12-11T17:45:37.831 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.826+0000 7f2baecde700 1 Processor -- start 2023-12-11T17:45:37.831 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.826+0000 7f2baecde700 1 -- start start 2023-12-11T17:45:37.831 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.827+0000 7f2baecde700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ba81014b0 0x7f2ba8075330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:37.831 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.827+0000 7f2baecde700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ba81026a0 0x7f2ba80739c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:37.831 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.827+0000 7f2baecde700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2ba8103040 0x7f2ba8073f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:37.832 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.827+0000 7f2baecde700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ba80746a0 con 0x7f2ba81026a0 2023-12-11T17:45:37.832 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.827+0000 7f2baecde700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ba8074810 con 0x7f2ba81014b0 2023-12-11T17:45:37.832 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.827+0000 7f2baecde700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ba81a1550 con 0x7f2ba8103040 2023-12-11T17:45:37.832 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.827+0000 7f2ba7fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ba81026a0 0x7f2ba80739c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:37.832 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.827+0000 7f2ba7fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ba81026a0 0x7f2ba80739c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33986/0 (socket says 172.21.15.59:33986) 2023-12-11T17:45:37.832 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.827+0000 7f2ba7fff700 1 -- 172.21.15.59:0/2746965614 learned_addr learned my addr 172.21.15.59:0/2746965614 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:45:37.832 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.827+0000 7f2baca7a700 1 --2- 172.21.15.59:0/2746965614 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ba81014b0 0x7f2ba8075330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:37.832 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.827+0000 7f2bad27b700 1 --2- 172.21.15.59:0/2746965614 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2ba8103040 0x7f2ba8073f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:37.832 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.827+0000 7f2ba7fff700 1 -- 172.21.15.59:0/2746965614 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2ba8103040 msgr2=0x7f2ba8073f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:37.832 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.827+0000 7f2ba7fff700 1 --2- 172.21.15.59:0/2746965614 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2ba8103040 0x7f2ba8073f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:37.832 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.827+0000 7f2ba7fff700 1 -- 172.21.15.59:0/2746965614 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ba81014b0 msgr2=0x7f2ba8075330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:37.832 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.827+0000 7f2ba7fff700 1 --2- 172.21.15.59:0/2746965614 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ba81014b0 0x7f2ba8075330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:37.833 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.827+0000 7f2ba7fff700 1 -- 172.21.15.59:0/2746965614 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ba0009d00 con 0x7f2ba81026a0 2023-12-11T17:45:37.833 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.827+0000 7f2ba7fff700 1 --2- 172.21.15.59:0/2746965614 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ba81026a0 0x7f2ba80739c0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f2b9400e9a0 tx=0x7f2b9400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:37.833 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.827+0000 7f2ba5ffb700 1 -- 172.21.15.59:0/2746965614 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b94009790 con 0x7f2ba81026a0 2023-12-11T17:45:37.833 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.827+0000 7f2ba5ffb700 1 -- 172.21.15.59:0/2746965614 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2b9400ce70 con 0x7f2ba81026a0 2023-12-11T17:45:37.833 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.827+0000 7f2baecde700 1 -- 172.21.15.59:0/2746965614 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ba81a17e0 con 0x7f2ba81026a0 2023-12-11T17:45:37.833 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.827+0000 7f2ba5ffb700 1 -- 172.21.15.59:0/2746965614 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b94018420 con 0x7f2ba81026a0 2023-12-11T17:45:37.833 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.828+0000 7f2baecde700 1 -- 172.21.15.59:0/2746965614 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ba81a1d40 con 0x7f2ba81026a0 2023-12-11T17:45:37.834 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.829+0000 7f2baecde700 1 -- 172.21.15.59:0/2746965614 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ba8108860 con 0x7f2ba81026a0 2023-12-11T17:45:37.834 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.830+0000 7f2ba5ffb700 1 -- 172.21.15.59:0/2746965614 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f2b940106f0 con 0x7f2ba81026a0 2023-12-11T17:45:37.834 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.831+0000 7f2ba5ffb700 1 --2- 172.21.15.59:0/2746965614 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f2b980a1590 0x7f2b980a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:37.834 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.831+0000 7f2baca7a700 1 --2- 172.21.15.59:0/2746965614 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f2b980a1590 0x7f2b980a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:37.834 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.831+0000 7f2ba5ffb700 1 -- 172.21.15.59:0/2746965614 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f2b94014070 con 0x7f2ba81026a0 2023-12-11T17:45:37.834 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.831+0000 7f2baca7a700 1 --2- 172.21.15.59:0/2746965614 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f2b980a1590 0x7f2b980a3a50 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2ba8102500 tx=0x7f2b9c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:37.837 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:37.834+0000 7f2ba5ffb700 1 -- 172.21.15.59:0/2746965614 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f2b94096900 con 0x7f2ba81026a0 2023-12-11T17:45:37.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:45:37 smithi181 ceph-mon[103627]: pgmap v299: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-12-11T17:45:37.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:45:37 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/2159073834' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-11T17:45:37.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:45:37 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/2877482521' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-11T17:45:37.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:45:37 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/2027507048' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-11T17:45:38.086 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.084+0000 7f2baecde700 1 -- 172.21.15.59:0/2746965614 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2ba8068680 con 0x7f2ba81026a0 2023-12-11T17:45:38.087 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.084+0000 7f2ba5ffb700 1 -- 172.21.15.59:0/2746965614 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+11325 (secure 0 0 0) 0x7f2b94099f20 con 0x7f2ba81026a0 2023-12-11T17:45:38.087 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-12-11T17:45:38.087 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/2184151593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":2184151593},{"type":"v1","addr":"172.21.15.59:6835","nonce":2184151593}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/2817614886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2817614886},{"type":"v1","addr":"172.21.15.59:6837","nonce":2817614886}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6839/229873490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":229873490},{"type":"v1","addr":"172.21.15.59:6839","nonce":229873490}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6841/967998755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":967998755},{"type":"v1","addr":"172.21.15.59:6841","nonce":967998755}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/395684042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":395684042},{"type":"v1","addr":"172.21.15.181:6833","nonce":395684042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6833/3214606191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3214606191},{"type":"v1","addr":"172.21.15.115:6833","nonce":3214606191}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-12-11T17:44:20.076660+0000","modified":"2023-12-11T17:44:47.770695+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24521,"mds_1":24484,"mds_2":24482},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/1874159657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1874159657},{"type":"v1","addr":"172.21.15.115:6835","nonce":1874159657}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24457":{"gid":24457,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/4221183620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":4221183620},{"type":"v1","addr":"172.21.15.181:6835","nonce":4221183620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24466":{"gid":24466,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6837/3580309407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":3580309407},{"type":"v1","addr":"172.21.15.181:6837","nonce":3580309407}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24482":{"gid":24482,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.115:6837/1209365460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":1209365460},{"type":"v1","addr":"172.21.15.115:6837","nonce":1209365460}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.181:6839/2214384806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":2214384806},{"type":"v1","addr":"172.21.15.181:6839","nonce":2214384806}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.115:6839/4122922866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":4122922866},{"type":"v1","addr":"172.21.15.115:6839","nonce":4122922866}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-11T17:45:38.089 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.086+0000 7f2baecde700 1 -- 172.21.15.59:0/2746965614 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f2b980a1590 msgr2=0x7f2b980a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:38.089 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.086+0000 7f2baecde700 1 --2- 172.21.15.59:0/2746965614 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f2b980a1590 0x7f2b980a3a50 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2ba8102500 tx=0x7f2b9c00b410 comp rx=0 tx=0).stop 2023-12-11T17:45:38.089 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.086+0000 7f2baecde700 1 -- 172.21.15.59:0/2746965614 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ba81026a0 msgr2=0x7f2ba80739c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:38.089 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.086+0000 7f2baecde700 1 --2- 172.21.15.59:0/2746965614 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ba81026a0 0x7f2ba80739c0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f2b9400e9a0 tx=0x7f2b9400ecb0 comp rx=0 tx=0).stop 2023-12-11T17:45:38.090 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.086+0000 7f2baecde700 1 -- 172.21.15.59:0/2746965614 shutdown_connections 2023-12-11T17:45:38.090 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.086+0000 7f2baecde700 1 --2- 172.21.15.59:0/2746965614 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f2b980a1590 0x7f2b980a3a50 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:38.090 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.086+0000 7f2baecde700 1 --2- 172.21.15.59:0/2746965614 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2ba8103040 0x7f2ba8073f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:38.090 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.086+0000 7f2baecde700 1 --2- 172.21.15.59:0/2746965614 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ba81014b0 0x7f2ba8075330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:38.090 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.086+0000 7f2baecde700 1 --2- 172.21.15.59:0/2746965614 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ba81026a0 0x7f2ba80739c0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:38.090 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.086+0000 7f2baecde700 1 -- 172.21.15.59:0/2746965614 >> 172.21.15.59:0/2746965614 conn(0x7f2ba80fca20 msgr2=0x7f2ba81059a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:45:38.090 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.087+0000 7f2baecde700 1 -- 172.21.15.59:0/2746965614 shutdown_connections 2023-12-11T17:45:38.090 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.087+0000 7f2baecde700 1 -- 172.21.15.59:0/2746965614 wait complete. 2023-12-11T17:45:38.090 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 24 2023-12-11T17:45:38.102 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-11T17:45:38.229 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.226+0000 7ff81a616700 1 -- 172.21.15.59:0/1092208675 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff814103030 msgr2=0x7ff8141098e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:38.229 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.226+0000 7ff81a616700 1 --2- 172.21.15.59:0/1092208675 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff814103030 0x7ff8141098e0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7ff80c00b0d0 tx=0x7ff80c00b3e0 comp rx=0 tx=0).stop 2023-12-11T17:45:38.229 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.226+0000 7ff81a616700 1 -- 172.21.15.59:0/1092208675 shutdown_connections 2023-12-11T17:45:38.229 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.226+0000 7ff81a616700 1 --2- 172.21.15.59:0/1092208675 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff814103030 0x7ff8141098e0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:38.229 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.226+0000 7ff81a616700 1 --2- 172.21.15.59:0/1092208675 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff8141014a0 0x7ff8141018c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:38.229 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.226+0000 7ff81a616700 1 --2- 172.21.15.59:0/1092208675 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff814102690 0x7ff814102af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:38.229 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.226+0000 7ff81a616700 1 -- 172.21.15.59:0/1092208675 >> 172.21.15.59:0/1092208675 conn(0x7ff8140fca30 msgr2=0x7ff8140fee50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:45:38.229 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.226+0000 7ff81a616700 1 -- 172.21.15.59:0/1092208675 shutdown_connections 2023-12-11T17:45:38.229 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.226+0000 7ff81a616700 1 -- 172.21.15.59:0/1092208675 wait complete. 2023-12-11T17:45:38.230 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.226+0000 7ff81a616700 1 Processor -- start 2023-12-11T17:45:38.230 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.226+0000 7ff81a616700 1 -- start start 2023-12-11T17:45:38.232 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.227+0000 7ff81a616700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff8141014a0 0x7ff814197540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:38.232 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.227+0000 7ff81a616700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff814102690 0x7ff814197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:38.232 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.227+0000 7ff81a616700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff814103030 0x7ff81419eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:38.232 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.227+0000 7ff81a616700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff81419f070 con 0x7ff814103030 2023-12-11T17:45:38.232 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.227+0000 7ff81a616700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff81419f1b0 con 0x7ff8141014a0 2023-12-11T17:45:38.232 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.227+0000 7ff81a616700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff81419f340 con 0x7ff814102690 2023-12-11T17:45:38.232 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.227+0000 7ff818bb3700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff814103030 0x7ff81419eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:38.232 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.227+0000 7ff818bb3700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff814103030 0x7ff81419eaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34002/0 (socket says 172.21.15.59:34002) 2023-12-11T17:45:38.232 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.227+0000 7ff818bb3700 1 -- 172.21.15.59:0/2719518936 learned_addr learned my addr 172.21.15.59:0/2719518936 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:45:38.233 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.227+0000 7ff813fff700 1 --2- 172.21.15.59:0/2719518936 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff8141014a0 0x7ff814197540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:38.233 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.227+0000 7ff8137fe700 1 --2- 172.21.15.59:0/2719518936 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff814102690 0x7ff814197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:38.233 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.227+0000 7ff818bb3700 1 -- 172.21.15.59:0/2719518936 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff814102690 msgr2=0x7ff814197a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:38.233 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.227+0000 7ff818bb3700 1 --2- 172.21.15.59:0/2719518936 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff814102690 0x7ff814197a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:38.233 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.227+0000 7ff818bb3700 1 -- 172.21.15.59:0/2719518936 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff8141014a0 msgr2=0x7ff814197540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:38.233 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.227+0000 7ff818bb3700 1 --2- 172.21.15.59:0/2719518936 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff8141014a0 0x7ff814197540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:38.233 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.227+0000 7ff818bb3700 1 -- 172.21.15.59:0/2719518936 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff80c009d00 con 0x7ff814103030 2023-12-11T17:45:38.233 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.227+0000 7ff813fff700 1 --2- 172.21.15.59:0/2719518936 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff8141014a0 0x7ff814197540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T17:45:38.233 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.227+0000 7ff818bb3700 1 --2- 172.21.15.59:0/2719518936 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff814103030 0x7ff81419eaa0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7ff80c007b60 tx=0x7ff80c008f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:38.233 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.227+0000 7ff8117fa700 1 -- 172.21.15.59:0/2719518936 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff80c00e040 con 0x7ff814103030 2023-12-11T17:45:38.233 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.228+0000 7ff8117fa700 1 -- 172.21.15.59:0/2719518936 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff80c00be70 con 0x7ff814103030 2023-12-11T17:45:38.233 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.228+0000 7ff81a616700 1 -- 172.21.15.59:0/2719518936 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff81419f5c0 con 0x7ff814103030 2023-12-11T17:45:38.233 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.228+0000 7ff8117fa700 1 -- 172.21.15.59:0/2719518936 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff80c01bb00 con 0x7ff814103030 2023-12-11T17:45:38.233 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.228+0000 7ff81a616700 1 -- 172.21.15.59:0/2719518936 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff81419fa40 con 0x7ff814103030 2023-12-11T17:45:38.233 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.228+0000 7ff81a616700 1 -- 172.21.15.59:0/2719518936 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff814108850 con 0x7ff814103030 2023-12-11T17:45:38.234 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.230+0000 7ff8117fa700 1 -- 172.21.15.59:0/2719518936 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7ff80c01d670 con 0x7ff814103030 2023-12-11T17:45:38.234 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.230+0000 7ff8117fa700 1 --2- 172.21.15.59:0/2719518936 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7ff7fc0a1510 0x7ff7fc0a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:38.234 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.230+0000 7ff813fff700 1 --2- 172.21.15.59:0/2719518936 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7ff7fc0a1510 0x7ff7fc0a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:38.234 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.230+0000 7ff8117fa700 1 -- 172.21.15.59:0/2719518936 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8999+0+0 (secure 0 0 0) 0x7ff80c0cee80 con 0x7ff814103030 2023-12-11T17:45:38.234 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.231+0000 7ff813fff700 1 --2- 172.21.15.59:0/2719518936 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7ff7fc0a1510 0x7ff7fc0a39d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ff8141024f0 tx=0x7ff804009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:38.236 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.234+0000 7ff8117fa700 1 -- 172.21.15.59:0/2719518936 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7ff80c098230 con 0x7ff814103030 2023-12-11T17:45:38.442 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.440+0000 7ff81a616700 1 -- 172.21.15.59:0/2719518936 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff81410cd10 con 0x7ff814103030 2023-12-11T17:45:38.444 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.441+0000 7ff8117fa700 1 -- 172.21.15.59:0/2719518936 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+24459 (secure 0 0 0) 0x7ff80c09b850 con 0x7ff814103030 2023-12-11T17:45:38.444 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-12-11T17:45:38.444 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":88,"fsid":"d0a07eaa-984b-11ee-95a3-87774f69a715","created":"2023-12-11T17:37:14.526539+0000","modified":"2023-12-11T17:45:15.870614+0000","last_up_change":"2023-12-11T17:43:38.535503+0000","last_in_change":"2023-12-11T17:43:21.014888+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-11T17:40:26.596029+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-11T17:44:18.586290+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-11T17:44:19.432952+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-11T17:44:23.097308+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3c0b3218-4b2b-40f8-8bef-ed8cd5633607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":1040759398},{"type":"v1","addr":"172.21.15.59:6803","nonce":1040759398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":1040759398},{"type":"v1","addr":"172.21.15.59:6805","nonce":1040759398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":1040759398},{"type":"v1","addr":"172.21.15.59:6809","nonce":1040759398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":1040759398},{"type":"v1","addr":"172.21.15.59:6807","nonce":1040759398}]},"public_addr":"172.21.15.59:6803/1040759398","cluster_addr":"172.21.15.59:6805/1040759398","heartbeat_back_addr":"172.21.15.59:6809/1040759398","heartbeat_front_addr":"172.21.15.59:6807/1040759398","state":["exists","up"]},{"osd":1,"uuid":"2ead7578-765b-4f5a-a72c-863731ed69e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":2776354245},{"type":"v1","addr":"172.21.15.115:6801","nonce":2776354245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":2776354245},{"type":"v1","addr":"172.21.15.115:6803","nonce":2776354245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":2776354245},{"type":"v1","addr":"172.21.15.115:6807","nonce":2776354245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":2776354245},{"type":"v1","addr":"172.21.15.115:6805","nonce":2776354245}]},"public_addr":"172.21.15.115:6801/2776354245","cluster_addr":"172.21.15.115:6803/2776354245","heartbeat_back_addr":"172.21.15.115:6807/2776354245","heartbeat_front_addr":"172.21.15.115:6805/2776354245","state":["exists","up"]},{"osd":2,"uuid":"8cb1d7ec-53cc-4672-b0b6-70d73797c44f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4287675481},{"type":"v1","addr":"172.21.15.181:6801","nonce":4287675481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4287675481},{"type":"v1","addr":"172.21.15.181:6803","nonce":4287675481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4287675481},{"type":"v1","addr":"172.21.15.181:6807","nonce":4287675481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4287675481},{"type":"v1","addr":"172.21.15.181:6805","nonce":4287675481}]},"public_addr":"172.21.15.181:6801/4287675481","cluster_addr":"172.21.15.181:6803/4287675481","heartbeat_back_addr":"172.21.15.181:6807/4287675481","heartbeat_front_addr":"172.21.15.181:6805/4287675481","state":["exists","up"]},{"osd":3,"uuid":"5d64d9af-831b-4d0a-8ace-a452169351d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":219909995},{"type":"v1","addr":"172.21.15.59:6811","nonce":219909995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":219909995},{"type":"v1","addr":"172.21.15.59:6813","nonce":219909995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":219909995},{"type":"v1","addr":"172.21.15.59:6817","nonce":219909995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":219909995},{"type":"v1","addr":"172.21.15.59:6815","nonce":219909995}]},"public_addr":"172.21.15.59:6811/219909995","cluster_addr":"172.21.15.59:6813/219909995","heartbeat_back_addr":"172.21.15.59:6817/219909995","heartbeat_front_addr":"172.21.15.59:6815/219909995","state":["exists","up"]},{"osd":4,"uuid":"ef8fad28-1db5-4cba-b5f2-10e54e8e1549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":1021473826},{"type":"v1","addr":"172.21.15.115:6809","nonce":1021473826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":1021473826},{"type":"v1","addr":"172.21.15.115:6811","nonce":1021473826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":1021473826},{"type":"v1","addr":"172.21.15.115:6815","nonce":1021473826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":1021473826},{"type":"v1","addr":"172.21.15.115:6813","nonce":1021473826}]},"public_addr":"172.21.15.115:6809/1021473826","cluster_addr":"172.21.15.115:6811/1021473826","heartbeat_back_addr":"172.21.15.115:6815/1021473826","heartbeat_front_addr":"172.21.15.115:6813/1021473826","state":["exists","up"]},{"osd":5,"uuid":"4233a512-9e8f-4db8-aed2-6c3af706e589","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":2405172521},{"type":"v1","addr":"172.21.15.181:6809","nonce":2405172521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":2405172521},{"type":"v1","addr":"172.21.15.181:6811","nonce":2405172521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2405172521},{"type":"v1","addr":"172.21.15.181:6815","nonce":2405172521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":2405172521},{"type":"v1","addr":"172.21.15.181:6813","nonce":2405172521}]},"public_addr":"172.21.15.181:6809/2405172521","cluster_addr":"172.21.15.181:6811/2405172521","heartbeat_back_addr":"172.21.15.181:6815/2405172521","heartbeat_front_addr":"172.21.15.181:6813/2405172521","state":["exists","up"]},{"osd":6,"uuid":"8faab332-016e-4c77-8370-8bef95fc6f34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":815374853},{"type":"v1","addr":"172.21.15.59:6819","nonce":815374853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":815374853},{"type":"v1","addr":"172.21.15.59:6821","nonce":815374853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":815374853},{"type":"v1","addr":"172.21.15.59:6825","nonce":815374853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":815374853},{"type":"v1","addr":"172.21.15.59:6823","nonce":815374853}]},"public_addr":"172.21.15.59:6819/815374853","cluster_addr":"172.21.15.59:6821/815374853","heartbeat_back_addr":"172.21.15.59:6825/815374853","heartbeat_front_addr":"172.21.15.59:6823/815374853","state":["exists","up"]},{"osd":7,"uuid":"f20f470a-0d40-4ed3-8e0e-fd58675cf143","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":1314971870},{"type":"v1","addr":"172.21.15.115:6817","nonce":1314971870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":1314971870},{"type":"v1","addr":"172.21.15.115:6819","nonce":1314971870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":1314971870},{"type":"v1","addr":"172.21.15.115:6823","nonce":1314971870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":1314971870},{"type":"v1","addr":"172.21.15.115:6821","nonce":1314971870}]},"public_addr":"172.21.15.115:6817/1314971870","cluster_addr":"172.21.15.115:6819/1314971870","heartbeat_back_addr":"172.21.15.115:6823/1314971870","heartbeat_front_addr":"172.21.15.115:6821/1314971870","state":["exists","up"]},{"osd":8,"uuid":"eb657d32-1a95-4fd7-a293-d0a970837fc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":3954509873},{"type":"v1","addr":"172.21.15.181:6817","nonce":3954509873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":3954509873},{"type":"v1","addr":"172.21.15.181:6819","nonce":3954509873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":3954509873},{"type":"v1","addr":"172.21.15.181:6823","nonce":3954509873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":3954509873},{"type":"v1","addr":"172.21.15.181:6821","nonce":3954509873}]},"public_addr":"172.21.15.181:6817/3954509873","cluster_addr":"172.21.15.181:6819/3954509873","heartbeat_back_addr":"172.21.15.181:6823/3954509873","heartbeat_front_addr":"172.21.15.181:6821/3954509873","state":["exists","up"]},{"osd":9,"uuid":"8f7717d4-b6f2-4ec1-9b58-62feb0c9a146","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":1163095677},{"type":"v1","addr":"172.21.15.59:6827","nonce":1163095677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":1163095677},{"type":"v1","addr":"172.21.15.59:6829","nonce":1163095677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":1163095677},{"type":"v1","addr":"172.21.15.59:6833","nonce":1163095677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":1163095677},{"type":"v1","addr":"172.21.15.59:6831","nonce":1163095677}]},"public_addr":"172.21.15.59:6827/1163095677","cluster_addr":"172.21.15.59:6829/1163095677","heartbeat_back_addr":"172.21.15.59:6833/1163095677","heartbeat_front_addr":"172.21.15.59:6831/1163095677","state":["exists","up"]},{"osd":10,"uuid":"b75c179a-875d-43bc-b4c1-e8eb80fb64fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":2199610269},{"type":"v1","addr":"172.21.15.115:6825","nonce":2199610269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":2199610269},{"type":"v1","addr":"172.21.15.115:6827","nonce":2199610269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":2199610269},{"type":"v1","addr":"172.21.15.115:6831","nonce":2199610269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":2199610269},{"type":"v1","addr":"172.21.15.115:6829","nonce":2199610269}]},"public_addr":"172.21.15.115:6825/2199610269","cluster_addr":"172.21.15.115:6827/2199610269","heartbeat_back_addr":"172.21.15.115:6831/2199610269","heartbeat_front_addr":"172.21.15.115:6829/2199610269","state":["exists","up"]},{"osd":11,"uuid":"5e141dfe-8a00-4bbc-8204-2dc9be24e18d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":226506989},{"type":"v1","addr":"172.21.15.181:6825","nonce":226506989}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":226506989},{"type":"v1","addr":"172.21.15.181:6827","nonce":226506989}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":226506989},{"type":"v1","addr":"172.21.15.181:6831","nonce":226506989}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":226506989},{"type":"v1","addr":"172.21.15.181:6829","nonce":226506989}]},"public_addr":"172.21.15.181:6825/226506989","cluster_addr":"172.21.15.181:6827/226506989","heartbeat_back_addr":"172.21.15.181:6831/226506989","heartbeat_front_addr":"172.21.15.181:6829/226506989","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:39:42.687084+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:40:03.903448+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:40:24.899908+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:40:46.600251+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:41:07.690037+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:41:28.488855+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:41:49.459274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:42:10.863881+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:42:32.248391+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:42:54.112265+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:43:15.634872+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:43:36.922534+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":7}]},{"pgid":"3.28","mappings":[{"from":5,"to":6},{"from":3,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.59:6800/1259916482":"2023-12-12T17:38:14.037922+0000","172.21.15.59:0/17457774":"2023-12-12T17:38:14.037922+0000","172.21.15.59:6801/1672530462":"2023-12-12T17:37:44.773925+0000","172.21.15.59:0/1053944547":"2023-12-12T17:38:14.037922+0000","172.21.15.59:6801/1259916482":"2023-12-12T17:38:14.037922+0000","172.21.15.59:6800/1672530462":"2023-12-12T17:37:44.773925+0000","172.21.15.59:0/3570767650":"2023-12-12T17:37:44.773925+0000","172.21.15.59:0/3853462899":"2023-12-12T17:37:44.773925+0000","172.21.15.59:0/2859415380":"2023-12-12T17:38:14.037922+0000","172.21.15.59:0/2192742819":"2023-12-12T17:37:44.773925+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-11T17:45:38.446 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.443+0000 7ff81a616700 1 -- 172.21.15.59:0/2719518936 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7ff7fc0a1510 msgr2=0x7ff7fc0a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:38.446 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.443+0000 7ff81a616700 1 --2- 172.21.15.59:0/2719518936 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7ff7fc0a1510 0x7ff7fc0a39d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ff8141024f0 tx=0x7ff804009260 comp rx=0 tx=0).stop 2023-12-11T17:45:38.446 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.443+0000 7ff81a616700 1 -- 172.21.15.59:0/2719518936 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff814103030 msgr2=0x7ff81419eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:38.446 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.443+0000 7ff81a616700 1 --2- 172.21.15.59:0/2719518936 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff814103030 0x7ff81419eaa0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7ff80c007b60 tx=0x7ff80c008f40 comp rx=0 tx=0).stop 2023-12-11T17:45:38.447 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.444+0000 7ff81a616700 1 -- 172.21.15.59:0/2719518936 shutdown_connections 2023-12-11T17:45:38.447 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.444+0000 7ff81a616700 1 --2- 172.21.15.59:0/2719518936 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7ff7fc0a1510 0x7ff7fc0a39d0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:38.447 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.444+0000 7ff81a616700 1 --2- 172.21.15.59:0/2719518936 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff814103030 0x7ff81419eaa0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:38.447 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.444+0000 7ff81a616700 1 --2- 172.21.15.59:0/2719518936 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff8141014a0 0x7ff814197540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:38.447 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.444+0000 7ff81a616700 1 --2- 172.21.15.59:0/2719518936 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff814102690 0x7ff814197a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:38.447 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.444+0000 7ff81a616700 1 -- 172.21.15.59:0/2719518936 >> 172.21.15.59:0/2719518936 conn(0x7ff8140fca30 msgr2=0x7ff814105990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:45:38.447 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.444+0000 7ff81a616700 1 -- 172.21.15.59:0/2719518936 shutdown_connections 2023-12-11T17:45:38.447 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.444+0000 7ff81a616700 1 -- 172.21.15.59:0/2719518936 wait complete. 2023-12-11T17:45:38.459 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-11T17:40:26.596029+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-11T17:44:18.586290+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-11T17:44:19.432952+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-12-11T17:44:23.097308+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-12-11T17:45:38.459 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-11T17:45:38.591 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.586+0000 7f28743d0700 1 -- 172.21.15.59:0/2594491089 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f286c107320 msgr2=0x7f286c109710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:38.591 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.586+0000 7f28743d0700 1 --2- 172.21.15.59:0/2594491089 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f286c107320 0x7f286c109710 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f286400b0a0 tx=0x7f286400b3b0 comp rx=0 tx=0).stop 2023-12-11T17:45:38.591 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.587+0000 7f28743d0700 1 -- 172.21.15.59:0/2594491089 shutdown_connections 2023-12-11T17:45:38.591 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.587+0000 7f28743d0700 1 --2- 172.21.15.59:0/2594491089 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f286c107320 0x7f286c109710 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:38.591 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.587+0000 7f28743d0700 1 --2- 172.21.15.59:0/2594491089 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f286c1020c0 0x7f286c1044b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:38.592 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.587+0000 7f28743d0700 1 --2- 172.21.15.59:0/2594491089 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f286c1049f0 0x7f286c106de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:38.592 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.587+0000 7f28743d0700 1 -- 172.21.15.59:0/2594491089 >> 172.21.15.59:0/2594491089 conn(0x7f286c0fb9f0 msgr2=0x7f286c0fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:45:38.592 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.587+0000 7f28743d0700 1 -- 172.21.15.59:0/2594491089 shutdown_connections 2023-12-11T17:45:38.592 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.587+0000 7f28743d0700 1 -- 172.21.15.59:0/2594491089 wait complete. 2023-12-11T17:45:38.592 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.587+0000 7f28743d0700 1 Processor -- start 2023-12-11T17:45:38.592 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.587+0000 7f28743d0700 1 -- start start 2023-12-11T17:45:38.592 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.587+0000 7f28743d0700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f286c1020c0 0x7f286c197610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:38.592 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.587+0000 7f28743d0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f286c1049f0 0x7f286c197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:38.592 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.587+0000 7f28743d0700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f286c107320 0x7f286c19eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:38.592 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.587+0000 7f28743d0700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f286c19f0f0 con 0x7f286c1020c0 2023-12-11T17:45:38.592 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.587+0000 7f28743d0700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f286c19f230 con 0x7f286c107320 2023-12-11T17:45:38.592 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.587+0000 7f28743d0700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f286c19f3c0 con 0x7f286c1049f0 2023-12-11T17:45:38.592 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.588+0000 7f287216c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f286c1020c0 0x7f286c197610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:38.592 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.588+0000 7f287216c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f286c1020c0 0x7f286c197610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34012/0 (socket says 172.21.15.59:34012) 2023-12-11T17:45:38.593 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.588+0000 7f287216c700 1 -- 172.21.15.59:0/4263854356 learned_addr learned my addr 172.21.15.59:0/4263854356 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:45:38.593 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.588+0000 7f287196b700 1 --2- 172.21.15.59:0/4263854356 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f286c1049f0 0x7f286c197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:38.593 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.588+0000 7f287296d700 1 --2- 172.21.15.59:0/4263854356 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f286c107320 0x7f286c19eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:38.593 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.588+0000 7f287216c700 1 -- 172.21.15.59:0/4263854356 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f286c1049f0 msgr2=0x7f286c197b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:38.593 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.588+0000 7f287216c700 1 --2- 172.21.15.59:0/4263854356 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f286c1049f0 0x7f286c197b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:38.593 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.588+0000 7f287216c700 1 -- 172.21.15.59:0/4263854356 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f286c107320 msgr2=0x7f286c19eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:38.593 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.588+0000 7f287216c700 1 --2- 172.21.15.59:0/4263854356 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f286c107320 0x7f286c19eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:38.593 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.588+0000 7f287216c700 1 -- 172.21.15.59:0/4263854356 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2864009d00 con 0x7f286c1020c0 2023-12-11T17:45:38.593 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.588+0000 7f287216c700 1 --2- 172.21.15.59:0/4263854356 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f286c1020c0 0x7f286c197610 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f286800c920 tx=0x7f286800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:38.593 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.588+0000 7f28637fe700 1 -- 172.21.15.59:0/4263854356 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2868007b00 con 0x7f286c1020c0 2023-12-11T17:45:38.593 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.588+0000 7f28637fe700 1 -- 172.21.15.59:0/4263854356 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f286800ce70 con 0x7f286c1020c0 2023-12-11T17:45:38.593 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.588+0000 7f28743d0700 1 -- 172.21.15.59:0/4263854356 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f286c19f6a0 con 0x7f286c1020c0 2023-12-11T17:45:38.593 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.588+0000 7f28637fe700 1 -- 172.21.15.59:0/4263854356 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28680054e0 con 0x7f286c1020c0 2023-12-11T17:45:38.593 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.588+0000 7f28743d0700 1 -- 172.21.15.59:0/4263854356 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f286c19fc00 con 0x7f286c1020c0 2023-12-11T17:45:38.594 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.589+0000 7f28743d0700 1 -- 172.21.15.59:0/4263854356 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f286c0fd570 con 0x7f286c1020c0 2023-12-11T17:45:38.594 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.590+0000 7f28637fe700 1 -- 172.21.15.59:0/4263854356 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f2868004750 con 0x7f286c1020c0 2023-12-11T17:45:38.595 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.591+0000 7f28637fe700 1 --2- 172.21.15.59:0/4263854356 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f28580a1590 0x7f28580a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:38.595 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.591+0000 7f287196b700 1 --2- 172.21.15.59:0/4263854356 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f28580a1590 0x7f28580a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:38.595 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.591+0000 7f28637fe700 1 -- 172.21.15.59:0/4263854356 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f28680cccd0 con 0x7f286c1020c0 2023-12-11T17:45:38.595 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.591+0000 7f287196b700 1 --2- 172.21.15.59:0/4263854356 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f28580a1590 0x7f28580a3a50 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f285c004630 tx=0x7f285c00b3c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:38.598 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.595+0000 7f28637fe700 1 -- 172.21.15.59:0/4263854356 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f2868095ed0 con 0x7f286c1020c0 2023-12-11T17:45:38.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:45:38 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/2657435861' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-11T17:45:38.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:45:38 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/2746965614' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T17:45:38.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:45:38 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/2719518936' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-11T17:45:38.798 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.796+0000 7f28743d0700 1 -- 172.21.15.59:0/4263854356 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f286c10d070 con 0x7f286c1020c0 2023-12-11T17:45:38.799 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.796+0000 7f28637fe700 1 -- 172.21.15.59:0/4263854356 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7f286801d0a0 con 0x7f286c1020c0 2023-12-11T17:45:38.799 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2023-12-11T17:45:38.801 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.799+0000 7f28743d0700 1 -- 172.21.15.59:0/4263854356 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f28580a1590 msgr2=0x7f28580a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:38.802 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.799+0000 7f28743d0700 1 --2- 172.21.15.59:0/4263854356 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f28580a1590 0x7f28580a3a50 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f285c004630 tx=0x7f285c00b3c0 comp rx=0 tx=0).stop 2023-12-11T17:45:38.802 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.799+0000 7f28743d0700 1 -- 172.21.15.59:0/4263854356 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f286c1020c0 msgr2=0x7f286c197610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:38.802 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.799+0000 7f28743d0700 1 --2- 172.21.15.59:0/4263854356 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f286c1020c0 0x7f286c197610 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f286800c920 tx=0x7f286800cc30 comp rx=0 tx=0).stop 2023-12-11T17:45:38.802 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.799+0000 7f28743d0700 1 -- 172.21.15.59:0/4263854356 shutdown_connections 2023-12-11T17:45:38.802 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.799+0000 7f28743d0700 1 --2- 172.21.15.59:0/4263854356 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f28580a1590 0x7f28580a3a50 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:38.802 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.799+0000 7f28743d0700 1 --2- 172.21.15.59:0/4263854356 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f286c107320 0x7f286c19eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:38.802 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.799+0000 7f28743d0700 1 --2- 172.21.15.59:0/4263854356 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f286c1020c0 0x7f286c197610 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:38.802 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.799+0000 7f28743d0700 1 --2- 172.21.15.59:0/4263854356 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f286c1049f0 0x7f286c197b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:38.802 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.799+0000 7f28743d0700 1 -- 172.21.15.59:0/4263854356 >> 172.21.15.59:0/4263854356 conn(0x7f286c0fb9f0 msgr2=0x7f286c102ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:45:38.802 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.799+0000 7f28743d0700 1 -- 172.21.15.59:0/4263854356 shutdown_connections 2023-12-11T17:45:38.802 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.799+0000 7f28743d0700 1 -- 172.21.15.59:0/4263854356 wait complete. 2023-12-11T17:45:38.813 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:45:38 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/2657435861' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-11T17:45:38.813 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:45:38 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/2746965614' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T17:45:38.813 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:45:38 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/2719518936' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-11T17:45:38.814 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-11T17:45:38.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:45:38 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/2657435861' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-11T17:45:38.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:45:38 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/2746965614' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T17:45:38.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:45:38 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/2719518936' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-11T17:45:38.944 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.941+0000 7f613d43c700 1 -- 172.21.15.59:0/699398205 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6138102040 msgr2=0x7f61381088f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:38.944 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.941+0000 7f613d43c700 1 --2- 172.21.15.59:0/699398205 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6138102040 0x7f61381088f0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f613000b0d0 tx=0x7f613000b3e0 comp rx=0 tx=0).stop 2023-12-11T17:45:38.944 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.941+0000 7f613d43c700 1 -- 172.21.15.59:0/699398205 shutdown_connections 2023-12-11T17:45:38.944 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.941+0000 7f613d43c700 1 --2- 172.21.15.59:0/699398205 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6138102040 0x7f61381088f0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:38.944 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.941+0000 7f613d43c700 1 --2- 172.21.15.59:0/699398205 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f61381004b0 0x7f61381008d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:38.945 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.941+0000 7f613d43c700 1 --2- 172.21.15.59:0/699398205 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61381016a0 0x7f6138101b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:38.945 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.941+0000 7f613d43c700 1 -- 172.21.15.59:0/699398205 >> 172.21.15.59:0/699398205 conn(0x7f61380fba20 msgr2=0x7f61380fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:45:38.945 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.941+0000 7f613d43c700 1 -- 172.21.15.59:0/699398205 shutdown_connections 2023-12-11T17:45:38.945 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.941+0000 7f613d43c700 1 -- 172.21.15.59:0/699398205 wait complete. 2023-12-11T17:45:38.945 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.942+0000 7f613d43c700 1 Processor -- start 2023-12-11T17:45:38.947 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.942+0000 7f613d43c700 1 -- start start 2023-12-11T17:45:38.947 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.942+0000 7f613d43c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f61381004b0 0x7f6138197610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:38.947 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.942+0000 7f613d43c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61381016a0 0x7f6138197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:38.947 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.942+0000 7f613d43c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6138102040 0x7f613819eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:38.947 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.942+0000 7f613d43c700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f613819f140 con 0x7f6138102040 2023-12-11T17:45:38.947 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.942+0000 7f613d43c700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f613819f280 con 0x7f61381004b0 2023-12-11T17:45:38.947 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.942+0000 7f613d43c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f613819f410 con 0x7f61381016a0 2023-12-11T17:45:38.947 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.942+0000 7f61377fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6138102040 0x7f613819eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:38.947 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.942+0000 7f61377fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6138102040 0x7f613819eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34030/0 (socket says 172.21.15.59:34030) 2023-12-11T17:45:38.947 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.942+0000 7f61377fe700 1 -- 172.21.15.59:0/3265124884 learned_addr learned my addr 172.21.15.59:0/3265124884 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:45:38.948 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.942+0000 7f6136ffd700 1 --2- 172.21.15.59:0/3265124884 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f61381004b0 0x7f6138197610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:38.948 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.943+0000 7f61377fe700 1 -- 172.21.15.59:0/3265124884 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61381016a0 msgr2=0x7f6138197b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:38.948 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.943+0000 7f61377fe700 1 --2- 172.21.15.59:0/3265124884 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61381016a0 0x7f6138197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:38.948 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.943+0000 7f61377fe700 1 -- 172.21.15.59:0/3265124884 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f61381004b0 msgr2=0x7f6138197610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:38.948 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.943+0000 7f61377fe700 1 --2- 172.21.15.59:0/3265124884 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f61381004b0 0x7f6138197610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:38.948 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.943+0000 7f61377fe700 1 -- 172.21.15.59:0/3265124884 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6130009d00 con 0x7f6138102040 2023-12-11T17:45:38.948 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.943+0000 7f61377fe700 1 --2- 172.21.15.59:0/3265124884 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6138102040 0x7f613819eb70 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f6130007b60 tx=0x7f6130008e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:38.948 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.943+0000 7f611ffff700 1 -- 172.21.15.59:0/3265124884 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f613000e040 con 0x7f6138102040 2023-12-11T17:45:38.948 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.943+0000 7f611ffff700 1 -- 172.21.15.59:0/3265124884 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f613000be30 con 0x7f6138102040 2023-12-11T17:45:38.948 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.943+0000 7f613d43c700 1 -- 172.21.15.59:0/3265124884 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f613819f690 con 0x7f6138102040 2023-12-11T17:45:38.948 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.943+0000 7f611ffff700 1 -- 172.21.15.59:0/3265124884 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f613001bb90 con 0x7f6138102040 2023-12-11T17:45:38.948 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.943+0000 7f613d43c700 1 -- 172.21.15.59:0/3265124884 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f613819fb10 con 0x7f6138102040 2023-12-11T17:45:38.948 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.944+0000 7f613d43c700 1 -- 172.21.15.59:0/3265124884 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6138068760 con 0x7f6138102040 2023-12-11T17:45:38.949 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.945+0000 7f611ffff700 1 -- 172.21.15.59:0/3265124884 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f613001d670 con 0x7f6138102040 2023-12-11T17:45:38.949 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.946+0000 7f611ffff700 1 --2- 172.21.15.59:0/3265124884 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f61200a1590 0x7f61200a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:38.949 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.946+0000 7f6136ffd700 1 --2- 172.21.15.59:0/3265124884 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f61200a1590 0x7f61200a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:38.949 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.946+0000 7f611ffff700 1 -- 172.21.15.59:0/3265124884 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f61300cf0e0 con 0x7f6138102040 2023-12-11T17:45:38.949 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.946+0000 7f6136ffd700 1 --2- 172.21.15.59:0/3265124884 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f61200a1590 0x7f61200a3a50 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f6138101500 tx=0x7f6128009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:38.951 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:38.949+0000 7f611ffff700 1 -- 172.21.15.59:0/3265124884 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f6130098370 con 0x7f6138102040 2023-12-11T17:45:39.156 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:39.153+0000 7f613d43c700 1 -- 172.21.15.59:0/3265124884 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f613819fde0 con 0x7f6138102040 2023-12-11T17:45:39.157 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:39.154+0000 7f611ffff700 1 -- 172.21.15.59:0/3265124884 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7f613009b990 con 0x7f6138102040 2023-12-11T17:45:39.157 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2023-12-11T17:45:39.159 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:39.156+0000 7f613d43c700 1 -- 172.21.15.59:0/3265124884 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f61200a1590 msgr2=0x7f61200a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:39.159 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:39.156+0000 7f613d43c700 1 --2- 172.21.15.59:0/3265124884 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f61200a1590 0x7f61200a3a50 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f6138101500 tx=0x7f6128009260 comp rx=0 tx=0).stop 2023-12-11T17:45:39.159 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:39.156+0000 7f613d43c700 1 -- 172.21.15.59:0/3265124884 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6138102040 msgr2=0x7f613819eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:39.160 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:39.156+0000 7f613d43c700 1 --2- 172.21.15.59:0/3265124884 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6138102040 0x7f613819eb70 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f6130007b60 tx=0x7f6130008e90 comp rx=0 tx=0).stop 2023-12-11T17:45:39.160 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:39.156+0000 7f613d43c700 1 -- 172.21.15.59:0/3265124884 shutdown_connections 2023-12-11T17:45:39.160 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:39.156+0000 7f613d43c700 1 --2- 172.21.15.59:0/3265124884 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f61200a1590 0x7f61200a3a50 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:39.160 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:39.156+0000 7f613d43c700 1 --2- 172.21.15.59:0/3265124884 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6138102040 0x7f613819eb70 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:39.160 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:39.156+0000 7f613d43c700 1 --2- 172.21.15.59:0/3265124884 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f61381004b0 0x7f6138197610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:39.160 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:39.156+0000 7f613d43c700 1 --2- 172.21.15.59:0/3265124884 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61381016a0 0x7f6138197b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:39.161 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:39.156+0000 7f613d43c700 1 -- 172.21.15.59:0/3265124884 >> 172.21.15.59:0/3265124884 conn(0x7f61380fba20 msgr2=0x7f61380fc050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:45:39.161 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:39.156+0000 7f613d43c700 1 -- 172.21.15.59:0/3265124884 shutdown_connections 2023-12-11T17:45:39.161 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:39.156+0000 7f613d43c700 1 -- 172.21.15.59:0/3265124884 wait complete. 2023-12-11T17:45:39.172 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-11T17:45:39.301 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:39.299+0000 7fd8a6644700 1 -- 172.21.15.59:0/906906812 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd8a0102040 msgr2=0x7fd8a01088f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:39.302 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:39.299+0000 7fd8a6644700 1 --2- 172.21.15.59:0/906906812 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd8a0102040 0x7fd8a01088f0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fd89800b0d0 tx=0x7fd89800b3e0 comp rx=0 tx=0).stop 2023-12-11T17:45:39.302 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:39.299+0000 7fd8a6644700 1 -- 172.21.15.59:0/906906812 shutdown_connections 2023-12-11T17:45:39.302 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:39.299+0000 7fd8a6644700 1 --2- 172.21.15.59:0/906906812 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd8a0102040 0x7fd8a01088f0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:39.302 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:39.299+0000 7fd8a6644700 1 --2- 172.21.15.59:0/906906812 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd8a01004b0 0x7fd8a01008d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:39.302 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:39.299+0000 7fd8a6644700 1 --2- 172.21.15.59:0/906906812 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd8a01016a0 0x7fd8a0101b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:39.302 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:39.299+0000 7fd8a6644700 1 -- 172.21.15.59:0/906906812 >> 172.21.15.59:0/906906812 conn(0x7fd8a00fba20 msgr2=0x7fd8a00fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:45:39.302 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:39.299+0000 7fd8a6644700 1 -- 172.21.15.59:0/906906812 shutdown_connections 2023-12-11T17:45:39.302 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:39.299+0000 7fd8a6644700 1 -- 172.21.15.59:0/906906812 wait complete. 2023-12-11T17:45:39.302 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:39.299+0000 7fd8a6644700 1 Processor -- start 2023-12-11T17:45:39.302 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:39.299+0000 7fd8a6644700 1 -- start start 2023-12-11T17:45:39.304 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:39.300+0000 7fd8a6644700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd8a01004b0 0x7fd8a01975d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:39.304 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:39.300+0000 7fd8a6644700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd8a01016a0 0x7fd8a0197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:39.304 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:39.300+0000 7fd8a6644700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd8a0102040 0x7fd8a019eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:39.305 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:39.300+0000 7fd8a6644700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8a019f100 con 0x7fd8a01016a0 2023-12-11T17:45:39.305 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:39.300+0000 7fd8a6644700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8a019f240 con 0x7fd8a01004b0 2023-12-11T17:45:39.305 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:39.300+0000 7fd8a6644700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8a019f3d0 con 0x7fd8a0102040 2023-12-11T17:45:39.305 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:39.300+0000 7fd89f7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd8a01016a0 0x7fd8a0197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:39.305 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:39.300+0000 7fd89f7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd8a01016a0 0x7fd8a0197b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34040/0 (socket says 172.21.15.59:34040) 2023-12-11T17:45:39.305 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:39.300+0000 7fd89f7fe700 1 -- 172.21.15.59:0/2419194253 learned_addr learned my addr 172.21.15.59:0/2419194253 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:45:39.305 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:39.300+0000 7fd89ffff700 1 --2- 172.21.15.59:0/2419194253 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd8a01004b0 0x7fd8a01975d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:39.305 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:39.300+0000 7fd8a4be1700 1 --2- 172.21.15.59:0/2419194253 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd8a0102040 0x7fd8a019eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:39.305 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:39.300+0000 7fd89f7fe700 1 -- 172.21.15.59:0/2419194253 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd8a0102040 msgr2=0x7fd8a019eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:39.305 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:39.300+0000 7fd89f7fe700 1 --2- 172.21.15.59:0/2419194253 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd8a0102040 0x7fd8a019eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:39.305 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:39.300+0000 7fd89f7fe700 1 -- 172.21.15.59:0/2419194253 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd8a01004b0 msgr2=0x7fd8a01975d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:39.305 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:39.300+0000 7fd89f7fe700 1 --2- 172.21.15.59:0/2419194253 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd8a01004b0 0x7fd8a01975d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:39.305 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:39.300+0000 7fd89f7fe700 1 -- 172.21.15.59:0/2419194253 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd898009d00 con 0x7fd8a01016a0 2023-12-11T17:45:39.306 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:39.300+0000 7fd8a4be1700 1 --2- 172.21.15.59:0/2419194253 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd8a0102040 0x7fd8a019eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T17:45:39.306 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:39.300+0000 7fd89f7fe700 1 --2- 172.21.15.59:0/2419194253 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd8a01016a0 0x7fd8a0197b10 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fd89400e9a0 tx=0x7fd89400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:39.306 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:39.300+0000 7fd89d7fa700 1 -- 172.21.15.59:0/2419194253 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd894009790 con 0x7fd8a01016a0 2023-12-11T17:45:39.306 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:39.300+0000 7fd8a6644700 1 -- 172.21.15.59:0/2419194253 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8a019f6b0 con 0x7fd8a01016a0 2023-12-11T17:45:39.306 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:39.300+0000 7fd89d7fa700 1 -- 172.21.15.59:0/2419194253 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd89400cbd0 con 0x7fd8a01016a0 2023-12-11T17:45:39.306 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:39.300+0000 7fd89d7fa700 1 -- 172.21.15.59:0/2419194253 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd894018680 con 0x7fd8a01016a0 2023-12-11T17:45:39.306 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:39.301+0000 7fd8a6644700 1 -- 172.21.15.59:0/2419194253 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8a019fbc0 con 0x7fd8a01016a0 2023-12-11T17:45:39.306 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:39.301+0000 7fd8a6644700 1 -- 172.21.15.59:0/2419194253 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8a01078b0 con 0x7fd8a01016a0 2023-12-11T17:45:39.307 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:39.303+0000 7fd89d7fa700 1 -- 172.21.15.59:0/2419194253 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7fd89400cd40 con 0x7fd8a01016a0 2023-12-11T17:45:39.307 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:39.303+0000 7fd89d7fa700 1 --2- 172.21.15.59:0/2419194253 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fd8880a1510 0x7fd8880a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:39.307 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:39.303+0000 7fd89ffff700 1 --2- 172.21.15.59:0/2419194253 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fd8880a1510 0x7fd8880a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:39.307 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:39.303+0000 7fd89d7fa700 1 -- 172.21.15.59:0/2419194253 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fd894014070 con 0x7fd8a01016a0 2023-12-11T17:45:39.307 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:39.304+0000 7fd89ffff700 1 --2- 172.21.15.59:0/2419194253 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fd8880a1510 0x7fd8880a39d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fd8a0101500 tx=0x7fd890009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:39.309 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:39.307+0000 7fd89d7fa700 1 -- 172.21.15.59:0/2419194253 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fd8940968e0 con 0x7fd8a01016a0 2023-12-11T17:45:39.513 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:39.510+0000 7fd8a6644700 1 -- 172.21.15.59:0/2419194253 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd8a010cf70 con 0x7fd8a01016a0 2023-12-11T17:45:39.513 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:39.511+0000 7fd89d7fa700 1 -- 172.21.15.59:0/2419194253 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd894099f00 con 0x7fd8a01016a0 2023-12-11T17:45:39.513 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2023-12-11T17:45:39.516 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:39.513+0000 7fd8a6644700 1 -- 172.21.15.59:0/2419194253 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fd8880a1510 msgr2=0x7fd8880a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:39.516 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:39.513+0000 7fd8a6644700 1 --2- 172.21.15.59:0/2419194253 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fd8880a1510 0x7fd8880a39d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fd8a0101500 tx=0x7fd890009260 comp rx=0 tx=0).stop 2023-12-11T17:45:39.516 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:39.513+0000 7fd8a6644700 1 -- 172.21.15.59:0/2419194253 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd8a01016a0 msgr2=0x7fd8a0197b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:39.516 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:39.513+0000 7fd8a6644700 1 --2- 172.21.15.59:0/2419194253 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd8a01016a0 0x7fd8a0197b10 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fd89400e9a0 tx=0x7fd89400ecb0 comp rx=0 tx=0).stop 2023-12-11T17:45:39.516 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:39.513+0000 7fd8a6644700 1 -- 172.21.15.59:0/2419194253 shutdown_connections 2023-12-11T17:45:39.516 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:39.513+0000 7fd8a6644700 1 --2- 172.21.15.59:0/2419194253 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fd8880a1510 0x7fd8880a39d0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:39.516 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:39.513+0000 7fd8a6644700 1 --2- 172.21.15.59:0/2419194253 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd8a0102040 0x7fd8a019eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:39.516 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:39.513+0000 7fd8a6644700 1 --2- 172.21.15.59:0/2419194253 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd8a01004b0 0x7fd8a01975d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:39.516 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:39.513+0000 7fd8a6644700 1 --2- 172.21.15.59:0/2419194253 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd8a01016a0 0x7fd8a0197b10 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:39.516 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:39.513+0000 7fd8a6644700 1 -- 172.21.15.59:0/2419194253 >> 172.21.15.59:0/2419194253 conn(0x7fd8a00fba20 msgr2=0x7fd8a01049f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:45:39.516 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:39.513+0000 7fd8a6644700 1 -- 172.21.15.59:0/2419194253 shutdown_connections 2023-12-11T17:45:39.516 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:39.513+0000 7fd8a6644700 1 -- 172.21.15.59:0/2419194253 wait complete. 2023-12-11T17:45:39.528 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-12-11T17:45:39.657 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:39.654+0000 7faf45250700 1 -- 172.21.15.59:0/1163591249 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf40100480 msgr2=0x7faf401008a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:39.657 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:39.654+0000 7faf45250700 1 --2- 172.21.15.59:0/1163591249 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf40100480 0x7faf401008a0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7faf340099c0 tx=0x7faf34009cd0 comp rx=0 tx=0).stop 2023-12-11T17:45:39.657 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:39.654+0000 7faf45250700 1 -- 172.21.15.59:0/1163591249 shutdown_connections 2023-12-11T17:45:39.658 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:39.654+0000 7faf45250700 1 --2- 172.21.15.59:0/1163591249 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf40102010 0x7faf401088c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:39.658 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:39.654+0000 7faf45250700 1 --2- 172.21.15.59:0/1163591249 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf40100480 0x7faf401008a0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:39.658 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:39.654+0000 7faf45250700 1 --2- 172.21.15.59:0/1163591249 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faf40101670 0x7faf40101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:39.658 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:39.654+0000 7faf45250700 1 -- 172.21.15.59:0/1163591249 >> 172.21.15.59:0/1163591249 conn(0x7faf400fb9f0 msgr2=0x7faf400fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:45:39.658 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:39.655+0000 7faf45250700 1 -- 172.21.15.59:0/1163591249 shutdown_connections 2023-12-11T17:45:39.658 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:39.655+0000 7faf45250700 1 -- 172.21.15.59:0/1163591249 wait complete. 2023-12-11T17:45:39.660 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:39.655+0000 7faf45250700 1 Processor -- start 2023-12-11T17:45:39.660 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:39.656+0000 7faf45250700 1 -- start start 2023-12-11T17:45:39.660 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:39.656+0000 7faf45250700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf40100480 0x7faf40197530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:39.660 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:39.656+0000 7faf45250700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf40101670 0x7faf40197a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:39.660 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:39.656+0000 7faf45250700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faf40102010 0x7faf4019ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:39.660 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:39.656+0000 7faf45250700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf4019f060 con 0x7faf40100480 2023-12-11T17:45:39.661 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:39.656+0000 7faf45250700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf4019f1a0 con 0x7faf40102010 2023-12-11T17:45:39.661 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:39.656+0000 7faf45250700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf4019f330 con 0x7faf40101670 2023-12-11T17:45:39.661 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:39.656+0000 7faf3ed9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf40100480 0x7faf40197530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:39.661 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:39.656+0000 7faf3ed9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf40100480 0x7faf40197530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34052/0 (socket says 172.21.15.59:34052) 2023-12-11T17:45:39.661 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:39.656+0000 7faf3ed9d700 1 -- 172.21.15.59:0/2250375854 learned_addr learned my addr 172.21.15.59:0/2250375854 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:45:39.661 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:39.656+0000 7faf3e59c700 1 --2- 172.21.15.59:0/2250375854 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf40101670 0x7faf40197a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:39.661 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:39.656+0000 7faf3f59e700 1 --2- 172.21.15.59:0/2250375854 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faf40102010 0x7faf4019ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:39.661 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:39.656+0000 7faf3ed9d700 1 -- 172.21.15.59:0/2250375854 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf40101670 msgr2=0x7faf40197a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:39.661 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:39.656+0000 7faf3ed9d700 1 --2- 172.21.15.59:0/2250375854 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf40101670 0x7faf40197a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:39.661 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:39.656+0000 7faf3ed9d700 1 -- 172.21.15.59:0/2250375854 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faf40102010 msgr2=0x7faf4019ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:39.661 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:39.657+0000 7faf3ed9d700 1 --2- 172.21.15.59:0/2250375854 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faf40102010 0x7faf4019ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:39.661 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:39.657+0000 7faf3ed9d700 1 -- 172.21.15.59:0/2250375854 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf340095f0 con 0x7faf40100480 2023-12-11T17:45:39.661 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:39.657+0000 7faf3ed9d700 1 --2- 172.21.15.59:0/2250375854 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf40100480 0x7faf40197530 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7faf34006010 tx=0x7faf3400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:39.661 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:39.657+0000 7faf27fff700 1 -- 172.21.15.59:0/2250375854 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf3401d070 con 0x7faf40100480 2023-12-11T17:45:39.661 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:39.657+0000 7faf45250700 1 -- 172.21.15.59:0/2250375854 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf4019f5b0 con 0x7faf40100480 2023-12-11T17:45:39.662 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:39.657+0000 7faf27fff700 1 -- 172.21.15.59:0/2250375854 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faf34005020 con 0x7faf40100480 2023-12-11T17:45:39.662 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:39.657+0000 7faf27fff700 1 -- 172.21.15.59:0/2250375854 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf34004b10 con 0x7faf40100480 2023-12-11T17:45:39.662 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:39.657+0000 7faf45250700 1 -- 172.21.15.59:0/2250375854 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf4019f970 con 0x7faf40100480 2023-12-11T17:45:39.662 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:39.658+0000 7faf45250700 1 -- 172.21.15.59:0/2250375854 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf40068310 con 0x7faf40100480 2023-12-11T17:45:39.662 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:39.660+0000 7faf27fff700 1 -- 172.21.15.59:0/2250375854 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7faf34005190 con 0x7faf40100480 2023-12-11T17:45:39.663 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:39.660+0000 7faf27fff700 1 --2- 172.21.15.59:0/2250375854 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7faf280a1510 0x7faf280a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:39.663 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:39.661+0000 7faf27fff700 1 -- 172.21.15.59:0/2250375854 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8999+0+0 (secure 0 0 0) 0x7faf340cdd50 con 0x7faf40100480 2023-12-11T17:45:39.663 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:39.661+0000 7faf3e59c700 1 --2- 172.21.15.59:0/2250375854 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7faf280a1510 0x7faf280a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:39.663 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:39.661+0000 7faf3e59c700 1 --2- 172.21.15.59:0/2250375854 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7faf280a1510 0x7faf280a39d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7faf30009c60 tx=0x7faf30009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:39.667 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:39.665+0000 7faf27fff700 1 -- 172.21.15.59:0/2250375854 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7faf34096fe0 con 0x7faf40100480 2023-12-11T17:45:39.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:45:39 smithi115 ceph-mon[102748]: pgmap v300: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-12-11T17:45:39.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:45:39 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/4263854356' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-11T17:45:39.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:45:39 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/3265124884' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-11T17:45:39.867 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:39.865+0000 7faf45250700 1 -- 172.21.15.59:0/2250375854 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7faf4019fd30 con 0x7faf40100480 2023-12-11T17:45:39.868 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:39.865+0000 7faf27fff700 1 -- 172.21.15.59:0/2250375854 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v88) v1 ==== 103+0+11 (secure 0 0 0) 0x7faf3409a600 con 0x7faf40100480 2023-12-11T17:45:39.868 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2023-12-11T17:45:39.870 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:39.867+0000 7faf45250700 1 -- 172.21.15.59:0/2250375854 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7faf280a1510 msgr2=0x7faf280a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:39.870 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:39.867+0000 7faf45250700 1 --2- 172.21.15.59:0/2250375854 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7faf280a1510 0x7faf280a39d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7faf30009c60 tx=0x7faf30009260 comp rx=0 tx=0).stop 2023-12-11T17:45:39.870 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:39.867+0000 7faf45250700 1 -- 172.21.15.59:0/2250375854 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf40100480 msgr2=0x7faf40197530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:39.870 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:39.867+0000 7faf45250700 1 --2- 172.21.15.59:0/2250375854 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf40100480 0x7faf40197530 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7faf34006010 tx=0x7faf3400f690 comp rx=0 tx=0).stop 2023-12-11T17:45:39.870 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:39.867+0000 7faf45250700 1 -- 172.21.15.59:0/2250375854 shutdown_connections 2023-12-11T17:45:39.870 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:39.867+0000 7faf45250700 1 --2- 172.21.15.59:0/2250375854 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7faf280a1510 0x7faf280a39d0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:39.870 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:39.867+0000 7faf45250700 1 --2- 172.21.15.59:0/2250375854 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faf40102010 0x7faf4019ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:39.870 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:39.867+0000 7faf45250700 1 --2- 172.21.15.59:0/2250375854 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf40100480 0x7faf40197530 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:39.871 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:39.867+0000 7faf45250700 1 --2- 172.21.15.59:0/2250375854 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf40101670 0x7faf40197a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:39.871 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:39.867+0000 7faf45250700 1 -- 172.21.15.59:0/2250375854 >> 172.21.15.59:0/2250375854 conn(0x7faf400fb9f0 msgr2=0x7faf40068640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:45:39.871 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:39.867+0000 7faf45250700 1 -- 172.21.15.59:0/2250375854 shutdown_connections 2023-12-11T17:45:39.871 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:39.867+0000 7faf45250700 1 -- 172.21.15.59:0/2250375854 wait complete. 2023-12-11T17:45:39.881 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:45:39 smithi059 ceph-mon[98847]: pgmap v300: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-12-11T17:45:39.881 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:45:39 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/4263854356' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-11T17:45:39.881 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:45:39 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/3265124884' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-11T17:45:39.882 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-11T17:45:39.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:45:39 smithi181 ceph-mon[103627]: pgmap v300: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-12-11T17:45:39.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:45:39 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/4263854356' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-11T17:45:39.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:45:39 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/3265124884' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-11T17:45:40.008 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.005+0000 7fbd30e68700 1 -- 172.21.15.59:0/1401642647 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd2c1014a0 msgr2=0x7fbd2c1018c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:40.008 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.005+0000 7fbd30e68700 1 --2- 172.21.15.59:0/1401642647 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd2c1014a0 0x7fbd2c1018c0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fbd1c0099c0 tx=0x7fbd1c009cd0 comp rx=0 tx=0).stop 2023-12-11T17:45:40.008 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.005+0000 7fbd30e68700 1 -- 172.21.15.59:0/1401642647 shutdown_connections 2023-12-11T17:45:40.008 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.005+0000 7fbd30e68700 1 --2- 172.21.15.59:0/1401642647 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbd2c103030 0x7fbd2c1098e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:40.008 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.005+0000 7fbd30e68700 1 --2- 172.21.15.59:0/1401642647 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd2c1014a0 0x7fbd2c1018c0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:40.008 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.005+0000 7fbd30e68700 1 --2- 172.21.15.59:0/1401642647 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbd2c102690 0x7fbd2c102af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:40.008 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.005+0000 7fbd30e68700 1 -- 172.21.15.59:0/1401642647 >> 172.21.15.59:0/1401642647 conn(0x7fbd2c0fca10 msgr2=0x7fbd2c0fee50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:45:40.008 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.005+0000 7fbd30e68700 1 -- 172.21.15.59:0/1401642647 shutdown_connections 2023-12-11T17:45:40.008 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.005+0000 7fbd30e68700 1 -- 172.21.15.59:0/1401642647 wait complete. 2023-12-11T17:45:40.011 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.006+0000 7fbd30e68700 1 Processor -- start 2023-12-11T17:45:40.011 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.006+0000 7fbd30e68700 1 -- start start 2023-12-11T17:45:40.012 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.006+0000 7fbd30e68700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbd2c1014a0 0x7fbd2c197420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:40.012 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.006+0000 7fbd30e68700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd2c102690 0x7fbd2c197960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:40.012 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.006+0000 7fbd30e68700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbd2c103030 0x7fbd2c19e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:40.012 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.006+0000 7fbd30e68700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd2c19ef50 con 0x7fbd2c102690 2023-12-11T17:45:40.012 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.006+0000 7fbd30e68700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd2c19f090 con 0x7fbd2c103030 2023-12-11T17:45:40.012 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.006+0000 7fbd30e68700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd2c19f220 con 0x7fbd2c1014a0 2023-12-11T17:45:40.012 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.006+0000 7fbd29d9b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd2c102690 0x7fbd2c197960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:40.012 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.006+0000 7fbd29d9b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd2c102690 0x7fbd2c197960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34068/0 (socket says 172.21.15.59:34068) 2023-12-11T17:45:40.014 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.006+0000 7fbd29d9b700 1 -- 172.21.15.59:0/1191111235 learned_addr learned my addr 172.21.15.59:0/1191111235 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:45:40.014 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.006+0000 7fbd2ad9d700 1 --2- 172.21.15.59:0/1191111235 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbd2c103030 0x7fbd2c19e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:40.014 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.006+0000 7fbd2a59c700 1 --2- 172.21.15.59:0/1191111235 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbd2c1014a0 0x7fbd2c197420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:40.014 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.006+0000 7fbd29d9b700 1 -- 172.21.15.59:0/1191111235 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbd2c1014a0 msgr2=0x7fbd2c197420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:40.014 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.006+0000 7fbd29d9b700 1 --2- 172.21.15.59:0/1191111235 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbd2c1014a0 0x7fbd2c197420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:40.014 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.006+0000 7fbd29d9b700 1 -- 172.21.15.59:0/1191111235 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbd2c103030 msgr2=0x7fbd2c19e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:40.014 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.006+0000 7fbd29d9b700 1 --2- 172.21.15.59:0/1191111235 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbd2c103030 0x7fbd2c19e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:40.014 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.006+0000 7fbd29d9b700 1 -- 172.21.15.59:0/1191111235 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd1c0095f0 con 0x7fbd2c102690 2023-12-11T17:45:40.015 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.007+0000 7fbd29d9b700 1 --2- 172.21.15.59:0/1191111235 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd2c102690 0x7fbd2c197960 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fbd1400d950 tx=0x7fbd1400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:40.015 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.007+0000 7fbd237fe700 1 -- 172.21.15.59:0/1191111235 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd1400f890 con 0x7fbd2c102690 2023-12-11T17:45:40.015 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.007+0000 7fbd30e68700 1 -- 172.21.15.59:0/1191111235 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd2c19f500 con 0x7fbd2c102690 2023-12-11T17:45:40.015 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.007+0000 7fbd237fe700 1 -- 172.21.15.59:0/1191111235 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbd14010460 con 0x7fbd2c102690 2023-12-11T17:45:40.015 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.007+0000 7fbd237fe700 1 -- 172.21.15.59:0/1191111235 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd1400e610 con 0x7fbd2c102690 2023-12-11T17:45:40.016 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.007+0000 7fbd30e68700 1 -- 172.21.15.59:0/1191111235 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd2c19fa10 con 0x7fbd2c102690 2023-12-11T17:45:40.016 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.008+0000 7fbd30e68700 1 -- 172.21.15.59:0/1191111235 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd2c1088c0 con 0x7fbd2c102690 2023-12-11T17:45:40.016 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.009+0000 7fbd237fe700 1 -- 172.21.15.59:0/1191111235 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7fbd140108d0 con 0x7fbd2c102690 2023-12-11T17:45:40.017 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.010+0000 7fbd237fe700 1 --2- 172.21.15.59:0/1191111235 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fbd180a1510 0x7fbd180a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:40.017 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.010+0000 7fbd2a59c700 1 --2- 172.21.15.59:0/1191111235 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fbd180a1510 0x7fbd180a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:40.017 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.010+0000 7fbd237fe700 1 -- 172.21.15.59:0/1191111235 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fbd140ccf60 con 0x7fbd2c102690 2023-12-11T17:45:40.017 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.010+0000 7fbd2a59c700 1 --2- 172.21.15.59:0/1191111235 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fbd180a1510 0x7fbd180a39d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fbd1c003820 tx=0x7fbd1c005920 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:40.017 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.013+0000 7fbd237fe700 1 -- 172.21.15.59:0/1191111235 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fbd14096160 con 0x7fbd2c102690 2023-12-11T17:45:40.266 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.264+0000 7fbd30e68700 1 -- 172.21.15.59:0/1191111235 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbd2c068230 con 0x7fbd2c102690 2023-12-11T17:45:40.267 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.265+0000 7fbd237fe700 1 -- 172.21.15.59:0/1191111235 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+11325 (secure 0 0 0) 0x7fbd14016090 con 0x7fbd2c102690 2023-12-11T17:45:40.267 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-12-11T17:45:40.267 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/2184151593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":2184151593},{"type":"v1","addr":"172.21.15.59:6835","nonce":2184151593}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/2817614886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2817614886},{"type":"v1","addr":"172.21.15.59:6837","nonce":2817614886}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6839/229873490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":229873490},{"type":"v1","addr":"172.21.15.59:6839","nonce":229873490}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6841/967998755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":967998755},{"type":"v1","addr":"172.21.15.59:6841","nonce":967998755}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/395684042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":395684042},{"type":"v1","addr":"172.21.15.181:6833","nonce":395684042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6833/3214606191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3214606191},{"type":"v1","addr":"172.21.15.115:6833","nonce":3214606191}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-12-11T17:44:20.076660+0000","modified":"2023-12-11T17:44:47.770695+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24521,"mds_1":24484,"mds_2":24482},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/1874159657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1874159657},{"type":"v1","addr":"172.21.15.115:6835","nonce":1874159657}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24457":{"gid":24457,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/4221183620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":4221183620},{"type":"v1","addr":"172.21.15.181:6835","nonce":4221183620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24466":{"gid":24466,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6837/3580309407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":3580309407},{"type":"v1","addr":"172.21.15.181:6837","nonce":3580309407}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24482":{"gid":24482,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.115:6837/1209365460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":1209365460},{"type":"v1","addr":"172.21.15.115:6837","nonce":1209365460}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.181:6839/2214384806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":2214384806},{"type":"v1","addr":"172.21.15.181:6839","nonce":2214384806}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.115:6839/4122922866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":4122922866},{"type":"v1","addr":"172.21.15.115:6839","nonce":4122922866}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-11T17:45:40.270 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.267+0000 7fbd30e68700 1 -- 172.21.15.59:0/1191111235 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fbd180a1510 msgr2=0x7fbd180a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:40.270 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.267+0000 7fbd30e68700 1 --2- 172.21.15.59:0/1191111235 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fbd180a1510 0x7fbd180a39d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fbd1c003820 tx=0x7fbd1c005920 comp rx=0 tx=0).stop 2023-12-11T17:45:40.270 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.267+0000 7fbd30e68700 1 -- 172.21.15.59:0/1191111235 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd2c102690 msgr2=0x7fbd2c197960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:40.270 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.267+0000 7fbd30e68700 1 --2- 172.21.15.59:0/1191111235 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd2c102690 0x7fbd2c197960 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fbd1400d950 tx=0x7fbd1400dc60 comp rx=0 tx=0).stop 2023-12-11T17:45:40.270 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.267+0000 7fbd30e68700 1 -- 172.21.15.59:0/1191111235 shutdown_connections 2023-12-11T17:45:40.270 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.267+0000 7fbd30e68700 1 --2- 172.21.15.59:0/1191111235 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fbd180a1510 0x7fbd180a39d0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:40.270 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.267+0000 7fbd30e68700 1 --2- 172.21.15.59:0/1191111235 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbd2c103030 0x7fbd2c19e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:40.270 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.267+0000 7fbd30e68700 1 --2- 172.21.15.59:0/1191111235 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbd2c1014a0 0x7fbd2c197420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:40.270 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.267+0000 7fbd30e68700 1 --2- 172.21.15.59:0/1191111235 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd2c102690 0x7fbd2c197960 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:40.270 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.267+0000 7fbd30e68700 1 -- 172.21.15.59:0/1191111235 >> 172.21.15.59:0/1191111235 conn(0x7fbd2c0fca10 msgr2=0x7fbd2c105a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:45:40.270 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.267+0000 7fbd30e68700 1 -- 172.21.15.59:0/1191111235 shutdown_connections 2023-12-11T17:45:40.271 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.267+0000 7fbd30e68700 1 -- 172.21.15.59:0/1191111235 wait complete. 2023-12-11T17:45:40.271 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 24 2023-12-11T17:45:40.282 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-11T17:45:40.408 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.406+0000 7fcb4470e700 1 -- 172.21.15.59:0/237079012 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcb3c100380 msgr2=0x7fcb3c1007a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:40.409 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.406+0000 7fcb4470e700 1 --2- 172.21.15.59:0/237079012 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcb3c100380 0x7fcb3c1007a0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fcb380099c0 tx=0x7fcb38009cd0 comp rx=0 tx=0).stop 2023-12-11T17:45:40.409 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.406+0000 7fcb4470e700 1 -- 172.21.15.59:0/237079012 shutdown_connections 2023-12-11T17:45:40.409 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.406+0000 7fcb4470e700 1 --2- 172.21.15.59:0/237079012 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb3c101f10 0x7fcb3c1087c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:40.409 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.406+0000 7fcb4470e700 1 --2- 172.21.15.59:0/237079012 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcb3c100380 0x7fcb3c1007a0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:40.409 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.406+0000 7fcb4470e700 1 --2- 172.21.15.59:0/237079012 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb3c101570 0x7fcb3c1019d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:40.409 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.406+0000 7fcb4470e700 1 -- 172.21.15.59:0/237079012 >> 172.21.15.59:0/237079012 conn(0x7fcb3c0fb930 msgr2=0x7fcb3c0fdd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:45:40.409 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.406+0000 7fcb4470e700 1 -- 172.21.15.59:0/237079012 shutdown_connections 2023-12-11T17:45:40.409 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.406+0000 7fcb4470e700 1 -- 172.21.15.59:0/237079012 wait complete. 2023-12-11T17:45:40.411 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.406+0000 7fcb4470e700 1 Processor -- start 2023-12-11T17:45:40.411 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.407+0000 7fcb4470e700 1 -- start start 2023-12-11T17:45:40.412 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.407+0000 7fcb4470e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb3c100380 0x7fcb3c196310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:40.412 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.407+0000 7fcb4470e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcb3c101570 0x7fcb3c196850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:40.412 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.407+0000 7fcb4470e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb3c101f10 0x7fcb3c19d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:40.412 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.407+0000 7fcb4470e700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb3c19de40 con 0x7fcb3c101570 2023-12-11T17:45:40.412 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.407+0000 7fcb4470e700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb3c19df80 con 0x7fcb3c101f10 2023-12-11T17:45:40.412 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.407+0000 7fcb4470e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb3c19e110 con 0x7fcb3c100380 2023-12-11T17:45:40.412 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.407+0000 7fcb41ca9700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcb3c101570 0x7fcb3c196850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:40.412 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.407+0000 7fcb41ca9700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcb3c101570 0x7fcb3c196850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34082/0 (socket says 172.21.15.59:34082) 2023-12-11T17:45:40.412 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.407+0000 7fcb41ca9700 1 -- 172.21.15.59:0/742325898 learned_addr learned my addr 172.21.15.59:0/742325898 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:45:40.412 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.407+0000 7fcb424aa700 1 --2- 172.21.15.59:0/742325898 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb3c100380 0x7fcb3c196310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:40.412 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.407+0000 7fcb42cab700 1 --2- 172.21.15.59:0/742325898 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb3c101f10 0x7fcb3c19d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:40.412 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.407+0000 7fcb41ca9700 1 -- 172.21.15.59:0/742325898 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb3c100380 msgr2=0x7fcb3c196310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:40.412 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.407+0000 7fcb41ca9700 1 --2- 172.21.15.59:0/742325898 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb3c100380 0x7fcb3c196310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:40.413 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.407+0000 7fcb41ca9700 1 -- 172.21.15.59:0/742325898 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb3c101f10 msgr2=0x7fcb3c19d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:40.413 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.407+0000 7fcb41ca9700 1 --2- 172.21.15.59:0/742325898 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb3c101f10 0x7fcb3c19d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:40.413 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.407+0000 7fcb41ca9700 1 -- 172.21.15.59:0/742325898 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb380095f0 con 0x7fcb3c101570 2023-12-11T17:45:40.413 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.407+0000 7fcb41ca9700 1 --2- 172.21.15.59:0/742325898 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcb3c101570 0x7fcb3c196850 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c00d950 tx=0x7fcb2c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:40.413 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.408+0000 7fcb337fe700 1 -- 172.21.15.59:0/742325898 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb2c009940 con 0x7fcb3c101570 2023-12-11T17:45:40.413 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.408+0000 7fcb337fe700 1 -- 172.21.15.59:0/742325898 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcb2c010460 con 0x7fcb3c101570 2023-12-11T17:45:40.413 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.408+0000 7fcb4470e700 1 -- 172.21.15.59:0/742325898 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb3c19e3f0 con 0x7fcb3c101570 2023-12-11T17:45:40.413 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.408+0000 7fcb337fe700 1 -- 172.21.15.59:0/742325898 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb2c00f620 con 0x7fcb3c101570 2023-12-11T17:45:40.413 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.408+0000 7fcb4470e700 1 -- 172.21.15.59:0/742325898 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb3c19e950 con 0x7fcb3c101570 2023-12-11T17:45:40.413 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.409+0000 7fcb4470e700 1 -- 172.21.15.59:0/742325898 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb3c107820 con 0x7fcb3c101570 2023-12-11T17:45:40.414 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.410+0000 7fcb337fe700 1 -- 172.21.15.59:0/742325898 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7fcb2c0108d0 con 0x7fcb3c101570 2023-12-11T17:45:40.414 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.410+0000 7fcb337fe700 1 --2- 172.21.15.59:0/742325898 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fcb280a1590 0x7fcb280a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:40.414 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.410+0000 7fcb337fe700 1 -- 172.21.15.59:0/742325898 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fcb2c0ccf60 con 0x7fcb3c101570 2023-12-11T17:45:40.414 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.410+0000 7fcb424aa700 1 --2- 172.21.15.59:0/742325898 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fcb280a1590 0x7fcb280a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:40.415 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.411+0000 7fcb424aa700 1 --2- 172.21.15.59:0/742325898 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fcb280a1590 0x7fcb280a3a50 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fcb38009990 tx=0x7fcb3801a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:40.417 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.415+0000 7fcb337fe700 1 -- 172.21.15.59:0/742325898 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fcb2c096160 con 0x7fcb3c101570 2023-12-11T17:45:40.625 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.623+0000 7fcb4470e700 1 -- 172.21.15.59:0/742325898 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcb3c068680 con 0x7fcb3c101570 2023-12-11T17:45:40.627 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.625+0000 7fcb337fe700 1 -- 172.21.15.59:0/742325898 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+24459 (secure 0 0 0) 0x7fcb2c016090 con 0x7fcb3c101570 2023-12-11T17:45:40.628 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-12-11T17:45:40.628 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":88,"fsid":"d0a07eaa-984b-11ee-95a3-87774f69a715","created":"2023-12-11T17:37:14.526539+0000","modified":"2023-12-11T17:45:15.870614+0000","last_up_change":"2023-12-11T17:43:38.535503+0000","last_in_change":"2023-12-11T17:43:21.014888+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-11T17:40:26.596029+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-11T17:44:18.586290+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-11T17:44:19.432952+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-11T17:44:23.097308+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3c0b3218-4b2b-40f8-8bef-ed8cd5633607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":1040759398},{"type":"v1","addr":"172.21.15.59:6803","nonce":1040759398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":1040759398},{"type":"v1","addr":"172.21.15.59:6805","nonce":1040759398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":1040759398},{"type":"v1","addr":"172.21.15.59:6809","nonce":1040759398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":1040759398},{"type":"v1","addr":"172.21.15.59:6807","nonce":1040759398}]},"public_addr":"172.21.15.59:6803/1040759398","cluster_addr":"172.21.15.59:6805/1040759398","heartbeat_back_addr":"172.21.15.59:6809/1040759398","heartbeat_front_addr":"172.21.15.59:6807/1040759398","state":["exists","up"]},{"osd":1,"uuid":"2ead7578-765b-4f5a-a72c-863731ed69e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":2776354245},{"type":"v1","addr":"172.21.15.115:6801","nonce":2776354245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":2776354245},{"type":"v1","addr":"172.21.15.115:6803","nonce":2776354245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":2776354245},{"type":"v1","addr":"172.21.15.115:6807","nonce":2776354245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":2776354245},{"type":"v1","addr":"172.21.15.115:6805","nonce":2776354245}]},"public_addr":"172.21.15.115:6801/2776354245","cluster_addr":"172.21.15.115:6803/2776354245","heartbeat_back_addr":"172.21.15.115:6807/2776354245","heartbeat_front_addr":"172.21.15.115:6805/2776354245","state":["exists","up"]},{"osd":2,"uuid":"8cb1d7ec-53cc-4672-b0b6-70d73797c44f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4287675481},{"type":"v1","addr":"172.21.15.181:6801","nonce":4287675481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4287675481},{"type":"v1","addr":"172.21.15.181:6803","nonce":4287675481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4287675481},{"type":"v1","addr":"172.21.15.181:6807","nonce":4287675481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4287675481},{"type":"v1","addr":"172.21.15.181:6805","nonce":4287675481}]},"public_addr":"172.21.15.181:6801/4287675481","cluster_addr":"172.21.15.181:6803/4287675481","heartbeat_back_addr":"172.21.15.181:6807/4287675481","heartbeat_front_addr":"172.21.15.181:6805/4287675481","state":["exists","up"]},{"osd":3,"uuid":"5d64d9af-831b-4d0a-8ace-a452169351d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":219909995},{"type":"v1","addr":"172.21.15.59:6811","nonce":219909995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":219909995},{"type":"v1","addr":"172.21.15.59:6813","nonce":219909995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":219909995},{"type":"v1","addr":"172.21.15.59:6817","nonce":219909995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":219909995},{"type":"v1","addr":"172.21.15.59:6815","nonce":219909995}]},"public_addr":"172.21.15.59:6811/219909995","cluster_addr":"172.21.15.59:6813/219909995","heartbeat_back_addr":"172.21.15.59:6817/219909995","heartbeat_front_addr":"172.21.15.59:6815/219909995","state":["exists","up"]},{"osd":4,"uuid":"ef8fad28-1db5-4cba-b5f2-10e54e8e1549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":1021473826},{"type":"v1","addr":"172.21.15.115:6809","nonce":1021473826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":1021473826},{"type":"v1","addr":"172.21.15.115:6811","nonce":1021473826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":1021473826},{"type":"v1","addr":"172.21.15.115:6815","nonce":1021473826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":1021473826},{"type":"v1","addr":"172.21.15.115:6813","nonce":1021473826}]},"public_addr":"172.21.15.115:6809/1021473826","cluster_addr":"172.21.15.115:6811/1021473826","heartbeat_back_addr":"172.21.15.115:6815/1021473826","heartbeat_front_addr":"172.21.15.115:6813/1021473826","state":["exists","up"]},{"osd":5,"uuid":"4233a512-9e8f-4db8-aed2-6c3af706e589","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":2405172521},{"type":"v1","addr":"172.21.15.181:6809","nonce":2405172521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":2405172521},{"type":"v1","addr":"172.21.15.181:6811","nonce":2405172521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2405172521},{"type":"v1","addr":"172.21.15.181:6815","nonce":2405172521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":2405172521},{"type":"v1","addr":"172.21.15.181:6813","nonce":2405172521}]},"public_addr":"172.21.15.181:6809/2405172521","cluster_addr":"172.21.15.181:6811/2405172521","heartbeat_back_addr":"172.21.15.181:6815/2405172521","heartbeat_front_addr":"172.21.15.181:6813/2405172521","state":["exists","up"]},{"osd":6,"uuid":"8faab332-016e-4c77-8370-8bef95fc6f34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":815374853},{"type":"v1","addr":"172.21.15.59:6819","nonce":815374853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":815374853},{"type":"v1","addr":"172.21.15.59:6821","nonce":815374853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":815374853},{"type":"v1","addr":"172.21.15.59:6825","nonce":815374853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":815374853},{"type":"v1","addr":"172.21.15.59:6823","nonce":815374853}]},"public_addr":"172.21.15.59:6819/815374853","cluster_addr":"172.21.15.59:6821/815374853","heartbeat_back_addr":"172.21.15.59:6825/815374853","heartbeat_front_addr":"172.21.15.59:6823/815374853","state":["exists","up"]},{"osd":7,"uuid":"f20f470a-0d40-4ed3-8e0e-fd58675cf143","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":1314971870},{"type":"v1","addr":"172.21.15.115:6817","nonce":1314971870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":1314971870},{"type":"v1","addr":"172.21.15.115:6819","nonce":1314971870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":1314971870},{"type":"v1","addr":"172.21.15.115:6823","nonce":1314971870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":1314971870},{"type":"v1","addr":"172.21.15.115:6821","nonce":1314971870}]},"public_addr":"172.21.15.115:6817/1314971870","cluster_addr":"172.21.15.115:6819/1314971870","heartbeat_back_addr":"172.21.15.115:6823/1314971870","heartbeat_front_addr":"172.21.15.115:6821/1314971870","state":["exists","up"]},{"osd":8,"uuid":"eb657d32-1a95-4fd7-a293-d0a970837fc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":3954509873},{"type":"v1","addr":"172.21.15.181:6817","nonce":3954509873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":3954509873},{"type":"v1","addr":"172.21.15.181:6819","nonce":3954509873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":3954509873},{"type":"v1","addr":"172.21.15.181:6823","nonce":3954509873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":3954509873},{"type":"v1","addr":"172.21.15.181:6821","nonce":3954509873}]},"public_addr":"172.21.15.181:6817/3954509873","cluster_addr":"172.21.15.181:6819/3954509873","heartbeat_back_addr":"172.21.15.181:6823/3954509873","heartbeat_front_addr":"172.21.15.181:6821/3954509873","state":["exists","up"]},{"osd":9,"uuid":"8f7717d4-b6f2-4ec1-9b58-62feb0c9a146","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":1163095677},{"type":"v1","addr":"172.21.15.59:6827","nonce":1163095677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":1163095677},{"type":"v1","addr":"172.21.15.59:6829","nonce":1163095677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":1163095677},{"type":"v1","addr":"172.21.15.59:6833","nonce":1163095677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":1163095677},{"type":"v1","addr":"172.21.15.59:6831","nonce":1163095677}]},"public_addr":"172.21.15.59:6827/1163095677","cluster_addr":"172.21.15.59:6829/1163095677","heartbeat_back_addr":"172.21.15.59:6833/1163095677","heartbeat_front_addr":"172.21.15.59:6831/1163095677","state":["exists","up"]},{"osd":10,"uuid":"b75c179a-875d-43bc-b4c1-e8eb80fb64fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":2199610269},{"type":"v1","addr":"172.21.15.115:6825","nonce":2199610269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":2199610269},{"type":"v1","addr":"172.21.15.115:6827","nonce":2199610269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":2199610269},{"type":"v1","addr":"172.21.15.115:6831","nonce":2199610269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":2199610269},{"type":"v1","addr":"172.21.15.115:6829","nonce":2199610269}]},"public_addr":"172.21.15.115:6825/2199610269","cluster_addr":"172.21.15.115:6827/2199610269","heartbeat_back_addr":"172.21.15.115:6831/2199610269","heartbeat_front_addr":"172.21.15.115:6829/2199610269","state":["exists","up"]},{"osd":11,"uuid":"5e141dfe-8a00-4bbc-8204-2dc9be24e18d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":226506989},{"type":"v1","addr":"172.21.15.181:6825","nonce":226506989}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":226506989},{"type":"v1","addr":"172.21.15.181:6827","nonce":226506989}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":226506989},{"type":"v1","addr":"172.21.15.181:6831","nonce":226506989}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":226506989},{"type":"v1","addr":"172.21.15.181:6829","nonce":226506989}]},"public_addr":"172.21.15.181:6825/226506989","cluster_addr":"172.21.15.181:6827/226506989","heartbeat_back_addr":"172.21.15.181:6831/226506989","heartbeat_front_addr":"172.21.15.181:6829/226506989","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:39:42.687084+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:40:03.903448+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:40:24.899908+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:40:46.600251+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:41:07.690037+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:41:28.488855+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:41:49.459274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:42:10.863881+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:42:32.248391+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:42:54.112265+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:43:15.634872+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:43:36.922534+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":7}]},{"pgid":"3.28","mappings":[{"from":5,"to":6},{"from":3,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.59:6800/1259916482":"2023-12-12T17:38:14.037922+0000","172.21.15.59:0/17457774":"2023-12-12T17:38:14.037922+0000","172.21.15.59:6801/1672530462":"2023-12-12T17:37:44.773925+0000","172.21.15.59:0/1053944547":"2023-12-12T17:38:14.037922+0000","172.21.15.59:6801/1259916482":"2023-12-12T17:38:14.037922+0000","172.21.15.59:6800/1672530462":"2023-12-12T17:37:44.773925+0000","172.21.15.59:0/3570767650":"2023-12-12T17:37:44.773925+0000","172.21.15.59:0/3853462899":"2023-12-12T17:37:44.773925+0000","172.21.15.59:0/2859415380":"2023-12-12T17:38:14.037922+0000","172.21.15.59:0/2192742819":"2023-12-12T17:37:44.773925+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-11T17:45:40.630 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.627+0000 7fcb4470e700 1 -- 172.21.15.59:0/742325898 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fcb280a1590 msgr2=0x7fcb280a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:40.630 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.627+0000 7fcb4470e700 1 --2- 172.21.15.59:0/742325898 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fcb280a1590 0x7fcb280a3a50 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fcb38009990 tx=0x7fcb3801a040 comp rx=0 tx=0).stop 2023-12-11T17:45:40.630 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.627+0000 7fcb4470e700 1 -- 172.21.15.59:0/742325898 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcb3c101570 msgr2=0x7fcb3c196850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:40.630 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.627+0000 7fcb4470e700 1 --2- 172.21.15.59:0/742325898 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcb3c101570 0x7fcb3c196850 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c00d950 tx=0x7fcb2c00dc60 comp rx=0 tx=0).stop 2023-12-11T17:45:40.630 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.627+0000 7fcb4470e700 1 -- 172.21.15.59:0/742325898 shutdown_connections 2023-12-11T17:45:40.630 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.627+0000 7fcb4470e700 1 --2- 172.21.15.59:0/742325898 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fcb280a1590 0x7fcb280a3a50 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:40.630 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.627+0000 7fcb4470e700 1 --2- 172.21.15.59:0/742325898 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb3c101f10 0x7fcb3c19d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:40.630 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.627+0000 7fcb4470e700 1 --2- 172.21.15.59:0/742325898 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb3c100380 0x7fcb3c196310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:40.631 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.627+0000 7fcb4470e700 1 --2- 172.21.15.59:0/742325898 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcb3c101570 0x7fcb3c196850 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:40.631 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.627+0000 7fcb4470e700 1 -- 172.21.15.59:0/742325898 >> 172.21.15.59:0/742325898 conn(0x7fcb3c0fb930 msgr2=0x7fcb3c104960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:45:40.631 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.627+0000 7fcb4470e700 1 -- 172.21.15.59:0/742325898 shutdown_connections 2023-12-11T17:45:40.631 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.627+0000 7fcb4470e700 1 -- 172.21.15.59:0/742325898 wait complete. 2023-12-11T17:45:40.641 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:45:40 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/2419194253' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-11T17:45:40.641 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:45:40 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/2250375854' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-11T17:45:40.641 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:45:40 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/1191111235' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T17:45:40.643 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-11T17:45:40.771 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.766+0000 7f005bf1e700 1 -- 172.21.15.59:0/730493856 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0054101570 msgr2=0x7f00541019d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:40.771 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.766+0000 7f005bf1e700 1 --2- 172.21.15.59:0/730493856 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0054101570 0x7f00541019d0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f00480099c0 tx=0x7f0048009cd0 comp rx=0 tx=0).stop 2023-12-11T17:45:40.771 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.766+0000 7f005bf1e700 1 -- 172.21.15.59:0/730493856 shutdown_connections 2023-12-11T17:45:40.771 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.766+0000 7f005bf1e700 1 --2- 172.21.15.59:0/730493856 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0054101f10 0x7f00541087c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:40.772 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.766+0000 7f005bf1e700 1 --2- 172.21.15.59:0/730493856 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0054100380 0x7f00541007a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:40.772 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.766+0000 7f005bf1e700 1 --2- 172.21.15.59:0/730493856 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0054101570 0x7f00541019d0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:40.772 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.766+0000 7f005bf1e700 1 -- 172.21.15.59:0/730493856 >> 172.21.15.59:0/730493856 conn(0x7f00540fb8f0 msgr2=0x7f00540fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:45:40.772 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.766+0000 7f005bf1e700 1 -- 172.21.15.59:0/730493856 shutdown_connections 2023-12-11T17:45:40.772 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.766+0000 7f005bf1e700 1 -- 172.21.15.59:0/730493856 wait complete. 2023-12-11T17:45:40.772 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.767+0000 7f005bf1e700 1 Processor -- start 2023-12-11T17:45:40.772 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.767+0000 7f005bf1e700 1 -- start start 2023-12-11T17:45:40.772 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.767+0000 7f005bf1e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0054100380 0x7f0054069130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:40.772 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.767+0000 7f005bf1e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0054101570 0x7f005419c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:40.772 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.767+0000 7f005bf1e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0054101f10 0x7f005419e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:40.772 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.767+0000 7f005bf1e700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f005419ee80 con 0x7f0054101f10 2023-12-11T17:45:40.772 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.767+0000 7f005bf1e700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f005419eff0 con 0x7f0054100380 2023-12-11T17:45:40.773 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.767+0000 7f005bf1e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f005419f1b0 con 0x7f0054101570 2023-12-11T17:45:40.773 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.767+0000 7f005a4bb700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0054101f10 0x7f005419e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:40.773 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.767+0000 7f0059cba700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0054100380 0x7f0054069130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:40.773 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.767+0000 7f005a4bb700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0054101f10 0x7f005419e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34098/0 (socket says 172.21.15.59:34098) 2023-12-11T17:45:40.773 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.767+0000 7f005a4bb700 1 -- 172.21.15.59:0/1054371705 learned_addr learned my addr 172.21.15.59:0/1054371705 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:45:40.773 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.767+0000 7f00594b9700 1 --2- 172.21.15.59:0/1054371705 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0054101570 0x7f005419c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:40.773 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.767+0000 7f005a4bb700 1 -- 172.21.15.59:0/1054371705 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0054101570 msgr2=0x7f005419c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:40.773 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.767+0000 7f005a4bb700 1 --2- 172.21.15.59:0/1054371705 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0054101570 0x7f005419c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:40.773 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.767+0000 7f005a4bb700 1 -- 172.21.15.59:0/1054371705 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0054100380 msgr2=0x7f0054069130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:40.774 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.767+0000 7f005a4bb700 1 --2- 172.21.15.59:0/1054371705 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0054100380 0x7f0054069130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:40.774 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.768+0000 7f005a4bb700 1 -- 172.21.15.59:0/1054371705 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00480095f0 con 0x7f0054101f10 2023-12-11T17:45:40.774 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.768+0000 7f0059cba700 1 --2- 172.21.15.59:0/1054371705 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0054100380 0x7f0054069130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T17:45:40.774 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.768+0000 7f005a4bb700 1 --2- 172.21.15.59:0/1054371705 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0054101f10 0x7f005419e6e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f004c00ef30 tx=0x7f004c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:40.774 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.768+0000 7f0046ffd700 1 -- 172.21.15.59:0/1054371705 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f004c018070 con 0x7f0054101f10 2023-12-11T17:45:40.774 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.768+0000 7f0046ffd700 1 -- 172.21.15.59:0/1054371705 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f004c00f040 con 0x7f0054101f10 2023-12-11T17:45:40.774 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.768+0000 7f0046ffd700 1 -- 172.21.15.59:0/1054371705 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f004c00cd00 con 0x7f0054101f10 2023-12-11T17:45:40.774 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.768+0000 7f005bf1e700 1 -- 172.21.15.59:0/1054371705 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f005419f430 con 0x7f0054101f10 2023-12-11T17:45:40.774 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.768+0000 7f005bf1e700 1 -- 172.21.15.59:0/1054371705 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f005419f910 con 0x7f0054101f10 2023-12-11T17:45:40.774 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.769+0000 7f005bf1e700 1 -- 172.21.15.59:0/1054371705 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00541077b0 con 0x7f0054101f10 2023-12-11T17:45:40.775 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.770+0000 7f0046ffd700 1 -- 172.21.15.59:0/1054371705 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f004c003a00 con 0x7f0054101f10 2023-12-11T17:45:40.775 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.770+0000 7f0046ffd700 1 --2- 172.21.15.59:0/1054371705 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f00400a1590 0x7f00400a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:40.775 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.771+0000 7f0046ffd700 1 -- 172.21.15.59:0/1054371705 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f004c0cd5b0 con 0x7f0054101f10 2023-12-11T17:45:40.775 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.771+0000 7f0059cba700 1 --2- 172.21.15.59:0/1054371705 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f00400a1590 0x7f00400a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:40.776 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.771+0000 7f0059cba700 1 --2- 172.21.15.59:0/1054371705 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f00400a1590 0x7f00400a3a50 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f00541013d0 tx=0x7f0050009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:40.776 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:40.774+0000 7f0046ffd700 1 -- 172.21.15.59:0/1054371705 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f004c096840 con 0x7f0054101f10 2023-12-11T17:45:40.782 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:45:40 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/2419194253' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-11T17:45:40.782 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:45:40 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/2250375854' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-11T17:45:40.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:45:40 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/1191111235' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T17:45:40.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:45:40 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/2419194253' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-11T17:45:40.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:45:40 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/2250375854' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-11T17:45:40.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:45:40 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/1191111235' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T17:45:41.023 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.021+0000 7f005bf1e700 1 -- 172.21.15.59:0/1054371705 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f005410bba0 con 0x7f0054101f10 2023-12-11T17:45:41.024 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.021+0000 7f0046ffd700 1 -- 172.21.15.59:0/1054371705 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+11325 (secure 0 0 0) 0x7f004c099e60 con 0x7f0054101f10 2023-12-11T17:45:41.024 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-12-11T17:45:41.024 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/2184151593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":2184151593},{"type":"v1","addr":"172.21.15.59:6835","nonce":2184151593}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/2817614886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2817614886},{"type":"v1","addr":"172.21.15.59:6837","nonce":2817614886}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6839/229873490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":229873490},{"type":"v1","addr":"172.21.15.59:6839","nonce":229873490}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6841/967998755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":967998755},{"type":"v1","addr":"172.21.15.59:6841","nonce":967998755}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/395684042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":395684042},{"type":"v1","addr":"172.21.15.181:6833","nonce":395684042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6833/3214606191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3214606191},{"type":"v1","addr":"172.21.15.115:6833","nonce":3214606191}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-12-11T17:44:20.076660+0000","modified":"2023-12-11T17:44:47.770695+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24521,"mds_1":24484,"mds_2":24482},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/1874159657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1874159657},{"type":"v1","addr":"172.21.15.115:6835","nonce":1874159657}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24457":{"gid":24457,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/4221183620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":4221183620},{"type":"v1","addr":"172.21.15.181:6835","nonce":4221183620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24466":{"gid":24466,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6837/3580309407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":3580309407},{"type":"v1","addr":"172.21.15.181:6837","nonce":3580309407}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24482":{"gid":24482,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.115:6837/1209365460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":1209365460},{"type":"v1","addr":"172.21.15.115:6837","nonce":1209365460}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.181:6839/2214384806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":2214384806},{"type":"v1","addr":"172.21.15.181:6839","nonce":2214384806}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.115:6839/4122922866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":4122922866},{"type":"v1","addr":"172.21.15.115:6839","nonce":4122922866}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-11T17:45:41.026 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.023+0000 7f005bf1e700 1 -- 172.21.15.59:0/1054371705 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f00400a1590 msgr2=0x7f00400a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:41.026 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.023+0000 7f005bf1e700 1 --2- 172.21.15.59:0/1054371705 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f00400a1590 0x7f00400a3a50 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f00541013d0 tx=0x7f0050009260 comp rx=0 tx=0).stop 2023-12-11T17:45:41.026 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.023+0000 7f005bf1e700 1 -- 172.21.15.59:0/1054371705 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0054101f10 msgr2=0x7f005419e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:41.026 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.023+0000 7f005bf1e700 1 --2- 172.21.15.59:0/1054371705 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0054101f10 0x7f005419e6e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f004c00ef30 tx=0x7f004c00c3a0 comp rx=0 tx=0).stop 2023-12-11T17:45:41.026 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.024+0000 7f005bf1e700 1 -- 172.21.15.59:0/1054371705 shutdown_connections 2023-12-11T17:45:41.027 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.024+0000 7f005bf1e700 1 --2- 172.21.15.59:0/1054371705 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f00400a1590 0x7f00400a3a50 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:41.027 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.024+0000 7f005bf1e700 1 --2- 172.21.15.59:0/1054371705 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0054101f10 0x7f005419e6e0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:41.027 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.024+0000 7f005bf1e700 1 --2- 172.21.15.59:0/1054371705 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0054100380 0x7f0054069130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:41.027 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.024+0000 7f005bf1e700 1 --2- 172.21.15.59:0/1054371705 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0054101570 0x7f005419c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:41.027 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.024+0000 7f005bf1e700 1 -- 172.21.15.59:0/1054371705 >> 172.21.15.59:0/1054371705 conn(0x7f00540fb8f0 msgr2=0x7f00541048f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:45:41.027 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.024+0000 7f005bf1e700 1 -- 172.21.15.59:0/1054371705 shutdown_connections 2023-12-11T17:45:41.027 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.024+0000 7f005bf1e700 1 -- 172.21.15.59:0/1054371705 wait complete. 2023-12-11T17:45:41.027 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 24 2023-12-11T17:45:41.039 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2023-12-11T17:44:20.076660+0000', 'modified': '2023-12-11T17:44:47.770695+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24521, 'mds_1': 24484, 'mds_2': 24482}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24449': {'gid': 24449, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.115:6835/1874159657', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6834', 'nonce': 1874159657}, {'type': 'v1', 'addr': '172.21.15.115:6835', 'nonce': 1874159657}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24457': {'gid': 24457, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.181:6835/4221183620', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6834', 'nonce': 4221183620}, {'type': 'v1', 'addr': '172.21.15.181:6835', 'nonce': 4221183620}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24466': {'gid': 24466, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.181:6837/3580309407', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6836', 'nonce': 3580309407}, {'type': 'v1', 'addr': '172.21.15.181:6837', 'nonce': 3580309407}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24482': {'gid': 24482, 'name': 'h', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.115:6837/1209365460', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6836', 'nonce': 1209365460}, {'type': 'v1', 'addr': '172.21.15.115:6837', 'nonce': 1209365460}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24484': {'gid': 24484, 'name': 'i', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.181:6839/2214384806', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6838', 'nonce': 2214384806}, {'type': 'v1', 'addr': '172.21.15.181:6839', 'nonce': 2214384806}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24521': {'gid': 24521, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.115:6839/4122922866', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6838', 'nonce': 4122922866}, {'type': 'v1', 'addr': '172.21.15.115:6839', 'nonce': 4122922866}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-12-11T17:45:41.039 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2023-12-11T17:45:41.039 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2023-12-11T17:45:41.169 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.164+0000 7f5de7541700 1 -- 172.21.15.59:0/3655361267 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5de0101670 msgr2=0x7f5de0101ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:41.169 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.164+0000 7f5de7541700 1 --2- 172.21.15.59:0/3655361267 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5de0101670 0x7f5de0101ad0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc0099c0 tx=0x7f5ddc009cd0 comp rx=0 tx=0).stop 2023-12-11T17:45:41.169 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.164+0000 7f5de7541700 1 -- 172.21.15.59:0/3655361267 shutdown_connections 2023-12-11T17:45:41.169 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.164+0000 7f5de7541700 1 --2- 172.21.15.59:0/3655361267 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5de0102010 0x7f5de01088c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:41.169 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.164+0000 7f5de7541700 1 --2- 172.21.15.59:0/3655361267 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5de0100480 0x7f5de01008a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:41.169 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.164+0000 7f5de7541700 1 --2- 172.21.15.59:0/3655361267 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5de0101670 0x7f5de0101ad0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:41.169 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.164+0000 7f5de7541700 1 -- 172.21.15.59:0/3655361267 >> 172.21.15.59:0/3655361267 conn(0x7f5de00fb9f0 msgr2=0x7f5de00fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:45:41.169 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.164+0000 7f5de7541700 1 -- 172.21.15.59:0/3655361267 shutdown_connections 2023-12-11T17:45:41.169 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.164+0000 7f5de7541700 1 -- 172.21.15.59:0/3655361267 wait complete. 2023-12-11T17:45:41.169 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.165+0000 7f5de7541700 1 Processor -- start 2023-12-11T17:45:41.169 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.165+0000 7f5de7541700 1 -- start start 2023-12-11T17:45:41.169 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.165+0000 7f5de7541700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5de0100480 0x7f5de01974b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:41.170 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.165+0000 7f5de7541700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5de0101670 0x7f5de01979f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:41.170 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.165+0000 7f5de7541700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5de0102010 0x7f5de019ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:41.170 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.165+0000 7f5de7541700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5de019efe0 con 0x7f5de0101670 2023-12-11T17:45:41.170 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.165+0000 7f5de7541700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5de019f120 con 0x7f5de0100480 2023-12-11T17:45:41.170 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.165+0000 7f5de7541700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5de019f2b0 con 0x7f5de0102010 2023-12-11T17:45:41.170 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.165+0000 7f5de4adc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5de0101670 0x7f5de01979f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:41.170 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.165+0000 7f5de4adc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5de0101670 0x7f5de01979f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34110/0 (socket says 172.21.15.59:34110) 2023-12-11T17:45:41.170 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.165+0000 7f5de4adc700 1 -- 172.21.15.59:0/1768487179 learned_addr learned my addr 172.21.15.59:0/1768487179 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:45:41.170 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.165+0000 7f5de52dd700 1 --2- 172.21.15.59:0/1768487179 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5de0100480 0x7f5de01974b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:41.170 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.165+0000 7f5de5ade700 1 --2- 172.21.15.59:0/1768487179 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5de0102010 0x7f5de019ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:41.170 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.165+0000 7f5de4adc700 1 -- 172.21.15.59:0/1768487179 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5de0102010 msgr2=0x7f5de019ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:41.170 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.165+0000 7f5de4adc700 1 --2- 172.21.15.59:0/1768487179 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5de0102010 0x7f5de019ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:41.170 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.165+0000 7f5de4adc700 1 -- 172.21.15.59:0/1768487179 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5de0100480 msgr2=0x7f5de01974b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:41.171 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.165+0000 7f5de4adc700 1 --2- 172.21.15.59:0/1768487179 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5de0100480 0x7f5de01974b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:41.171 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.165+0000 7f5de4adc700 1 -- 172.21.15.59:0/1768487179 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ddc0095f0 con 0x7f5de0101670 2023-12-11T17:45:41.171 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.166+0000 7f5de4adc700 1 --2- 172.21.15.59:0/1768487179 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5de0101670 0x7f5de01979f0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc0036f0 tx=0x7f5ddc0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:41.171 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.166+0000 7f5dd67fc700 1 -- 172.21.15.59:0/1768487179 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ddc01d070 con 0x7f5de0101670 2023-12-11T17:45:41.171 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.166+0000 7f5de7541700 1 -- 172.21.15.59:0/1768487179 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5de019f530 con 0x7f5de0101670 2023-12-11T17:45:41.171 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.166+0000 7f5dd67fc700 1 -- 172.21.15.59:0/1768487179 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5ddc004110 con 0x7f5de0101670 2023-12-11T17:45:41.171 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.166+0000 7f5dd67fc700 1 -- 172.21.15.59:0/1768487179 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ddc022e10 con 0x7f5de0101670 2023-12-11T17:45:41.171 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.166+0000 7f5de7541700 1 -- 172.21.15.59:0/1768487179 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5de019fa30 con 0x7f5de0101670 2023-12-11T17:45:41.178 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.168+0000 7f5dd67fc700 1 -- 172.21.15.59:0/1768487179 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f5ddc022470 con 0x7f5de0101670 2023-12-11T17:45:41.178 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.169+0000 7f5dd67fc700 1 --2- 172.21.15.59:0/1768487179 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f5dcc0a1510 0x7f5dcc0a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:41.179 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.169+0000 7f5de52dd700 1 --2- 172.21.15.59:0/1768487179 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f5dcc0a1510 0x7f5dcc0a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:41.179 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.169+0000 7f5dd67fc700 1 -- 172.21.15.59:0/1768487179 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f5ddc0ce550 con 0x7f5de0101670 2023-12-11T17:45:41.179 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.169+0000 7f5de52dd700 1 --2- 172.21.15.59:0/1768487179 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f5dcc0a1510 0x7f5dcc0a39d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f5dd0005950 tx=0x7f5dd00058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:41.179 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.170+0000 7f5df373ab80 1 Processor -- start 2023-12-11T17:45:41.179 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.170+0000 7f5df373ab80 1 -- start start 2023-12-11T17:45:41.179 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.170+0000 7f5df373ab80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bcc1c2a380 0x55bcc1c2a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:41.179 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.170+0000 7f5df373ab80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bcc1c2ace0 0x55bcc1c2b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:41.179 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.170+0000 7f5df373ab80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bcc1c2bbd0 0x55bcc1c2c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:41.179 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.170+0000 7f5df373ab80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcc1b85b00 con 0x55bcc1c2ace0 2023-12-11T17:45:41.179 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.170+0000 7f5df373ab80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcc1b83420 con 0x55bcc1c2a380 2023-12-11T17:45:41.179 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.170+0000 7f5df373ab80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcc1b7ed10 con 0x55bcc1c2bbd0 2023-12-11T17:45:41.179 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.170+0000 7f5de4adc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bcc1c2ace0 0x55bcc1c2b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:41.179 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.170+0000 7f5de4adc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bcc1c2ace0 0x55bcc1c2b5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34118/0 (socket says 172.21.15.59:34118) 2023-12-11T17:45:41.179 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.170+0000 7f5de4adc700 1 -- 172.21.15.59:0/2019273252 learned_addr learned my addr 172.21.15.59:0/2019273252 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:45:41.180 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.170+0000 7f5de5ade700 1 --2- 172.21.15.59:0/2019273252 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bcc1c2bbd0 0x55bcc1c2c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:41.180 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.170+0000 7f5de52dd700 1 --2- 172.21.15.59:0/2019273252 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bcc1c2a380 0x55bcc1c2a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:41.180 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.170+0000 7f5de4adc700 1 -- 172.21.15.59:0/2019273252 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bcc1c2bbd0 msgr2=0x55bcc1c2c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T17:45:41.180 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.170+0000 7f5de4adc700 1 --2- 172.21.15.59:0/2019273252 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bcc1c2bbd0 0x55bcc1c2c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:41.180 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.170+0000 7f5de4adc700 1 -- 172.21.15.59:0/2019273252 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bcc1c2a380 msgr2=0x55bcc1c2a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T17:45:41.180 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.170+0000 7f5de4adc700 1 --2- 172.21.15.59:0/2019273252 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bcc1c2a380 0x55bcc1c2a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:41.180 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.170+0000 7f5de4adc700 1 -- 172.21.15.59:0/2019273252 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bcc19fb5e0 con 0x55bcc1c2ace0 2023-12-11T17:45:41.180 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.170+0000 7f5de4adc700 1 --2- 172.21.15.59:0/2019273252 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bcc1c2ace0 0x55bcc1c2b5f0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc097eb0 tx=0x7f5ddc0037c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72a4b5b96c9c31f6 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:41.180 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.171+0000 7f5dcb7fe700 1 -- 172.21.15.59:0/2019273252 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ddc09bb40 con 0x55bcc1c2ace0 2023-12-11T17:45:41.180 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.171+0000 7f5dcb7fe700 1 -- 172.21.15.59:0/2019273252 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5ddc09b440 con 0x55bcc1c2ace0 2023-12-11T17:45:41.180 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.171+0000 7f5dcb7fe700 1 -- 172.21.15.59:0/2019273252 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ddc09f170 con 0x55bcc1c2ace0 2023-12-11T17:45:41.180 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.171+0000 7f5df373ab80 1 -- 172.21.15.59:0/2019273252 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bcc1c2ace0 msgr2=0x55bcc1c2b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:41.180 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.171+0000 7f5df373ab80 1 --2- 172.21.15.59:0/2019273252 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bcc1c2ace0 0x55bcc1c2b5f0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc097eb0 tx=0x7f5ddc0037c0 comp rx=0 tx=0).stop 2023-12-11T17:45:41.180 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.171+0000 7f5df373ab80 1 -- 172.21.15.59:0/2019273252 shutdown_connections 2023-12-11T17:45:41.180 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.171+0000 7f5df373ab80 1 --2- 172.21.15.59:0/2019273252 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bcc1c2bbd0 0x55bcc1c2c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:41.181 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.171+0000 7f5df373ab80 1 --2- 172.21.15.59:0/2019273252 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bcc1c2a380 0x55bcc1c2a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:41.181 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.171+0000 7f5df373ab80 1 --2- 172.21.15.59:0/2019273252 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bcc1c2ace0 0x55bcc1c2b5f0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:41.181 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.171+0000 7f5df373ab80 1 -- 172.21.15.59:0/2019273252 >> 172.21.15.59:0/2019273252 conn(0x55bcc1b59790 msgr2=0x55bcc1c26d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:45:41.181 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.171+0000 7f5df373ab80 1 -- 172.21.15.59:0/2019273252 shutdown_connections 2023-12-11T17:45:41.181 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.171+0000 7f5df373ab80 1 -- 172.21.15.59:0/2019273252 wait complete. 2023-12-11T17:45:41.181 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.172+0000 7f5df373ab80 1 -- start start 2023-12-11T17:45:41.181 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.172+0000 7f5df373ab80 1 Processor -- start 2023-12-11T17:45:41.181 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.172+0000 7f5df373ab80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bcc1c2a380 0x55bcc1c33e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:41.181 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.172+0000 7f5df373ab80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bcc1c2ace0 0x55bcc1c343d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:41.181 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.172+0000 7f5df373ab80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bcc1c2bbd0 0x55bcc1c3b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:41.181 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.172+0000 7f5df373ab80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcc1b38510 con 0x55bcc1c2a380 2023-12-11T17:45:41.181 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.172+0000 7f5df373ab80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcc1b381b0 con 0x55bcc1c2bbd0 2023-12-11T17:45:41.181 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.172+0000 7f5df373ab80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcc19fd8d0 con 0x55bcc1c2ace0 2023-12-11T17:45:41.182 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.172+0000 7f5df373ab80 10 client.0 _ll_register_callbacks cb 0x55bcc1a63a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T17:45:41.182 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.172+0000 7f5de52dd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bcc1c2a380 0x55bcc1c33e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:41.182 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.172+0000 7f5de52dd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bcc1c2a380 0x55bcc1c33e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34122/0 (socket says 172.21.15.59:34122) 2023-12-11T17:45:41.182 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.172+0000 7f5de52dd700 1 -- 172.21.15.59:0/4006375518 learned_addr learned my addr 172.21.15.59:0/4006375518 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:45:41.182 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.173+0000 7f5de5ade700 1 --2- 172.21.15.59:0/4006375518 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bcc1c2bbd0 0x55bcc1c3b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:41.182 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.173+0000 7f5de4adc700 1 --2- 172.21.15.59:0/4006375518 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bcc1c2ace0 0x55bcc1c343d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:41.182 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.173+0000 7f5de52dd700 1 -- 172.21.15.59:0/4006375518 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bcc1c2ace0 msgr2=0x55bcc1c343d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T17:45:41.182 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.173+0000 7f5de52dd700 1 --2- 172.21.15.59:0/4006375518 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bcc1c2ace0 0x55bcc1c343d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:41.182 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.173+0000 7f5de52dd700 1 -- 172.21.15.59:0/4006375518 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bcc1c2bbd0 msgr2=0x55bcc1c3b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T17:45:41.182 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.173+0000 7f5de52dd700 1 --2- 172.21.15.59:0/4006375518 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bcc1c2bbd0 0x55bcc1c3b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:41.182 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.173+0000 7f5de52dd700 1 -- 172.21.15.59:0/4006375518 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5de019fa30 con 0x55bcc1c2a380 2023-12-11T17:45:41.182 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.173+0000 7f5de52dd700 1 --2- 172.21.15.59:0/4006375518 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bcc1c2a380 0x55bcc1c33e90 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f5dd000ff60 tx=0x7f5dd0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1e090a754f24a91 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:41.182 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.173+0000 7f5dc8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2023-12-11T17:45:41.182 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.173+0000 7f5dc8ff9700 1 -- 172.21.15.59:0/4006375518 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd0028670 con 0x55bcc1c2a380 2023-12-11T17:45:41.183 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.173+0000 7f5dc8ff9700 1 -- 172.21.15.59:0/4006375518 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f5dd00287d0 con 0x55bcc1c2a380 2023-12-11T17:45:41.183 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.173+0000 7f5df373ab80 1 -- 172.21.15.59:0/4006375518 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bcc1c1daf0 con 0x55bcc1c2a380 2023-12-11T17:45:41.183 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.173+0000 7f5dc8ff9700 1 -- 172.21.15.59:0/4006375518 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5dd0011e70 con 0x55bcc1c2a380 2023-12-11T17:45:41.183 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.173+0000 7f5df373ab80 10 client.15174 fetch_fsmap learned FSMap version 24 2023-12-11T17:45:41.183 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.173+0000 7f5df373ab80 1 -- 172.21.15.59:0/4006375518 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55bcc19a7d90 con 0x55bcc1c2a380 2023-12-11T17:45:41.183 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.173+0000 7f5dc8ff9700 1 -- 172.21.15.59:0/4006375518 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55bcc1c1daf0 con 0x55bcc1c2a380 2023-12-11T17:45:41.183 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.174+0000 7f5dc8ff9700 1 -- 172.21.15.59:0/4006375518 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f5dd001fb60 con 0x55bcc1c2a380 2023-12-11T17:45:41.183 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.174+0000 7f5df373ab80 10 client.15174 fetch_fsmap finished waiting for FSMap version 24 2023-12-11T17:45:41.183 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.174+0000 7f5df373ab80 10 client.15174 resolve_mds: resolved name 'h' to daemon mds.h 2023-12-11T17:45:41.183 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.174+0000 7f5df373ab80 20 client.15174 populate_metadata read hostname 'smithi059' 2023-12-11T17:45:41.183 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.174+0000 7f5df373ab80 1 --2- 172.21.15.59:0/4006375518 >> [v2:172.21.15.115:6836/1209365460,v1:172.21.15.115:6837/1209365460] conn(0x55bcc1c3fb60 0x55bcc1c3e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:41.183 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.174+0000 7f5df373ab80 4 client.15174 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-11T17:45:41.183 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.174+0000 7f5df373ab80 1 -- 172.21.15.59:0/4006375518 --> [v2:172.21.15.115:6836/1209365460,v1:172.21.15.115:6837/1209365460] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bcc1bf9ef0 con 0x55bcc1c3fb60 2023-12-11T17:45:41.184 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.174+0000 7f5de4adc700 1 --2- 172.21.15.59:0/4006375518 >> [v2:172.21.15.115:6836/1209365460,v1:172.21.15.115:6837/1209365460] conn(0x55bcc1c3fb60 0x55bcc1c3e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:41.184 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.174+0000 7f5de4adc700 1 --2- 172.21.15.59:0/4006375518 >> [v2:172.21.15.115:6836/1209365460,v1:172.21.15.115:6837/1209365460] conn(0x55bcc1c3fb60 0x55bcc1c3e610 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6b9db56acf6da6ea server_cookie=9973785e71fb8fc3 in_seq=0 out_seq=0 2023-12-11T17:45:41.184 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.174+0000 7f5dc8ff9700 10 client.15174 ms_handle_connect on v2:172.21.15.115:6836/1209365460 2023-12-11T17:45:41.184 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.175+0000 7f5dc8ff9700 1 -- 172.21.15.59:0/4006375518 <== mds.2 v2:172.21.15.115:6836/1209365460 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55bcc1bf9ef0 con 0x55bcc1c3fb60 2023-12-11T17:45:41.184 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.175+0000 7f5dc8ff9700 10 client.15174 handle_command_reply: tid=0 2023-12-11T17:45:41.184 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.175+0000 7f5df373ab80 1 client.15174 shutdown 2023-12-11T17:45:41.184 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.175+0000 7f5df373ab80 1 -- 172.21.15.59:0/4006375518 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bcc1c2a380 msgr2=0x55bcc1c33e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:41.184 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.175+0000 7f5df373ab80 1 --2- 172.21.15.59:0/4006375518 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bcc1c2a380 0x55bcc1c33e90 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f5dd000ff60 tx=0x7f5dd0004750 comp rx=0 tx=0).stop 2023-12-11T17:45:41.184 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.175+0000 7f5df373ab80 1 -- 172.21.15.59:0/4006375518 shutdown_connections 2023-12-11T17:45:41.184 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.175+0000 7f5df373ab80 1 --2- 172.21.15.59:0/4006375518 >> [v2:172.21.15.115:6836/1209365460,v1:172.21.15.115:6837/1209365460] conn(0x55bcc1c3fb60 0x55bcc1c3e610 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:41.184 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.175+0000 7f5df373ab80 1 --2- 172.21.15.59:0/4006375518 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bcc1c2bbd0 0x55bcc1c3b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:41.184 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.175+0000 7f5df373ab80 1 --2- 172.21.15.59:0/4006375518 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bcc1c2a380 0x55bcc1c33e90 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:41.184 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.175+0000 7f5dc8ff9700 0 client.15174 ms_handle_reset on v2:172.21.15.115:6836/1209365460 2023-12-11T17:45:41.185 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.175+0000 7f5df373ab80 1 --2- 172.21.15.59:0/4006375518 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bcc1c2ace0 0x55bcc1c343d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:41.185 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.175+0000 7f5df373ab80 1 -- 172.21.15.59:0/4006375518 >> 172.21.15.59:0/4006375518 conn(0x55bcc1b59790 msgr2=0x55bcc1c259f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:45:41.185 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.176+0000 7f5df373ab80 1 -- 172.21.15.59:0/4006375518 shutdown_connections 2023-12-11T17:45:41.185 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.176+0000 7f5df373ab80 1 -- 172.21.15.59:0/4006375518 wait complete. 2023-12-11T17:45:41.185 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.176+0000 7f5df373ab80 20 client.15174 trim_cache size 0 max 16384 2023-12-11T17:45:41.198 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.194+0000 7f5df373ab80 1 Processor -- start 2023-12-11T17:45:41.198 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.194+0000 7f5df373ab80 1 -- start start 2023-12-11T17:45:41.198 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.194+0000 7f5df373ab80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bcc1c2a380 0x55bcc1c34af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:41.198 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.194+0000 7f5df373ab80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bcc1c2ace0 0x55bcc1c3b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:41.198 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.194+0000 7f5df373ab80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bcc1c2bbd0 0x55bcc1c321f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:41.198 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.194+0000 7f5df373ab80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcc1b23b50 con 0x55bcc1c2a380 2023-12-11T17:45:41.198 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.194+0000 7f5df373ab80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcc1b1a360 con 0x55bcc1c2bbd0 2023-12-11T17:45:41.198 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.194+0000 7f5df373ab80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcc1b21830 con 0x55bcc1c2ace0 2023-12-11T17:45:41.198 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.194+0000 7f5de52dd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bcc1c2a380 0x55bcc1c34af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:41.198 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.194+0000 7f5de52dd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bcc1c2a380 0x55bcc1c34af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34130/0 (socket says 172.21.15.59:34130) 2023-12-11T17:45:41.199 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.194+0000 7f5de52dd700 1 -- 172.21.15.59:0/3915820239 learned_addr learned my addr 172.21.15.59:0/3915820239 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:45:41.199 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.194+0000 7f5de52dd700 1 -- 172.21.15.59:0/3915820239 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bcc1c2ace0 msgr2=0x55bcc1c3b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T17:45:41.199 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.194+0000 7f5de52dd700 1 --2- 172.21.15.59:0/3915820239 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bcc1c2ace0 0x55bcc1c3b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:41.199 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.194+0000 7f5de52dd700 1 -- 172.21.15.59:0/3915820239 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bcc1c2bbd0 msgr2=0x55bcc1c321f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T17:45:41.199 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.194+0000 7f5de52dd700 1 --2- 172.21.15.59:0/3915820239 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bcc1c2bbd0 0x55bcc1c321f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:41.199 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.194+0000 7f5de52dd700 1 -- 172.21.15.59:0/3915820239 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bcc1990c10 con 0x55bcc1c2a380 2023-12-11T17:45:41.199 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.194+0000 7f5de52dd700 1 --2- 172.21.15.59:0/3915820239 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bcc1c2a380 0x55bcc1c34af0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f5dd000f770 tx=0x7f5dd000f7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21103c81dda86725 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:41.199 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.194+0000 7f5dcbfff700 1 -- 172.21.15.59:0/3915820239 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd001f630 con 0x55bcc1c2a380 2023-12-11T17:45:41.199 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.195+0000 7f5dcbfff700 1 -- 172.21.15.59:0/3915820239 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5dd0011ab0 con 0x55bcc1c2a380 2023-12-11T17:45:41.199 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.195+0000 7f5dcbfff700 1 -- 172.21.15.59:0/3915820239 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd001ee10 con 0x55bcc1c2a380 2023-12-11T17:45:41.199 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.195+0000 7f5df373ab80 1 -- 172.21.15.59:0/3915820239 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bcc1c2a380 msgr2=0x55bcc1c34af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:41.199 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.195+0000 7f5df373ab80 1 --2- 172.21.15.59:0/3915820239 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bcc1c2a380 0x55bcc1c34af0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f5dd000f770 tx=0x7f5dd000f7a0 comp rx=0 tx=0).stop 2023-12-11T17:45:41.199 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.195+0000 7f5df373ab80 1 -- 172.21.15.59:0/3915820239 shutdown_connections 2023-12-11T17:45:41.199 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.195+0000 7f5df373ab80 1 --2- 172.21.15.59:0/3915820239 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bcc1c2bbd0 0x55bcc1c321f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:41.199 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.195+0000 7f5df373ab80 1 --2- 172.21.15.59:0/3915820239 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bcc1c2a380 0x55bcc1c34af0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:41.199 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.195+0000 7f5df373ab80 1 --2- 172.21.15.59:0/3915820239 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bcc1c2ace0 0x55bcc1c3b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:41.200 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.195+0000 7f5df373ab80 1 -- 172.21.15.59:0/3915820239 >> 172.21.15.59:0/3915820239 conn(0x55bcc1b59790 msgr2=0x55bcc1c33050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:45:41.200 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.195+0000 7f5df373ab80 1 -- 172.21.15.59:0/3915820239 shutdown_connections 2023-12-11T17:45:41.200 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.195+0000 7f5df373ab80 1 -- 172.21.15.59:0/3915820239 wait complete. 2023-12-11T17:45:41.200 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.195+0000 7f5df373ab80 1 -- start start 2023-12-11T17:45:41.200 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.196+0000 7f5df373ab80 1 Processor -- start 2023-12-11T17:45:41.203 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.196+0000 7f5df373ab80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bcc1c2a380 0x55bcc1c3ad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:41.203 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.196+0000 7f5df373ab80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bcc1c2ace0 0x55bcc1c3b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:41.203 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.196+0000 7f5df373ab80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bcc1c2bbd0 0x55bcc1c378f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:41.203 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.196+0000 7f5df373ab80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcc1c37f90 con 0x55bcc1c2bbd0 2023-12-11T17:45:41.203 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.196+0000 7f5df373ab80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcc1c38100 con 0x55bcc1c2ace0 2023-12-11T17:45:41.204 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.196+0000 7f5df373ab80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcc1c382c0 con 0x55bcc1c2a380 2023-12-11T17:45:41.204 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.196+0000 7f5df373ab80 10 client.0 _ll_register_callbacks cb 0x55bcc1a63a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T17:45:41.204 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.196+0000 7f5de5ade700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bcc1c2bbd0 0x55bcc1c378f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:41.204 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.196+0000 7f5de5ade700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bcc1c2bbd0 0x55bcc1c378f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34140/0 (socket says 172.21.15.59:34140) 2023-12-11T17:45:41.204 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.196+0000 7f5de5ade700 1 -- 172.21.15.59:0/28662863 learned_addr learned my addr 172.21.15.59:0/28662863 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:45:41.204 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.196+0000 7f5de4adc700 1 --2- 172.21.15.59:0/28662863 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bcc1c2ace0 0x55bcc1c3b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:41.204 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.196+0000 7f5de52dd700 1 --2- 172.21.15.59:0/28662863 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bcc1c2a380 0x55bcc1c3ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:41.204 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.196+0000 7f5de5ade700 1 -- 172.21.15.59:0/28662863 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bcc1c2a380 msgr2=0x55bcc1c3ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T17:45:41.204 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.196+0000 7f5de5ade700 1 --2- 172.21.15.59:0/28662863 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bcc1c2a380 0x55bcc1c3ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:41.204 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.196+0000 7f5de5ade700 1 -- 172.21.15.59:0/28662863 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bcc1c2ace0 msgr2=0x55bcc1c3b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T17:45:41.204 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.196+0000 7f5de5ade700 1 --2- 172.21.15.59:0/28662863 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bcc1c2ace0 0x55bcc1c3b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:41.204 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.196+0000 7f5de5ade700 1 -- 172.21.15.59:0/28662863 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bcc19a7d90 con 0x55bcc1c2bbd0 2023-12-11T17:45:41.204 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.197+0000 7f5de5ade700 1 --2- 172.21.15.59:0/28662863 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bcc1c2bbd0 0x55bcc1c378f0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f5dd8007b20 tx=0x7f5dd8007e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f05bcf317594017e server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:41.204 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.197+0000 7f5dc8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2023-12-11T17:45:41.204 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.197+0000 7f5dc8ff9700 1 -- 172.21.15.59:0/28662863 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd8004bb0 con 0x55bcc1c2bbd0 2023-12-11T17:45:41.205 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.197+0000 7f5df373ab80 1 -- 172.21.15.59:0/28662863 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5ddc09b440 con 0x55bcc1c2bbd0 2023-12-11T17:45:41.205 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.197+0000 7f5dc8ff9700 1 -- 172.21.15.59:0/28662863 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f5dd8004d10 con 0x55bcc1c2bbd0 2023-12-11T17:45:41.205 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.197+0000 7f5dc8ff9700 1 -- 172.21.15.59:0/28662863 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5dd8014a00 con 0x55bcc1c2bbd0 2023-12-11T17:45:41.205 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.197+0000 7f5df373ab80 10 client.15180 fetch_fsmap learned FSMap version 24 2023-12-11T17:45:41.205 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.197+0000 7f5df373ab80 1 -- 172.21.15.59:0/28662863 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55bcc19ce200 con 0x55bcc1c2bbd0 2023-12-11T17:45:41.205 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.197+0000 7f5dc8ff9700 1 -- 172.21.15.59:0/28662863 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5ddc09b440 con 0x55bcc1c2bbd0 2023-12-11T17:45:41.205 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.197+0000 7f5dc8ff9700 1 -- 172.21.15.59:0/28662863 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f5dd8017cf0 con 0x55bcc1c2bbd0 2023-12-11T17:45:41.205 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.197+0000 7f5df373ab80 10 client.15180 fetch_fsmap finished waiting for FSMap version 24 2023-12-11T17:45:41.205 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.197+0000 7f5df373ab80 10 client.15180 resolve_mds: resolved name 'h' to daemon mds.h 2023-12-11T17:45:41.205 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.197+0000 7f5df373ab80 20 client.15180 populate_metadata read hostname 'smithi059' 2023-12-11T17:45:41.205 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.197+0000 7f5df373ab80 1 --2- 172.21.15.59:0/28662863 >> [v2:172.21.15.115:6836/1209365460,v1:172.21.15.115:6837/1209365460] conn(0x55bcc1c9ac50 0x55bcc1c970f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:41.205 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.197+0000 7f5df373ab80 4 client.15180 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "status"}] 2023-12-11T17:45:41.205 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.197+0000 7f5df373ab80 1 -- 172.21.15.59:0/28662863 --> [v2:172.21.15.115:6836/1209365460,v1:172.21.15.115:6837/1209365460] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55bcc1b44f80 con 0x55bcc1c9ac50 2023-12-11T17:45:41.205 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.198+0000 7f5de52dd700 1 --2- 172.21.15.59:0/28662863 >> [v2:172.21.15.115:6836/1209365460,v1:172.21.15.115:6837/1209365460] conn(0x55bcc1c9ac50 0x55bcc1c970f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:41.205 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.198+0000 7f5de52dd700 1 --2- 172.21.15.59:0/28662863 >> [v2:172.21.15.115:6836/1209365460,v1:172.21.15.115:6837/1209365460] conn(0x55bcc1c9ac50 0x55bcc1c970f0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=79b017d9491cf670 server_cookie=dc3421e4d46d6d8f in_seq=0 out_seq=0 2023-12-11T17:45:41.206 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.198+0000 7f5dc8ff9700 10 client.15180 ms_handle_connect on v2:172.21.15.115:6836/1209365460 2023-12-11T17:45:41.206 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.198+0000 7f5dc8ff9700 1 -- 172.21.15.59:0/28662863 <== mds.2 v2:172.21.15.115:6836/1209365460 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55bcc1b44f80 con 0x55bcc1c9ac50 2023-12-11T17:45:41.206 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.198+0000 7f5dc8ff9700 10 client.15180 handle_command_reply: tid=0 2023-12-11T17:45:41.206 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.198+0000 7f5df373ab80 1 client.15180 shutdown 2023-12-11T17:45:41.206 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.199+0000 7f5df373ab80 1 -- 172.21.15.59:0/28662863 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bcc1c2bbd0 msgr2=0x55bcc1c378f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:41.206 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.199+0000 7f5df373ab80 1 --2- 172.21.15.59:0/28662863 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bcc1c2bbd0 0x55bcc1c378f0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f5dd8007b20 tx=0x7f5dd8007e30 comp rx=0 tx=0).stop 2023-12-11T17:45:41.206 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.199+0000 7f5df373ab80 1 -- 172.21.15.59:0/28662863 shutdown_connections 2023-12-11T17:45:41.206 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.199+0000 7f5df373ab80 1 --2- 172.21.15.59:0/28662863 >> [v2:172.21.15.115:6836/1209365460,v1:172.21.15.115:6837/1209365460] conn(0x55bcc1c9ac50 0x55bcc1c970f0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:41.206 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.199+0000 7f5df373ab80 1 --2- 172.21.15.59:0/28662863 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bcc1c2bbd0 0x55bcc1c378f0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:41.206 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.199+0000 7f5dc8ff9700 0 client.15180 ms_handle_reset on v2:172.21.15.115:6836/1209365460 2023-12-11T17:45:41.206 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.199+0000 7f5df373ab80 1 --2- 172.21.15.59:0/28662863 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bcc1c2a380 0x55bcc1c3ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:41.206 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.199+0000 7f5df373ab80 1 --2- 172.21.15.59:0/28662863 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55bcc1c2ace0 0x55bcc1c3b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:41.206 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.199+0000 7f5df373ab80 1 -- 172.21.15.59:0/28662863 >> 172.21.15.59:0/28662863 conn(0x55bcc1b59790 msgr2=0x55bcc1c33050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:45:41.206 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.199+0000 7f5df373ab80 1 -- 172.21.15.59:0/28662863 shutdown_connections 2023-12-11T17:45:41.206 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.199+0000 7f5df373ab80 1 -- 172.21.15.59:0/28662863 wait complete. 2023-12-11T17:45:41.207 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.199+0000 7f5df373ab80 20 client.15180 trim_cache size 0 max 16384 2023-12-11T17:45:41.207 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2023-12-11T17:45:41.207 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_fsid": "d0a07eaa-984b-11ee-95a3-87774f69a715", 2023-12-11T17:45:41.207 INFO:teuthology.orchestra.run.smithi059.stdout: "whoami": 2, 2023-12-11T17:45:41.207 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 24482, 2023-12-11T17:45:41.207 INFO:teuthology.orchestra.run.smithi059.stdout: "want_state": "up:active", 2023-12-11T17:45:41.207 INFO:teuthology.orchestra.run.smithi059.stdout: "state": "up:active", 2023-12-11T17:45:41.207 INFO:teuthology.orchestra.run.smithi059.stdout: "fs_name": "cephfs", 2023-12-11T17:45:41.207 INFO:teuthology.orchestra.run.smithi059.stdout: "rank_uptime": 54.433248372000001, 2023-12-11T17:45:41.207 INFO:teuthology.orchestra.run.smithi059.stdout: "mdsmap_epoch": 24, 2023-12-11T17:45:41.207 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch": 88, 2023-12-11T17:45:41.208 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch_barrier": 86, 2023-12-11T17:45:41.208 INFO:teuthology.orchestra.run.smithi059.stdout: "uptime": 95.237445520999998 2023-12-11T17:45:41.208 INFO:teuthology.orchestra.run.smithi059.stdout:} 2023-12-11T17:45:41.209 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.199+0000 7f5de7541700 1 -- 172.21.15.59:0/1768487179 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f5dcc0a1510 msgr2=0x7f5dcc0a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:41.209 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.199+0000 7f5de7541700 1 --2- 172.21.15.59:0/1768487179 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f5dcc0a1510 0x7f5dcc0a39d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f5dd0005950 tx=0x7f5dd00058e0 comp rx=0 tx=0).stop 2023-12-11T17:45:41.209 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.199+0000 7f5de7541700 1 -- 172.21.15.59:0/1768487179 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5de0101670 msgr2=0x7f5de01979f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:41.209 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.199+0000 7f5de7541700 1 --2- 172.21.15.59:0/1768487179 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5de0101670 0x7f5de01979f0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc0036f0 tx=0x7f5ddc0047b0 comp rx=0 tx=0).stop 2023-12-11T17:45:41.209 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.199+0000 7f5de7541700 1 -- 172.21.15.59:0/1768487179 shutdown_connections 2023-12-11T17:45:41.209 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.199+0000 7f5de7541700 1 --2- 172.21.15.59:0/1768487179 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f5dcc0a1510 0x7f5dcc0a39d0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:41.209 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.199+0000 7f5de7541700 1 --2- 172.21.15.59:0/1768487179 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5de0102010 0x7f5de019ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:41.209 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.199+0000 7f5de7541700 1 --2- 172.21.15.59:0/1768487179 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5de0100480 0x7f5de01974b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:41.209 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.199+0000 7f5de7541700 1 --2- 172.21.15.59:0/1768487179 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5de0101670 0x7f5de01979f0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:41.209 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.199+0000 7f5de7541700 1 -- 172.21.15.59:0/1768487179 >> 172.21.15.59:0/1768487179 conn(0x7f5de00fb9f0 msgr2=0x7f5de00fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:45:41.209 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.199+0000 7f5de7541700 1 -- 172.21.15.59:0/1768487179 shutdown_connections 2023-12-11T17:45:41.209 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.199+0000 7f5de7541700 1 -- 172.21.15.59:0/1768487179 wait complete. 2023-12-11T17:45:41.214 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2023-12-11T17:45:41.341 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.338+0000 7f53c3b08700 1 -- 172.21.15.59:0/188801436 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53bc1004b0 msgr2=0x7f53bc1008d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:41.341 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.338+0000 7f53c3b08700 1 --2- 172.21.15.59:0/188801436 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53bc1004b0 0x7f53bc1008d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f53ac009990 tx=0x7f53ac009ca0 comp rx=0 tx=0).stop 2023-12-11T17:45:41.341 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.338+0000 7f53c3b08700 1 -- 172.21.15.59:0/188801436 shutdown_connections 2023-12-11T17:45:41.341 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.338+0000 7f53c3b08700 1 --2- 172.21.15.59:0/188801436 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f53bc102040 0x7f53bc1088f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:41.341 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.338+0000 7f53c3b08700 1 --2- 172.21.15.59:0/188801436 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53bc1004b0 0x7f53bc1008d0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:41.341 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.338+0000 7f53c3b08700 1 --2- 172.21.15.59:0/188801436 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f53bc1016a0 0x7f53bc101b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:41.342 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.338+0000 7f53c3b08700 1 -- 172.21.15.59:0/188801436 >> 172.21.15.59:0/188801436 conn(0x7f53bc0fba20 msgr2=0x7f53bc0fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:45:41.342 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.338+0000 7f53c3b08700 1 -- 172.21.15.59:0/188801436 shutdown_connections 2023-12-11T17:45:41.342 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.338+0000 7f53c3b08700 1 -- 172.21.15.59:0/188801436 wait complete. 2023-12-11T17:45:41.344 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.339+0000 7f53c3b08700 1 Processor -- start 2023-12-11T17:45:41.344 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.339+0000 7f53c3b08700 1 -- start start 2023-12-11T17:45:41.344 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.339+0000 7f53c3b08700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f53bc1004b0 0x7f53bc196440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:41.344 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.339+0000 7f53c3b08700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f53bc1016a0 0x7f53bc196980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:41.344 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.339+0000 7f53c3b08700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53bc102040 0x7f53bc19d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:41.344 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.339+0000 7f53c3b08700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53bc19df70 con 0x7f53bc1016a0 2023-12-11T17:45:41.344 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.339+0000 7f53c3b08700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53bc19e0b0 con 0x7f53bc1004b0 2023-12-11T17:45:41.344 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.339+0000 7f53c3b08700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53bc19e240 con 0x7f53bc102040 2023-12-11T17:45:41.344 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.340+0000 7f53c10a3700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f53bc1016a0 0x7f53bc196980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:41.345 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.340+0000 7f53c10a3700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f53bc1016a0 0x7f53bc196980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34152/0 (socket says 172.21.15.59:34152) 2023-12-11T17:45:41.345 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.340+0000 7f53c10a3700 1 -- 172.21.15.59:0/2842434543 learned_addr learned my addr 172.21.15.59:0/2842434543 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:45:41.345 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.340+0000 7f53c18a4700 1 --2- 172.21.15.59:0/2842434543 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f53bc1004b0 0x7f53bc196440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:41.345 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.340+0000 7f53c20a5700 1 --2- 172.21.15.59:0/2842434543 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53bc102040 0x7f53bc19d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:41.345 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.340+0000 7f53c10a3700 1 -- 172.21.15.59:0/2842434543 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53bc102040 msgr2=0x7f53bc19d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:41.345 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.340+0000 7f53c10a3700 1 --2- 172.21.15.59:0/2842434543 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53bc102040 0x7f53bc19d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:41.345 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.340+0000 7f53c10a3700 1 -- 172.21.15.59:0/2842434543 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f53bc1004b0 msgr2=0x7f53bc196440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:41.345 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.340+0000 7f53c10a3700 1 --2- 172.21.15.59:0/2842434543 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f53bc1004b0 0x7f53bc196440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:41.345 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.340+0000 7f53c10a3700 1 -- 172.21.15.59:0/2842434543 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53b8009520 con 0x7f53bc1016a0 2023-12-11T17:45:41.345 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.340+0000 7f53c18a4700 1 --2- 172.21.15.59:0/2842434543 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f53bc1004b0 0x7f53bc196440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T17:45:41.345 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.340+0000 7f53c10a3700 1 --2- 172.21.15.59:0/2842434543 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f53bc1016a0 0x7f53bc196980 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f53b800eb30 tx=0x7f53b800eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:41.345 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.340+0000 7f53b2ffd700 1 -- 172.21.15.59:0/2842434543 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53b8009880 con 0x7f53bc1016a0 2023-12-11T17:45:41.345 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.340+0000 7f53c3b08700 1 -- 172.21.15.59:0/2842434543 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53ac0095f0 con 0x7f53bc1016a0 2023-12-11T17:45:41.345 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.340+0000 7f53b2ffd700 1 -- 172.21.15.59:0/2842434543 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f53b80044e0 con 0x7f53bc1016a0 2023-12-11T17:45:41.345 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.340+0000 7f53b2ffd700 1 -- 172.21.15.59:0/2842434543 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53b8010430 con 0x7f53bc1016a0 2023-12-11T17:45:41.346 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.340+0000 7f53c3b08700 1 -- 172.21.15.59:0/2842434543 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53bc069a50 con 0x7f53bc1016a0 2023-12-11T17:45:41.353 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.343+0000 7f53b2ffd700 1 -- 172.21.15.59:0/2842434543 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f53b8003ee0 con 0x7f53bc1016a0 2023-12-11T17:45:41.353 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.343+0000 7f53b2ffd700 1 --2- 172.21.15.59:0/2842434543 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f53a80a1510 0x7f53a80a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:41.353 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.343+0000 7f53b2ffd700 1 -- 172.21.15.59:0/2842434543 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f53b8014070 con 0x7f53bc1016a0 2023-12-11T17:45:41.353 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.343+0000 7f53c18a4700 1 --2- 172.21.15.59:0/2842434543 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f53a80a1510 0x7f53a80a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:41.353 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.344+0000 7f53c18a4700 1 --2- 172.21.15.59:0/2842434543 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f53a80a1510 0x7f53a80a39d0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f53ac006010 tx=0x7f53ac00b770 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:41.353 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.344+0000 7f53cfd01b80 1 Processor -- start 2023-12-11T17:45:41.353 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.344+0000 7f53cfd01b80 1 -- start start 2023-12-11T17:45:41.353 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.344+0000 7f53cfd01b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5635e2726790 0x5635e2726bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:41.353 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.344+0000 7f53cfd01b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5635e27270f0 0x5635e2727a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:41.353 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.344+0000 7f53cfd01b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5635e2727fe0 0x5635e2728460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:41.353 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.344+0000 7f53cfd01b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635e267db00 con 0x5635e2726790 2023-12-11T17:45:41.353 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.344+0000 7f53cfd01b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635e267b420 con 0x5635e2727fe0 2023-12-11T17:45:41.354 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.344+0000 7f53cfd01b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635e2676d10 con 0x5635e27270f0 2023-12-11T17:45:41.354 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.344+0000 7f53c18a4700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5635e2726790 0x5635e2726bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:41.354 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.344+0000 7f53c18a4700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5635e2726790 0x5635e2726bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34158/0 (socket says 172.21.15.59:34158) 2023-12-11T17:45:41.354 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.344+0000 7f53c18a4700 1 -- 172.21.15.59:0/4001168841 learned_addr learned my addr 172.21.15.59:0/4001168841 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:45:41.354 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.345+0000 7f53c10a3700 1 --2- 172.21.15.59:0/4001168841 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5635e27270f0 0x5635e2727a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:41.354 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.345+0000 7f53c20a5700 1 --2- 172.21.15.59:0/4001168841 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5635e2727fe0 0x5635e2728460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:41.354 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.345+0000 7f53c18a4700 1 -- 172.21.15.59:0/4001168841 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5635e27270f0 msgr2=0x5635e2727a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T17:45:41.354 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.345+0000 7f53c18a4700 1 --2- 172.21.15.59:0/4001168841 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5635e27270f0 0x5635e2727a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:41.354 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.345+0000 7f53c18a4700 1 -- 172.21.15.59:0/4001168841 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5635e2727fe0 msgr2=0x5635e2728460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T17:45:41.354 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.345+0000 7f53c18a4700 1 --2- 172.21.15.59:0/4001168841 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5635e2727fe0 0x5635e2728460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:41.354 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.345+0000 7f53c18a4700 1 -- 172.21.15.59:0/4001168841 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5635e24f35e0 con 0x5635e2726790 2023-12-11T17:45:41.354 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.345+0000 7f53c18a4700 1 --2- 172.21.15.59:0/4001168841 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5635e2726790 0x5635e2726bb0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f53ac015000 tx=0x7f53ac01f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9533d950293325b6 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:41.354 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.345+0000 7f53a7fff700 1 -- 172.21.15.59:0/4001168841 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53ac02b070 con 0x5635e2726790 2023-12-11T17:45:41.354 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.345+0000 7f53a7fff700 1 -- 172.21.15.59:0/4001168841 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f53ac01e040 con 0x5635e2726790 2023-12-11T17:45:41.355 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.345+0000 7f53a7fff700 1 -- 172.21.15.59:0/4001168841 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53ac00f4e0 con 0x5635e2726790 2023-12-11T17:45:41.355 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.346+0000 7f53cfd01b80 1 -- 172.21.15.59:0/4001168841 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5635e2726790 msgr2=0x5635e2726bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:41.355 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.346+0000 7f53cfd01b80 1 --2- 172.21.15.59:0/4001168841 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5635e2726790 0x5635e2726bb0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f53ac015000 tx=0x7f53ac01f000 comp rx=0 tx=0).stop 2023-12-11T17:45:41.355 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.346+0000 7f53cfd01b80 1 -- 172.21.15.59:0/4001168841 shutdown_connections 2023-12-11T17:45:41.355 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.346+0000 7f53cfd01b80 1 --2- 172.21.15.59:0/4001168841 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5635e2727fe0 0x5635e2728460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:41.355 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.346+0000 7f53cfd01b80 1 --2- 172.21.15.59:0/4001168841 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5635e2726790 0x5635e2726bb0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:41.355 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.346+0000 7f53cfd01b80 1 --2- 172.21.15.59:0/4001168841 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5635e27270f0 0x5635e2727a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:41.355 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.346+0000 7f53cfd01b80 1 -- 172.21.15.59:0/4001168841 >> 172.21.15.59:0/4001168841 conn(0x5635e2651790 msgr2=0x5635e271ed40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:45:41.355 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.346+0000 7f53cfd01b80 1 -- 172.21.15.59:0/4001168841 shutdown_connections 2023-12-11T17:45:41.355 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.346+0000 7f53cfd01b80 1 -- 172.21.15.59:0/4001168841 wait complete. 2023-12-11T17:45:41.355 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.346+0000 7f53cfd01b80 1 -- start start 2023-12-11T17:45:41.355 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.346+0000 7f53cfd01b80 1 Processor -- start 2023-12-11T17:45:41.355 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.347+0000 7f53cfd01b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5635e2726790 0x5635e2730500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:41.355 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.347+0000 7f53cfd01b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5635e27270f0 0x5635e2730a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:41.356 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.347+0000 7f53cfd01b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5635e2727fe0 0x5635e2737a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:41.356 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.347+0000 7f53cfd01b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635e2630510 con 0x5635e2726790 2023-12-11T17:45:41.356 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.347+0000 7f53cfd01b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635e26301b0 con 0x5635e27270f0 2023-12-11T17:45:41.356 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.347+0000 7f53cfd01b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635e24f58d0 con 0x5635e2727fe0 2023-12-11T17:45:41.356 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.347+0000 7f53cfd01b80 10 client.0 _ll_register_callbacks cb 0x5635e255ba30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T17:45:41.356 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.347+0000 7f53c18a4700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5635e2726790 0x5635e2730500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:41.356 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.347+0000 7f53c18a4700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5635e2726790 0x5635e2730500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34164/0 (socket says 172.21.15.59:34164) 2023-12-11T17:45:41.356 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.347+0000 7f53c18a4700 1 -- 172.21.15.59:0/2658823958 learned_addr learned my addr 172.21.15.59:0/2658823958 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:45:41.356 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.347+0000 7f53c20a5700 1 --2- 172.21.15.59:0/2658823958 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5635e2727fe0 0x5635e2737a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:41.356 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.347+0000 7f53c10a3700 1 --2- 172.21.15.59:0/2658823958 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5635e27270f0 0x5635e2730a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:41.356 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.347+0000 7f53c18a4700 1 -- 172.21.15.59:0/2658823958 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5635e2727fe0 msgr2=0x5635e2737a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T17:45:41.356 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.347+0000 7f53c18a4700 1 --2- 172.21.15.59:0/2658823958 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5635e2727fe0 0x5635e2737a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:41.356 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.347+0000 7f53c18a4700 1 -- 172.21.15.59:0/2658823958 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5635e27270f0 msgr2=0x5635e2730a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T17:45:41.356 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.347+0000 7f53c18a4700 1 --2- 172.21.15.59:0/2658823958 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5635e27270f0 0x5635e2730a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:41.356 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.347+0000 7f53c18a4700 1 -- 172.21.15.59:0/2658823958 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53bc109240 con 0x5635e2726790 2023-12-11T17:45:41.357 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.347+0000 7f53c18a4700 1 --2- 172.21.15.59:0/2658823958 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5635e2726790 0x5635e2730500 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f53ac009990 tx=0x7f53ac0053e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d233393379174b7d server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:41.357 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.347+0000 7f53a57fa700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2023-12-11T17:45:41.357 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.348+0000 7f53a57fa700 1 -- 172.21.15.59:0/2658823958 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53ac02b040 con 0x5635e2726790 2023-12-11T17:45:41.357 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.348+0000 7f53a57fa700 1 -- 172.21.15.59:0/2658823958 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f53ac00f8f0 con 0x5635e2726790 2023-12-11T17:45:41.357 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.348+0000 7f53cfd01b80 1 -- 172.21.15.59:0/2658823958 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5635e2715af0 con 0x5635e2726790 2023-12-11T17:45:41.357 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.348+0000 7f53a57fa700 1 -- 172.21.15.59:0/2658823958 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f53ac005940 con 0x5635e2726790 2023-12-11T17:45:41.357 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.348+0000 7f53cfd01b80 10 client.15192 fetch_fsmap learned FSMap version 24 2023-12-11T17:45:41.357 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.348+0000 7f53cfd01b80 1 -- 172.21.15.59:0/2658823958 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5635e249fd90 con 0x5635e2726790 2023-12-11T17:45:41.358 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.348+0000 7f53a57fa700 1 -- 172.21.15.59:0/2658823958 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f53ac040080 con 0x5635e2726790 2023-12-11T17:45:41.358 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.348+0000 7f53a57fa700 1 -- 172.21.15.59:0/2658823958 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f53ac024940 con 0x5635e2726790 2023-12-11T17:45:41.358 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.348+0000 7f53cfd01b80 10 client.15192 fetch_fsmap finished waiting for FSMap version 24 2023-12-11T17:45:41.358 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.348+0000 7f53cfd01b80 10 client.15192 resolve_mds: resolved name 'i' to daemon mds.i 2023-12-11T17:45:41.358 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.348+0000 7f53cfd01b80 20 client.15192 populate_metadata read hostname 'smithi059' 2023-12-11T17:45:41.358 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.348+0000 7f53cfd01b80 1 --2- 172.21.15.59:0/2658823958 >> [v2:172.21.15.181:6838/2214384806,v1:172.21.15.181:6839/2214384806] conn(0x5635e273bc70 0x5635e273aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:41.358 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.348+0000 7f53cfd01b80 4 client.15192 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-11T17:45:41.358 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.348+0000 7f53cfd01b80 1 -- 172.21.15.59:0/2658823958 --> [v2:172.21.15.181:6838/2214384806,v1:172.21.15.181:6839/2214384806] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5635e26f1ef0 con 0x5635e273bc70 2023-12-11T17:45:41.358 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.349+0000 7f53c10a3700 1 --2- 172.21.15.59:0/2658823958 >> [v2:172.21.15.181:6838/2214384806,v1:172.21.15.181:6839/2214384806] conn(0x5635e273bc70 0x5635e273aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:41.358 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.349+0000 7f53a57fa700 10 client.15192 ms_handle_connect on v2:172.21.15.181:6838/2214384806 2023-12-11T17:45:41.358 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.349+0000 7f53c10a3700 1 --2- 172.21.15.59:0/2658823958 >> [v2:172.21.15.181:6838/2214384806,v1:172.21.15.181:6839/2214384806] conn(0x5635e273bc70 0x5635e273aa50 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bd928043c3feaf4d server_cookie=a884464d6638b4f3 in_seq=0 out_seq=0 2023-12-11T17:45:41.358 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.350+0000 7f53a57fa700 1 -- 172.21.15.59:0/2658823958 <== mds.1 v2:172.21.15.181:6838/2214384806 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5635e26f1ef0 con 0x5635e273bc70 2023-12-11T17:45:41.358 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.350+0000 7f53a57fa700 10 client.15192 handle_command_reply: tid=0 2023-12-11T17:45:41.358 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.350+0000 7f53cfd01b80 1 client.15192 shutdown 2023-12-11T17:45:41.358 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.350+0000 7f53cfd01b80 1 -- 172.21.15.59:0/2658823958 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5635e2726790 msgr2=0x5635e2730500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:41.359 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.350+0000 7f53cfd01b80 1 --2- 172.21.15.59:0/2658823958 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5635e2726790 0x5635e2730500 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f53ac009990 tx=0x7f53ac0053e0 comp rx=0 tx=0).stop 2023-12-11T17:45:41.359 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.350+0000 7f53cfd01b80 1 -- 172.21.15.59:0/2658823958 shutdown_connections 2023-12-11T17:45:41.359 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.350+0000 7f53cfd01b80 1 --2- 172.21.15.59:0/2658823958 >> [v2:172.21.15.181:6838/2214384806,v1:172.21.15.181:6839/2214384806] conn(0x5635e273bc70 0x5635e273aa50 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:41.359 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.350+0000 7f53cfd01b80 1 --2- 172.21.15.59:0/2658823958 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5635e2727fe0 0x5635e2737a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:41.359 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.350+0000 7f53a57fa700 0 client.15192 ms_handle_reset on v2:172.21.15.181:6838/2214384806 2023-12-11T17:45:41.359 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.350+0000 7f53cfd01b80 1 --2- 172.21.15.59:0/2658823958 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5635e2726790 0x5635e2730500 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:41.359 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.350+0000 7f53cfd01b80 1 --2- 172.21.15.59:0/2658823958 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5635e27270f0 0x5635e2730a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:41.359 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.350+0000 7f53cfd01b80 1 -- 172.21.15.59:0/2658823958 >> 172.21.15.59:0/2658823958 conn(0x5635e2651790 msgr2=0x5635e271d9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:45:41.359 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.350+0000 7f53cfd01b80 1 -- 172.21.15.59:0/2658823958 shutdown_connections 2023-12-11T17:45:41.359 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.350+0000 7f53cfd01b80 1 -- 172.21.15.59:0/2658823958 wait complete. 2023-12-11T17:45:41.359 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.350+0000 7f53cfd01b80 20 client.15192 trim_cache size 0 max 16384 2023-12-11T17:45:41.376 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.369+0000 7f53cfd01b80 1 Processor -- start 2023-12-11T17:45:41.376 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.369+0000 7f53cfd01b80 1 -- start start 2023-12-11T17:45:41.376 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.369+0000 7f53cfd01b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5635e2726790 0x5635e272af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:41.377 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.369+0000 7f53cfd01b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5635e27270f0 0x5635e272b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:41.377 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.369+0000 7f53cfd01b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5635e2727fe0 0x5635e2737a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:41.377 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.369+0000 7f53cfd01b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635e2672e00 con 0x5635e27270f0 2023-12-11T17:45:41.377 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.369+0000 7f53cfd01b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635e261bb50 con 0x5635e2727fe0 2023-12-11T17:45:41.377 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.369+0000 7f53cfd01b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635e2619830 con 0x5635e2726790 2023-12-11T17:45:41.377 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.369+0000 7f53c10a3700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5635e27270f0 0x5635e272b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:41.377 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.369+0000 7f53c10a3700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5635e27270f0 0x5635e272b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34174/0 (socket says 172.21.15.59:34174) 2023-12-11T17:45:41.377 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.369+0000 7f53c10a3700 1 -- 172.21.15.59:0/1955995886 learned_addr learned my addr 172.21.15.59:0/1955995886 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:45:41.377 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.369+0000 7f53c18a4700 1 --2- 172.21.15.59:0/1955995886 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5635e2726790 0x5635e272af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:41.377 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.369+0000 7f53c20a5700 1 --2- 172.21.15.59:0/1955995886 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5635e2727fe0 0x5635e2737a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:41.378 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.369+0000 7f53c10a3700 1 -- 172.21.15.59:0/1955995886 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5635e2726790 msgr2=0x5635e272af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T17:45:41.378 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.369+0000 7f53c10a3700 1 --2- 172.21.15.59:0/1955995886 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5635e2726790 0x5635e272af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:41.378 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.369+0000 7f53c10a3700 1 -- 172.21.15.59:0/1955995886 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5635e2727fe0 msgr2=0x5635e2737a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T17:45:41.378 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.369+0000 7f53c10a3700 1 --2- 172.21.15.59:0/1955995886 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5635e2727fe0 0x5635e2737a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:41.379 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.369+0000 7f53c10a3700 1 -- 172.21.15.59:0/1955995886 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5635e2488c10 con 0x5635e27270f0 2023-12-11T17:45:41.379 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.369+0000 7f53c18a4700 1 --2- 172.21.15.59:0/1955995886 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5635e2726790 0x5635e272af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T17:45:41.379 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.369+0000 7f53c20a5700 1 --2- 172.21.15.59:0/1955995886 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5635e2727fe0 0x5635e2737a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-11T17:45:41.379 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.369+0000 7f53c10a3700 1 --2- 172.21.15.59:0/1955995886 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5635e27270f0 0x5635e272b450 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f53b8000c00 tx=0x7f53b800cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eef774803428f441 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:41.379 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.369+0000 7f53b0d82700 1 -- 172.21.15.59:0/1955995886 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53b800cd40 con 0x5635e27270f0 2023-12-11T17:45:41.379 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.370+0000 7f53b0d82700 1 -- 172.21.15.59:0/1955995886 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f53b809a550 con 0x5635e27270f0 2023-12-11T17:45:41.379 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.370+0000 7f53b0d82700 1 -- 172.21.15.59:0/1955995886 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53b80988f0 con 0x5635e27270f0 2023-12-11T17:45:41.379 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.370+0000 7f53cfd01b80 1 -- 172.21.15.59:0/1955995886 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5635e27270f0 msgr2=0x5635e272b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:41.379 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.370+0000 7f53cfd01b80 1 --2- 172.21.15.59:0/1955995886 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5635e27270f0 0x5635e272b450 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f53b8000c00 tx=0x7f53b800cf40 comp rx=0 tx=0).stop 2023-12-11T17:45:41.379 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.370+0000 7f53cfd01b80 1 -- 172.21.15.59:0/1955995886 shutdown_connections 2023-12-11T17:45:41.380 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.370+0000 7f53cfd01b80 1 --2- 172.21.15.59:0/1955995886 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5635e2727fe0 0x5635e2737a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:41.380 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.370+0000 7f53cfd01b80 1 --2- 172.21.15.59:0/1955995886 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5635e2726790 0x5635e272af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:41.380 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.370+0000 7f53cfd01b80 1 --2- 172.21.15.59:0/1955995886 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5635e27270f0 0x5635e272b450 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:41.380 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.370+0000 7f53cfd01b80 1 -- 172.21.15.59:0/1955995886 >> 172.21.15.59:0/1955995886 conn(0x5635e2651790 msgr2=0x5635e27445d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:45:41.380 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.370+0000 7f53cfd01b80 1 -- 172.21.15.59:0/1955995886 shutdown_connections 2023-12-11T17:45:41.380 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.370+0000 7f53cfd01b80 1 -- 172.21.15.59:0/1955995886 wait complete. 2023-12-11T17:45:41.380 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.371+0000 7f53cfd01b80 1 -- start start 2023-12-11T17:45:41.380 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.371+0000 7f53cfd01b80 1 Processor -- start 2023-12-11T17:45:41.380 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.371+0000 7f53cfd01b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5635e2726790 0x5635e27321e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:41.380 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.371+0000 7f53cfd01b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5635e27270f0 0x5635e2732720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:41.380 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.371+0000 7f53cfd01b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5635e2727fe0 0x5635e2737570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:41.380 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.371+0000 7f53cfd01b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635e2737ab0 con 0x5635e2726790 2023-12-11T17:45:41.380 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.371+0000 7f53cfd01b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635e2737c20 con 0x5635e27270f0 2023-12-11T17:45:41.380 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.371+0000 7f53cfd01b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635e2737de0 con 0x5635e2727fe0 2023-12-11T17:45:41.381 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.371+0000 7f53cfd01b80 10 client.0 _ll_register_callbacks cb 0x5635e255ba30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T17:45:41.381 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.371+0000 7f53c18a4700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5635e2726790 0x5635e27321e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:41.381 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.371+0000 7f53c18a4700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5635e2726790 0x5635e27321e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34178/0 (socket says 172.21.15.59:34178) 2023-12-11T17:45:41.381 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.371+0000 7f53c18a4700 1 -- 172.21.15.59:0/2404231530 learned_addr learned my addr 172.21.15.59:0/2404231530 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:45:41.381 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.371+0000 7f53c10a3700 1 --2- 172.21.15.59:0/2404231530 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5635e27270f0 0x5635e2732720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:41.381 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.371+0000 7f53c20a5700 1 --2- 172.21.15.59:0/2404231530 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5635e2727fe0 0x5635e2737570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:41.381 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.371+0000 7f53c18a4700 1 -- 172.21.15.59:0/2404231530 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5635e2727fe0 msgr2=0x5635e2737570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T17:45:41.381 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.371+0000 7f53c18a4700 1 --2- 172.21.15.59:0/2404231530 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5635e2727fe0 0x5635e2737570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:41.381 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.371+0000 7f53c18a4700 1 -- 172.21.15.59:0/2404231530 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5635e27270f0 msgr2=0x5635e2732720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T17:45:41.381 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.371+0000 7f53c18a4700 1 --2- 172.21.15.59:0/2404231530 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5635e27270f0 0x5635e2732720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:41.381 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.371+0000 7f53c18a4700 1 -- 172.21.15.59:0/2404231530 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53b8096040 con 0x5635e2726790 2023-12-11T17:45:41.381 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.371+0000 7f53c10a3700 1 --2- 172.21.15.59:0/2404231530 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5635e27270f0 0x5635e2732720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T17:45:41.381 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.371+0000 7f53c20a5700 1 --2- 172.21.15.59:0/2404231530 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5635e2727fe0 0x5635e2737570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-11T17:45:41.381 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.372+0000 7f53c18a4700 1 --2- 172.21.15.59:0/2404231530 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5635e2726790 0x5635e27321e0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f53ac00b680 tx=0x7f53ac025f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36780fe22ed63a25 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:41.382 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.372+0000 7f53a57fa700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2023-12-11T17:45:41.382 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.372+0000 7f53a57fa700 1 -- 172.21.15.59:0/2404231530 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53ac02b070 con 0x5635e2726790 2023-12-11T17:45:41.382 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.372+0000 7f53cfd01b80 1 -- 172.21.15.59:0/2404231530 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f53ac01e040 con 0x5635e2726790 2023-12-11T17:45:41.382 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.372+0000 7f53a57fa700 1 -- 172.21.15.59:0/2404231530 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f53ac005940 con 0x5635e2726790 2023-12-11T17:45:41.382 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.372+0000 7f53a57fa700 1 -- 172.21.15.59:0/2404231530 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f53ac0046d0 con 0x5635e2726790 2023-12-11T17:45:41.382 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.372+0000 7f53cfd01b80 10 client.15198 fetch_fsmap learned FSMap version 24 2023-12-11T17:45:41.382 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.372+0000 7f53cfd01b80 1 -- 172.21.15.59:0/2404231530 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5635e24c6200 con 0x5635e2726790 2023-12-11T17:45:41.382 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.372+0000 7f53a57fa700 1 -- 172.21.15.59:0/2404231530 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f53ac04d080 con 0x5635e2726790 2023-12-11T17:45:41.382 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.372+0000 7f53a57fa700 1 -- 172.21.15.59:0/2404231530 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f53ac02f680 con 0x5635e2726790 2023-12-11T17:45:41.382 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.372+0000 7f53cfd01b80 10 client.15198 fetch_fsmap finished waiting for FSMap version 24 2023-12-11T17:45:41.382 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.372+0000 7f53cfd01b80 10 client.15198 resolve_mds: resolved name 'i' to daemon mds.i 2023-12-11T17:45:41.382 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.372+0000 7f53cfd01b80 20 client.15198 populate_metadata read hostname 'smithi059' 2023-12-11T17:45:41.382 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.372+0000 7f53cfd01b80 1 --2- 172.21.15.59:0/2404231530 >> [v2:172.21.15.181:6838/2214384806,v1:172.21.15.181:6839/2214384806] conn(0x5635e27968c0 0x5635e2792d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:41.382 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.372+0000 7f53cfd01b80 4 client.15198 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "status"}] 2023-12-11T17:45:41.382 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.372+0000 7f53cfd01b80 1 -- 172.21.15.59:0/2404231530 --> [v2:172.21.15.181:6838/2214384806,v1:172.21.15.181:6839/2214384806] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5635e263cf80 con 0x5635e27968c0 2023-12-11T17:45:41.383 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.373+0000 7f53c10a3700 1 --2- 172.21.15.59:0/2404231530 >> [v2:172.21.15.181:6838/2214384806,v1:172.21.15.181:6839/2214384806] conn(0x5635e27968c0 0x5635e2792d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:41.383 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.373+0000 7f53a57fa700 10 client.15198 ms_handle_connect on v2:172.21.15.181:6838/2214384806 2023-12-11T17:45:41.383 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.373+0000 7f53c10a3700 1 --2- 172.21.15.59:0/2404231530 >> [v2:172.21.15.181:6838/2214384806,v1:172.21.15.181:6839/2214384806] conn(0x5635e27968c0 0x5635e2792d00 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a20a770431f0268f server_cookie=4af5e27cfdfac29f in_seq=0 out_seq=0 2023-12-11T17:45:41.383 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.373+0000 7f53a57fa700 1 -- 172.21.15.59:0/2404231530 <== mds.1 v2:172.21.15.181:6838/2214384806 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x5635e263cf80 con 0x5635e27968c0 2023-12-11T17:45:41.383 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.373+0000 7f53a57fa700 10 client.15198 handle_command_reply: tid=0 2023-12-11T17:45:41.383 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.373+0000 7f53cfd01b80 1 client.15198 shutdown 2023-12-11T17:45:41.383 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.373+0000 7f53cfd01b80 1 -- 172.21.15.59:0/2404231530 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5635e2726790 msgr2=0x5635e27321e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:41.383 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.373+0000 7f53cfd01b80 1 --2- 172.21.15.59:0/2404231530 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5635e2726790 0x5635e27321e0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f53ac00b680 tx=0x7f53ac025f40 comp rx=0 tx=0).stop 2023-12-11T17:45:41.383 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.373+0000 7f53cfd01b80 1 -- 172.21.15.59:0/2404231530 shutdown_connections 2023-12-11T17:45:41.383 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.373+0000 7f53cfd01b80 1 --2- 172.21.15.59:0/2404231530 >> [v2:172.21.15.181:6838/2214384806,v1:172.21.15.181:6839/2214384806] conn(0x5635e27968c0 0x5635e2792d00 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:41.383 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.373+0000 7f53cfd01b80 1 --2- 172.21.15.59:0/2404231530 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5635e2727fe0 0x5635e2737570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:41.383 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.373+0000 7f53a57fa700 0 client.15198 ms_handle_reset on v2:172.21.15.181:6838/2214384806 2023-12-11T17:45:41.383 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.373+0000 7f53cfd01b80 1 --2- 172.21.15.59:0/2404231530 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5635e2726790 0x5635e27321e0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:41.383 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.373+0000 7f53cfd01b80 1 --2- 172.21.15.59:0/2404231530 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5635e27270f0 0x5635e2732720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:41.383 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.373+0000 7f53cfd01b80 1 -- 172.21.15.59:0/2404231530 >> 172.21.15.59:0/2404231530 conn(0x5635e2651790 msgr2=0x5635e272af10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:45:41.384 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.374+0000 7f53cfd01b80 1 -- 172.21.15.59:0/2404231530 shutdown_connections 2023-12-11T17:45:41.384 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.374+0000 7f53cfd01b80 1 -- 172.21.15.59:0/2404231530 wait complete. 2023-12-11T17:45:41.384 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.374+0000 7f53cfd01b80 20 client.15198 trim_cache size 0 max 16384 2023-12-11T17:45:41.384 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2023-12-11T17:45:41.384 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_fsid": "d0a07eaa-984b-11ee-95a3-87774f69a715", 2023-12-11T17:45:41.384 INFO:teuthology.orchestra.run.smithi059.stdout: "whoami": 1, 2023-12-11T17:45:41.384 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 24484, 2023-12-11T17:45:41.384 INFO:teuthology.orchestra.run.smithi059.stdout: "want_state": "up:active", 2023-12-11T17:45:41.384 INFO:teuthology.orchestra.run.smithi059.stdout: "state": "up:active", 2023-12-11T17:45:41.384 INFO:teuthology.orchestra.run.smithi059.stdout: "fs_name": "cephfs", 2023-12-11T17:45:41.385 INFO:teuthology.orchestra.run.smithi059.stdout: "rank_uptime": 55.619991026999998, 2023-12-11T17:45:41.385 INFO:teuthology.orchestra.run.smithi059.stdout: "mdsmap_epoch": 24, 2023-12-11T17:45:41.385 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch": 88, 2023-12-11T17:45:41.385 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch_barrier": 86, 2023-12-11T17:45:41.385 INFO:teuthology.orchestra.run.smithi059.stdout: "uptime": 93.123512027000004 2023-12-11T17:45:41.385 INFO:teuthology.orchestra.run.smithi059.stdout:} 2023-12-11T17:45:41.386 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.374+0000 7f53c3b08700 1 -- 172.21.15.59:0/2842434543 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f53a80a1510 msgr2=0x7f53a80a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:41.386 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.374+0000 7f53c3b08700 1 --2- 172.21.15.59:0/2842434543 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f53a80a1510 0x7f53a80a39d0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f53ac006010 tx=0x7f53ac00b770 comp rx=0 tx=0).stop 2023-12-11T17:45:41.386 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.374+0000 7f53c3b08700 1 -- 172.21.15.59:0/2842434543 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f53bc1016a0 msgr2=0x7f53bc196980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:41.386 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.374+0000 7f53c3b08700 1 --2- 172.21.15.59:0/2842434543 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f53bc1016a0 0x7f53bc196980 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f53b800eb30 tx=0x7f53b800eef0 comp rx=0 tx=0).stop 2023-12-11T17:45:41.386 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.374+0000 7f53c3b08700 1 -- 172.21.15.59:0/2842434543 shutdown_connections 2023-12-11T17:45:41.386 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.374+0000 7f53c3b08700 1 --2- 172.21.15.59:0/2842434543 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f53a80a1510 0x7f53a80a39d0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:41.386 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.374+0000 7f53c3b08700 1 --2- 172.21.15.59:0/2842434543 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53bc102040 0x7f53bc19d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:41.386 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.374+0000 7f53c3b08700 1 --2- 172.21.15.59:0/2842434543 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f53bc1004b0 0x7f53bc196440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:41.386 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.374+0000 7f53c3b08700 1 --2- 172.21.15.59:0/2842434543 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f53bc1016a0 0x7f53bc196980 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:41.386 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.374+0000 7f53c3b08700 1 -- 172.21.15.59:0/2842434543 >> 172.21.15.59:0/2842434543 conn(0x7f53bc0fba20 msgr2=0x7f53bc104ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:45:41.386 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.374+0000 7f53c3b08700 1 -- 172.21.15.59:0/2842434543 shutdown_connections 2023-12-11T17:45:41.386 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.374+0000 7f53c3b08700 1 -- 172.21.15.59:0/2842434543 wait complete. 2023-12-11T17:45:41.389 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-12-11T17:45:41.519 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.515+0000 7f1432086700 1 -- 172.21.15.59:0/1499193370 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f142c102000 msgr2=0x7f142c1043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:41.520 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.515+0000 7f1432086700 1 --2- 172.21.15.59:0/1499193370 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f142c102000 0x7f142c1043f0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f141c009990 tx=0x7f141c009ca0 comp rx=0 tx=0).stop 2023-12-11T17:45:41.520 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.515+0000 7f1432086700 1 -- 172.21.15.59:0/1499193370 shutdown_connections 2023-12-11T17:45:41.520 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.515+0000 7f1432086700 1 --2- 172.21.15.59:0/1499193370 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f142c107260 0x7f142c1096d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:41.520 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.515+0000 7f1432086700 1 --2- 172.21.15.59:0/1499193370 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f142c102000 0x7f142c1043f0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:41.520 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.515+0000 7f1432086700 1 --2- 172.21.15.59:0/1499193370 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f142c104930 0x7f142c106d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:41.520 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.515+0000 7f1432086700 1 -- 172.21.15.59:0/1499193370 >> 172.21.15.59:0/1499193370 conn(0x7f142c0fb9f0 msgr2=0x7f142c0fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:45:41.520 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.515+0000 7f1432086700 1 -- 172.21.15.59:0/1499193370 shutdown_connections 2023-12-11T17:45:41.520 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.515+0000 7f1432086700 1 -- 172.21.15.59:0/1499193370 wait complete. 2023-12-11T17:45:41.520 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.515+0000 7f1432086700 1 Processor -- start 2023-12-11T17:45:41.520 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.516+0000 7f1432086700 1 -- start start 2023-12-11T17:45:41.520 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.516+0000 7f1432086700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f142c102000 0x7f142c1974a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:41.520 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.516+0000 7f1432086700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f142c104930 0x7f142c1979e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:41.521 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.516+0000 7f1432086700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f142c107260 0x7f142c19ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:41.521 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.516+0000 7f1432086700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f142c19efd0 con 0x7f142c107260 2023-12-11T17:45:41.521 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.516+0000 7f1432086700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f142c19f110 con 0x7f142c102000 2023-12-11T17:45:41.521 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.516+0000 7f1432086700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f142c19f2a0 con 0x7f142c104930 2023-12-11T17:45:41.521 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.516+0000 7f142bfff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f142c107260 0x7f142c19ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:41.521 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.516+0000 7f142bfff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f142c107260 0x7f142c19ea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34196/0 (socket says 172.21.15.59:34196) 2023-12-11T17:45:41.521 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.516+0000 7f142b7fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f142c102000 0x7f142c1974a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:41.521 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.516+0000 7f142bfff700 1 -- 172.21.15.59:0/3898658731 learned_addr learned my addr 172.21.15.59:0/3898658731 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:45:41.521 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.516+0000 7f142affd700 1 --2- 172.21.15.59:0/3898658731 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f142c104930 0x7f142c1979e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:41.521 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.516+0000 7f142bfff700 1 -- 172.21.15.59:0/3898658731 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f142c104930 msgr2=0x7f142c1979e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:41.521 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.516+0000 7f142bfff700 1 --2- 172.21.15.59:0/3898658731 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f142c104930 0x7f142c1979e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:41.521 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.516+0000 7f142bfff700 1 -- 172.21.15.59:0/3898658731 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f142c102000 msgr2=0x7f142c1974a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:41.521 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.516+0000 7f142bfff700 1 --2- 172.21.15.59:0/3898658731 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f142c102000 0x7f142c1974a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:41.521 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.516+0000 7f142bfff700 1 -- 172.21.15.59:0/3898658731 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f141c0095f0 con 0x7f142c107260 2023-12-11T17:45:41.522 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.516+0000 7f142b7fe700 1 --2- 172.21.15.59:0/3898658731 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f142c102000 0x7f142c1974a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T17:45:41.522 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.516+0000 7f142bfff700 1 --2- 172.21.15.59:0/3898658731 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f142c107260 0x7f142c19ea00 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f142400ef30 tx=0x7f142400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:41.522 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.517+0000 7f1428ff9700 1 -- 172.21.15.59:0/3898658731 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1424018070 con 0x7f142c107260 2023-12-11T17:45:41.522 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.517+0000 7f1432086700 1 -- 172.21.15.59:0/3898658731 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f142c19f520 con 0x7f142c107260 2023-12-11T17:45:41.522 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.517+0000 7f1428ff9700 1 -- 172.21.15.59:0/3898658731 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f142400f040 con 0x7f142c107260 2023-12-11T17:45:41.522 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.517+0000 7f1428ff9700 1 -- 172.21.15.59:0/3898658731 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f142400cdb0 con 0x7f142c107260 2023-12-11T17:45:41.522 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.517+0000 7f1432086700 1 -- 172.21.15.59:0/3898658731 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f142c19f9e0 con 0x7f142c107260 2023-12-11T17:45:41.525 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.519+0000 7f1428ff9700 1 -- 172.21.15.59:0/3898658731 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f1424003a00 con 0x7f142c107260 2023-12-11T17:45:41.525 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.519+0000 7f1428ff9700 1 --2- 172.21.15.59:0/3898658731 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f14180a1510 0x7f14180a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:41.525 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.520+0000 7f1428ff9700 1 -- 172.21.15.59:0/3898658731 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f14240cca00 con 0x7f142c107260 2023-12-11T17:45:41.525 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.520+0000 7f142b7fe700 1 --2- 172.21.15.59:0/3898658731 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f14180a1510 0x7f14180a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:41.525 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.520+0000 7f142b7fe700 1 --2- 172.21.15.59:0/3898658731 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f14180a1510 0x7f14180a39d0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f141c00b2e0 tx=0x7f141c005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:41.525 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.520+0000 7f143e27fb80 1 Processor -- start 2023-12-11T17:45:41.525 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.520+0000 7f143e27fb80 1 -- start start 2023-12-11T17:45:41.525 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.521+0000 7f143e27fb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555fbd588380 0x555fbd5887a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:41.525 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.521+0000 7f143e27fb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555fbd588ce0 0x555fbd5895f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:41.525 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.521+0000 7f143e27fb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555fbd589bd0 0x555fbd58a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:41.526 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.521+0000 7f143e27fb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fbd4e3b00 con 0x555fbd588380 2023-12-11T17:45:41.526 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.521+0000 7f143e27fb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fbd4e1420 con 0x555fbd589bd0 2023-12-11T17:45:41.526 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.521+0000 7f143e27fb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fbd4dcd10 con 0x555fbd588ce0 2023-12-11T17:45:41.526 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.521+0000 7f142b7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555fbd588380 0x555fbd5887a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:41.526 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.521+0000 7f142b7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555fbd588380 0x555fbd5887a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34200/0 (socket says 172.21.15.59:34200) 2023-12-11T17:45:41.526 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.521+0000 7f142b7fe700 1 -- 172.21.15.59:0/1030583476 learned_addr learned my addr 172.21.15.59:0/1030583476 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:45:41.526 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.521+0000 7f142affd700 1 --2- 172.21.15.59:0/1030583476 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555fbd588ce0 0x555fbd5895f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:41.526 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.521+0000 7f142bfff700 1 --2- 172.21.15.59:0/1030583476 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555fbd589bd0 0x555fbd58a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:41.526 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.521+0000 7f142b7fe700 1 -- 172.21.15.59:0/1030583476 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555fbd588ce0 msgr2=0x555fbd5895f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T17:45:41.526 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.521+0000 7f142b7fe700 1 --2- 172.21.15.59:0/1030583476 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555fbd588ce0 0x555fbd5895f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:41.526 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.521+0000 7f142b7fe700 1 -- 172.21.15.59:0/1030583476 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555fbd589bd0 msgr2=0x555fbd58a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T17:45:41.526 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.521+0000 7f142b7fe700 1 --2- 172.21.15.59:0/1030583476 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555fbd589bd0 0x555fbd58a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:41.526 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.521+0000 7f142b7fe700 1 -- 172.21.15.59:0/1030583476 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555fbd3595e0 con 0x555fbd588380 2023-12-11T17:45:41.526 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.521+0000 7f142bfff700 1 --2- 172.21.15.59:0/1030583476 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555fbd589bd0 0x555fbd58a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T17:45:41.527 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.521+0000 7f142b7fe700 1 --2- 172.21.15.59:0/1030583476 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555fbd588380 0x555fbd5887a0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f141c015000 tx=0x7f141c020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=443c4a7e938d50 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:41.527 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.522+0000 7f1421ffb700 1 -- 172.21.15.59:0/1030583476 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f141c00fd50 con 0x555fbd588380 2023-12-11T17:45:41.527 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.522+0000 7f1421ffb700 1 -- 172.21.15.59:0/1030583476 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f141c01e040 con 0x555fbd588380 2023-12-11T17:45:41.527 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.522+0000 7f1421ffb700 1 -- 172.21.15.59:0/1030583476 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f141c024670 con 0x555fbd588380 2023-12-11T17:45:41.527 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.522+0000 7f143e27fb80 1 -- 172.21.15.59:0/1030583476 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555fbd588380 msgr2=0x555fbd5887a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:41.527 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.522+0000 7f143e27fb80 1 --2- 172.21.15.59:0/1030583476 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555fbd588380 0x555fbd5887a0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f141c015000 tx=0x7f141c020000 comp rx=0 tx=0).stop 2023-12-11T17:45:41.527 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.522+0000 7f143e27fb80 1 -- 172.21.15.59:0/1030583476 shutdown_connections 2023-12-11T17:45:41.527 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.522+0000 7f143e27fb80 1 --2- 172.21.15.59:0/1030583476 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555fbd589bd0 0x555fbd58a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:41.527 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.522+0000 7f143e27fb80 1 --2- 172.21.15.59:0/1030583476 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555fbd588380 0x555fbd5887a0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:41.527 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.522+0000 7f143e27fb80 1 --2- 172.21.15.59:0/1030583476 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555fbd588ce0 0x555fbd5895f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:41.527 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.522+0000 7f143e27fb80 1 -- 172.21.15.59:0/1030583476 >> 172.21.15.59:0/1030583476 conn(0x555fbd4b7790 msgr2=0x555fbd584d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:45:41.527 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.522+0000 7f143e27fb80 1 -- 172.21.15.59:0/1030583476 shutdown_connections 2023-12-11T17:45:41.527 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.522+0000 7f143e27fb80 1 -- 172.21.15.59:0/1030583476 wait complete. 2023-12-11T17:45:41.527 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.523+0000 7f143e27fb80 1 -- start start 2023-12-11T17:45:41.531 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.523+0000 7f143e27fb80 1 Processor -- start 2023-12-11T17:45:41.531 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.523+0000 7f143e27fb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555fbd588380 0x555fbd592130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:41.531 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.523+0000 7f143e27fb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555fbd588ce0 0x555fbd592670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:41.531 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.523+0000 7f143e27fb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555fbd589bd0 0x555fbd5996b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:41.532 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.523+0000 7f143e27fb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fbd496510 con 0x555fbd588ce0 2023-12-11T17:45:41.532 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.523+0000 7f143e27fb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fbd4961b0 con 0x555fbd589bd0 2023-12-11T17:45:41.532 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.523+0000 7f143e27fb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fbd35b8d0 con 0x555fbd588380 2023-12-11T17:45:41.532 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.523+0000 7f143e27fb80 10 client.0 _ll_register_callbacks cb 0x555fbd3c1a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T17:45:41.532 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.523+0000 7f142affd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555fbd588ce0 0x555fbd592670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:41.532 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.524+0000 7f142affd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555fbd588ce0 0x555fbd592670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34208/0 (socket says 172.21.15.59:34208) 2023-12-11T17:45:41.532 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.524+0000 7f142affd700 1 -- 172.21.15.59:0/306274200 learned_addr learned my addr 172.21.15.59:0/306274200 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:45:41.532 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.524+0000 7f142b7fe700 1 --2- 172.21.15.59:0/306274200 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555fbd588380 0x555fbd592130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:41.532 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.524+0000 7f142bfff700 1 --2- 172.21.15.59:0/306274200 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555fbd589bd0 0x555fbd5996b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:41.532 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.524+0000 7f142affd700 1 -- 172.21.15.59:0/306274200 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555fbd588380 msgr2=0x555fbd592130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T17:45:41.532 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.524+0000 7f142affd700 1 --2- 172.21.15.59:0/306274200 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555fbd588380 0x555fbd592130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:41.532 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.524+0000 7f142affd700 1 -- 172.21.15.59:0/306274200 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555fbd589bd0 msgr2=0x555fbd5996b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T17:45:41.532 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.524+0000 7f142affd700 1 --2- 172.21.15.59:0/306274200 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555fbd589bd0 0x555fbd5996b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:41.532 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.524+0000 7f142affd700 1 -- 172.21.15.59:0/306274200 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f142c10a2c0 con 0x555fbd588ce0 2023-12-11T17:45:41.532 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.524+0000 7f142b7fe700 1 --2- 172.21.15.59:0/306274200 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555fbd588380 0x555fbd592130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T17:45:41.533 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.524+0000 7f142bfff700 1 --2- 172.21.15.59:0/306274200 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555fbd589bd0 0x555fbd5996b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T17:45:41.533 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.524+0000 7f142affd700 1 --2- 172.21.15.59:0/306274200 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555fbd588ce0 0x555fbd592670 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f1414012700 tx=0x7f1414012a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9c957569be84472 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:41.533 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.524+0000 7f140affd700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2023-12-11T17:45:41.533 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.524+0000 7f140affd700 1 -- 172.21.15.59:0/306274200 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1414017070 con 0x555fbd588ce0 2023-12-11T17:45:41.533 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.524+0000 7f143e27fb80 1 -- 172.21.15.59:0/306274200 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555fbd57baf0 con 0x555fbd588ce0 2023-12-11T17:45:41.533 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.524+0000 7f140affd700 1 -- 172.21.15.59:0/306274200 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f1414004260 con 0x555fbd588ce0 2023-12-11T17:45:41.533 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.524+0000 7f140affd700 1 -- 172.21.15.59:0/306274200 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1414004420 con 0x555fbd588ce0 2023-12-11T17:45:41.533 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.524+0000 7f143e27fb80 10 client.15210 fetch_fsmap learned FSMap version 24 2023-12-11T17:45:41.533 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.524+0000 7f143e27fb80 1 -- 172.21.15.59:0/306274200 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x555fbd305d90 con 0x555fbd588ce0 2023-12-11T17:45:41.533 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.525+0000 7f140affd700 1 -- 172.21.15.59:0/306274200 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555fbd57baf0 con 0x555fbd588ce0 2023-12-11T17:45:41.533 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.525+0000 7f140affd700 1 -- 172.21.15.59:0/306274200 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f1414011860 con 0x555fbd588ce0 2023-12-11T17:45:41.533 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.525+0000 7f143e27fb80 10 client.15210 fetch_fsmap finished waiting for FSMap version 24 2023-12-11T17:45:41.533 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.525+0000 7f143e27fb80 10 client.15210 resolve_mds: resolved name 'b' to daemon mds.b 2023-12-11T17:45:41.533 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.525+0000 7f143e27fb80 20 client.15210 populate_metadata read hostname 'smithi059' 2023-12-11T17:45:41.534 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.525+0000 7f143e27fb80 1 --2- 172.21.15.59:0/306274200 >> [v2:172.21.15.115:6838/4122922866,v1:172.21.15.115:6839/4122922866] conn(0x555fbd59de00 0x555fbd59c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:41.534 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.525+0000 7f143e27fb80 4 client.15210 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-11T17:45:41.534 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.525+0000 7f143e27fb80 1 -- 172.21.15.59:0/306274200 --> [v2:172.21.15.115:6838/4122922866,v1:172.21.15.115:6839/4122922866] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555fbd557ef0 con 0x555fbd59de00 2023-12-11T17:45:41.534 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.525+0000 7f142b7fe700 1 --2- 172.21.15.59:0/306274200 >> [v2:172.21.15.115:6838/4122922866,v1:172.21.15.115:6839/4122922866] conn(0x555fbd59de00 0x555fbd59c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:41.534 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.526+0000 7f140affd700 10 client.15210 ms_handle_connect on v2:172.21.15.115:6838/4122922866 2023-12-11T17:45:41.534 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.526+0000 7f142b7fe700 1 --2- 172.21.15.59:0/306274200 >> [v2:172.21.15.115:6838/4122922866,v1:172.21.15.115:6839/4122922866] conn(0x555fbd59de00 0x555fbd59c8b0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6f113e01ede172e server_cookie=291242d0b778af32 in_seq=0 out_seq=0 2023-12-11T17:45:41.534 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.526+0000 7f140affd700 1 -- 172.21.15.59:0/306274200 <== mds.0 v2:172.21.15.115:6838/4122922866 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555fbd557ef0 con 0x555fbd59de00 2023-12-11T17:45:41.534 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.526+0000 7f140affd700 10 client.15210 handle_command_reply: tid=0 2023-12-11T17:45:41.534 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.526+0000 7f143e27fb80 1 client.15210 shutdown 2023-12-11T17:45:41.534 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.526+0000 7f143e27fb80 1 -- 172.21.15.59:0/306274200 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555fbd588ce0 msgr2=0x555fbd592670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:41.534 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.526+0000 7f143e27fb80 1 --2- 172.21.15.59:0/306274200 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555fbd588ce0 0x555fbd592670 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f1414012700 tx=0x7f1414012a10 comp rx=0 tx=0).stop 2023-12-11T17:45:41.534 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.526+0000 7f143e27fb80 1 -- 172.21.15.59:0/306274200 shutdown_connections 2023-12-11T17:45:41.534 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.526+0000 7f143e27fb80 1 --2- 172.21.15.59:0/306274200 >> [v2:172.21.15.115:6838/4122922866,v1:172.21.15.115:6839/4122922866] conn(0x555fbd59de00 0x555fbd59c8b0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:41.534 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.526+0000 7f143e27fb80 1 --2- 172.21.15.59:0/306274200 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555fbd589bd0 0x555fbd5996b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:41.534 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.526+0000 7f143e27fb80 1 --2- 172.21.15.59:0/306274200 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555fbd588380 0x555fbd592130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:41.535 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.526+0000 7f140affd700 0 client.15210 ms_handle_reset on v2:172.21.15.115:6838/4122922866 2023-12-11T17:45:41.535 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.526+0000 7f143e27fb80 1 --2- 172.21.15.59:0/306274200 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555fbd588ce0 0x555fbd592670 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:41.535 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.526+0000 7f143e27fb80 1 -- 172.21.15.59:0/306274200 >> 172.21.15.59:0/306274200 conn(0x555fbd4b7790 msgr2=0x555fbd5839f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:45:41.535 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.527+0000 7f143e27fb80 1 -- 172.21.15.59:0/306274200 shutdown_connections 2023-12-11T17:45:41.535 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.527+0000 7f143e27fb80 1 -- 172.21.15.59:0/306274200 wait complete. 2023-12-11T17:45:41.535 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.527+0000 7f143e27fb80 20 client.15210 trim_cache size 0 max 16384 2023-12-11T17:45:41.553 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.544+0000 7f143e27fb80 1 Processor -- start 2023-12-11T17:45:41.553 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.545+0000 7f143e27fb80 1 -- start start 2023-12-11T17:45:41.553 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.545+0000 7f143e27fb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555fbd588380 0x555fbd590190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:41.553 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.545+0000 7f143e27fb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555fbd588ce0 0x555fbd58cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:41.553 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.545+0000 7f143e27fb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555fbd589bd0 0x555fbd58d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:41.553 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.545+0000 7f143e27fb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fbd47f870 con 0x555fbd589bd0 2023-12-11T17:45:41.553 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.545+0000 7f143e27fb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fbd57f740 con 0x555fbd588380 2023-12-11T17:45:41.553 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.545+0000 7f143e27fb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fbd57bad0 con 0x555fbd588ce0 2023-12-11T17:45:41.553 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.545+0000 7f142bfff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555fbd589bd0 0x555fbd58d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:41.553 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.545+0000 7f142bfff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555fbd589bd0 0x555fbd58d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34218/0 (socket says 172.21.15.59:34218) 2023-12-11T17:45:41.553 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.545+0000 7f142bfff700 1 -- 172.21.15.59:0/3395573926 learned_addr learned my addr 172.21.15.59:0/3395573926 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:45:41.553 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.545+0000 7f142affd700 1 --2- 172.21.15.59:0/3395573926 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555fbd588ce0 0x555fbd58cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:41.554 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.545+0000 7f142b7fe700 1 --2- 172.21.15.59:0/3395573926 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555fbd588380 0x555fbd590190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:41.554 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.545+0000 7f142bfff700 1 -- 172.21.15.59:0/3395573926 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555fbd588ce0 msgr2=0x555fbd58cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T17:45:41.554 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.545+0000 7f142bfff700 1 --2- 172.21.15.59:0/3395573926 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555fbd588ce0 0x555fbd58cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:41.554 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.545+0000 7f142bfff700 1 -- 172.21.15.59:0/3395573926 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555fbd588380 msgr2=0x555fbd590190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T17:45:41.554 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.545+0000 7f142bfff700 1 --2- 172.21.15.59:0/3395573926 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555fbd588380 0x555fbd590190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:41.554 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.545+0000 7f142bfff700 1 -- 172.21.15.59:0/3395573926 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555fbd2eec10 con 0x555fbd589bd0 2023-12-11T17:45:41.554 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.545+0000 7f142b7fe700 1 --2- 172.21.15.59:0/3395573926 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555fbd588380 0x555fbd590190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-11T17:45:41.554 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.545+0000 7f142bfff700 1 --2- 172.21.15.59:0/3395573926 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555fbd589bd0 0x555fbd58d0b0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f1424099650 tx=0x7f1424099330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12abe0c759b564f9 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:41.554 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.545+0000 7f14227fc700 1 -- 172.21.15.59:0/3395573926 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1424099950 con 0x555fbd589bd0 2023-12-11T17:45:41.554 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.545+0000 7f14227fc700 1 -- 172.21.15.59:0/3395573926 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f142409daa0 con 0x555fbd589bd0 2023-12-11T17:45:41.554 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.545+0000 7f14227fc700 1 -- 172.21.15.59:0/3395573926 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1424099e10 con 0x555fbd589bd0 2023-12-11T17:45:41.554 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.546+0000 7f143e27fb80 1 -- 172.21.15.59:0/3395573926 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555fbd589bd0 msgr2=0x555fbd58d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:41.554 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.546+0000 7f143e27fb80 1 --2- 172.21.15.59:0/3395573926 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555fbd589bd0 0x555fbd58d0b0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f1424099650 tx=0x7f1424099330 comp rx=0 tx=0).stop 2023-12-11T17:45:41.554 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.546+0000 7f143e27fb80 1 -- 172.21.15.59:0/3395573926 shutdown_connections 2023-12-11T17:45:41.554 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.546+0000 7f143e27fb80 1 --2- 172.21.15.59:0/3395573926 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555fbd589bd0 0x555fbd58d0b0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:41.555 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.546+0000 7f143e27fb80 1 --2- 172.21.15.59:0/3395573926 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555fbd588380 0x555fbd590190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:41.555 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.546+0000 7f143e27fb80 1 --2- 172.21.15.59:0/3395573926 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555fbd588ce0 0x555fbd58cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:41.555 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.546+0000 7f143e27fb80 1 -- 172.21.15.59:0/3395573926 >> 172.21.15.59:0/3395573926 conn(0x555fbd4b7790 msgr2=0x555fbd591640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:45:41.555 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.546+0000 7f143e27fb80 1 -- 172.21.15.59:0/3395573926 shutdown_connections 2023-12-11T17:45:41.555 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.546+0000 7f143e27fb80 1 -- 172.21.15.59:0/3395573926 wait complete. 2023-12-11T17:45:41.555 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.546+0000 7f143e27fb80 1 -- start start 2023-12-11T17:45:41.555 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.547+0000 7f143e27fb80 1 Processor -- start 2023-12-11T17:45:41.555 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.547+0000 7f143e27fb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555fbd588380 0x555fbd5f00f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:41.555 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.547+0000 7f143e27fb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555fbd588ce0 0x555fbd5f0630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:41.555 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.547+0000 7f143e27fb80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555fbd589bd0 0x555fbd5f72e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:41.555 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.547+0000 7f143e27fb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fbd5f7820 con 0x555fbd588380 2023-12-11T17:45:41.555 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.547+0000 7f143e27fb80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fbd5f7960 con 0x555fbd589bd0 2023-12-11T17:45:41.555 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.547+0000 7f143e27fb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fbd5f7b20 con 0x555fbd588ce0 2023-12-11T17:45:41.555 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.547+0000 7f143e27fb80 10 client.0 _ll_register_callbacks cb 0x555fbd3c1a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T17:45:41.556 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.547+0000 7f142b7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555fbd588380 0x555fbd5f00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:41.556 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.547+0000 7f142b7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555fbd588380 0x555fbd5f00f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34220/0 (socket says 172.21.15.59:34220) 2023-12-11T17:45:41.556 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.547+0000 7f142b7fe700 1 -- 172.21.15.59:0/649825894 learned_addr learned my addr 172.21.15.59:0/649825894 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:45:41.556 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.547+0000 7f142affd700 1 --2- 172.21.15.59:0/649825894 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555fbd588ce0 0x555fbd5f0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:41.556 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.547+0000 7f142bfff700 1 --2- 172.21.15.59:0/649825894 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555fbd589bd0 0x555fbd5f72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:41.556 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.547+0000 7f142b7fe700 1 -- 172.21.15.59:0/649825894 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555fbd588ce0 msgr2=0x555fbd5f0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T17:45:41.556 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.547+0000 7f142b7fe700 1 --2- 172.21.15.59:0/649825894 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555fbd588ce0 0x555fbd5f0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:41.556 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.547+0000 7f142b7fe700 1 -- 172.21.15.59:0/649825894 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555fbd589bd0 msgr2=0x555fbd5f72e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T17:45:41.556 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.547+0000 7f142b7fe700 1 --2- 172.21.15.59:0/649825894 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555fbd589bd0 0x555fbd5f72e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:41.556 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.547+0000 7f142b7fe700 1 -- 172.21.15.59:0/649825894 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f142c19f9e0 con 0x555fbd588380 2023-12-11T17:45:41.556 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.547+0000 7f142bfff700 1 --2- 172.21.15.59:0/649825894 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555fbd589bd0 0x555fbd5f72e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T17:45:41.556 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.548+0000 7f142b7fe700 1 --2- 172.21.15.59:0/649825894 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555fbd588380 0x555fbd5f00f0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f141c0098e0 tx=0x7f141c005ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90ec968bf56f88ab server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:41.556 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.548+0000 7f140affd700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2023-12-11T17:45:41.556 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.548+0000 7f140affd700 1 -- 172.21.15.59:0/649825894 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f141c03eea0 con 0x555fbd588380 2023-12-11T17:45:41.556 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.548+0000 7f140affd700 1 -- 172.21.15.59:0/649825894 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f141c024920 con 0x555fbd588380 2023-12-11T17:45:41.557 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.548+0000 7f143e27fb80 1 -- 172.21.15.59:0/649825894 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f141c01e040 con 0x555fbd588380 2023-12-11T17:45:41.557 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.548+0000 7f140affd700 1 -- 172.21.15.59:0/649825894 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f141c029040 con 0x555fbd588380 2023-12-11T17:45:41.557 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.548+0000 7f143e27fb80 10 client.15216 fetch_fsmap learned FSMap version 24 2023-12-11T17:45:41.557 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.548+0000 7f143e27fb80 1 -- 172.21.15.59:0/649825894 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x555fbd32c200 con 0x555fbd588380 2023-12-11T17:45:41.557 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.548+0000 7f140affd700 1 -- 172.21.15.59:0/649825894 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f141c01e040 con 0x555fbd588380 2023-12-11T17:45:41.557 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.548+0000 7f140affd700 1 -- 172.21.15.59:0/649825894 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f141c02f9e0 con 0x555fbd588380 2023-12-11T17:45:41.557 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.548+0000 7f143e27fb80 10 client.15216 fetch_fsmap finished waiting for FSMap version 24 2023-12-11T17:45:41.557 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.548+0000 7f143e27fb80 10 client.15216 resolve_mds: resolved name 'b' to daemon mds.b 2023-12-11T17:45:41.557 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.548+0000 7f143e27fb80 20 client.15216 populate_metadata read hostname 'smithi059' 2023-12-11T17:45:41.557 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.548+0000 7f143e27fb80 1 --2- 172.21.15.59:0/649825894 >> [v2:172.21.15.115:6838/4122922866,v1:172.21.15.115:6839/4122922866] conn(0x555fbd5fcaa0 0x555fbd5f8f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:41.557 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.548+0000 7f143e27fb80 4 client.15216 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "status"}] 2023-12-11T17:45:41.557 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.548+0000 7f143e27fb80 1 -- 172.21.15.59:0/649825894 --> [v2:172.21.15.115:6838/4122922866,v1:172.21.15.115:6839/4122922866] -- command(tid 0: {"prefix": "status"}) v1 -- 0x555fbd4a2f80 con 0x555fbd5fcaa0 2023-12-11T17:45:41.557 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.549+0000 7f142affd700 1 --2- 172.21.15.59:0/649825894 >> [v2:172.21.15.115:6838/4122922866,v1:172.21.15.115:6839/4122922866] conn(0x555fbd5fcaa0 0x555fbd5f8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:41.557 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.549+0000 7f140affd700 10 client.15216 ms_handle_connect on v2:172.21.15.115:6838/4122922866 2023-12-11T17:45:41.557 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.549+0000 7f142affd700 1 --2- 172.21.15.59:0/649825894 >> [v2:172.21.15.115:6838/4122922866,v1:172.21.15.115:6839/4122922866] conn(0x555fbd5fcaa0 0x555fbd5f8f20 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=290928038c2008e7 server_cookie=82a87441f832faf in_seq=0 out_seq=0 2023-12-11T17:45:41.558 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.549+0000 7f140affd700 1 -- 172.21.15.59:0/649825894 <== mds.0 v2:172.21.15.115:6838/4122922866 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x555fbd4a2f80 con 0x555fbd5fcaa0 2023-12-11T17:45:41.558 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.549+0000 7f140affd700 10 client.15216 handle_command_reply: tid=0 2023-12-11T17:45:41.558 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.549+0000 7f143e27fb80 1 client.15216 shutdown 2023-12-11T17:45:41.558 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.549+0000 7f143e27fb80 1 -- 172.21.15.59:0/649825894 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555fbd588380 msgr2=0x555fbd5f00f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:41.558 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.549+0000 7f143e27fb80 1 --2- 172.21.15.59:0/649825894 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555fbd588380 0x555fbd5f00f0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f141c0098e0 tx=0x7f141c005ad0 comp rx=0 tx=0).stop 2023-12-11T17:45:41.558 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.549+0000 7f143e27fb80 1 -- 172.21.15.59:0/649825894 shutdown_connections 2023-12-11T17:45:41.558 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.549+0000 7f143e27fb80 1 --2- 172.21.15.59:0/649825894 >> [v2:172.21.15.115:6838/4122922866,v1:172.21.15.115:6839/4122922866] conn(0x555fbd5fcaa0 0x555fbd5f8f20 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:41.558 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.549+0000 7f143e27fb80 1 --2- 172.21.15.59:0/649825894 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555fbd589bd0 0x555fbd5f72e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:41.558 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.549+0000 7f143e27fb80 1 --2- 172.21.15.59:0/649825894 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555fbd588380 0x555fbd5f00f0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:41.558 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.549+0000 7f140affd700 0 client.15216 ms_handle_reset on v2:172.21.15.115:6838/4122922866 2023-12-11T17:45:41.558 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.549+0000 7f143e27fb80 1 --2- 172.21.15.59:0/649825894 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555fbd588ce0 0x555fbd5f0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:41.558 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.549+0000 7f143e27fb80 1 -- 172.21.15.59:0/649825894 >> 172.21.15.59:0/649825894 conn(0x555fbd4b7790 msgr2=0x555fbd58cb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:45:41.558 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.550+0000 7f143e27fb80 1 -- 172.21.15.59:0/649825894 shutdown_connections 2023-12-11T17:45:41.559 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.550+0000 7f143e27fb80 1 -- 172.21.15.59:0/649825894 wait complete. 2023-12-11T17:45:41.559 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.550+0000 7f143e27fb80 20 client.15216 trim_cache size 0 max 16384 2023-12-11T17:45:41.559 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2023-12-11T17:45:41.559 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_fsid": "d0a07eaa-984b-11ee-95a3-87774f69a715", 2023-12-11T17:45:41.559 INFO:teuthology.orchestra.run.smithi059.stdout: "whoami": 0, 2023-12-11T17:45:41.559 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 24521, 2023-12-11T17:45:41.559 INFO:teuthology.orchestra.run.smithi059.stdout: "want_state": "up:active", 2023-12-11T17:45:41.559 INFO:teuthology.orchestra.run.smithi059.stdout: "state": "up:active", 2023-12-11T17:45:41.559 INFO:teuthology.orchestra.run.smithi059.stdout: "fs_name": "cephfs", 2023-12-11T17:45:41.559 INFO:teuthology.orchestra.run.smithi059.stdout: "rank_uptime": 80.902826465000004, 2023-12-11T17:45:41.559 INFO:teuthology.orchestra.run.smithi059.stdout: "mdsmap_epoch": 24, 2023-12-11T17:45:41.559 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch": 88, 2023-12-11T17:45:41.560 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch_barrier": 86, 2023-12-11T17:45:41.560 INFO:teuthology.orchestra.run.smithi059.stdout: "uptime": 86.382282407999995 2023-12-11T17:45:41.560 INFO:teuthology.orchestra.run.smithi059.stdout:} 2023-12-11T17:45:41.560 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.550+0000 7f1432086700 1 -- 172.21.15.59:0/3898658731 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f14180a1510 msgr2=0x7f14180a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:41.560 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.550+0000 7f1432086700 1 --2- 172.21.15.59:0/3898658731 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f14180a1510 0x7f14180a39d0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f141c00b2e0 tx=0x7f141c005fd0 comp rx=0 tx=0).stop 2023-12-11T17:45:41.561 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.550+0000 7f1432086700 1 -- 172.21.15.59:0/3898658731 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f142c107260 msgr2=0x7f142c19ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:41.561 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.550+0000 7f1432086700 1 --2- 172.21.15.59:0/3898658731 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f142c107260 0x7f142c19ea00 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f142400ef30 tx=0x7f142400c3a0 comp rx=0 tx=0).stop 2023-12-11T17:45:41.561 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.550+0000 7f1432086700 1 -- 172.21.15.59:0/3898658731 shutdown_connections 2023-12-11T17:45:41.561 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.550+0000 7f1432086700 1 --2- 172.21.15.59:0/3898658731 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f14180a1510 0x7f14180a39d0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:41.561 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.550+0000 7f1432086700 1 --2- 172.21.15.59:0/3898658731 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f142c107260 0x7f142c19ea00 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:41.561 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.550+0000 7f1432086700 1 --2- 172.21.15.59:0/3898658731 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f142c102000 0x7f142c1974a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:41.561 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.550+0000 7f1432086700 1 --2- 172.21.15.59:0/3898658731 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f142c104930 0x7f142c1979e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:41.561 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.550+0000 7f1432086700 1 -- 172.21.15.59:0/3898658731 >> 172.21.15.59:0/3898658731 conn(0x7f142c0fb9f0 msgr2=0x7f142c102ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:45:41.561 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.550+0000 7f1432086700 1 -- 172.21.15.59:0/3898658731 shutdown_connections 2023-12-11T17:45:41.561 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:41.550+0000 7f1432086700 1 -- 172.21.15.59:0/3898658731 wait complete. 2023-12-11T17:45:41.565 INFO:tasks.cephadm:Setup complete, yielding 2023-12-11T17:45:41.565 INFO:teuthology.run_tasks:Running task cephadm.shell... 2023-12-11T17:45:41.576 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi059.front.sepia.ceph.com 2023-12-11T17:45:41.577 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d0a07eaa-984b-11ee-95a3-87774f69a715 -- bash -c 'ceph orch status' 2023-12-11T17:45:41.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:45:41 smithi059 ceph-mon[98847]: pgmap v301: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-12-11T17:45:41.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:45:41 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/742325898' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-11T17:45:41.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:45:41 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/1054371705' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T17:45:41.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:45:41 smithi115 ceph-mon[102748]: pgmap v301: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-12-11T17:45:41.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:45:41 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/742325898' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-11T17:45:41.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:45:41 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/1054371705' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T17:45:41.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:45:41 smithi181 ceph-mon[103627]: pgmap v301: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-12-11T17:45:41.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:45:41 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/742325898' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-11T17:45:41.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:45:41 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/1054371705' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T17:45:42.181 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/mon.a/config 2023-12-11T17:45:43.403 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:43.399+0000 7fe984185700 1 -- 172.21.15.59:0/3079063567 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe97c101620 msgr2=0x7fe97c101a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:43.403 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:43.399+0000 7fe984185700 1 --2- 172.21.15.59:0/3079063567 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe97c101620 0x7fe97c101a40 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fe9780099c0 tx=0x7fe978009cd0 comp rx=0 tx=0).stop 2023-12-11T17:45:43.404 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:43.399+0000 7fe984185700 1 -- 172.21.15.59:0/3079063567 shutdown_connections 2023-12-11T17:45:43.404 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:43.399+0000 7fe984185700 1 --2- 172.21.15.59:0/3079063567 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe97c1031c0 0x7fe97c109a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:43.404 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:43.399+0000 7fe984185700 1 --2- 172.21.15.59:0/3079063567 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe97c101620 0x7fe97c101a40 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:43.404 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:43.399+0000 7fe984185700 1 --2- 172.21.15.59:0/3079063567 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe97c102820 0x7fe97c102c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:43.404 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:43.399+0000 7fe984185700 1 -- 172.21.15.59:0/3079063567 >> 172.21.15.59:0/3079063567 conn(0x7fe97c0fcbc0 msgr2=0x7fe97c0ff000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:45:43.404 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:43.399+0000 7fe984185700 1 -- 172.21.15.59:0/3079063567 shutdown_connections 2023-12-11T17:45:43.404 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:43.399+0000 7fe984185700 1 -- 172.21.15.59:0/3079063567 wait complete. 2023-12-11T17:45:43.404 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:43.400+0000 7fe984185700 1 Processor -- start 2023-12-11T17:45:43.404 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:43.400+0000 7fe984185700 1 -- start start 2023-12-11T17:45:43.404 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:43.400+0000 7fe984185700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe97c101620 0x7fe97c075320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:43.404 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:43.400+0000 7fe984185700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe97c102820 0x7fe97c0739c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:43.404 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:43.400+0000 7fe984185700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe97c1031c0 0x7fe97c073f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:43.404 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:43.400+0000 7fe984185700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe97c0746a0 con 0x7fe97c101620 2023-12-11T17:45:43.405 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:43.400+0000 7fe984185700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe97c074810 con 0x7fe97c1031c0 2023-12-11T17:45:43.405 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:43.400+0000 7fe984185700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe97c1a1520 con 0x7fe97c102820 2023-12-11T17:45:43.405 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:43.400+0000 7fe981f21700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe97c101620 0x7fe97c075320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:43.405 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:43.400+0000 7fe981f21700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe97c101620 0x7fe97c075320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34232/0 (socket says 172.21.15.59:34232) 2023-12-11T17:45:43.405 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:43.400+0000 7fe981f21700 1 -- 172.21.15.59:0/733398169 learned_addr learned my addr 172.21.15.59:0/733398169 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:45:43.405 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:43.400+0000 7fe982722700 1 --2- 172.21.15.59:0/733398169 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe97c1031c0 0x7fe97c073f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:43.405 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:43.400+0000 7fe981720700 1 --2- 172.21.15.59:0/733398169 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe97c102820 0x7fe97c0739c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:43.405 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:43.400+0000 7fe981f21700 1 -- 172.21.15.59:0/733398169 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe97c102820 msgr2=0x7fe97c0739c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:43.405 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:43.400+0000 7fe981f21700 1 --2- 172.21.15.59:0/733398169 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe97c102820 0x7fe97c0739c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:43.405 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:43.400+0000 7fe981f21700 1 -- 172.21.15.59:0/733398169 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe97c1031c0 msgr2=0x7fe97c073f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:43.405 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:43.400+0000 7fe981f21700 1 --2- 172.21.15.59:0/733398169 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe97c1031c0 0x7fe97c073f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:43.405 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:43.400+0000 7fe981f21700 1 -- 172.21.15.59:0/733398169 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9780095f0 con 0x7fe97c101620 2023-12-11T17:45:43.405 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:43.401+0000 7fe981f21700 1 --2- 172.21.15.59:0/733398169 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe97c101620 0x7fe97c075320 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fe978006010 tx=0x7fe978005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:43.405 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:43.401+0000 7fe972ffd700 1 -- 172.21.15.59:0/733398169 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe97801d070 con 0x7fe97c101620 2023-12-11T17:45:43.405 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:43.401+0000 7fe972ffd700 1 -- 172.21.15.59:0/733398169 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe978004010 con 0x7fe97c101620 2023-12-11T17:45:43.406 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:43.401+0000 7fe984185700 1 -- 172.21.15.59:0/733398169 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe97c1a1750 con 0x7fe97c101620 2023-12-11T17:45:43.406 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:43.401+0000 7fe972ffd700 1 -- 172.21.15.59:0/733398169 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe978022cd0 con 0x7fe97c101620 2023-12-11T17:45:43.407 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:43.401+0000 7fe984185700 1 -- 172.21.15.59:0/733398169 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe97c1a1bf0 con 0x7fe97c101620 2023-12-11T17:45:43.407 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:43.402+0000 7fe984185700 1 -- 172.21.15.59:0/733398169 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe97c108a00 con 0x7fe97c101620 2023-12-11T17:45:43.407 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:43.403+0000 7fe972ffd700 1 -- 172.21.15.59:0/733398169 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7fe978022470 con 0x7fe97c101620 2023-12-11T17:45:43.407 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:43.404+0000 7fe972ffd700 1 --2- 172.21.15.59:0/733398169 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fe9680a1510 0x7fe9680a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:43.407 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:43.404+0000 7fe981720700 1 --2- 172.21.15.59:0/733398169 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fe9680a1510 0x7fe9680a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:43.407 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:43.404+0000 7fe972ffd700 1 -- 172.21.15.59:0/733398169 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fe9780cecf0 con 0x7fe97c101620 2023-12-11T17:45:43.407 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:43.404+0000 7fe981720700 1 --2- 172.21.15.59:0/733398169 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fe9680a1510 0x7fe9680a39d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe96c005ea0 tx=0x7fe96c005e30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:43.409 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:43.407+0000 7fe972ffd700 1 -- 172.21.15.59:0/733398169 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fe978097f20 con 0x7fe97c101620 2023-12-11T17:45:43.629 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:43.627+0000 7fe984185700 1 -- 172.21.15.59:0/733398169 --> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe97c061a10 con 0x7fe9680a1510 2023-12-11T17:45:43.630 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:43.628+0000 7fe972ffd700 1 -- 172.21.15.59:0/733398169 <== mgr.14150 v2:172.21.15.59:6800/3627728353 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7fe97c061a10 con 0x7fe9680a1510 2023-12-11T17:45:43.631 INFO:teuthology.orchestra.run.smithi059.stdout:Backend: cephadm 2023-12-11T17:45:43.631 INFO:teuthology.orchestra.run.smithi059.stdout:Available: Yes 2023-12-11T17:45:43.631 INFO:teuthology.orchestra.run.smithi059.stdout:Paused: No 2023-12-11T17:45:43.633 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:43.630+0000 7fe984185700 1 -- 172.21.15.59:0/733398169 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fe9680a1510 msgr2=0x7fe9680a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:43.633 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:43.630+0000 7fe984185700 1 --2- 172.21.15.59:0/733398169 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fe9680a1510 0x7fe9680a39d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe96c005ea0 tx=0x7fe96c005e30 comp rx=0 tx=0).stop 2023-12-11T17:45:43.633 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:43.630+0000 7fe984185700 1 -- 172.21.15.59:0/733398169 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe97c101620 msgr2=0x7fe97c075320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:43.633 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:43.630+0000 7fe984185700 1 --2- 172.21.15.59:0/733398169 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe97c101620 0x7fe97c075320 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fe978006010 tx=0x7fe978005740 comp rx=0 tx=0).stop 2023-12-11T17:45:43.633 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:43.630+0000 7fe984185700 1 -- 172.21.15.59:0/733398169 shutdown_connections 2023-12-11T17:45:43.633 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:43.630+0000 7fe984185700 1 --2- 172.21.15.59:0/733398169 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fe9680a1510 0x7fe9680a39d0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:43.633 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:43.630+0000 7fe984185700 1 --2- 172.21.15.59:0/733398169 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe97c1031c0 0x7fe97c073f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:43.633 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:43.630+0000 7fe984185700 1 --2- 172.21.15.59:0/733398169 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe97c101620 0x7fe97c075320 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:43.633 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:43.630+0000 7fe984185700 1 --2- 172.21.15.59:0/733398169 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe97c102820 0x7fe97c0739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:43.633 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:43.630+0000 7fe984185700 1 -- 172.21.15.59:0/733398169 >> 172.21.15.59:0/733398169 conn(0x7fe97c0fcbc0 msgr2=0x7fe97c105b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:45:43.633 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:43.631+0000 7fe984185700 1 -- 172.21.15.59:0/733398169 shutdown_connections 2023-12-11T17:45:43.634 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:43.631+0000 7fe984185700 1 -- 172.21.15.59:0/733398169 wait complete. 2023-12-11T17:45:43.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:45:43 smithi115 ceph-mon[102748]: pgmap v302: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-12-11T17:45:43.792 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:45:43 smithi059 ceph-mon[98847]: pgmap v302: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-12-11T17:45:43.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:45:43 smithi181 ceph-mon[103627]: pgmap v302: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-12-11T17:45:44.046 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d0a07eaa-984b-11ee-95a3-87774f69a715 -- bash -c 'ceph orch ps' 2023-12-11T17:45:44.651 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/mon.a/config 2023-12-11T17:45:44.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:45:44 smithi115 ceph-mon[102748]: from='client.15222 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T17:45:44.874 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:45:44 smithi059 ceph-mon[98847]: from='client.15222 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T17:45:44.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:45:44 smithi181 ceph-mon[103627]: from='client.15222 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T17:45:45.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:45:45 smithi115 ceph-mon[102748]: pgmap v303: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-12-11T17:45:45.893 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:45.888+0000 7fe9cfd94700 1 -- 172.21.15.59:0/3609410404 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9c81020b0 msgr2=0x7fe9c8108960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:45.893 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:45.888+0000 7fe9cfd94700 1 --2- 172.21.15.59:0/3609410404 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9c81020b0 0x7fe9c8108960 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fe9c000b0d0 tx=0x7fe9c000b3e0 comp rx=0 tx=0).stop 2023-12-11T17:45:45.893 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:45.888+0000 7fe9cfd94700 1 -- 172.21.15.59:0/3609410404 shutdown_connections 2023-12-11T17:45:45.893 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:45.888+0000 7fe9cfd94700 1 --2- 172.21.15.59:0/3609410404 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9c81020b0 0x7fe9c8108960 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:45.893 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:45.888+0000 7fe9cfd94700 1 --2- 172.21.15.59:0/3609410404 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe9c8100510 0x7fe9c8100930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:45.893 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:45.888+0000 7fe9cfd94700 1 --2- 172.21.15.59:0/3609410404 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe9c8101710 0x7fe9c8101b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:45.893 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:45.888+0000 7fe9cfd94700 1 -- 172.21.15.59:0/3609410404 >> 172.21.15.59:0/3609410404 conn(0x7fe9c80fbad0 msgr2=0x7fe9c80fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:45:45.893 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:45.888+0000 7fe9cfd94700 1 -- 172.21.15.59:0/3609410404 shutdown_connections 2023-12-11T17:45:45.894 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:45.888+0000 7fe9cfd94700 1 -- 172.21.15.59:0/3609410404 wait complete. 2023-12-11T17:45:45.894 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:45.889+0000 7fe9cfd94700 1 Processor -- start 2023-12-11T17:45:45.894 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:45.889+0000 7fe9cfd94700 1 -- start start 2023-12-11T17:45:45.894 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:45.889+0000 7fe9cfd94700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe9c8100510 0x7fe9c81964f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:45.894 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:45.889+0000 7fe9cfd94700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe9c8101710 0x7fe9c8196a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:45.894 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:45.889+0000 7fe9cfd94700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9c81020b0 0x7fe9c819da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:45.894 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:45.889+0000 7fe9cfd94700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c819dfe0 con 0x7fe9c81020b0 2023-12-11T17:45:45.894 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:45.889+0000 7fe9cfd94700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c819e120 con 0x7fe9c8100510 2023-12-11T17:45:45.894 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:45.889+0000 7fe9cfd94700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c819e2b0 con 0x7fe9c8101710 2023-12-11T17:45:45.894 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:45.889+0000 7fe9ce331700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9c81020b0 0x7fe9c819da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:45.894 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:45.889+0000 7fe9ce331700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9c81020b0 0x7fe9c819da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34250/0 (socket says 172.21.15.59:34250) 2023-12-11T17:45:45.894 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:45.889+0000 7fe9ce331700 1 -- 172.21.15.59:0/2536739075 learned_addr learned my addr 172.21.15.59:0/2536739075 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:45:45.894 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:45.889+0000 7fe9cdb30700 1 --2- 172.21.15.59:0/2536739075 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe9c8100510 0x7fe9c81964f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:45.894 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:45.890+0000 7fe9cd32f700 1 --2- 172.21.15.59:0/2536739075 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe9c8101710 0x7fe9c8196a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:45.894 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:45.890+0000 7fe9ce331700 1 -- 172.21.15.59:0/2536739075 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe9c8101710 msgr2=0x7fe9c8196a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:45.895 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:45.890+0000 7fe9ce331700 1 --2- 172.21.15.59:0/2536739075 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe9c8101710 0x7fe9c8196a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:45.895 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:45.890+0000 7fe9ce331700 1 -- 172.21.15.59:0/2536739075 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe9c8100510 msgr2=0x7fe9c81964f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:45.895 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:45.890+0000 7fe9ce331700 1 --2- 172.21.15.59:0/2536739075 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe9c8100510 0x7fe9c81964f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:45.895 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:45.890+0000 7fe9ce331700 1 -- 172.21.15.59:0/2536739075 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9c0009d00 con 0x7fe9c81020b0 2023-12-11T17:45:45.895 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:45.890+0000 7fe9cdb30700 1 --2- 172.21.15.59:0/2536739075 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe9c8100510 0x7fe9c81964f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T17:45:45.895 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:45.890+0000 7fe9cd32f700 1 --2- 172.21.15.59:0/2536739075 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe9c8101710 0x7fe9c8196a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-11T17:45:45.895 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:45.890+0000 7fe9ce331700 1 --2- 172.21.15.59:0/2536739075 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9c81020b0 0x7fe9c819da10 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fe9c0007a00 tx=0x7fe9c00094f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:45.895 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:45.890+0000 7fe9beffd700 1 -- 172.21.15.59:0/2536739075 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c000e040 con 0x7fe9c81020b0 2023-12-11T17:45:45.895 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:45.890+0000 7fe9beffd700 1 -- 172.21.15.59:0/2536739075 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe9c0012dc0 con 0x7fe9c81020b0 2023-12-11T17:45:45.895 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:45.890+0000 7fe9beffd700 1 -- 172.21.15.59:0/2536739075 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c001b450 con 0x7fe9c81020b0 2023-12-11T17:45:45.895 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:45.890+0000 7fe9cfd94700 1 -- 172.21.15.59:0/2536739075 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9c819e530 con 0x7fe9c81020b0 2023-12-11T17:45:45.896 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:45.890+0000 7fe9cfd94700 1 -- 172.21.15.59:0/2536739075 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9c819e950 con 0x7fe9c81020b0 2023-12-11T17:45:45.896 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:45:45 smithi181 ceph-mon[103627]: pgmap v303: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-12-11T17:45:45.897 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:45.891+0000 7fe9cfd94700 1 -- 172.21.15.59:0/2536739075 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9c81078d0 con 0x7fe9c81020b0 2023-12-11T17:45:45.897 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:45.892+0000 7fe9beffd700 1 -- 172.21.15.59:0/2536739075 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7fe9c0012900 con 0x7fe9c81020b0 2023-12-11T17:45:45.897 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:45.893+0000 7fe9beffd700 1 --2- 172.21.15.59:0/2536739075 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fe9b40a1510 0x7fe9b40a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:45.897 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:45.893+0000 7fe9cdb30700 1 --2- 172.21.15.59:0/2536739075 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fe9b40a1510 0x7fe9b40a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:45.897 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:45.893+0000 7fe9beffd700 1 -- 172.21.15.59:0/2536739075 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fe9c00cf180 con 0x7fe9c81020b0 2023-12-11T17:45:45.897 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:45.894+0000 7fe9cdb30700 1 --2- 172.21.15.59:0/2536739075 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fe9b40a1510 0x7fe9b40a39d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fe9c8101570 tx=0x7fe9b800a680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:45.900 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:45.898+0000 7fe9beffd700 1 -- 172.21.15.59:0/2536739075 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fe9c0098410 con 0x7fe9c81020b0 2023-12-11T17:45:45.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:45:45 smithi059 ceph-mon[98847]: pgmap v303: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-12-11T17:45:46.110 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:46.108+0000 7fe9cfd94700 1 -- 172.21.15.59:0/2536739075 --> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fe9c8061a10 con 0x7fe9b40a1510 2023-12-11T17:45:46.132 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:46.130+0000 7fe9beffd700 1 -- 172.21.15.59:0/2536739075 <== mgr.14150 v2:172.21.15.59:6800/3627728353 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+4154 (secure 0 0 0) 0x7fe9c8061a10 con 0x7fe9b40a1510 2023-12-11T17:45:46.132 INFO:teuthology.orchestra.run.smithi059.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2023-12-11T17:45:46.132 INFO:teuthology.orchestra.run.smithi059.stdout:mds.a smithi059 running (93s) 15s ago 93s 15.2M - 19.0.0-119-ga62fda73 0b0a5e4a17a7 d6d01ee907b4 2023-12-11T17:45:46.133 INFO:teuthology.orchestra.run.smithi059.stdout:mds.b smithi115 running (91s) 20s ago 90s 33.0M - 19.0.0-119-ga62fda73 0b0a5e4a17a7 c68d62e00c79 2023-12-11T17:45:46.133 INFO:teuthology.orchestra.run.smithi059.stdout:mds.c smithi181 running (102s) 20s ago 102s 20.5M - 19.0.0-119-ga62fda73 0b0a5e4a17a7 9a72aa6edc69 2023-12-11T17:45:46.133 INFO:teuthology.orchestra.run.smithi059.stdout:mds.d smithi059 running (109s) 15s ago 109s 14.3M - 19.0.0-119-ga62fda73 0b0a5e4a17a7 12731884662c 2023-12-11T17:45:46.133 INFO:teuthology.orchestra.run.smithi059.stdout:mds.e smithi115 running (107s) 20s ago 106s 16.7M - 19.0.0-119-ga62fda73 0b0a5e4a17a7 3613dbeb01a5 2023-12-11T17:45:46.133 INFO:teuthology.orchestra.run.smithi059.stdout:mds.f smithi181 running (105s) 20s ago 104s 17.8M - 19.0.0-119-ga62fda73 0b0a5e4a17a7 500dc1c7c0a8 2023-12-11T17:45:46.133 INFO:teuthology.orchestra.run.smithi059.stdout:mds.g smithi059 running (95s) 15s ago 95s 17.6M - 19.0.0-119-ga62fda73 0b0a5e4a17a7 23e954a0e0d8 2023-12-11T17:45:46.133 INFO:teuthology.orchestra.run.smithi059.stdout:mds.h smithi115 running (100s) 20s ago 100s 21.2M - 19.0.0-119-ga62fda73 0b0a5e4a17a7 4e1cbc7bbb6f 2023-12-11T17:45:46.133 INFO:teuthology.orchestra.run.smithi059.stdout:mds.i smithi181 running (98s) 20s ago 97s 19.0M - 19.0.0-119-ga62fda73 0b0a5e4a17a7 3de1d6fedb90 2023-12-11T17:45:46.133 INFO:teuthology.orchestra.run.smithi059.stdout:mds.j smithi059 running (111s) 15s ago 111s 17.7M - 19.0.0-119-ga62fda73 0b0a5e4a17a7 934c2b8b45fe 2023-12-11T17:45:46.133 INFO:teuthology.orchestra.run.smithi059.stdout:mds.k smithi115 running (113s) 20s ago 113s 17.4M - 19.0.0-119-ga62fda73 0b0a5e4a17a7 7b2f668a080e 2023-12-11T17:45:46.133 INFO:teuthology.orchestra.run.smithi059.stdout:mds.l smithi181 running (116s) 20s ago 115s 17.7M - 19.0.0-119-ga62fda73 0b0a5e4a17a7 256fa9247ebd 2023-12-11T17:45:46.133 INFO:teuthology.orchestra.run.smithi059.stdout:mgr.x smithi059 *:9283,8765 running (8m) 15s ago 8m 526M - 19.0.0-119-ga62fda73 0b0a5e4a17a7 ffa6f4d32753 2023-12-11T17:45:46.133 INFO:teuthology.orchestra.run.smithi059.stdout:mgr.y smithi115 *:8443,8765 running (6m) 20s ago 6m 452M - 19.0.0-119-ga62fda73 0b0a5e4a17a7 ac4263c0c699 2023-12-11T17:45:46.133 INFO:teuthology.orchestra.run.smithi059.stdout:mgr.z smithi181 *:8443,8765 running (6m) 20s ago 6m 450M - 19.0.0-119-ga62fda73 0b0a5e4a17a7 650264cab371 2023-12-11T17:45:46.133 INFO:teuthology.orchestra.run.smithi059.stdout:mon.a smithi059 running (8m) 15s ago 8m 96.8M 2048M 19.0.0-119-ga62fda73 0b0a5e4a17a7 05668a55c7a9 2023-12-11T17:45:46.134 INFO:teuthology.orchestra.run.smithi059.stdout:mon.b smithi115 running (6m) 20s ago 6m 62.7M 2048M 19.0.0-119-ga62fda73 0b0a5e4a17a7 0f99ecd93664 2023-12-11T17:45:46.134 INFO:teuthology.orchestra.run.smithi059.stdout:mon.c smithi181 running (6m) 20s ago 6m 62.7M 2048M 19.0.0-119-ga62fda73 0b0a5e4a17a7 d4faf4e50007 2023-12-11T17:45:46.134 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0 smithi059 running (6m) 15s ago 6m 71.4M 4293M 19.0.0-119-ga62fda73 0b0a5e4a17a7 b2ab57f7b32e 2023-12-11T17:45:46.134 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1 smithi115 running (5m) 20s ago 5m 78.2M 4292M 19.0.0-119-ga62fda73 0b0a5e4a17a7 78710ddde772 2023-12-11T17:45:46.134 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2 smithi181 running (5m) 20s ago 5m 73.7M 4292M 19.0.0-119-ga62fda73 0b0a5e4a17a7 0859ec8d797f 2023-12-11T17:45:46.134 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3 smithi059 running (5m) 15s ago 5m 65.5M 4293M 19.0.0-119-ga62fda73 0b0a5e4a17a7 c9ba71c1c4f7 2023-12-11T17:45:46.134 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4 smithi115 running (4m) 20s ago 4m 64.1M 4292M 19.0.0-119-ga62fda73 0b0a5e4a17a7 384db3c2b2bb 2023-12-11T17:45:46.134 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5 smithi181 running (4m) 20s ago 4m 69.2M 4292M 19.0.0-119-ga62fda73 0b0a5e4a17a7 95cee85a887b 2023-12-11T17:45:46.134 INFO:teuthology.orchestra.run.smithi059.stdout:osd.6 smithi059 running (4m) 15s ago 4m 61.4M 4293M 19.0.0-119-ga62fda73 0b0a5e4a17a7 90474b5cc88a 2023-12-11T17:45:46.134 INFO:teuthology.orchestra.run.smithi059.stdout:osd.7 smithi115 running (3m) 20s ago 3m 65.7M 4292M 19.0.0-119-ga62fda73 0b0a5e4a17a7 2fb1b07b3858 2023-12-11T17:45:46.134 INFO:teuthology.orchestra.run.smithi059.stdout:osd.8 smithi181 running (3m) 20s ago 3m 64.4M 4292M 19.0.0-119-ga62fda73 0b0a5e4a17a7 db40779c8413 2023-12-11T17:45:46.134 INFO:teuthology.orchestra.run.smithi059.stdout:osd.9 smithi059 running (2m) 15s ago 2m 63.2M 4293M 19.0.0-119-ga62fda73 0b0a5e4a17a7 00c15450d577 2023-12-11T17:45:46.134 INFO:teuthology.orchestra.run.smithi059.stdout:osd.10 smithi115 running (2m) 20s ago 2m 64.4M 4292M 19.0.0-119-ga62fda73 0b0a5e4a17a7 4bab53c77384 2023-12-11T17:45:46.134 INFO:teuthology.orchestra.run.smithi059.stdout:osd.11 smithi181 running (2m) 20s ago 2m 57.5M 4292M 19.0.0-119-ga62fda73 0b0a5e4a17a7 0c0eae781f3a 2023-12-11T17:45:46.135 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:46.132+0000 7fe9cfd94700 1 -- 172.21.15.59:0/2536739075 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fe9b40a1510 msgr2=0x7fe9b40a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:46.135 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:46.132+0000 7fe9cfd94700 1 --2- 172.21.15.59:0/2536739075 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fe9b40a1510 0x7fe9b40a39d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fe9c8101570 tx=0x7fe9b800a680 comp rx=0 tx=0).stop 2023-12-11T17:45:46.135 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:46.132+0000 7fe9cfd94700 1 -- 172.21.15.59:0/2536739075 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9c81020b0 msgr2=0x7fe9c819da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:46.135 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:46.132+0000 7fe9cfd94700 1 --2- 172.21.15.59:0/2536739075 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9c81020b0 0x7fe9c819da10 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fe9c0007a00 tx=0x7fe9c00094f0 comp rx=0 tx=0).stop 2023-12-11T17:45:46.135 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:46.132+0000 7fe9cfd94700 1 -- 172.21.15.59:0/2536739075 shutdown_connections 2023-12-11T17:45:46.135 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:46.132+0000 7fe9cfd94700 1 --2- 172.21.15.59:0/2536739075 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fe9b40a1510 0x7fe9b40a39d0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:46.135 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:46.132+0000 7fe9cfd94700 1 --2- 172.21.15.59:0/2536739075 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9c81020b0 0x7fe9c819da10 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:46.136 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:46.132+0000 7fe9cfd94700 1 --2- 172.21.15.59:0/2536739075 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe9c8100510 0x7fe9c81964f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:46.136 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:46.132+0000 7fe9cfd94700 1 --2- 172.21.15.59:0/2536739075 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe9c8101710 0x7fe9c8196a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:46.136 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:46.132+0000 7fe9cfd94700 1 -- 172.21.15.59:0/2536739075 >> 172.21.15.59:0/2536739075 conn(0x7fe9c80fbad0 msgr2=0x7fe9c8104a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:45:46.136 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:46.132+0000 7fe9cfd94700 1 -- 172.21.15.59:0/2536739075 shutdown_connections 2023-12-11T17:45:46.136 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:46.132+0000 7fe9cfd94700 1 -- 172.21.15.59:0/2536739075 wait complete. 2023-12-11T17:45:46.726 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d0a07eaa-984b-11ee-95a3-87774f69a715 -- bash -c 'ceph orch ls' 2023-12-11T17:45:47.331 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/mon.a/config 2023-12-11T17:45:47.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:45:47 smithi115 ceph-mon[102748]: from='client.15228 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T17:45:47.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:45:47 smithi115 ceph-mon[102748]: pgmap v304: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-12-11T17:45:47.808 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:45:47 smithi059 ceph-mon[98847]: from='client.15228 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T17:45:47.809 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:45:47 smithi059 ceph-mon[98847]: pgmap v304: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-12-11T17:45:47.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:45:47 smithi181 ceph-mon[103627]: from='client.15228 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T17:45:47.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:45:47 smithi181 ceph-mon[103627]: pgmap v304: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-12-11T17:45:48.560 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:48.555+0000 7f36bf61e700 1 -- 172.21.15.59:0/3141778547 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f36b8102820 msgr2=0x7f36b8102c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:48.560 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:48.555+0000 7f36bf61e700 1 --2- 172.21.15.59:0/3141778547 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f36b8102820 0x7f36b8102c80 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f36b40099c0 tx=0x7f36b4009cd0 comp rx=0 tx=0).stop 2023-12-11T17:45:48.560 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:48.555+0000 7f36bf61e700 1 -- 172.21.15.59:0/3141778547 shutdown_connections 2023-12-11T17:45:48.560 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:48.555+0000 7f36bf61e700 1 --2- 172.21.15.59:0/3141778547 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f36b81031c0 0x7f36b8109a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:48.560 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:48.555+0000 7f36bf61e700 1 --2- 172.21.15.59:0/3141778547 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36b8101620 0x7f36b8101a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:48.560 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:48.555+0000 7f36bf61e700 1 --2- 172.21.15.59:0/3141778547 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f36b8102820 0x7f36b8102c80 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:48.560 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:48.555+0000 7f36bf61e700 1 -- 172.21.15.59:0/3141778547 >> 172.21.15.59:0/3141778547 conn(0x7f36b80fcbe0 msgr2=0x7f36b80ff000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:45:48.561 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:48.555+0000 7f36bf61e700 1 -- 172.21.15.59:0/3141778547 shutdown_connections 2023-12-11T17:45:48.561 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:48.555+0000 7f36bf61e700 1 -- 172.21.15.59:0/3141778547 wait complete. 2023-12-11T17:45:48.561 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:48.556+0000 7f36bf61e700 1 Processor -- start 2023-12-11T17:45:48.561 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:48.556+0000 7f36bf61e700 1 -- start start 2023-12-11T17:45:48.561 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:48.556+0000 7f36bf61e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f36b8101620 0x7f36b81975d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:48.561 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:48.556+0000 7f36bf61e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f36b8102820 0x7f36b8197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:48.561 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:48.556+0000 7f36bf61e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36b81031c0 0x7f36b819eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:48.561 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:48.556+0000 7f36bf61e700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36b819f110 con 0x7f36b8101620 2023-12-11T17:45:48.561 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:48.556+0000 7f36bf61e700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36b819f250 con 0x7f36b8102820 2023-12-11T17:45:48.561 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:48.556+0000 7f36bf61e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36b819f3e0 con 0x7f36b81031c0 2023-12-11T17:45:48.561 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:48.556+0000 7f36bd3ba700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f36b8101620 0x7f36b81975d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:48.561 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:48.556+0000 7f36bd3ba700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f36b8101620 0x7f36b81975d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34258/0 (socket says 172.21.15.59:34258) 2023-12-11T17:45:48.561 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:48.556+0000 7f36bd3ba700 1 -- 172.21.15.59:0/2682114853 learned_addr learned my addr 172.21.15.59:0/2682114853 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:45:48.561 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:48.557+0000 7f36bdbbb700 1 --2- 172.21.15.59:0/2682114853 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36b81031c0 0x7f36b819eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:48.562 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:48.557+0000 7f36bcbb9700 1 --2- 172.21.15.59:0/2682114853 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f36b8102820 0x7f36b8197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:48.562 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:48.557+0000 7f36bd3ba700 1 -- 172.21.15.59:0/2682114853 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36b81031c0 msgr2=0x7f36b819eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:48.562 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:48.557+0000 7f36bd3ba700 1 --2- 172.21.15.59:0/2682114853 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36b81031c0 0x7f36b819eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:48.562 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:48.557+0000 7f36bd3ba700 1 -- 172.21.15.59:0/2682114853 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f36b8102820 msgr2=0x7f36b8197b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:48.562 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:48.557+0000 7f36bd3ba700 1 --2- 172.21.15.59:0/2682114853 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f36b8102820 0x7f36b8197b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:48.562 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:48.557+0000 7f36bd3ba700 1 -- 172.21.15.59:0/2682114853 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36b40095f0 con 0x7f36b8101620 2023-12-11T17:45:48.562 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:48.557+0000 7f36bcbb9700 1 --2- 172.21.15.59:0/2682114853 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f36b8102820 0x7f36b8197b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-11T17:45:48.562 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:48.557+0000 7f36bd3ba700 1 --2- 172.21.15.59:0/2682114853 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f36b8101620 0x7f36b81975d0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f36a800c9a0 tx=0x7f36a800cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:48.562 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:48.557+0000 7f36ae7fc700 1 -- 172.21.15.59:0/2682114853 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36a8004220 con 0x7f36b8101620 2023-12-11T17:45:48.562 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:48.557+0000 7f36ae7fc700 1 -- 172.21.15.59:0/2682114853 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f36a8007d20 con 0x7f36b8101620 2023-12-11T17:45:48.562 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:48.557+0000 7f36bf61e700 1 -- 172.21.15.59:0/2682114853 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36b819f6c0 con 0x7f36b8101620 2023-12-11T17:45:48.562 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:48.557+0000 7f36ae7fc700 1 -- 172.21.15.59:0/2682114853 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36a8003d00 con 0x7f36b8101620 2023-12-11T17:45:48.562 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:48.557+0000 7f36bf61e700 1 -- 172.21.15.59:0/2682114853 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36b819fbc0 con 0x7f36b8101620 2023-12-11T17:45:48.563 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:48.558+0000 7f36bf61e700 1 -- 172.21.15.59:0/2682114853 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36b8108b00 con 0x7f36b8101620 2023-12-11T17:45:48.563 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:48.560+0000 7f36ae7fc700 1 -- 172.21.15.59:0/2682114853 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f36a8004380 con 0x7f36b8101620 2023-12-11T17:45:48.564 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:48.560+0000 7f36ae7fc700 1 --2- 172.21.15.59:0/2682114853 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f36a40a1510 0x7f36a40a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:48.564 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:48.560+0000 7f36bcbb9700 1 --2- 172.21.15.59:0/2682114853 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f36a40a1510 0x7f36a40a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:48.564 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:48.560+0000 7f36ae7fc700 1 -- 172.21.15.59:0/2682114853 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f36a80cd1d0 con 0x7f36b8101620 2023-12-11T17:45:48.564 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:48.561+0000 7f36bcbb9700 1 --2- 172.21.15.59:0/2682114853 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f36a40a1510 0x7f36a40a39d0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f36b4003820 tx=0x7f36b4000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:48.566 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:48.563+0000 7f36ae7fc700 1 -- 172.21.15.59:0/2682114853 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f36a8096370 con 0x7f36b8101620 2023-12-11T17:45:48.777 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:48.775+0000 7f36bf61e700 1 -- 172.21.15.59:0/2682114853 --> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f36b8061a10 con 0x7f36a40a1510 2023-12-11T17:45:48.785 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:48.783+0000 7f36ae7fc700 1 -- 172.21.15.59:0/2682114853 <== mgr.14150 v2:172.21.15.59:6800/3627728353 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+980 (secure 0 0 0) 0x7f36b8061a10 con 0x7f36a40a1510 2023-12-11T17:45:48.786 INFO:teuthology.orchestra.run.smithi059.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2023-12-11T17:45:48.786 INFO:teuthology.orchestra.run.smithi059.stdout:mds.all 12/12 23s ago 2m smithi059=a;smithi059=d;smithi059=g;smithi059=j;smithi115=b;smithi115=e;smithi115=h;smithi115=k;smithi181=c;smithi181=f;smithi181=i;smithi181=l;count:12 2023-12-11T17:45:48.786 INFO:teuthology.orchestra.run.smithi059.stdout:mgr 3/3 23s ago 6m smithi059=x;smithi115=y;smithi181=z;count:3 2023-12-11T17:45:48.786 INFO:teuthology.orchestra.run.smithi059.stdout:mon 3/3 23s ago 6m smithi059:172.21.15.59=a;smithi115:172.21.15.115=b;smithi181:172.21.15.181=c;count:3 2023-12-11T17:45:48.787 INFO:teuthology.orchestra.run.smithi059.stdout:osd 12 23s ago - 2023-12-11T17:45:48.788 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:48.785+0000 7f36bf61e700 1 -- 172.21.15.59:0/2682114853 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f36a40a1510 msgr2=0x7f36a40a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:48.788 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:48.785+0000 7f36bf61e700 1 --2- 172.21.15.59:0/2682114853 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f36a40a1510 0x7f36a40a39d0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f36b4003820 tx=0x7f36b4000bc0 comp rx=0 tx=0).stop 2023-12-11T17:45:48.788 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:48.785+0000 7f36bf61e700 1 -- 172.21.15.59:0/2682114853 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f36b8101620 msgr2=0x7f36b81975d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:48.788 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:48.785+0000 7f36bf61e700 1 --2- 172.21.15.59:0/2682114853 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f36b8101620 0x7f36b81975d0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f36a800c9a0 tx=0x7f36a800cd60 comp rx=0 tx=0).stop 2023-12-11T17:45:48.788 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:48.785+0000 7f36bf61e700 1 -- 172.21.15.59:0/2682114853 shutdown_connections 2023-12-11T17:45:48.788 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:48.785+0000 7f36bf61e700 1 --2- 172.21.15.59:0/2682114853 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f36a40a1510 0x7f36a40a39d0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:48.788 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:48.785+0000 7f36bf61e700 1 --2- 172.21.15.59:0/2682114853 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36b81031c0 0x7f36b819eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:48.789 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:48.785+0000 7f36bf61e700 1 --2- 172.21.15.59:0/2682114853 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f36b8101620 0x7f36b81975d0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:48.789 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:48.785+0000 7f36bf61e700 1 --2- 172.21.15.59:0/2682114853 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f36b8102820 0x7f36b8197b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:48.789 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:48.785+0000 7f36bf61e700 1 -- 172.21.15.59:0/2682114853 >> 172.21.15.59:0/2682114853 conn(0x7f36b80fcbe0 msgr2=0x7f36b8105c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:45:48.789 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:48.785+0000 7f36bf61e700 1 -- 172.21.15.59:0/2682114853 shutdown_connections 2023-12-11T17:45:48.789 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:48.785+0000 7f36bf61e700 1 -- 172.21.15.59:0/2682114853 wait complete. 2023-12-11T17:45:49.202 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d0a07eaa-984b-11ee-95a3-87774f69a715 -- bash -c 'ceph orch host ls' 2023-12-11T17:45:49.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:45:49 smithi115 ceph-mon[102748]: pgmap v305: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-12-11T17:45:49.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:45:49 smithi115 ceph-mon[102748]: from='client.15234 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T17:45:49.804 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/mon.a/config 2023-12-11T17:45:49.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:45:49 smithi181 ceph-mon[103627]: pgmap v305: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-12-11T17:45:49.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:45:49 smithi181 ceph-mon[103627]: from='client.15234 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T17:45:49.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:45:49 smithi059 ceph-mon[98847]: pgmap v305: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-12-11T17:45:49.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:45:49 smithi059 ceph-mon[98847]: from='client.15234 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T17:45:50.921 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:50.916+0000 7f0686ea1700 1 -- 172.21.15.59:0/3990790628 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0680101710 msgr2=0x7f0680101b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:50.922 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:50.916+0000 7f0686ea1700 1 --2- 172.21.15.59:0/3990790628 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0680101710 0x7f0680101b70 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f066c0099c0 tx=0x7f066c009cd0 comp rx=0 tx=0).stop 2023-12-11T17:45:50.922 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:50.917+0000 7f0686ea1700 1 -- 172.21.15.59:0/3990790628 shutdown_connections 2023-12-11T17:45:50.922 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:50.917+0000 7f0686ea1700 1 --2- 172.21.15.59:0/3990790628 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f06801020b0 0x7f0680108960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:50.922 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:50.917+0000 7f0686ea1700 1 --2- 172.21.15.59:0/3990790628 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0680100510 0x7f0680100930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:50.922 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:50.917+0000 7f0686ea1700 1 --2- 172.21.15.59:0/3990790628 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0680101710 0x7f0680101b70 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:50.922 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:50.917+0000 7f0686ea1700 1 -- 172.21.15.59:0/3990790628 >> 172.21.15.59:0/3990790628 conn(0x7f06800fbad0 msgr2=0x7f06800fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:45:50.922 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:50.917+0000 7f0686ea1700 1 -- 172.21.15.59:0/3990790628 shutdown_connections 2023-12-11T17:45:50.922 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:50.917+0000 7f0686ea1700 1 -- 172.21.15.59:0/3990790628 wait complete. 2023-12-11T17:45:50.922 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:50.917+0000 7f0686ea1700 1 Processor -- start 2023-12-11T17:45:50.922 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:50.917+0000 7f0686ea1700 1 -- start start 2023-12-11T17:45:50.922 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:50.917+0000 7f0686ea1700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0680100510 0x7f0680197480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:50.922 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:50.917+0000 7f0686ea1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0680101710 0x7f06801979c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:50.923 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:50.917+0000 7f0686ea1700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f06801020b0 0x7f068019e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:50.923 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:50.917+0000 7f0686ea1700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f068019efc0 con 0x7f0680100510 2023-12-11T17:45:50.923 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:50.917+0000 7f0686ea1700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f068019f100 con 0x7f06801020b0 2023-12-11T17:45:50.923 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:50.917+0000 7f0686ea1700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f068019f290 con 0x7f0680101710 2023-12-11T17:45:50.923 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:50.918+0000 7f0684c3d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0680100510 0x7f0680197480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:50.923 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:50.918+0000 7f0684c3d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0680100510 0x7f0680197480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34270/0 (socket says 172.21.15.59:34270) 2023-12-11T17:45:50.923 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:50.918+0000 7f0684c3d700 1 -- 172.21.15.59:0/2181935794 learned_addr learned my addr 172.21.15.59:0/2181935794 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:45:50.923 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:50.918+0000 7f067ffff700 1 --2- 172.21.15.59:0/2181935794 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0680101710 0x7f06801979c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:50.923 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:50.918+0000 7f068543e700 1 --2- 172.21.15.59:0/2181935794 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f06801020b0 0x7f068019e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:50.923 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:50.918+0000 7f0684c3d700 1 -- 172.21.15.59:0/2181935794 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0680101710 msgr2=0x7f06801979c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:50.923 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:50.918+0000 7f0684c3d700 1 --2- 172.21.15.59:0/2181935794 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0680101710 0x7f06801979c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:50.923 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:50.918+0000 7f0684c3d700 1 -- 172.21.15.59:0/2181935794 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f06801020b0 msgr2=0x7f068019e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:50.924 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:50.918+0000 7f0684c3d700 1 --2- 172.21.15.59:0/2181935794 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f06801020b0 0x7f068019e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:50.924 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:50.918+0000 7f0684c3d700 1 -- 172.21.15.59:0/2181935794 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f066c0095f0 con 0x7f0680100510 2023-12-11T17:45:50.924 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:50.918+0000 7f067ffff700 1 --2- 172.21.15.59:0/2181935794 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0680101710 0x7f06801979c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T17:45:50.924 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:50.918+0000 7f068543e700 1 --2- 172.21.15.59:0/2181935794 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f06801020b0 0x7f068019e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-11T17:45:50.924 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:50.918+0000 7f0684c3d700 1 --2- 172.21.15.59:0/2181935794 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0680100510 0x7f0680197480 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f067400d9b0 tx=0x7f067400dd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:50.924 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:50.918+0000 7f067dffb700 1 -- 172.21.15.59:0/2181935794 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f067400be80 con 0x7f0680100510 2023-12-11T17:45:50.924 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:50.918+0000 7f0686ea1700 1 -- 172.21.15.59:0/2181935794 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f068019f570 con 0x7f0680100510 2023-12-11T17:45:50.924 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:50.918+0000 7f067dffb700 1 -- 172.21.15.59:0/2181935794 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f06740044e0 con 0x7f0680100510 2023-12-11T17:45:50.924 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:50.918+0000 7f067dffb700 1 -- 172.21.15.59:0/2181935794 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0674003b40 con 0x7f0680100510 2023-12-11T17:45:50.924 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:50.918+0000 7f0686ea1700 1 -- 172.21.15.59:0/2181935794 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f068019fac0 con 0x7f0680100510 2023-12-11T17:45:50.925 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:50.919+0000 7f0686ea1700 1 -- 172.21.15.59:0/2181935794 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f068004f290 con 0x7f0680100510 2023-12-11T17:45:50.925 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:50.920+0000 7f067dffb700 1 -- 172.21.15.59:0/2181935794 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f0674009940 con 0x7f0680100510 2023-12-11T17:45:50.925 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:50.921+0000 7f067dffb700 1 --2- 172.21.15.59:0/2181935794 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f06700a1510 0x7f06700a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:50.925 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:50.921+0000 7f067ffff700 1 --2- 172.21.15.59:0/2181935794 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f06700a1510 0x7f06700a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:50.925 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:50.921+0000 7f067dffb700 1 -- 172.21.15.59:0/2181935794 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f06740ccf20 con 0x7f0680100510 2023-12-11T17:45:50.925 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:50.922+0000 7f067ffff700 1 --2- 172.21.15.59:0/2181935794 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f06700a1510 0x7f06700a39d0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f066c000c00 tx=0x7f066c005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:50.928 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:50.926+0000 7f067dffb700 1 -- 172.21.15.59:0/2181935794 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f0674096120 con 0x7f0680100510 2023-12-11T17:45:51.136 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:51.134+0000 7f0686ea1700 1 -- 172.21.15.59:0/2181935794 --> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f06800fdc80 con 0x7f06700a1510 2023-12-11T17:45:51.137 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:51.135+0000 7f067dffb700 1 -- 172.21.15.59:0/2181935794 <== mgr.14150 v2:172.21.15.59:6800/3627728353 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+191 (secure 0 0 0) 0x7f06800fdc80 con 0x7f06700a1510 2023-12-11T17:45:51.138 INFO:teuthology.orchestra.run.smithi059.stdout:HOST ADDR LABELS STATUS 2023-12-11T17:45:51.138 INFO:teuthology.orchestra.run.smithi059.stdout:smithi059 172.21.15.59 2023-12-11T17:45:51.138 INFO:teuthology.orchestra.run.smithi059.stdout:smithi115 172.21.15.115 2023-12-11T17:45:51.138 INFO:teuthology.orchestra.run.smithi059.stdout:smithi181 172.21.15.181 2023-12-11T17:45:51.138 INFO:teuthology.orchestra.run.smithi059.stdout:3 hosts in cluster 2023-12-11T17:45:51.140 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:51.137+0000 7f0686ea1700 1 -- 172.21.15.59:0/2181935794 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f06700a1510 msgr2=0x7f06700a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:51.140 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:51.137+0000 7f0686ea1700 1 --2- 172.21.15.59:0/2181935794 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f06700a1510 0x7f06700a39d0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f066c000c00 tx=0x7f066c005fd0 comp rx=0 tx=0).stop 2023-12-11T17:45:51.140 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:51.137+0000 7f0686ea1700 1 -- 172.21.15.59:0/2181935794 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0680100510 msgr2=0x7f0680197480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:51.140 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:51.137+0000 7f0686ea1700 1 --2- 172.21.15.59:0/2181935794 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0680100510 0x7f0680197480 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f067400d9b0 tx=0x7f067400dd70 comp rx=0 tx=0).stop 2023-12-11T17:45:51.140 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:51.137+0000 7f0686ea1700 1 -- 172.21.15.59:0/2181935794 shutdown_connections 2023-12-11T17:45:51.140 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:51.137+0000 7f0686ea1700 1 --2- 172.21.15.59:0/2181935794 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f06700a1510 0x7f06700a39d0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:51.141 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:51.137+0000 7f0686ea1700 1 --2- 172.21.15.59:0/2181935794 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f06801020b0 0x7f068019e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:51.141 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:51.137+0000 7f0686ea1700 1 --2- 172.21.15.59:0/2181935794 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0680100510 0x7f0680197480 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:51.141 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:51.137+0000 7f0686ea1700 1 --2- 172.21.15.59:0/2181935794 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0680101710 0x7f06801979c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:51.141 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:51.137+0000 7f0686ea1700 1 -- 172.21.15.59:0/2181935794 >> 172.21.15.59:0/2181935794 conn(0x7f06800fbad0 msgr2=0x7f06800fd570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:45:51.141 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:51.137+0000 7f0686ea1700 1 -- 172.21.15.59:0/2181935794 shutdown_connections 2023-12-11T17:45:51.141 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:51.137+0000 7f0686ea1700 1 -- 172.21.15.59:0/2181935794 wait complete. 2023-12-11T17:45:51.580 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d0a07eaa-984b-11ee-95a3-87774f69a715 -- bash -c 'ceph orch device ls' 2023-12-11T17:45:51.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:45:51 smithi115 ceph-mon[102748]: pgmap v306: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-12-11T17:45:51.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:45:51 smithi181 ceph-mon[103627]: pgmap v306: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-12-11T17:45:51.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:45:51 smithi059 ceph-mon[98847]: pgmap v306: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-12-11T17:45:52.183 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/mon.a/config 2023-12-11T17:45:52.716 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:45:52 smithi059 ceph-mon[98847]: from='client.15240 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T17:45:52.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:45:52 smithi115 ceph-mon[102748]: from='client.15240 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T17:45:52.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:45:52 smithi181 ceph-mon[103627]: from='client.15240 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T17:45:53.392 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:53.387+0000 7fdd16a76700 1 -- 172.21.15.59:0/2824862887 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdd10101620 msgr2=0x7fdd10101a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:53.392 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:53.387+0000 7fdd16a76700 1 --2- 172.21.15.59:0/2824862887 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdd10101620 0x7fdd10101a40 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fdd040099c0 tx=0x7fdd04009cd0 comp rx=0 tx=0).stop 2023-12-11T17:45:53.392 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:53.387+0000 7fdd16a76700 1 -- 172.21.15.59:0/2824862887 shutdown_connections 2023-12-11T17:45:53.392 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:53.387+0000 7fdd16a76700 1 --2- 172.21.15.59:0/2824862887 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd101031c0 0x7fdd10109a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:53.392 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:53.387+0000 7fdd16a76700 1 --2- 172.21.15.59:0/2824862887 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdd10101620 0x7fdd10101a40 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:53.392 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:53.387+0000 7fdd16a76700 1 --2- 172.21.15.59:0/2824862887 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdd10102820 0x7fdd10102c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:53.392 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:53.387+0000 7fdd16a76700 1 -- 172.21.15.59:0/2824862887 >> 172.21.15.59:0/2824862887 conn(0x7fdd100fcbc0 msgr2=0x7fdd100ff000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:45:53.393 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:53.387+0000 7fdd16a76700 1 -- 172.21.15.59:0/2824862887 shutdown_connections 2023-12-11T17:45:53.393 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:53.387+0000 7fdd16a76700 1 -- 172.21.15.59:0/2824862887 wait complete. 2023-12-11T17:45:53.393 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:53.388+0000 7fdd16a76700 1 Processor -- start 2023-12-11T17:45:53.393 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:53.388+0000 7fdd16a76700 1 -- start start 2023-12-11T17:45:53.393 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:53.388+0000 7fdd16a76700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdd10101620 0x7fdd10197580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:53.393 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:53.388+0000 7fdd16a76700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd10102820 0x7fdd10197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:53.393 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:53.388+0000 7fdd16a76700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdd101031c0 0x7fdd1019eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:53.393 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:53.388+0000 7fdd16a76700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd1019f0c0 con 0x7fdd10101620 2023-12-11T17:45:53.393 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:53.388+0000 7fdd16a76700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd1019f200 con 0x7fdd101031c0 2023-12-11T17:45:53.393 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:53.388+0000 7fdd16a76700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd1019f390 con 0x7fdd10102820 2023-12-11T17:45:53.393 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:53.388+0000 7fdd14812700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdd10101620 0x7fdd10197580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:53.393 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:53.388+0000 7fdd14812700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdd10101620 0x7fdd10197580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34282/0 (socket says 172.21.15.59:34282) 2023-12-11T17:45:53.393 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:53.388+0000 7fdd14812700 1 -- 172.21.15.59:0/810065400 learned_addr learned my addr 172.21.15.59:0/810065400 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:45:53.393 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:53.389+0000 7fdd0ffff700 1 --2- 172.21.15.59:0/810065400 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd10102820 0x7fdd10197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:53.393 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:53.389+0000 7fdd15013700 1 --2- 172.21.15.59:0/810065400 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdd101031c0 0x7fdd1019eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:53.394 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:53.389+0000 7fdd14812700 1 -- 172.21.15.59:0/810065400 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd10102820 msgr2=0x7fdd10197ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:53.394 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:53.389+0000 7fdd14812700 1 --2- 172.21.15.59:0/810065400 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd10102820 0x7fdd10197ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:53.394 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:53.389+0000 7fdd14812700 1 -- 172.21.15.59:0/810065400 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdd101031c0 msgr2=0x7fdd1019eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:53.394 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:53.389+0000 7fdd14812700 1 --2- 172.21.15.59:0/810065400 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdd101031c0 0x7fdd1019eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:53.394 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:53.389+0000 7fdd14812700 1 -- 172.21.15.59:0/810065400 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd040095f0 con 0x7fdd10101620 2023-12-11T17:45:53.394 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:53.389+0000 7fdd15013700 1 --2- 172.21.15.59:0/810065400 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdd101031c0 0x7fdd1019eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T17:45:53.394 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:53.389+0000 7fdd14812700 1 --2- 172.21.15.59:0/810065400 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdd10101620 0x7fdd10197580 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fdd04009f50 tx=0x7fdd04004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:53.394 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:53.389+0000 7fdd0dffb700 1 -- 172.21.15.59:0/810065400 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd0401d070 con 0x7fdd10101620 2023-12-11T17:45:53.394 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:53.389+0000 7fdd16a76700 1 -- 172.21.15.59:0/810065400 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd1019f610 con 0x7fdd10101620 2023-12-11T17:45:53.394 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:53.389+0000 7fdd0dffb700 1 -- 172.21.15.59:0/810065400 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdd040054f0 con 0x7fdd10101620 2023-12-11T17:45:53.394 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:53.389+0000 7fdd0dffb700 1 -- 172.21.15.59:0/810065400 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd04004a00 con 0x7fdd10101620 2023-12-11T17:45:53.394 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:53.389+0000 7fdd16a76700 1 -- 172.21.15.59:0/810065400 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd1019fb00 con 0x7fdd10101620 2023-12-11T17:45:53.395 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:53.391+0000 7fdd16a76700 1 -- 172.21.15.59:0/810065400 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd10108b50 con 0x7fdd10101620 2023-12-11T17:45:53.395 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:53.391+0000 7fdd0dffb700 1 -- 172.21.15.59:0/810065400 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7fdd04003b50 con 0x7fdd10101620 2023-12-11T17:45:53.395 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:53.392+0000 7fdd0dffb700 1 --2- 172.21.15.59:0/810065400 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fdd000a1510 0x7fdd000a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:53.395 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:53.392+0000 7fdd0ffff700 1 --2- 172.21.15.59:0/810065400 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fdd000a1510 0x7fdd000a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:53.395 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:53.392+0000 7fdd0dffb700 1 -- 172.21.15.59:0/810065400 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fdd040cdf50 con 0x7fdd10101620 2023-12-11T17:45:53.395 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:53.392+0000 7fdd0ffff700 1 --2- 172.21.15.59:0/810065400 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fdd000a1510 0x7fdd000a39d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fdcfc005950 tx=0x7fdcfc00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:53.399 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:53.396+0000 7fdd0dffb700 1 -- 172.21.15.59:0/810065400 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fdd040971e0 con 0x7fdd10101620 2023-12-11T17:45:53.604 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:53.601+0000 7fdd16a76700 1 -- 172.21.15.59:0/810065400 --> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fdd10061a10 con 0x7fdd000a1510 2023-12-11T17:45:53.606 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:53.604+0000 7fdd0dffb700 1 -- 172.21.15.59:0/810065400 <== mgr.14150 v2:172.21.15.59:6800/3627728353 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+532 (secure 0 0 0) 0x7fdd10061a10 con 0x7fdd000a1510 2023-12-11T17:45:53.606 INFO:teuthology.orchestra.run.smithi059.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2023-12-11T17:45:53.606 INFO:teuthology.orchestra.run.smithi059.stdout:smithi059 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT5330009L400BGN 372G No 2m ago Has a FileSystem, LVM detected 2023-12-11T17:45:53.606 INFO:teuthology.orchestra.run.smithi059.stdout:smithi115 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400UU400BGN 372G No 2m ago Has a FileSystem, LVM detected 2023-12-11T17:45:53.606 INFO:teuthology.orchestra.run.smithi059.stdout:smithi181 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6246000E400BGN 372G No 2m ago Has a FileSystem, LVM detected 2023-12-11T17:45:53.608 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:53.605+0000 7fdd16a76700 1 -- 172.21.15.59:0/810065400 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fdd000a1510 msgr2=0x7fdd000a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:53.608 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:53.605+0000 7fdd16a76700 1 --2- 172.21.15.59:0/810065400 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fdd000a1510 0x7fdd000a39d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fdcfc005950 tx=0x7fdcfc00b410 comp rx=0 tx=0).stop 2023-12-11T17:45:53.608 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:53.605+0000 7fdd16a76700 1 -- 172.21.15.59:0/810065400 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdd10101620 msgr2=0x7fdd10197580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:53.608 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:53.605+0000 7fdd16a76700 1 --2- 172.21.15.59:0/810065400 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdd10101620 0x7fdd10197580 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fdd04009f50 tx=0x7fdd04004730 comp rx=0 tx=0).stop 2023-12-11T17:45:53.608 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:53.606+0000 7fdd16a76700 1 -- 172.21.15.59:0/810065400 shutdown_connections 2023-12-11T17:45:53.609 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:53.606+0000 7fdd16a76700 1 --2- 172.21.15.59:0/810065400 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fdd000a1510 0x7fdd000a39d0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:53.609 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:53.606+0000 7fdd16a76700 1 --2- 172.21.15.59:0/810065400 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdd101031c0 0x7fdd1019eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:53.609 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:53.606+0000 7fdd16a76700 1 --2- 172.21.15.59:0/810065400 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdd10101620 0x7fdd10197580 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:53.609 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:53.606+0000 7fdd16a76700 1 --2- 172.21.15.59:0/810065400 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd10102820 0x7fdd10197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:53.609 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:53.606+0000 7fdd16a76700 1 -- 172.21.15.59:0/810065400 >> 172.21.15.59:0/810065400 conn(0x7fdd100fcbc0 msgr2=0x7fdd10105c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:45:53.609 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:53.606+0000 7fdd16a76700 1 -- 172.21.15.59:0/810065400 shutdown_connections 2023-12-11T17:45:53.609 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:53.606+0000 7fdd16a76700 1 -- 172.21.15.59:0/810065400 wait complete. 2023-12-11T17:45:53.783 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:45:53 smithi059 ceph-mon[98847]: pgmap v307: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-12-11T17:45:53.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:45:53 smithi115 ceph-mon[102748]: pgmap v307: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-12-11T17:45:53.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:45:53 smithi181 ceph-mon[103627]: pgmap v307: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-12-11T17:45:54.056 INFO:teuthology.run_tasks:Running task cephadm.shell... 2023-12-11T17:45:54.065 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi059.front.sepia.ceph.com 2023-12-11T17:45:54.065 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d0a07eaa-984b-11ee-95a3-87774f69a715 -- bash -c 'ceph fs dump' 2023-12-11T17:45:54.672 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/mon.a/config 2023-12-11T17:45:54.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:45:54 smithi115 ceph-mon[102748]: from='client.15246 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T17:45:54.881 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:45:54 smithi059 ceph-mon[98847]: from='client.15246 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T17:45:54.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:45:54 smithi181 ceph-mon[103627]: from='client.15246 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T17:45:55.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:45:55 smithi115 ceph-mon[102748]: pgmap v308: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-11T17:45:55.839 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:45:55 smithi059 ceph-mon[98847]: pgmap v308: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-11T17:45:55.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:45:55 smithi181 ceph-mon[103627]: pgmap v308: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-11T17:45:56.043 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:56.040+0000 7f586a87d700 1 -- 172.21.15.59:0/3182821086 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5864101b90 msgr2=0x7f5864101fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:56.043 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:56.040+0000 7f586a87d700 1 --2- 172.21.15.59:0/3182821086 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5864101b90 0x7f5864101fb0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f5854009990 tx=0x7f5854009ca0 comp rx=0 tx=0).stop 2023-12-11T17:45:56.043 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:56.040+0000 7f586a87d700 1 -- 172.21.15.59:0/3182821086 shutdown_connections 2023-12-11T17:45:56.043 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:56.040+0000 7f586a87d700 1 --2- 172.21.15.59:0/3182821086 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5864101120 0x7f58641089c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:56.043 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:56.040+0000 7f586a87d700 1 --2- 172.21.15.59:0/3182821086 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5864101b90 0x7f5864101fb0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:56.043 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:56.040+0000 7f586a87d700 1 --2- 172.21.15.59:0/3182821086 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5864100760 0x7f5864100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:56.044 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:56.040+0000 7f586a87d700 1 -- 172.21.15.59:0/3182821086 >> 172.21.15.59:0/3182821086 conn(0x7f58640fb230 msgr2=0x7f58640fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:45:56.044 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:56.040+0000 7f586a87d700 1 -- 172.21.15.59:0/3182821086 shutdown_connections 2023-12-11T17:45:56.044 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:56.040+0000 7f586a87d700 1 -- 172.21.15.59:0/3182821086 wait complete. 2023-12-11T17:45:56.044 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:56.041+0000 7f586a87d700 1 Processor -- start 2023-12-11T17:45:56.044 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:56.041+0000 7f586a87d700 1 -- start start 2023-12-11T17:45:56.045 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:56.041+0000 7f586a87d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5864100760 0x7f5864199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:56.045 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:56.041+0000 7f586a87d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5864101120 0x7f5864199c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:56.046 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:56.041+0000 7f586a87d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5864101b90 0x7f58641a0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:56.046 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:56.041+0000 7f586a87d700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58641a1280 con 0x7f5864100760 2023-12-11T17:45:56.046 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:56.041+0000 7f586a87d700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58641a13c0 con 0x7f5864101120 2023-12-11T17:45:56.046 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:56.041+0000 7f586a87d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58641a1550 con 0x7f5864101b90 2023-12-11T17:45:56.046 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:56.041+0000 7f5863fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5864100760 0x7f5864199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:56.046 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:56.041+0000 7f5863fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5864100760 0x7f5864199740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34296/0 (socket says 172.21.15.59:34296) 2023-12-11T17:45:56.046 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:56.041+0000 7f5863fff700 1 -- 172.21.15.59:0/3431036878 learned_addr learned my addr 172.21.15.59:0/3431036878 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:45:56.046 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:56.041+0000 7f58637fe700 1 --2- 172.21.15.59:0/3431036878 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5864101120 0x7f5864199c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:56.046 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:56.041+0000 7f5868e1a700 1 --2- 172.21.15.59:0/3431036878 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5864101b90 0x7f58641a0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:56.046 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:56.041+0000 7f5863fff700 1 -- 172.21.15.59:0/3431036878 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5864101b90 msgr2=0x7f58641a0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:56.046 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:56.041+0000 7f5863fff700 1 --2- 172.21.15.59:0/3431036878 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5864101b90 0x7f58641a0cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:56.046 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:56.041+0000 7f5863fff700 1 -- 172.21.15.59:0/3431036878 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5864101120 msgr2=0x7f5864199c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:56.046 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:56.041+0000 7f5863fff700 1 --2- 172.21.15.59:0/3431036878 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5864101120 0x7f5864199c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:56.046 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:56.041+0000 7f5863fff700 1 -- 172.21.15.59:0/3431036878 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58540095f0 con 0x7f5864100760 2023-12-11T17:45:56.047 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:56.042+0000 7f5863fff700 1 --2- 172.21.15.59:0/3431036878 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5864100760 0x7f5864199740 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f5854005f50 tx=0x7f585400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:56.047 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:56.042+0000 7f58617fa700 1 -- 172.21.15.59:0/3431036878 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f585401d070 con 0x7f5864100760 2023-12-11T17:45:56.047 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:56.042+0000 7f58617fa700 1 -- 172.21.15.59:0/3431036878 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5854005510 con 0x7f5864100760 2023-12-11T17:45:56.047 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:56.042+0000 7f586a87d700 1 -- 172.21.15.59:0/3431036878 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58641a17d0 con 0x7f5864100760 2023-12-11T17:45:56.047 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:56.042+0000 7f58617fa700 1 -- 172.21.15.59:0/3431036878 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5854004c10 con 0x7f5864100760 2023-12-11T17:45:56.047 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:56.042+0000 7f586a87d700 1 -- 172.21.15.59:0/3431036878 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58641a1c40 con 0x7f5864100760 2023-12-11T17:45:56.047 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:56.043+0000 7f586a87d700 1 -- 172.21.15.59:0/3431036878 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f586404f290 con 0x7f5864100760 2023-12-11T17:45:56.048 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:56.044+0000 7f58617fa700 1 -- 172.21.15.59:0/3431036878 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f585400fb80 con 0x7f5864100760 2023-12-11T17:45:56.048 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:56.045+0000 7f58617fa700 1 --2- 172.21.15.59:0/3431036878 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f58500a1590 0x7f58500a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:56.048 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:56.045+0000 7f58637fe700 1 --2- 172.21.15.59:0/3431036878 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f58500a1590 0x7f58500a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:56.048 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:56.045+0000 7f58617fa700 1 -- 172.21.15.59:0/3431036878 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f58540ce060 con 0x7f5864100760 2023-12-11T17:45:56.048 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:56.045+0000 7f58637fe700 1 --2- 172.21.15.59:0/3431036878 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f58500a1590 0x7f58500a3a50 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f584c005950 tx=0x7f584c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:56.052 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:56.049+0000 7f58617fa700 1 -- 172.21.15.59:0/3431036878 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f58540972f0 con 0x7f5864100760 2023-12-11T17:45:56.296 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:56.293+0000 7f586a87d700 1 -- 172.21.15.59:0/3431036878 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f586410bd30 con 0x7f5864100760 2023-12-11T17:45:56.296 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:56.293+0000 7f58617fa700 1 -- 172.21.15.59:0/3431036878 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 24 v24) v1 ==== 76+0+2947 (secure 0 0 0) 0x7f585409a910 con 0x7f5864100760 2023-12-11T17:45:56.296 INFO:teuthology.orchestra.run.smithi059.stdout:e24 2023-12-11T17:45:56.296 INFO:teuthology.orchestra.run.smithi059.stdout:enable_multiple, ever_enabled_multiple: 1,1 2023-12-11T17:45:56.296 INFO:teuthology.orchestra.run.smithi059.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-12-11T17:45:56.296 INFO:teuthology.orchestra.run.smithi059.stdout:legacy client fscid: 1 2023-12-11T17:45:56.296 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-12-11T17:45:56.296 INFO:teuthology.orchestra.run.smithi059.stdout:Filesystem 'cephfs' (1) 2023-12-11T17:45:56.296 INFO:teuthology.orchestra.run.smithi059.stdout:fs_name cephfs 2023-12-11T17:45:56.296 INFO:teuthology.orchestra.run.smithi059.stdout:epoch 24 2023-12-11T17:45:56.297 INFO:teuthology.orchestra.run.smithi059.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2023-12-11T17:45:56.297 INFO:teuthology.orchestra.run.smithi059.stdout:created 2023-12-11T17:44:20.076660+0000 2023-12-11T17:45:56.297 INFO:teuthology.orchestra.run.smithi059.stdout:modified 2023-12-11T17:44:47.770695+0000 2023-12-11T17:45:56.297 INFO:teuthology.orchestra.run.smithi059.stdout:tableserver 0 2023-12-11T17:45:56.297 INFO:teuthology.orchestra.run.smithi059.stdout:root 0 2023-12-11T17:45:56.297 INFO:teuthology.orchestra.run.smithi059.stdout:session_timeout 300 2023-12-11T17:45:56.297 INFO:teuthology.orchestra.run.smithi059.stdout:session_autoclose 300 2023-12-11T17:45:56.297 INFO:teuthology.orchestra.run.smithi059.stdout:max_file_size 1099511627776 2023-12-11T17:45:56.297 INFO:teuthology.orchestra.run.smithi059.stdout:max_xattr_size 65536 2023-12-11T17:45:56.297 INFO:teuthology.orchestra.run.smithi059.stdout:required_client_features {} 2023-12-11T17:45:56.297 INFO:teuthology.orchestra.run.smithi059.stdout:last_failure 0 2023-12-11T17:45:56.298 INFO:teuthology.orchestra.run.smithi059.stdout:last_failure_osd_epoch 0 2023-12-11T17:45:56.298 INFO:teuthology.orchestra.run.smithi059.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-12-11T17:45:56.298 INFO:teuthology.orchestra.run.smithi059.stdout:max_mds 3 2023-12-11T17:45:56.298 INFO:teuthology.orchestra.run.smithi059.stdout:in 0,1,2 2023-12-11T17:45:56.298 INFO:teuthology.orchestra.run.smithi059.stdout:up {0=24521,1=24484,2=24482} 2023-12-11T17:45:56.298 INFO:teuthology.orchestra.run.smithi059.stdout:failed 2023-12-11T17:45:56.298 INFO:teuthology.orchestra.run.smithi059.stdout:damaged 2023-12-11T17:45:56.298 INFO:teuthology.orchestra.run.smithi059.stdout:stopped 2023-12-11T17:45:56.298 INFO:teuthology.orchestra.run.smithi059.stdout:data_pools [3,4] 2023-12-11T17:45:56.298 INFO:teuthology.orchestra.run.smithi059.stdout:metadata_pool 2 2023-12-11T17:45:56.298 INFO:teuthology.orchestra.run.smithi059.stdout:inline_data disabled 2023-12-11T17:45:56.298 INFO:teuthology.orchestra.run.smithi059.stdout:balancer 2023-12-11T17:45:56.298 INFO:teuthology.orchestra.run.smithi059.stdout:bal_rank_mask -1 2023-12-11T17:45:56.298 INFO:teuthology.orchestra.run.smithi059.stdout:standby_count_wanted 0 2023-12-11T17:45:56.299 INFO:teuthology.orchestra.run.smithi059.stdout:[mds.b{0:24521} state up:active seq 3 addr [v2:172.21.15.115:6838/4122922866,v1:172.21.15.115:6839/4122922866] compat {c=[1],r=[1],i=[fff]}] 2023-12-11T17:45:56.299 INFO:teuthology.orchestra.run.smithi059.stdout:[mds.c{0:24466} state up:standby-replay seq 1 addr [v2:172.21.15.181:6836/3580309407,v1:172.21.15.181:6837/3580309407] compat {c=[1],r=[1],i=[fff]}] 2023-12-11T17:45:56.299 INFO:teuthology.orchestra.run.smithi059.stdout:[mds.i{1:24484} state up:active seq 11 addr [v2:172.21.15.181:6838/2214384806,v1:172.21.15.181:6839/2214384806] compat {c=[1],r=[1],i=[fff]}] 2023-12-11T17:45:56.299 INFO:teuthology.orchestra.run.smithi059.stdout:[mds.f{1:24457} state up:standby-replay seq 1 addr [v2:172.21.15.181:6834/4221183620,v1:172.21.15.181:6835/4221183620] compat {c=[1],r=[1],i=[fff]}] 2023-12-11T17:45:56.299 INFO:teuthology.orchestra.run.smithi059.stdout:[mds.h{2:24482} state up:active seq 12 addr [v2:172.21.15.115:6836/1209365460,v1:172.21.15.115:6837/1209365460] compat {c=[1],r=[1],i=[fff]}] 2023-12-11T17:45:56.299 INFO:teuthology.orchestra.run.smithi059.stdout:[mds.e{2:24449} state up:standby-replay seq 1 addr [v2:172.21.15.115:6834/1874159657,v1:172.21.15.115:6835/1874159657] compat {c=[1],r=[1],i=[fff]}] 2023-12-11T17:45:56.299 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-12-11T17:45:56.299 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-12-11T17:45:56.299 INFO:teuthology.orchestra.run.smithi059.stdout:Standby daemons: 2023-12-11T17:45:56.299 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-12-11T17:45:56.299 INFO:teuthology.orchestra.run.smithi059.stdout:[mds.j{-1:14526} state up:standby seq 1 addr [v2:172.21.15.59:6834/2184151593,v1:172.21.15.59:6835/2184151593] compat {c=[1],r=[1],i=[fff]}] 2023-12-11T17:45:56.299 INFO:teuthology.orchestra.run.smithi059.stdout:[mds.d{-1:14532} state up:standby seq 1 addr [v2:172.21.15.59:6836/2817614886,v1:172.21.15.59:6837/2817614886] compat {c=[1],r=[1],i=[fff]}] 2023-12-11T17:45:56.299 INFO:teuthology.orchestra.run.smithi059.stdout:[mds.g{-1:14586} state up:standby seq 1 addr [v2:172.21.15.59:6838/229873490,v1:172.21.15.59:6839/229873490] compat {c=[1],r=[1],i=[fff]}] 2023-12-11T17:45:56.299 INFO:teuthology.orchestra.run.smithi059.stdout:[mds.a{-1:14598} state up:standby seq 1 addr [v2:172.21.15.59:6840/967998755,v1:172.21.15.59:6841/967998755] compat {c=[1],r=[1],i=[fff]}] 2023-12-11T17:45:56.299 INFO:teuthology.orchestra.run.smithi059.stdout:[mds.l{-1:24418} state up:standby seq 1 addr [v2:172.21.15.181:6832/395684042,v1:172.21.15.181:6833/395684042] compat {c=[1],r=[1],i=[fff]}] 2023-12-11T17:45:56.299 INFO:teuthology.orchestra.run.smithi059.stdout:[mds.k{-1:24422} state up:standby seq 1 addr [v2:172.21.15.115:6832/3214606191,v1:172.21.15.115:6833/3214606191] compat {c=[1],r=[1],i=[fff]}] 2023-12-11T17:45:56.300 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:56.295+0000 7f586a87d700 1 -- 172.21.15.59:0/3431036878 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f58500a1590 msgr2=0x7f58500a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:56.300 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:56.295+0000 7f586a87d700 1 --2- 172.21.15.59:0/3431036878 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f58500a1590 0x7f58500a3a50 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f584c005950 tx=0x7f584c00b410 comp rx=0 tx=0).stop 2023-12-11T17:45:56.300 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:56.295+0000 7f586a87d700 1 -- 172.21.15.59:0/3431036878 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5864100760 msgr2=0x7f5864199740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:56.300 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:56.295+0000 7f586a87d700 1 --2- 172.21.15.59:0/3431036878 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5864100760 0x7f5864199740 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f5854005f50 tx=0x7f585400f690 comp rx=0 tx=0).stop 2023-12-11T17:45:56.300 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:56.295+0000 7f586a87d700 1 -- 172.21.15.59:0/3431036878 shutdown_connections 2023-12-11T17:45:56.300 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:56.295+0000 7f586a87d700 1 --2- 172.21.15.59:0/3431036878 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f58500a1590 0x7f58500a3a50 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:56.301 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:56.295+0000 7f586a87d700 1 --2- 172.21.15.59:0/3431036878 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5864101b90 0x7f58641a0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:56.301 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:56.295+0000 7f586a87d700 1 --2- 172.21.15.59:0/3431036878 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5864100760 0x7f5864199740 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:56.301 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:56.295+0000 7f586a87d700 1 --2- 172.21.15.59:0/3431036878 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5864101120 0x7f5864199c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:56.301 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:56.295+0000 7f586a87d700 1 -- 172.21.15.59:0/3431036878 >> 172.21.15.59:0/3431036878 conn(0x7f58640fb230 msgr2=0x7f58640ff680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:45:56.301 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:56.295+0000 7f586a87d700 1 -- 172.21.15.59:0/3431036878 shutdown_connections 2023-12-11T17:45:56.301 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:56.295+0000 7f586a87d700 1 -- 172.21.15.59:0/3431036878 wait complete. 2023-12-11T17:45:56.301 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 24 2023-12-11T17:45:56.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:45:56 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/3431036878' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-12-11T17:45:56.750 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d0a07eaa-984b-11ee-95a3-87774f69a715 -- bash -c 'ceph osd dump' 2023-12-11T17:45:56.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:45:56 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/3431036878' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-12-11T17:45:56.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:45:56 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/3431036878' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-12-11T17:45:57.356 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/mon.a/config 2023-12-11T17:45:57.781 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:45:57 smithi059 ceph-mon[98847]: pgmap v309: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-12-11T17:45:57.782 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:45:57 smithi115 ceph-mon[102748]: pgmap v309: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-12-11T17:45:57.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:45:57 smithi181 ceph-mon[103627]: pgmap v309: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-12-11T17:45:58.717 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:58.711+0000 7ffb118ae700 1 -- 172.21.15.59:0/856387945 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffb0c1031a0 msgr2=0x7ffb0c109a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:58.717 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:58.711+0000 7ffb118ae700 1 --2- 172.21.15.59:0/856387945 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffb0c1031a0 0x7ffb0c109a50 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7ffb0400b0d0 tx=0x7ffb0400b3e0 comp rx=0 tx=0).stop 2023-12-11T17:45:58.717 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:58.711+0000 7ffb118ae700 1 -- 172.21.15.59:0/856387945 shutdown_connections 2023-12-11T17:45:58.718 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:58.711+0000 7ffb118ae700 1 --2- 172.21.15.59:0/856387945 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffb0c1031a0 0x7ffb0c109a50 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:58.718 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:58.711+0000 7ffb118ae700 1 --2- 172.21.15.59:0/856387945 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffb0c101600 0x7ffb0c101a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:58.718 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:58.711+0000 7ffb118ae700 1 --2- 172.21.15.59:0/856387945 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb0c102800 0x7ffb0c102c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:58.718 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:58.711+0000 7ffb118ae700 1 -- 172.21.15.59:0/856387945 >> 172.21.15.59:0/856387945 conn(0x7ffb0c0fcbe0 msgr2=0x7ffb0c0ff000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:45:58.718 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:58.712+0000 7ffb118ae700 1 -- 172.21.15.59:0/856387945 shutdown_connections 2023-12-11T17:45:58.718 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:58.712+0000 7ffb118ae700 1 -- 172.21.15.59:0/856387945 wait complete. 2023-12-11T17:45:58.718 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:58.712+0000 7ffb118ae700 1 Processor -- start 2023-12-11T17:45:58.718 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:58.712+0000 7ffb118ae700 1 -- start start 2023-12-11T17:45:58.718 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:58.713+0000 7ffb118ae700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffb0c101600 0x7ffb0c0752d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:58.718 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:58.713+0000 7ffb118ae700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb0c102800 0x7ffb0c0739c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:58.718 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:58.713+0000 7ffb118ae700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffb0c1031a0 0x7ffb0c073f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:58.718 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:58.713+0000 7ffb118ae700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb0c0745e0 con 0x7ffb0c1031a0 2023-12-11T17:45:58.719 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:58.713+0000 7ffb118ae700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb0c074750 con 0x7ffb0c101600 2023-12-11T17:45:58.719 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:58.713+0000 7ffb118ae700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb0c1a1650 con 0x7ffb0c102800 2023-12-11T17:45:58.719 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:58.713+0000 7ffb0b7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffb0c1031a0 0x7ffb0c073f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:58.719 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:58.713+0000 7ffb0affd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffb0c101600 0x7ffb0c0752d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:58.719 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:58.713+0000 7ffb0b7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffb0c1031a0 0x7ffb0c073f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34314/0 (socket says 172.21.15.59:34314) 2023-12-11T17:45:58.719 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:58.713+0000 7ffb0affd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffb0c101600 0x7ffb0c0752d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.59:59038/0 (socket says 172.21.15.59:59038) 2023-12-11T17:45:58.719 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:58.713+0000 7ffb0b7fe700 1 -- 172.21.15.59:0/1180287458 learned_addr learned my addr 172.21.15.59:0/1180287458 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:45:58.719 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:58.713+0000 7ffb0a7fc700 1 --2- 172.21.15.59:0/1180287458 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb0c102800 0x7ffb0c0739c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:58.719 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:58.713+0000 7ffb0b7fe700 1 -- 172.21.15.59:0/1180287458 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb0c102800 msgr2=0x7ffb0c0739c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:58.719 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:58.713+0000 7ffb0b7fe700 1 --2- 172.21.15.59:0/1180287458 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb0c102800 0x7ffb0c0739c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:58.719 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:58.713+0000 7ffb0b7fe700 1 -- 172.21.15.59:0/1180287458 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffb0c101600 msgr2=0x7ffb0c0752d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:58.719 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:58.713+0000 7ffb0b7fe700 1 --2- 172.21.15.59:0/1180287458 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffb0c101600 0x7ffb0c0752d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:58.719 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:58.713+0000 7ffb0b7fe700 1 -- 172.21.15.59:0/1180287458 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb04009d00 con 0x7ffb0c1031a0 2023-12-11T17:45:58.720 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:58.713+0000 7ffb0b7fe700 1 --2- 172.21.15.59:0/1180287458 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffb0c1031a0 0x7ffb0c073f00 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7ffb04007bd0 tx=0x7ffb04009440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:58.720 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:58.713+0000 7ffb108ac700 1 -- 172.21.15.59:0/1180287458 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb0400e040 con 0x7ffb0c1031a0 2023-12-11T17:45:58.720 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:58.713+0000 7ffb118ae700 1 -- 172.21.15.59:0/1180287458 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb0c1a1850 con 0x7ffb0c1031a0 2023-12-11T17:45:58.720 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:58.713+0000 7ffb108ac700 1 -- 172.21.15.59:0/1180287458 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffb0401dd50 con 0x7ffb0c1031a0 2023-12-11T17:45:58.720 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:58.714+0000 7ffb108ac700 1 -- 172.21.15.59:0/1180287458 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb040123f0 con 0x7ffb0c1031a0 2023-12-11T17:45:58.720 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:58.714+0000 7ffb118ae700 1 -- 172.21.15.59:0/1180287458 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb0c1a1cc0 con 0x7ffb0c1031a0 2023-12-11T17:45:58.721 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:58.715+0000 7ffb118ae700 1 -- 172.21.15.59:0/1180287458 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb0c1089c0 con 0x7ffb0c1031a0 2023-12-11T17:45:58.721 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:58.716+0000 7ffb108ac700 1 -- 172.21.15.59:0/1180287458 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7ffb0401d890 con 0x7ffb0c1031a0 2023-12-11T17:45:58.721 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:58.716+0000 7ffb108ac700 1 --2- 172.21.15.59:0/1180287458 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7ffaf40a1510 0x7ffaf40a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:58.721 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:58.716+0000 7ffb108ac700 1 -- 172.21.15.59:0/1180287458 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8999+0+0 (secure 0 0 0) 0x7ffb040ceeb0 con 0x7ffb0c1031a0 2023-12-11T17:45:58.721 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:58.716+0000 7ffb0affd700 1 --2- 172.21.15.59:0/1180287458 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7ffaf40a1510 0x7ffaf40a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:58.721 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:58.717+0000 7ffb0affd700 1 --2- 172.21.15.59:0/1180287458 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7ffaf40a1510 0x7ffaf40a39d0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c102660 tx=0x7ffafc00a3b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:58.724 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:58.721+0000 7ffb108ac700 1 -- 172.21.15.59:0/1180287458 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7ffb04098140 con 0x7ffb0c1031a0 2023-12-11T17:45:58.926 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:58.923+0000 7ffb118ae700 1 -- 172.21.15.59:0/1180287458 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump"} v 0) v1 -- 0x7ffb0c1083a0 con 0x7ffb0c1031a0 2023-12-11T17:45:58.927 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:58.924+0000 7ffb108ac700 1 -- 172.21.15.59:0/1180287458 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v88) v1 ==== 56+0+5580 (secure 0 0 0) 0x7ffb0409b760 con 0x7ffb0c1031a0 2023-12-11T17:45:58.928 INFO:teuthology.orchestra.run.smithi059.stdout:epoch 88 2023-12-11T17:45:58.928 INFO:teuthology.orchestra.run.smithi059.stdout:fsid d0a07eaa-984b-11ee-95a3-87774f69a715 2023-12-11T17:45:58.928 INFO:teuthology.orchestra.run.smithi059.stdout:created 2023-12-11T17:37:14.526539+0000 2023-12-11T17:45:58.928 INFO:teuthology.orchestra.run.smithi059.stdout:modified 2023-12-11T17:45:15.870614+0000 2023-12-11T17:45:58.928 INFO:teuthology.orchestra.run.smithi059.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2023-12-11T17:45:58.928 INFO:teuthology.orchestra.run.smithi059.stdout:crush_version 27 2023-12-11T17:45:58.928 INFO:teuthology.orchestra.run.smithi059.stdout:full_ratio 0.95 2023-12-11T17:45:58.928 INFO:teuthology.orchestra.run.smithi059.stdout:backfillfull_ratio 0.9 2023-12-11T17:45:58.928 INFO:teuthology.orchestra.run.smithi059.stdout:nearfull_ratio 0.85 2023-12-11T17:45:58.928 INFO:teuthology.orchestra.run.smithi059.stdout:require_min_compat_client luminous 2023-12-11T17:45:58.928 INFO:teuthology.orchestra.run.smithi059.stdout:min_compat_client luminous 2023-12-11T17:45:58.928 INFO:teuthology.orchestra.run.smithi059.stdout:require_osd_release reef 2023-12-11T17:45:58.929 INFO:teuthology.orchestra.run.smithi059.stdout:stretch_mode_enabled false 2023-12-11T17:45:58.929 INFO:teuthology.orchestra.run.smithi059.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2023-12-11T17:45:58.929 INFO:teuthology.orchestra.run.smithi059.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 81 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2023-12-11T17:45:58.929 INFO:teuthology.orchestra.run.smithi059.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 82 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2023-12-11T17:45:58.929 INFO:teuthology.orchestra.run.smithi059.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 86 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2023-12-11T17:45:58.929 INFO:teuthology.orchestra.run.smithi059.stdout:max_osd 12 2023-12-11T17:45:58.929 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0 up in weight 1 up_from 8 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.59:6802/1040759398,v1:172.21.15.59:6803/1040759398] [v2:172.21.15.59:6804/1040759398,v1:172.21.15.59:6805/1040759398] exists,up 3c0b3218-4b2b-40f8-8bef-ed8cd5633607 2023-12-11T17:45:58.929 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1 up in weight 1 up_from 13 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.115:6800/2776354245,v1:172.21.15.115:6801/2776354245] [v2:172.21.15.115:6802/2776354245,v1:172.21.15.115:6803/2776354245] exists,up 2ead7578-765b-4f5a-a72c-863731ed69e6 2023-12-11T17:45:58.929 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2 up in weight 1 up_from 18 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.181:6800/4287675481,v1:172.21.15.181:6801/4287675481] [v2:172.21.15.181:6802/4287675481,v1:172.21.15.181:6803/4287675481] exists,up 8cb1d7ec-53cc-4672-b0b6-70d73797c44f 2023-12-11T17:45:58.929 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3 up in weight 1 up_from 25 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.59:6810/219909995,v1:172.21.15.59:6811/219909995] [v2:172.21.15.59:6812/219909995,v1:172.21.15.59:6813/219909995] exists,up 5d64d9af-831b-4d0a-8ace-a452169351d6 2023-12-11T17:45:58.929 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4 up in weight 1 up_from 32 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.115:6808/1021473826,v1:172.21.15.115:6809/1021473826] [v2:172.21.15.115:6810/1021473826,v1:172.21.15.115:6811/1021473826] exists,up ef8fad28-1db5-4cba-b5f2-10e54e8e1549 2023-12-11T17:45:58.929 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5 up in weight 1 up_from 37 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.181:6808/2405172521,v1:172.21.15.181:6809/2405172521] [v2:172.21.15.181:6810/2405172521,v1:172.21.15.181:6811/2405172521] exists,up 4233a512-9e8f-4db8-aed2-6c3af706e589 2023-12-11T17:45:58.929 INFO:teuthology.orchestra.run.smithi059.stdout:osd.6 up in weight 1 up_from 45 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.59:6818/815374853,v1:172.21.15.59:6819/815374853] [v2:172.21.15.59:6820/815374853,v1:172.21.15.59:6821/815374853] exists,up 8faab332-016e-4c77-8370-8bef95fc6f34 2023-12-11T17:45:58.930 INFO:teuthology.orchestra.run.smithi059.stdout:osd.7 up in weight 1 up_from 50 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.115:6816/1314971870,v1:172.21.15.115:6817/1314971870] [v2:172.21.15.115:6818/1314971870,v1:172.21.15.115:6819/1314971870] exists,up f20f470a-0d40-4ed3-8e0e-fd58675cf143 2023-12-11T17:45:58.930 INFO:teuthology.orchestra.run.smithi059.stdout:osd.8 up in weight 1 up_from 57 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.181:6816/3954509873,v1:172.21.15.181:6817/3954509873] [v2:172.21.15.181:6818/3954509873,v1:172.21.15.181:6819/3954509873] exists,up eb657d32-1a95-4fd7-a293-d0a970837fc3 2023-12-11T17:45:58.930 INFO:teuthology.orchestra.run.smithi059.stdout:osd.9 up in weight 1 up_from 65 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.59:6826/1163095677,v1:172.21.15.59:6827/1163095677] [v2:172.21.15.59:6828/1163095677,v1:172.21.15.59:6829/1163095677] exists,up 8f7717d4-b6f2-4ec1-9b58-62feb0c9a146 2023-12-11T17:45:58.930 INFO:teuthology.orchestra.run.smithi059.stdout:osd.10 up in weight 1 up_from 70 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.115:6824/2199610269,v1:172.21.15.115:6825/2199610269] [v2:172.21.15.115:6826/2199610269,v1:172.21.15.115:6827/2199610269] exists,up b75c179a-875d-43bc-b4c1-e8eb80fb64fa 2023-12-11T17:45:58.930 INFO:teuthology.orchestra.run.smithi059.stdout:osd.11 up in weight 1 up_from 77 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.181:6824/226506989,v1:172.21.15.181:6825/226506989] [v2:172.21.15.181:6826/226506989,v1:172.21.15.181:6827/226506989] exists,up 5e141dfe-8a00-4bbc-8204-2dc9be24e18d 2023-12-11T17:45:58.930 INFO:teuthology.orchestra.run.smithi059.stdout:pg_upmap_items 2.b [2,4] 2023-12-11T17:45:58.931 INFO:teuthology.orchestra.run.smithi059.stdout:pg_upmap_items 2.1b [2,1] 2023-12-11T17:45:58.931 INFO:teuthology.orchestra.run.smithi059.stdout:pg_upmap_items 2.1c [8,11] 2023-12-11T17:45:58.931 INFO:teuthology.orchestra.run.smithi059.stdout:pg_upmap_items 2.1f [7,11] 2023-12-11T17:45:58.931 INFO:teuthology.orchestra.run.smithi059.stdout:pg_upmap_items 3.2 [3,9] 2023-12-11T17:45:58.931 INFO:teuthology.orchestra.run.smithi059.stdout:pg_upmap_items 3.17 [4,7] 2023-12-11T17:45:58.931 INFO:teuthology.orchestra.run.smithi059.stdout:pg_upmap_items 3.28 [5,6,3,8] 2023-12-11T17:45:58.931 INFO:teuthology.orchestra.run.smithi059.stdout:pg_upmap_items 3.36 [4,9] 2023-12-11T17:45:58.931 INFO:teuthology.orchestra.run.smithi059.stdout:pg_upmap_items 3.3a [4,8] 2023-12-11T17:45:58.931 INFO:teuthology.orchestra.run.smithi059.stdout:blocklist 172.21.15.59:6800/1259916482 expires 2023-12-12T17:38:14.037922+0000 2023-12-11T17:45:58.931 INFO:teuthology.orchestra.run.smithi059.stdout:blocklist 172.21.15.59:0/17457774 expires 2023-12-12T17:38:14.037922+0000 2023-12-11T17:45:58.931 INFO:teuthology.orchestra.run.smithi059.stdout:blocklist 172.21.15.59:6801/1672530462 expires 2023-12-12T17:37:44.773925+0000 2023-12-11T17:45:58.931 INFO:teuthology.orchestra.run.smithi059.stdout:blocklist 172.21.15.59:0/1053944547 expires 2023-12-12T17:38:14.037922+0000 2023-12-11T17:45:58.931 INFO:teuthology.orchestra.run.smithi059.stdout:blocklist 172.21.15.59:6801/1259916482 expires 2023-12-12T17:38:14.037922+0000 2023-12-11T17:45:58.932 INFO:teuthology.orchestra.run.smithi059.stdout:blocklist 172.21.15.59:6800/1672530462 expires 2023-12-12T17:37:44.773925+0000 2023-12-11T17:45:58.932 INFO:teuthology.orchestra.run.smithi059.stdout:blocklist 172.21.15.59:0/3570767650 expires 2023-12-12T17:37:44.773925+0000 2023-12-11T17:45:58.932 INFO:teuthology.orchestra.run.smithi059.stdout:blocklist 172.21.15.59:0/3853462899 expires 2023-12-12T17:37:44.773925+0000 2023-12-11T17:45:58.932 INFO:teuthology.orchestra.run.smithi059.stdout:blocklist 172.21.15.59:0/2859415380 expires 2023-12-12T17:38:14.037922+0000 2023-12-11T17:45:58.932 INFO:teuthology.orchestra.run.smithi059.stdout:blocklist 172.21.15.59:0/2192742819 expires 2023-12-12T17:37:44.773925+0000 2023-12-11T17:45:58.933 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:58.926+0000 7ffb118ae700 1 -- 172.21.15.59:0/1180287458 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7ffaf40a1510 msgr2=0x7ffaf40a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:58.933 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:58.926+0000 7ffb118ae700 1 --2- 172.21.15.59:0/1180287458 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7ffaf40a1510 0x7ffaf40a39d0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c102660 tx=0x7ffafc00a3b0 comp rx=0 tx=0).stop 2023-12-11T17:45:58.933 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:58.926+0000 7ffb118ae700 1 -- 172.21.15.59:0/1180287458 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffb0c1031a0 msgr2=0x7ffb0c073f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:58.933 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:58.926+0000 7ffb118ae700 1 --2- 172.21.15.59:0/1180287458 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffb0c1031a0 0x7ffb0c073f00 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7ffb04007bd0 tx=0x7ffb04009440 comp rx=0 tx=0).stop 2023-12-11T17:45:58.933 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:58.926+0000 7ffb118ae700 1 -- 172.21.15.59:0/1180287458 shutdown_connections 2023-12-11T17:45:58.933 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:58.926+0000 7ffb118ae700 1 --2- 172.21.15.59:0/1180287458 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7ffaf40a1510 0x7ffaf40a39d0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:58.933 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:58.926+0000 7ffb118ae700 1 --2- 172.21.15.59:0/1180287458 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffb0c1031a0 0x7ffb0c073f00 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:58.933 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:58.926+0000 7ffb118ae700 1 --2- 172.21.15.59:0/1180287458 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffb0c101600 0x7ffb0c0752d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:58.933 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:58.926+0000 7ffb118ae700 1 --2- 172.21.15.59:0/1180287458 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb0c102800 0x7ffb0c0739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:58.933 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:58.926+0000 7ffb118ae700 1 -- 172.21.15.59:0/1180287458 >> 172.21.15.59:0/1180287458 conn(0x7ffb0c0fcbe0 msgr2=0x7ffb0c105b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:45:58.933 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:58.927+0000 7ffb118ae700 1 -- 172.21.15.59:0/1180287458 shutdown_connections 2023-12-11T17:45:58.933 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:58.927+0000 7ffb118ae700 1 -- 172.21.15.59:0/1180287458 wait complete. 2023-12-11T17:45:59.328 INFO:teuthology.run_tasks:Running task fs.ready... 2023-12-11T17:45:59.340 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-11T17:45:59.472 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:59.466+0000 7f22a7ab1700 1 -- 172.21.15.59:0/4152242090 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f22a0101070 msgr2=0x7f22a01014f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:59.472 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:59.466+0000 7f22a7ab1700 1 --2- 172.21.15.59:0/4152242090 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f22a0101070 0x7f22a01014f0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f229800b0d0 tx=0x7f229800b3e0 comp rx=0 tx=0).stop 2023-12-11T17:45:59.472 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:59.466+0000 7f22a7ab1700 1 -- 172.21.15.59:0/4152242090 shutdown_connections 2023-12-11T17:45:59.472 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:59.466+0000 7f22a7ab1700 1 --2- 172.21.15.59:0/4152242090 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f22a0101070 0x7f22a01014f0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:59.472 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:59.466+0000 7f22a7ab1700 1 --2- 172.21.15.59:0/4152242090 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f22a00ffd50 0x7f22a0100170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:59.472 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:59.466+0000 7f22a7ab1700 1 --2- 172.21.15.59:0/4152242090 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f22a01006b0 0x7f22a0100b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:59.472 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:59.466+0000 7f22a7ab1700 1 -- 172.21.15.59:0/4152242090 >> 172.21.15.59:0/4152242090 conn(0x7f22a00fb8f0 msgr2=0x7f22a00fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:45:59.472 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:59.466+0000 7f22a7ab1700 1 -- 172.21.15.59:0/4152242090 shutdown_connections 2023-12-11T17:45:59.472 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:59.466+0000 7f22a7ab1700 1 -- 172.21.15.59:0/4152242090 wait complete. 2023-12-11T17:45:59.473 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:59.467+0000 7f22a7ab1700 1 Processor -- start 2023-12-11T17:45:59.473 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:59.467+0000 7f22a7ab1700 1 -- start start 2023-12-11T17:45:59.473 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:59.467+0000 7f22a7ab1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f22a00ffd50 0x7f22a01996b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:59.473 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:59.467+0000 7f22a7ab1700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f22a01006b0 0x7f22a0199bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:59.473 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:59.467+0000 7f22a7ab1700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f22a0101070 0x7f22a01a0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:59.473 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:59.467+0000 7f22a7ab1700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22a01a11e0 con 0x7f22a01006b0 2023-12-11T17:45:59.473 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:59.467+0000 7f22a7ab1700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22a01a1320 con 0x7f22a0101070 2023-12-11T17:45:59.473 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:59.467+0000 7f22a7ab1700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22a01a14b0 con 0x7f22a00ffd50 2023-12-11T17:45:59.473 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:59.467+0000 7f22a504c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f22a01006b0 0x7f22a0199bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:59.473 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:59.467+0000 7f22a504c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f22a01006b0 0x7f22a0199bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34326/0 (socket says 172.21.15.59:34326) 2023-12-11T17:45:59.473 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:59.467+0000 7f22a504c700 1 -- 172.21.15.59:0/2628105238 learned_addr learned my addr 172.21.15.59:0/2628105238 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:45:59.473 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:59.467+0000 7f22a584d700 1 --2- 172.21.15.59:0/2628105238 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f22a00ffd50 0x7f22a01996b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:59.473 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:59.467+0000 7f22a604e700 1 --2- 172.21.15.59:0/2628105238 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f22a0101070 0x7f22a01a0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:59.473 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:59.468+0000 7f22a504c700 1 -- 172.21.15.59:0/2628105238 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f22a00ffd50 msgr2=0x7f22a01996b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:59.473 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:59.468+0000 7f22a504c700 1 --2- 172.21.15.59:0/2628105238 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f22a00ffd50 0x7f22a01996b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:59.474 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:59.468+0000 7f22a504c700 1 -- 172.21.15.59:0/2628105238 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f22a0101070 msgr2=0x7f22a01a0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:59.474 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:59.468+0000 7f22a504c700 1 --2- 172.21.15.59:0/2628105238 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f22a0101070 0x7f22a01a0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:59.474 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:59.468+0000 7f22a504c700 1 -- 172.21.15.59:0/2628105238 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2298009d00 con 0x7f22a01006b0 2023-12-11T17:45:59.474 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:59.468+0000 7f22a504c700 1 --2- 172.21.15.59:0/2628105238 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f22a01006b0 0x7f22a0199bf0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f229000d950 tx=0x7f229000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:59.474 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:59.468+0000 7f2296ffd700 1 -- 172.21.15.59:0/2628105238 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f229000f890 con 0x7f22a01006b0 2023-12-11T17:45:59.474 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:59.468+0000 7f22a7ab1700 1 -- 172.21.15.59:0/2628105238 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22a01a1790 con 0x7f22a01006b0 2023-12-11T17:45:59.474 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:59.468+0000 7f2296ffd700 1 -- 172.21.15.59:0/2628105238 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2290010460 con 0x7f22a01006b0 2023-12-11T17:45:59.474 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:59.468+0000 7f2296ffd700 1 -- 172.21.15.59:0/2628105238 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f229000e610 con 0x7f22a01006b0 2023-12-11T17:45:59.474 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:59.468+0000 7f22a7ab1700 1 -- 172.21.15.59:0/2628105238 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22a01a1ca0 con 0x7f22a01006b0 2023-12-11T17:45:59.475 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:59.469+0000 7f22a7ab1700 1 -- 172.21.15.59:0/2628105238 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22a010ad10 con 0x7f22a01006b0 2023-12-11T17:45:59.475 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:59.470+0000 7f2296ffd700 1 -- 172.21.15.59:0/2628105238 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f22900108d0 con 0x7f22a01006b0 2023-12-11T17:45:59.475 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:59.471+0000 7f2296ffd700 1 --2- 172.21.15.59:0/2628105238 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f228c0a1510 0x7f228c0a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:59.475 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:59.471+0000 7f22a584d700 1 --2- 172.21.15.59:0/2628105238 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f228c0a1510 0x7f228c0a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:59.475 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:59.471+0000 7f2296ffd700 1 -- 172.21.15.59:0/2628105238 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f22900ccfc0 con 0x7f22a01006b0 2023-12-11T17:45:59.475 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:59.471+0000 7f22a584d700 1 --2- 172.21.15.59:0/2628105238 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f228c0a1510 0x7f228c0a39d0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f229c005950 tx=0x7f229c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:59.477 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:59.474+0000 7f2296ffd700 1 -- 172.21.15.59:0/2628105238 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f2290096160 con 0x7f22a01006b0 2023-12-11T17:45:59.683 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:59.680+0000 7f22a7ab1700 1 -- 172.21.15.59:0/2628105238 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f22a010ee70 con 0x7f22a01006b0 2023-12-11T17:45:59.704 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:59.681+0000 7f2296ffd700 1 -- 172.21.15.59:0/2628105238 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+24459 (secure 0 0 0) 0x7f2290016090 con 0x7f22a01006b0 2023-12-11T17:45:59.705 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-12-11T17:45:59.705 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":88,"fsid":"d0a07eaa-984b-11ee-95a3-87774f69a715","created":"2023-12-11T17:37:14.526539+0000","modified":"2023-12-11T17:45:15.870614+0000","last_up_change":"2023-12-11T17:43:38.535503+0000","last_in_change":"2023-12-11T17:43:21.014888+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-11T17:40:26.596029+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-11T17:44:18.586290+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-11T17:44:19.432952+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-11T17:44:23.097308+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3c0b3218-4b2b-40f8-8bef-ed8cd5633607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":1040759398},{"type":"v1","addr":"172.21.15.59:6803","nonce":1040759398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":1040759398},{"type":"v1","addr":"172.21.15.59:6805","nonce":1040759398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":1040759398},{"type":"v1","addr":"172.21.15.59:6809","nonce":1040759398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":1040759398},{"type":"v1","addr":"172.21.15.59:6807","nonce":1040759398}]},"public_addr":"172.21.15.59:6803/1040759398","cluster_addr":"172.21.15.59:6805/1040759398","heartbeat_back_addr":"172.21.15.59:6809/1040759398","heartbeat_front_addr":"172.21.15.59:6807/1040759398","state":["exists","up"]},{"osd":1,"uuid":"2ead7578-765b-4f5a-a72c-863731ed69e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":2776354245},{"type":"v1","addr":"172.21.15.115:6801","nonce":2776354245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":2776354245},{"type":"v1","addr":"172.21.15.115:6803","nonce":2776354245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":2776354245},{"type":"v1","addr":"172.21.15.115:6807","nonce":2776354245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":2776354245},{"type":"v1","addr":"172.21.15.115:6805","nonce":2776354245}]},"public_addr":"172.21.15.115:6801/2776354245","cluster_addr":"172.21.15.115:6803/2776354245","heartbeat_back_addr":"172.21.15.115:6807/2776354245","heartbeat_front_addr":"172.21.15.115:6805/2776354245","state":["exists","up"]},{"osd":2,"uuid":"8cb1d7ec-53cc-4672-b0b6-70d73797c44f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4287675481},{"type":"v1","addr":"172.21.15.181:6801","nonce":4287675481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4287675481},{"type":"v1","addr":"172.21.15.181:6803","nonce":4287675481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4287675481},{"type":"v1","addr":"172.21.15.181:6807","nonce":4287675481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4287675481},{"type":"v1","addr":"172.21.15.181:6805","nonce":4287675481}]},"public_addr":"172.21.15.181:6801/4287675481","cluster_addr":"172.21.15.181:6803/4287675481","heartbeat_back_addr":"172.21.15.181:6807/4287675481","heartbeat_front_addr":"172.21.15.181:6805/4287675481","state":["exists","up"]},{"osd":3,"uuid":"5d64d9af-831b-4d0a-8ace-a452169351d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":219909995},{"type":"v1","addr":"172.21.15.59:6811","nonce":219909995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":219909995},{"type":"v1","addr":"172.21.15.59:6813","nonce":219909995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":219909995},{"type":"v1","addr":"172.21.15.59:6817","nonce":219909995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":219909995},{"type":"v1","addr":"172.21.15.59:6815","nonce":219909995}]},"public_addr":"172.21.15.59:6811/219909995","cluster_addr":"172.21.15.59:6813/219909995","heartbeat_back_addr":"172.21.15.59:6817/219909995","heartbeat_front_addr":"172.21.15.59:6815/219909995","state":["exists","up"]},{"osd":4,"uuid":"ef8fad28-1db5-4cba-b5f2-10e54e8e1549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":1021473826},{"type":"v1","addr":"172.21.15.115:6809","nonce":1021473826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":1021473826},{"type":"v1","addr":"172.21.15.115:6811","nonce":1021473826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":1021473826},{"type":"v1","addr":"172.21.15.115:6815","nonce":1021473826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":1021473826},{"type":"v1","addr":"172.21.15.115:6813","nonce":1021473826}]},"public_addr":"172.21.15.115:6809/1021473826","cluster_addr":"172.21.15.115:6811/1021473826","heartbeat_back_addr":"172.21.15.115:6815/1021473826","heartbeat_front_addr":"172.21.15.115:6813/1021473826","state":["exists","up"]},{"osd":5,"uuid":"4233a512-9e8f-4db8-aed2-6c3af706e589","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":2405172521},{"type":"v1","addr":"172.21.15.181:6809","nonce":2405172521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":2405172521},{"type":"v1","addr":"172.21.15.181:6811","nonce":2405172521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2405172521},{"type":"v1","addr":"172.21.15.181:6815","nonce":2405172521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":2405172521},{"type":"v1","addr":"172.21.15.181:6813","nonce":2405172521}]},"public_addr":"172.21.15.181:6809/2405172521","cluster_addr":"172.21.15.181:6811/2405172521","heartbeat_back_addr":"172.21.15.181:6815/2405172521","heartbeat_front_addr":"172.21.15.181:6813/2405172521","state":["exists","up"]},{"osd":6,"uuid":"8faab332-016e-4c77-8370-8bef95fc6f34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":815374853},{"type":"v1","addr":"172.21.15.59:6819","nonce":815374853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":815374853},{"type":"v1","addr":"172.21.15.59:6821","nonce":815374853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":815374853},{"type":"v1","addr":"172.21.15.59:6825","nonce":815374853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":815374853},{"type":"v1","addr":"172.21.15.59:6823","nonce":815374853}]},"public_addr":"172.21.15.59:6819/815374853","cluster_addr":"172.21.15.59:6821/815374853","heartbeat_back_addr":"172.21.15.59:6825/815374853","heartbeat_front_addr":"172.21.15.59:6823/815374853","state":["exists","up"]},{"osd":7,"uuid":"f20f470a-0d40-4ed3-8e0e-fd58675cf143","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":1314971870},{"type":"v1","addr":"172.21.15.115:6817","nonce":1314971870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":1314971870},{"type":"v1","addr":"172.21.15.115:6819","nonce":1314971870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":1314971870},{"type":"v1","addr":"172.21.15.115:6823","nonce":1314971870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":1314971870},{"type":"v1","addr":"172.21.15.115:6821","nonce":1314971870}]},"public_addr":"172.21.15.115:6817/1314971870","cluster_addr":"172.21.15.115:6819/1314971870","heartbeat_back_addr":"172.21.15.115:6823/1314971870","heartbeat_front_addr":"172.21.15.115:6821/1314971870","state":["exists","up"]},{"osd":8,"uuid":"eb657d32-1a95-4fd7-a293-d0a970837fc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":3954509873},{"type":"v1","addr":"172.21.15.181:6817","nonce":3954509873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":3954509873},{"type":"v1","addr":"172.21.15.181:6819","nonce":3954509873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":3954509873},{"type":"v1","addr":"172.21.15.181:6823","nonce":3954509873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":3954509873},{"type":"v1","addr":"172.21.15.181:6821","nonce":3954509873}]},"public_addr":"172.21.15.181:6817/3954509873","cluster_addr":"172.21.15.181:6819/3954509873","heartbeat_back_addr":"172.21.15.181:6823/3954509873","heartbeat_front_addr":"172.21.15.181:6821/3954509873","state":["exists","up"]},{"osd":9,"uuid":"8f7717d4-b6f2-4ec1-9b58-62feb0c9a146","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":1163095677},{"type":"v1","addr":"172.21.15.59:6827","nonce":1163095677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":1163095677},{"type":"v1","addr":"172.21.15.59:6829","nonce":1163095677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":1163095677},{"type":"v1","addr":"172.21.15.59:6833","nonce":1163095677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":1163095677},{"type":"v1","addr":"172.21.15.59:6831","nonce":1163095677}]},"public_addr":"172.21.15.59:6827/1163095677","cluster_addr":"172.21.15.59:6829/1163095677","heartbeat_back_addr":"172.21.15.59:6833/1163095677","heartbeat_front_addr":"172.21.15.59:6831/1163095677","state":["exists","up"]},{"osd":10,"uuid":"b75c179a-875d-43bc-b4c1-e8eb80fb64fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":2199610269},{"type":"v1","addr":"172.21.15.115:6825","nonce":2199610269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":2199610269},{"type":"v1","addr":"172.21.15.115:6827","nonce":2199610269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":2199610269},{"type":"v1","addr":"172.21.15.115:6831","nonce":2199610269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":2199610269},{"type":"v1","addr":"172.21.15.115:6829","nonce":2199610269}]},"public_addr":"172.21.15.115:6825/2199610269","cluster_addr":"172.21.15.115:6827/2199610269","heartbeat_back_addr":"172.21.15.115:6831/2199610269","heartbeat_front_addr":"172.21.15.115:6829/2199610269","state":["exists","up"]},{"osd":11,"uuid":"5e141dfe-8a00-4bbc-8204-2dc9be24e18d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":226506989},{"type":"v1","addr":"172.21.15.181:6825","nonce":226506989}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":226506989},{"type":"v1","addr":"172.21.15.181:6827","nonce":226506989}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":226506989},{"type":"v1","addr":"172.21.15.181:6831","nonce":226506989}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":226506989},{"type":"v1","addr":"172.21.15.181:6829","nonce":226506989}]},"public_addr":"172.21.15.181:6825/226506989","cluster_addr":"172.21.15.181:6827/226506989","heartbeat_back_addr":"172.21.15.181:6831/226506989","heartbeat_front_addr":"172.21.15.181:6829/226506989","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:39:42.687084+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:40:03.903448+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:40:24.899908+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:40:46.600251+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:41:07.690037+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:41:28.488855+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:41:49.459274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:42:10.863881+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:42:32.248391+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:42:54.112265+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:43:15.634872+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:43:36.922534+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":7}]},{"pgid":"3.28","mappings":[{"from":5,"to":6},{"from":3,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.59:6800/1259916482":"2023-12-12T17:38:14.037922+0000","172.21.15.59:0/17457774":"2023-12-12T17:38:14.037922+0000","172.21.15.59:6801/1672530462":"2023-12-12T17:37:44.773925+0000","172.21.15.59:0/1053944547":"2023-12-12T17:38:14.037922+0000","172.21.15.59:6801/1259916482":"2023-12-12T17:38:14.037922+0000","172.21.15.59:6800/1672530462":"2023-12-12T17:37:44.773925+0000","172.21.15.59:0/3570767650":"2023-12-12T17:37:44.773925+0000","172.21.15.59:0/3853462899":"2023-12-12T17:37:44.773925+0000","172.21.15.59:0/2859415380":"2023-12-12T17:38:14.037922+0000","172.21.15.59:0/2192742819":"2023-12-12T17:37:44.773925+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-11T17:45:59.706 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:59.683+0000 7f22a7ab1700 1 -- 172.21.15.59:0/2628105238 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f228c0a1510 msgr2=0x7f228c0a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:59.706 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:59.683+0000 7f22a7ab1700 1 --2- 172.21.15.59:0/2628105238 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f228c0a1510 0x7f228c0a39d0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f229c005950 tx=0x7f229c00b410 comp rx=0 tx=0).stop 2023-12-11T17:45:59.706 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:59.683+0000 7f22a7ab1700 1 -- 172.21.15.59:0/2628105238 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f22a01006b0 msgr2=0x7f22a0199bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:59.706 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:59.683+0000 7f22a7ab1700 1 --2- 172.21.15.59:0/2628105238 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f22a01006b0 0x7f22a0199bf0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f229000d950 tx=0x7f229000dc60 comp rx=0 tx=0).stop 2023-12-11T17:45:59.706 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:59.683+0000 7f22a7ab1700 1 -- 172.21.15.59:0/2628105238 shutdown_connections 2023-12-11T17:45:59.707 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:59.683+0000 7f22a7ab1700 1 --2- 172.21.15.59:0/2628105238 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f228c0a1510 0x7f228c0a39d0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:59.707 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:59.683+0000 7f22a7ab1700 1 --2- 172.21.15.59:0/2628105238 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f22a0101070 0x7f22a01a0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:59.707 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:59.683+0000 7f22a7ab1700 1 --2- 172.21.15.59:0/2628105238 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f22a00ffd50 0x7f22a01996b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:59.707 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:59.683+0000 7f22a7ab1700 1 --2- 172.21.15.59:0/2628105238 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f22a01006b0 0x7f22a0199bf0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:59.707 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:59.683+0000 7f22a7ab1700 1 -- 172.21.15.59:0/2628105238 >> 172.21.15.59:0/2628105238 conn(0x7f22a00fb8f0 msgr2=0x7f22a00fd060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:45:59.707 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:59.684+0000 7f22a7ab1700 1 -- 172.21.15.59:0/2628105238 shutdown_connections 2023-12-11T17:45:59.707 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:59.684+0000 7f22a7ab1700 1 -- 172.21.15.59:0/2628105238 wait complete. 2023-12-11T17:45:59.708 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-11T17:40:26.596029+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-11T17:44:18.586290+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-11T17:44:19.432952+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-12-11T17:44:23.097308+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-12-11T17:45:59.708 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-11T17:45:59.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:45:59 smithi115 ceph-mon[102748]: pgmap v310: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-12-11T17:45:59.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:45:59 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/1180287458' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-12-11T17:45:59.837 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:59.833+0000 7f9128cf4700 1 -- 172.21.15.59:0/2611154143 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9124101a90 msgr2=0x7f9124101eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:59.837 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:59.833+0000 7f9128cf4700 1 --2- 172.21.15.59:0/2611154143 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9124101a90 0x7f9124101eb0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f9114009990 tx=0x7f9114009ca0 comp rx=0 tx=0).stop 2023-12-11T17:45:59.837 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:59.833+0000 7f9128cf4700 1 -- 172.21.15.59:0/2611154143 shutdown_connections 2023-12-11T17:45:59.837 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:59.833+0000 7f9128cf4700 1 --2- 172.21.15.59:0/2611154143 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f91241010a0 0x7f9124108940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:59.837 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:59.833+0000 7f9128cf4700 1 --2- 172.21.15.59:0/2611154143 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9124101a90 0x7f9124101eb0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:59.837 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:59.833+0000 7f9128cf4700 1 --2- 172.21.15.59:0/2611154143 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f91241006e0 0x7f9124100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:59.838 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:59.833+0000 7f9128cf4700 1 -- 172.21.15.59:0/2611154143 >> 172.21.15.59:0/2611154143 conn(0x7f91240fb170 msgr2=0x7f91240fd5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:45:59.838 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:59.833+0000 7f9128cf4700 1 -- 172.21.15.59:0/2611154143 shutdown_connections 2023-12-11T17:45:59.838 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:59.833+0000 7f9128cf4700 1 -- 172.21.15.59:0/2611154143 wait complete. 2023-12-11T17:45:59.840 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:59.834+0000 7f9128cf4700 1 Processor -- start 2023-12-11T17:45:59.840 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:59.834+0000 7f9128cf4700 1 -- start start 2023-12-11T17:45:59.841 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:59.834+0000 7f9128cf4700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f91241006e0 0x7f91241964d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:59.841 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:59.834+0000 7f9128cf4700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f91241010a0 0x7f9124196a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:59.841 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:59.834+0000 7f9128cf4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9124101a90 0x7f912419da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:59.841 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:59.834+0000 7f9128cf4700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f912419e000 con 0x7f91241006e0 2023-12-11T17:45:59.841 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:59.834+0000 7f9128cf4700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f912419e140 con 0x7f91241010a0 2023-12-11T17:45:59.841 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:59.834+0000 7f9128cf4700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f912419e2d0 con 0x7f9124101a90 2023-12-11T17:45:59.841 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:59.834+0000 7f912259c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f91241006e0 0x7f91241964d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:59.841 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:59.835+0000 7f912259c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f91241006e0 0x7f91241964d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34338/0 (socket says 172.21.15.59:34338) 2023-12-11T17:45:59.841 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:59.835+0000 7f912259c700 1 -- 172.21.15.59:0/4270870238 learned_addr learned my addr 172.21.15.59:0/4270870238 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:45:59.841 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:59.835+0000 7f9121d9b700 1 --2- 172.21.15.59:0/4270870238 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f91241010a0 0x7f9124196a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:59.841 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:59.835+0000 7f912259c700 1 -- 172.21.15.59:0/4270870238 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9124101a90 msgr2=0x7f912419da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:59.841 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:59.835+0000 7f9122d9d700 1 --2- 172.21.15.59:0/4270870238 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9124101a90 0x7f912419da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:59.842 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:59.835+0000 7f912259c700 1 --2- 172.21.15.59:0/4270870238 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9124101a90 0x7f912419da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:59.842 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:59.835+0000 7f912259c700 1 -- 172.21.15.59:0/4270870238 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f91241010a0 msgr2=0x7f9124196a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:45:59.842 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:59.835+0000 7f912259c700 1 --2- 172.21.15.59:0/4270870238 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f91241010a0 0x7f9124196a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:45:59.842 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:59.835+0000 7f912259c700 1 -- 172.21.15.59:0/4270870238 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91140095f0 con 0x7f91241006e0 2023-12-11T17:45:59.842 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:59.835+0000 7f9122d9d700 1 --2- 172.21.15.59:0/4270870238 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9124101a90 0x7f912419da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-11T17:45:59.842 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:59.835+0000 7f9121d9b700 1 --2- 172.21.15.59:0/4270870238 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f91241010a0 0x7f9124196a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-11T17:45:59.842 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:59.835+0000 7f912259c700 1 --2- 172.21.15.59:0/4270870238 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f91241006e0 0x7f91241964d0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f91140098e0 tx=0x7f911401a960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:59.842 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:59.835+0000 7f91137fe700 1 -- 172.21.15.59:0/4270870238 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f911401f070 con 0x7f91241006e0 2023-12-11T17:45:59.842 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:59.835+0000 7f91137fe700 1 -- 172.21.15.59:0/4270870238 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f911401ac40 con 0x7f91241006e0 2023-12-11T17:45:59.843 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:59.835+0000 7f9128cf4700 1 -- 172.21.15.59:0/4270870238 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91240ff5d0 con 0x7f91241006e0 2023-12-11T17:45:59.843 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:59.835+0000 7f91137fe700 1 -- 172.21.15.59:0/4270870238 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9114025710 con 0x7f91241006e0 2023-12-11T17:45:59.843 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:59.835+0000 7f9128cf4700 1 -- 172.21.15.59:0/4270870238 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91240ffa80 con 0x7f91241006e0 2023-12-11T17:45:59.843 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:59.836+0000 7f9128cf4700 1 -- 172.21.15.59:0/4270870238 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9124067a80 con 0x7f91241006e0 2023-12-11T17:45:59.843 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:59.837+0000 7f91137fe700 1 -- 172.21.15.59:0/4270870238 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f9114023070 con 0x7f91241006e0 2023-12-11T17:45:59.844 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:59.838+0000 7f91137fe700 1 --2- 172.21.15.59:0/4270870238 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f910c0a1510 0x7f910c0a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:45:59.844 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:59.838+0000 7f91137fe700 1 -- 172.21.15.59:0/4270870238 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f91140d0a10 con 0x7f91241006e0 2023-12-11T17:45:59.844 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:59.838+0000 7f9121d9b700 1 --2- 172.21.15.59:0/4270870238 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f910c0a1510 0x7f910c0a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:45:59.844 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:59.838+0000 7f9121d9b700 1 --2- 172.21.15.59:0/4270870238 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f910c0a1510 0x7f910c0a39d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f9118007900 tx=0x7f9118008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:45:59.844 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:45:59.841+0000 7f91137fe700 1 -- 172.21.15.59:0/4270870238 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f9114099ca0 con 0x7f91241006e0 2023-12-11T17:45:59.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:45:59 smithi181 ceph-mon[103627]: pgmap v310: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-12-11T17:45:59.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:45:59 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/1180287458' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-12-11T17:45:59.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:45:59 smithi059 ceph-mon[98847]: pgmap v310: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-12-11T17:45:59.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:45:59 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/1180287458' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-12-11T17:46:00.050 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.047+0000 7f9128cf4700 1 -- 172.21.15.59:0/4270870238 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f912410bf10 con 0x7f91241006e0 2023-12-11T17:46:00.050 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.047+0000 7f91137fe700 1 -- 172.21.15.59:0/4270870238 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7f911409d2c0 con 0x7f91241006e0 2023-12-11T17:46:00.051 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2023-12-11T17:46:00.053 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.049+0000 7f9128cf4700 1 -- 172.21.15.59:0/4270870238 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f910c0a1510 msgr2=0x7f910c0a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:46:00.053 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.049+0000 7f9128cf4700 1 --2- 172.21.15.59:0/4270870238 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f910c0a1510 0x7f910c0a39d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f9118007900 tx=0x7f9118008040 comp rx=0 tx=0).stop 2023-12-11T17:46:00.053 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.049+0000 7f9128cf4700 1 -- 172.21.15.59:0/4270870238 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f91241006e0 msgr2=0x7f91241964d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:46:00.053 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.049+0000 7f9128cf4700 1 --2- 172.21.15.59:0/4270870238 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f91241006e0 0x7f91241964d0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f91140098e0 tx=0x7f911401a960 comp rx=0 tx=0).stop 2023-12-11T17:46:00.053 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.049+0000 7f9128cf4700 1 -- 172.21.15.59:0/4270870238 shutdown_connections 2023-12-11T17:46:00.053 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.049+0000 7f9128cf4700 1 --2- 172.21.15.59:0/4270870238 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f910c0a1510 0x7f910c0a39d0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:00.053 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.049+0000 7f9128cf4700 1 --2- 172.21.15.59:0/4270870238 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9124101a90 0x7f912419da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:00.054 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.049+0000 7f9128cf4700 1 --2- 172.21.15.59:0/4270870238 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f91241006e0 0x7f91241964d0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:00.054 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.049+0000 7f9128cf4700 1 --2- 172.21.15.59:0/4270870238 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f91241010a0 0x7f9124196a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:00.054 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.049+0000 7f9128cf4700 1 -- 172.21.15.59:0/4270870238 >> 172.21.15.59:0/4270870238 conn(0x7f91240fb170 msgr2=0x7f9124104ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:46:00.054 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.049+0000 7f9128cf4700 1 -- 172.21.15.59:0/4270870238 shutdown_connections 2023-12-11T17:46:00.054 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.049+0000 7f9128cf4700 1 -- 172.21.15.59:0/4270870238 wait complete. 2023-12-11T17:46:00.067 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-11T17:46:00.199 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.195+0000 7f075d26c700 1 -- 172.21.15.59:0/1898226357 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0758101a10 msgr2=0x7f0758101e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:46:00.199 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.195+0000 7f075d26c700 1 --2- 172.21.15.59:0/1898226357 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0758101a10 0x7f0758101e30 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f0748009990 tx=0x7f0748009ca0 comp rx=0 tx=0).stop 2023-12-11T17:46:00.199 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.195+0000 7f075d26c700 1 -- 172.21.15.59:0/1898226357 shutdown_connections 2023-12-11T17:46:00.199 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.195+0000 7f075d26c700 1 --2- 172.21.15.59:0/1898226357 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f07581010e0 0x7f07581089a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:00.199 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.195+0000 7f075d26c700 1 --2- 172.21.15.59:0/1898226357 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0758101a10 0x7f0758101e30 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:00.199 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.195+0000 7f075d26c700 1 --2- 172.21.15.59:0/1898226357 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0758100740 0x7f0758100ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:00.199 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.195+0000 7f075d26c700 1 -- 172.21.15.59:0/1898226357 >> 172.21.15.59:0/1898226357 conn(0x7f07580fb170 msgr2=0x7f07580fd5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:46:00.199 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.195+0000 7f075d26c700 1 -- 172.21.15.59:0/1898226357 shutdown_connections 2023-12-11T17:46:00.199 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.195+0000 7f075d26c700 1 -- 172.21.15.59:0/1898226357 wait complete. 2023-12-11T17:46:00.199 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.196+0000 7f075d26c700 1 Processor -- start 2023-12-11T17:46:00.202 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.196+0000 7f075d26c700 1 -- start start 2023-12-11T17:46:00.202 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.196+0000 7f075d26c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0758100740 0x7f0758196370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:00.202 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.196+0000 7f075d26c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f07581010e0 0x7f07581968b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:00.202 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.196+0000 7f075d26c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0758101a10 0x7f075819d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:00.202 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.196+0000 7f075d26c700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f075819dea0 con 0x7f0758100740 2023-12-11T17:46:00.202 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.196+0000 7f075d26c700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f075819dfe0 con 0x7f07581010e0 2023-12-11T17:46:00.203 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.196+0000 7f075d26c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f075819e170 con 0x7f0758101a10 2023-12-11T17:46:00.203 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.196+0000 7f0756ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0758100740 0x7f0758196370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:46:00.203 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.196+0000 7f0756ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0758100740 0x7f0758196370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34352/0 (socket says 172.21.15.59:34352) 2023-12-11T17:46:00.203 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.196+0000 7f0756ffd700 1 -- 172.21.15.59:0/3845825209 learned_addr learned my addr 172.21.15.59:0/3845825209 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:46:00.203 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.196+0000 7f07577fe700 1 --2- 172.21.15.59:0/3845825209 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0758101a10 0x7f075819d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:46:00.203 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.197+0000 7f07567fc700 1 --2- 172.21.15.59:0/3845825209 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f07581010e0 0x7f07581968b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:46:00.203 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.197+0000 7f0756ffd700 1 -- 172.21.15.59:0/3845825209 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0758101a10 msgr2=0x7f075819d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:46:00.203 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.197+0000 7f0756ffd700 1 --2- 172.21.15.59:0/3845825209 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0758101a10 0x7f075819d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:00.203 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.197+0000 7f0756ffd700 1 -- 172.21.15.59:0/3845825209 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f07581010e0 msgr2=0x7f07581968b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:46:00.203 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.197+0000 7f0756ffd700 1 --2- 172.21.15.59:0/3845825209 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f07581010e0 0x7f07581968b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:00.203 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.197+0000 7f0756ffd700 1 -- 172.21.15.59:0/3845825209 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07480095f0 con 0x7f0758100740 2023-12-11T17:46:00.203 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.197+0000 7f07567fc700 1 --2- 172.21.15.59:0/3845825209 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f07581010e0 0x7f07581968b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-11T17:46:00.203 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.197+0000 7f0756ffd700 1 --2- 172.21.15.59:0/3845825209 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0758100740 0x7f0758196370 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f074800bc60 tx=0x7f074800f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:46:00.203 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.197+0000 7f074ffff700 1 -- 172.21.15.59:0/3845825209 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f074801d070 con 0x7f0758100740 2023-12-11T17:46:00.204 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.197+0000 7f074ffff700 1 -- 172.21.15.59:0/3845825209 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f074800fc30 con 0x7f0758100740 2023-12-11T17:46:00.204 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.197+0000 7f075d26c700 1 -- 172.21.15.59:0/3845825209 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f075819e3f0 con 0x7f0758100740 2023-12-11T17:46:00.204 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.197+0000 7f074ffff700 1 -- 172.21.15.59:0/3845825209 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0748017650 con 0x7f0758100740 2023-12-11T17:46:00.204 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.197+0000 7f075d26c700 1 -- 172.21.15.59:0/3845825209 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f075819e870 con 0x7f0758100740 2023-12-11T17:46:00.204 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.198+0000 7f075d26c700 1 -- 172.21.15.59:0/3845825209 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0758107930 con 0x7f0758100740 2023-12-11T17:46:00.204 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.199+0000 7f074ffff700 1 -- 172.21.15.59:0/3845825209 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f0748022470 con 0x7f0758100740 2023-12-11T17:46:00.205 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.200+0000 7f074ffff700 1 --2- 172.21.15.59:0/3845825209 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f07440a1590 0x7f07440a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:00.205 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.200+0000 7f07567fc700 1 --2- 172.21.15.59:0/3845825209 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f07440a1590 0x7f07440a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:46:00.205 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.200+0000 7f074ffff700 1 -- 172.21.15.59:0/3845825209 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f07480ce620 con 0x7f0758100740 2023-12-11T17:46:00.205 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.200+0000 7f07567fc700 1 --2- 172.21.15.59:0/3845825209 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f07440a1590 0x7f07440a3a50 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f0740005fd0 tx=0x7f0740005e20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:46:00.208 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.204+0000 7f074ffff700 1 -- 172.21.15.59:0/3845825209 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f07480978b0 con 0x7f0758100740 2023-12-11T17:46:00.410 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.406+0000 7f075d26c700 1 -- 172.21.15.59:0/3845825209 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0758067a80 con 0x7f0758100740 2023-12-11T17:46:00.410 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.407+0000 7f074ffff700 1 -- 172.21.15.59:0/3845825209 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0758067a80 con 0x7f0758100740 2023-12-11T17:46:00.410 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2023-12-11T17:46:00.412 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.409+0000 7f075d26c700 1 -- 172.21.15.59:0/3845825209 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f07440a1590 msgr2=0x7f07440a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:46:00.413 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.409+0000 7f075d26c700 1 --2- 172.21.15.59:0/3845825209 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f07440a1590 0x7f07440a3a50 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f0740005fd0 tx=0x7f0740005e20 comp rx=0 tx=0).stop 2023-12-11T17:46:00.413 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.409+0000 7f075d26c700 1 -- 172.21.15.59:0/3845825209 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0758100740 msgr2=0x7f0758196370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:46:00.413 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.409+0000 7f075d26c700 1 --2- 172.21.15.59:0/3845825209 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0758100740 0x7f0758196370 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f074800bc60 tx=0x7f074800f740 comp rx=0 tx=0).stop 2023-12-11T17:46:00.413 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.409+0000 7f075d26c700 1 -- 172.21.15.59:0/3845825209 shutdown_connections 2023-12-11T17:46:00.413 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.409+0000 7f075d26c700 1 --2- 172.21.15.59:0/3845825209 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f07440a1590 0x7f07440a3a50 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:00.413 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.409+0000 7f075d26c700 1 --2- 172.21.15.59:0/3845825209 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0758101a10 0x7f075819d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:00.413 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.409+0000 7f075d26c700 1 --2- 172.21.15.59:0/3845825209 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0758100740 0x7f0758196370 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:00.413 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.409+0000 7f075d26c700 1 --2- 172.21.15.59:0/3845825209 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f07581010e0 0x7f07581968b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:00.413 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.409+0000 7f075d26c700 1 -- 172.21.15.59:0/3845825209 >> 172.21.15.59:0/3845825209 conn(0x7f07580fb170 msgr2=0x7f0758104a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:46:00.413 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.409+0000 7f075d26c700 1 -- 172.21.15.59:0/3845825209 shutdown_connections 2023-12-11T17:46:00.413 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.409+0000 7f075d26c700 1 -- 172.21.15.59:0/3845825209 wait complete. 2023-12-11T17:46:00.424 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-11T17:46:00.556 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.551+0000 7f2501136700 1 -- 172.21.15.59:0/1215907765 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24fc1072c0 msgr2=0x7f24fc1096d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:46:00.556 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.551+0000 7f2501136700 1 --2- 172.21.15.59:0/1215907765 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24fc1072c0 0x7f24fc1096d0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f24f400b0a0 tx=0x7f24f400b3b0 comp rx=0 tx=0).stop 2023-12-11T17:46:00.556 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.552+0000 7f2501136700 1 -- 172.21.15.59:0/1215907765 shutdown_connections 2023-12-11T17:46:00.556 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.552+0000 7f2501136700 1 --2- 172.21.15.59:0/1215907765 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24fc1072c0 0x7f24fc1096d0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:00.556 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.552+0000 7f2501136700 1 --2- 172.21.15.59:0/1215907765 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f24fc102060 0x7f24fc104450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:00.556 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.552+0000 7f2501136700 1 --2- 172.21.15.59:0/1215907765 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f24fc104990 0x7f24fc106d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:00.556 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.552+0000 7f2501136700 1 -- 172.21.15.59:0/1215907765 >> 172.21.15.59:0/1215907765 conn(0x7f24fc0fb9f0 msgr2=0x7f24fc0fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:46:00.556 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.552+0000 7f2501136700 1 -- 172.21.15.59:0/1215907765 shutdown_connections 2023-12-11T17:46:00.556 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.552+0000 7f2501136700 1 -- 172.21.15.59:0/1215907765 wait complete. 2023-12-11T17:46:00.556 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.552+0000 7f2501136700 1 Processor -- start 2023-12-11T17:46:00.558 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.553+0000 7f2501136700 1 -- start start 2023-12-11T17:46:00.558 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.553+0000 7f2501136700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f24fc102060 0x7f24fc1976a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:00.559 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.553+0000 7f2501136700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f24fc104990 0x7f24fc197be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:00.559 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.553+0000 7f2501136700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24fc1072c0 0x7f24fc19eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:00.559 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.553+0000 7f2501136700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24fc19f130 con 0x7f24fc1072c0 2023-12-11T17:46:00.559 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.553+0000 7f2501136700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24fc19f270 con 0x7f24fc104990 2023-12-11T17:46:00.559 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.553+0000 7f2501136700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24fc19f400 con 0x7f24fc102060 2023-12-11T17:46:00.559 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.553+0000 7f24fb59e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24fc1072c0 0x7f24fc19eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:46:00.559 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.553+0000 7f24fb59e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24fc1072c0 0x7f24fc19eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34370/0 (socket says 172.21.15.59:34370) 2023-12-11T17:46:00.559 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.553+0000 7f24fb59e700 1 -- 172.21.15.59:0/1525683277 learned_addr learned my addr 172.21.15.59:0/1525683277 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:46:00.559 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.553+0000 7f24fad9d700 1 --2- 172.21.15.59:0/1525683277 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f24fc102060 0x7f24fc1976a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:46:00.559 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.553+0000 7f24fa59c700 1 --2- 172.21.15.59:0/1525683277 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f24fc104990 0x7f24fc197be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:46:00.559 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.553+0000 7f24fb59e700 1 -- 172.21.15.59:0/1525683277 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f24fc102060 msgr2=0x7f24fc1976a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:46:00.559 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.553+0000 7f24fb59e700 1 --2- 172.21.15.59:0/1525683277 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f24fc102060 0x7f24fc1976a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:00.559 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.553+0000 7f24fb59e700 1 -- 172.21.15.59:0/1525683277 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f24fc104990 msgr2=0x7f24fc197be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:46:00.559 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.553+0000 7f24fb59e700 1 --2- 172.21.15.59:0/1525683277 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f24fc104990 0x7f24fc197be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:00.560 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.553+0000 7f24fb59e700 1 -- 172.21.15.59:0/1525683277 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24f4009d00 con 0x7f24fc1072c0 2023-12-11T17:46:00.560 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.553+0000 7f24fa59c700 1 --2- 172.21.15.59:0/1525683277 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f24fc104990 0x7f24fc197be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T17:46:00.560 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.553+0000 7f24fb59e700 1 --2- 172.21.15.59:0/1525683277 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24fc1072c0 0x7f24fc19eb60 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f24f4007a60 tx=0x7f24f400bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:46:00.560 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.554+0000 7f24e3fff700 1 -- 172.21.15.59:0/1525683277 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24f400e040 con 0x7f24fc1072c0 2023-12-11T17:46:00.560 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.554+0000 7f2501136700 1 -- 172.21.15.59:0/1525683277 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24fc19f680 con 0x7f24fc1072c0 2023-12-11T17:46:00.560 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.554+0000 7f24e3fff700 1 -- 172.21.15.59:0/1525683277 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f24f40094c0 con 0x7f24fc1072c0 2023-12-11T17:46:00.560 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.554+0000 7f24e3fff700 1 -- 172.21.15.59:0/1525683277 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24f40052a0 con 0x7f24fc1072c0 2023-12-11T17:46:00.560 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.554+0000 7f2501136700 1 -- 172.21.15.59:0/1525683277 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24fc19fb00 con 0x7f24fc1072c0 2023-12-11T17:46:00.560 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.555+0000 7f2501136700 1 -- 172.21.15.59:0/1525683277 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24fc0fd570 con 0x7f24fc1072c0 2023-12-11T17:46:00.561 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.556+0000 7f24e3fff700 1 -- 172.21.15.59:0/1525683277 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f24f401d470 con 0x7f24fc1072c0 2023-12-11T17:46:00.561 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.556+0000 7f24e3fff700 1 --2- 172.21.15.59:0/1525683277 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f24e40a1510 0x7f24e40a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:00.561 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.556+0000 7f24fad9d700 1 --2- 172.21.15.59:0/1525683277 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f24e40a1510 0x7f24e40a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:46:00.561 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.556+0000 7f24e3fff700 1 -- 172.21.15.59:0/1525683277 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f24f40cea80 con 0x7f24fc1072c0 2023-12-11T17:46:00.561 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.557+0000 7f24fad9d700 1 --2- 172.21.15.59:0/1525683277 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f24e40a1510 0x7f24e40a39d0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f24ec005950 tx=0x7f24ec00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:46:00.563 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.560+0000 7f24e3fff700 1 -- 172.21.15.59:0/1525683277 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f24f4097d10 con 0x7f24fc1072c0 2023-12-11T17:46:00.769 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.766+0000 7f2501136700 1 -- 172.21.15.59:0/1525683277 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f24fc10cf40 con 0x7f24fc1072c0 2023-12-11T17:46:00.770 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.766+0000 7f24e3fff700 1 -- 172.21.15.59:0/1525683277 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7f24f409b330 con 0x7f24fc1072c0 2023-12-11T17:46:00.770 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2023-12-11T17:46:00.772 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.769+0000 7f2501136700 1 -- 172.21.15.59:0/1525683277 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f24e40a1510 msgr2=0x7f24e40a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:46:00.772 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.769+0000 7f2501136700 1 --2- 172.21.15.59:0/1525683277 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f24e40a1510 0x7f24e40a39d0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f24ec005950 tx=0x7f24ec00b410 comp rx=0 tx=0).stop 2023-12-11T17:46:00.772 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.769+0000 7f2501136700 1 -- 172.21.15.59:0/1525683277 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24fc1072c0 msgr2=0x7f24fc19eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:46:00.773 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.769+0000 7f2501136700 1 --2- 172.21.15.59:0/1525683277 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24fc1072c0 0x7f24fc19eb60 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f24f4007a60 tx=0x7f24f400bce0 comp rx=0 tx=0).stop 2023-12-11T17:46:00.773 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.769+0000 7f2501136700 1 -- 172.21.15.59:0/1525683277 shutdown_connections 2023-12-11T17:46:00.773 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.769+0000 7f2501136700 1 --2- 172.21.15.59:0/1525683277 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f24e40a1510 0x7f24e40a39d0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:00.773 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.769+0000 7f2501136700 1 --2- 172.21.15.59:0/1525683277 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24fc1072c0 0x7f24fc19eb60 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:00.773 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.769+0000 7f2501136700 1 --2- 172.21.15.59:0/1525683277 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f24fc102060 0x7f24fc1976a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:00.773 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.769+0000 7f2501136700 1 --2- 172.21.15.59:0/1525683277 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f24fc104990 0x7f24fc197be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:00.773 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.769+0000 7f2501136700 1 -- 172.21.15.59:0/1525683277 >> 172.21.15.59:0/1525683277 conn(0x7f24fc0fb9f0 msgr2=0x7f24fc1029a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:46:00.773 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.769+0000 7f2501136700 1 -- 172.21.15.59:0/1525683277 shutdown_connections 2023-12-11T17:46:00.773 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.769+0000 7f2501136700 1 -- 172.21.15.59:0/1525683277 wait complete. 2023-12-11T17:46:00.783 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:00 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/2628105238' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-11T17:46:00.784 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:00 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/4270870238' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-11T17:46:00.784 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:00 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/3845825209' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-11T17:46:00.785 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-12-11T17:46:00.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:00 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/2628105238' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-11T17:46:00.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:00 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/4270870238' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-11T17:46:00.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:00 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/3845825209' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-11T17:46:00.914 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.910+0000 7f154ca7b700 1 -- 172.21.15.59:0/1478748843 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1548102790 msgr2=0x7f1548102bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:46:00.914 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.910+0000 7f154ca7b700 1 --2- 172.21.15.59:0/1478748843 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1548102790 0x7f1548102bf0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f153c0099c0 tx=0x7f153c009cd0 comp rx=0 tx=0).stop 2023-12-11T17:46:00.914 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.910+0000 7f154ca7b700 1 -- 172.21.15.59:0/1478748843 shutdown_connections 2023-12-11T17:46:00.914 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.910+0000 7f154ca7b700 1 --2- 172.21.15.59:0/1478748843 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1548103130 0x7f15481099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:00.914 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.910+0000 7f154ca7b700 1 --2- 172.21.15.59:0/1478748843 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f15481015a0 0x7f15481019c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:00.914 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.910+0000 7f154ca7b700 1 --2- 172.21.15.59:0/1478748843 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1548102790 0x7f1548102bf0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:00.914 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.910+0000 7f154ca7b700 1 -- 172.21.15.59:0/1478748843 >> 172.21.15.59:0/1478748843 conn(0x7f15480fcb10 msgr2=0x7f15480fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:46:00.915 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.910+0000 7f154ca7b700 1 -- 172.21.15.59:0/1478748843 shutdown_connections 2023-12-11T17:46:00.915 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.910+0000 7f154ca7b700 1 -- 172.21.15.59:0/1478748843 wait complete. 2023-12-11T17:46:00.917 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.911+0000 7f154ca7b700 1 Processor -- start 2023-12-11T17:46:00.917 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.911+0000 7f154ca7b700 1 -- start start 2023-12-11T17:46:00.917 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.911+0000 7f154ca7b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f15481015a0 0x7f1548075330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:00.917 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.911+0000 7f154ca7b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1548102790 0x7f15480739c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:00.917 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.911+0000 7f154ca7b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1548103130 0x7f1548073f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:00.917 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.911+0000 7f154ca7b700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15480746a0 con 0x7f1548102790 2023-12-11T17:46:00.917 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.911+0000 7f154ca7b700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1548074810 con 0x7f1548103130 2023-12-11T17:46:00.917 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.911+0000 7f154ca7b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15481a1620 con 0x7f15481015a0 2023-12-11T17:46:00.917 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.911+0000 7f1545d9b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1548102790 0x7f15480739c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:46:00.917 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.911+0000 7f1545d9b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1548102790 0x7f15480739c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34382/0 (socket says 172.21.15.59:34382) 2023-12-11T17:46:00.917 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.911+0000 7f1545d9b700 1 -- 172.21.15.59:0/3492812958 learned_addr learned my addr 172.21.15.59:0/3492812958 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:46:00.917 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.911+0000 7f154659c700 1 --2- 172.21.15.59:0/3492812958 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f15481015a0 0x7f1548075330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:46:00.918 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.911+0000 7f1546d9d700 1 --2- 172.21.15.59:0/3492812958 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1548103130 0x7f1548073f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:46:00.918 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.911+0000 7f1545d9b700 1 -- 172.21.15.59:0/3492812958 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f15481015a0 msgr2=0x7f1548075330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:46:00.918 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.911+0000 7f1545d9b700 1 --2- 172.21.15.59:0/3492812958 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f15481015a0 0x7f1548075330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:00.918 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.912+0000 7f1545d9b700 1 -- 172.21.15.59:0/3492812958 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1548103130 msgr2=0x7f1548073f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:46:00.918 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.912+0000 7f1545d9b700 1 --2- 172.21.15.59:0/3492812958 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1548103130 0x7f1548073f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:00.918 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.912+0000 7f1545d9b700 1 -- 172.21.15.59:0/3492812958 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f153c0095f0 con 0x7f1548102790 2023-12-11T17:46:00.918 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.912+0000 7f1545d9b700 1 --2- 172.21.15.59:0/3492812958 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1548102790 0x7f15480739c0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f153c006010 tx=0x7f153c005af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:46:00.918 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.912+0000 7f15377fe700 1 -- 172.21.15.59:0/3492812958 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f153c01d070 con 0x7f1548102790 2023-12-11T17:46:00.918 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.912+0000 7f15377fe700 1 -- 172.21.15.59:0/3492812958 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f153c004150 con 0x7f1548102790 2023-12-11T17:46:00.918 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.912+0000 7f15377fe700 1 -- 172.21.15.59:0/3492812958 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f153c00f460 con 0x7f1548102790 2023-12-11T17:46:00.918 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.912+0000 7f154ca7b700 1 -- 172.21.15.59:0/3492812958 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15481a1850 con 0x7f1548102790 2023-12-11T17:46:00.918 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.912+0000 7f154ca7b700 1 -- 172.21.15.59:0/3492812958 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15481a1bc0 con 0x7f1548102790 2023-12-11T17:46:00.918 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.913+0000 7f154ca7b700 1 -- 172.21.15.59:0/3492812958 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1548108a20 con 0x7f1548102790 2023-12-11T17:46:00.919 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.914+0000 7f15377fe700 1 -- 172.21.15.59:0/3492812958 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f153c022700 con 0x7f1548102790 2023-12-11T17:46:00.919 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.915+0000 7f15377fe700 1 --2- 172.21.15.59:0/3492812958 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f15300a1510 0x7f15300a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:00.919 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.915+0000 7f15377fe700 1 -- 172.21.15.59:0/3492812958 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f153c0cea10 con 0x7f1548102790 2023-12-11T17:46:00.919 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.915+0000 7f154659c700 1 --2- 172.21.15.59:0/3492812958 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f15300a1510 0x7f15300a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:46:00.919 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.915+0000 7f154659c700 1 --2- 172.21.15.59:0/3492812958 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f15300a1510 0x7f15300a39d0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f1538005950 tx=0x7f15380058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:46:00.921 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:00.918+0000 7f15377fe700 1 -- 172.21.15.59:0/3492812958 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f153c097ca0 con 0x7f1548102790 2023-12-11T17:46:01.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:00 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/2628105238' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-11T17:46:01.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:00 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/4270870238' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-11T17:46:01.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:00 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/3845825209' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-11T17:46:01.125 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:01.122+0000 7f154ca7b700 1 -- 172.21.15.59:0/3492812958 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f154810cf90 con 0x7f1548102790 2023-12-11T17:46:01.125 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:01.122+0000 7f15377fe700 1 -- 172.21.15.59:0/3492812958 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v88) v1 ==== 103+0+11 (secure 0 0 0) 0x7f153c09b2c0 con 0x7f1548102790 2023-12-11T17:46:01.126 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2023-12-11T17:46:01.128 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:01.124+0000 7f154ca7b700 1 -- 172.21.15.59:0/3492812958 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f15300a1510 msgr2=0x7f15300a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:46:01.128 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:01.124+0000 7f154ca7b700 1 --2- 172.21.15.59:0/3492812958 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f15300a1510 0x7f15300a39d0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f1538005950 tx=0x7f15380058e0 comp rx=0 tx=0).stop 2023-12-11T17:46:01.128 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:01.124+0000 7f154ca7b700 1 -- 172.21.15.59:0/3492812958 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1548102790 msgr2=0x7f15480739c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:46:01.128 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:01.124+0000 7f154ca7b700 1 --2- 172.21.15.59:0/3492812958 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1548102790 0x7f15480739c0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f153c006010 tx=0x7f153c005af0 comp rx=0 tx=0).stop 2023-12-11T17:46:01.128 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:01.124+0000 7f154ca7b700 1 -- 172.21.15.59:0/3492812958 shutdown_connections 2023-12-11T17:46:01.128 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:01.124+0000 7f154ca7b700 1 --2- 172.21.15.59:0/3492812958 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f15300a1510 0x7f15300a39d0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:01.128 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:01.124+0000 7f154ca7b700 1 --2- 172.21.15.59:0/3492812958 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1548103130 0x7f1548073f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:01.128 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:01.124+0000 7f154ca7b700 1 --2- 172.21.15.59:0/3492812958 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f15481015a0 0x7f1548075330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:01.128 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:01.124+0000 7f154ca7b700 1 --2- 172.21.15.59:0/3492812958 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1548102790 0x7f15480739c0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:01.129 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:01.124+0000 7f154ca7b700 1 -- 172.21.15.59:0/3492812958 >> 172.21.15.59:0/3492812958 conn(0x7f15480fcb10 msgr2=0x7f1548105b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:46:01.129 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:01.124+0000 7f154ca7b700 1 -- 172.21.15.59:0/3492812958 shutdown_connections 2023-12-11T17:46:01.129 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:01.124+0000 7f154ca7b700 1 -- 172.21.15.59:0/3492812958 wait complete. 2023-12-11T17:46:01.140 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-11T17:46:01.267 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:01.262+0000 7f3e7ec98700 1 -- 172.21.15.59:0/3199358603 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e78102690 msgr2=0x7f3e78102af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:46:01.268 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:01.262+0000 7f3e7ec98700 1 --2- 172.21.15.59:0/3199358603 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e78102690 0x7f3e78102af0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f3e640099c0 tx=0x7f3e64009cd0 comp rx=0 tx=0).stop 2023-12-11T17:46:01.268 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:01.262+0000 7f3e7ec98700 1 -- 172.21.15.59:0/3199358603 shutdown_connections 2023-12-11T17:46:01.268 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:01.262+0000 7f3e7ec98700 1 --2- 172.21.15.59:0/3199358603 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3e78103030 0x7f3e781098e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:01.268 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:01.262+0000 7f3e7ec98700 1 --2- 172.21.15.59:0/3199358603 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3e781014a0 0x7f3e781018c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:01.268 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:01.262+0000 7f3e7ec98700 1 --2- 172.21.15.59:0/3199358603 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e78102690 0x7f3e78102af0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:01.268 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:01.262+0000 7f3e7ec98700 1 -- 172.21.15.59:0/3199358603 >> 172.21.15.59:0/3199358603 conn(0x7f3e780fca30 msgr2=0x7f3e780fee50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:46:01.268 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:01.262+0000 7f3e7ec98700 1 -- 172.21.15.59:0/3199358603 shutdown_connections 2023-12-11T17:46:01.268 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:01.262+0000 7f3e7ec98700 1 -- 172.21.15.59:0/3199358603 wait complete. 2023-12-11T17:46:01.268 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:01.262+0000 7f3e7ec98700 1 Processor -- start 2023-12-11T17:46:01.268 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:01.262+0000 7f3e7ec98700 1 -- start start 2023-12-11T17:46:01.268 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:01.263+0000 7f3e7ec98700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3e781014a0 0x7f3e78075330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:01.268 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:01.263+0000 7f3e7ec98700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e78102690 0x7f3e780739c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:01.269 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:01.263+0000 7f3e7ec98700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3e78103030 0x7f3e78073f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:01.269 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:01.263+0000 7f3e7ec98700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e780746a0 con 0x7f3e78102690 2023-12-11T17:46:01.269 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:01.263+0000 7f3e7ec98700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e78074810 con 0x7f3e78103030 2023-12-11T17:46:01.269 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:01.263+0000 7f3e7ec98700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e781a1410 con 0x7f3e781014a0 2023-12-11T17:46:01.269 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:01.263+0000 7f3e77fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e78102690 0x7f3e780739c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:46:01.269 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:01.263+0000 7f3e77fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e78102690 0x7f3e780739c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34396/0 (socket says 172.21.15.59:34396) 2023-12-11T17:46:01.269 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:01.263+0000 7f3e77fff700 1 -- 172.21.15.59:0/1081546782 learned_addr learned my addr 172.21.15.59:0/1081546782 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:46:01.269 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:01.263+0000 7f3e7ca34700 1 --2- 172.21.15.59:0/1081546782 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3e781014a0 0x7f3e78075330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:46:01.269 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:01.263+0000 7f3e7d235700 1 --2- 172.21.15.59:0/1081546782 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3e78103030 0x7f3e78073f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:46:01.269 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:01.263+0000 7f3e77fff700 1 -- 172.21.15.59:0/1081546782 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3e781014a0 msgr2=0x7f3e78075330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:46:01.269 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:01.263+0000 7f3e77fff700 1 --2- 172.21.15.59:0/1081546782 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3e781014a0 0x7f3e78075330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:01.269 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:01.263+0000 7f3e77fff700 1 -- 172.21.15.59:0/1081546782 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3e78103030 msgr2=0x7f3e78073f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:46:01.269 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:01.263+0000 7f3e77fff700 1 --2- 172.21.15.59:0/1081546782 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3e78103030 0x7f3e78073f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:01.269 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:01.263+0000 7f3e77fff700 1 -- 172.21.15.59:0/1081546782 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e640095f0 con 0x7f3e78102690 2023-12-11T17:46:01.269 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:01.263+0000 7f3e7d235700 1 --2- 172.21.15.59:0/1081546782 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3e78103030 0x7f3e78073f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T17:46:01.270 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:01.263+0000 7f3e77fff700 1 --2- 172.21.15.59:0/1081546782 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e78102690 0x7f3e780739c0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f3e6400b580 tx=0x7f3e64009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:46:01.270 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:01.263+0000 7f3e75ffb700 1 -- 172.21.15.59:0/1081546782 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e6401d070 con 0x7f3e78102690 2023-12-11T17:46:01.270 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:01.263+0000 7f3e75ffb700 1 -- 172.21.15.59:0/1081546782 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3e64004010 con 0x7f3e78102690 2023-12-11T17:46:01.270 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:01.263+0000 7f3e7ec98700 1 -- 172.21.15.59:0/1081546782 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e781a1640 con 0x7f3e78102690 2023-12-11T17:46:01.270 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:01.263+0000 7f3e75ffb700 1 -- 172.21.15.59:0/1081546782 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e64022d10 con 0x7f3e78102690 2023-12-11T17:46:01.270 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:01.264+0000 7f3e7ec98700 1 -- 172.21.15.59:0/1081546782 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e781a1a90 con 0x7f3e78102690 2023-12-11T17:46:01.271 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:01.265+0000 7f3e7ec98700 1 -- 172.21.15.59:0/1081546782 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e781088d0 con 0x7f3e78102690 2023-12-11T17:46:01.271 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:01.266+0000 7f3e75ffb700 1 -- 172.21.15.59:0/1081546782 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f3e64022470 con 0x7f3e78102690 2023-12-11T17:46:01.271 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:01.266+0000 7f3e75ffb700 1 --2- 172.21.15.59:0/1081546782 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f3e680a1510 0x7f3e680a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:01.271 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:01.266+0000 7f3e7ca34700 1 --2- 172.21.15.59:0/1081546782 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f3e680a1510 0x7f3e680a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:46:01.271 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:01.266+0000 7f3e75ffb700 1 -- 172.21.15.59:0/1081546782 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f3e640ced50 con 0x7f3e78102690 2023-12-11T17:46:01.271 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:01.267+0000 7f3e7ca34700 1 --2- 172.21.15.59:0/1081546782 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f3e680a1510 0x7f3e680a39d0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f3e781024f0 tx=0x7f3e6c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:46:01.273 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:01.270+0000 7f3e75ffb700 1 -- 172.21.15.59:0/1081546782 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f3e64097fe0 con 0x7f3e78102690 2023-12-11T17:46:01.526 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:01.523+0000 7f3e7ec98700 1 -- 172.21.15.59:0/1081546782 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3e7810cdc0 con 0x7f3e78102690 2023-12-11T17:46:01.527 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:01.524+0000 7f3e75ffb700 1 -- 172.21.15.59:0/1081546782 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+11325 (secure 0 0 0) 0x7f3e64027090 con 0x7f3e78102690 2023-12-11T17:46:01.527 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-12-11T17:46:01.527 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/2184151593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":2184151593},{"type":"v1","addr":"172.21.15.59:6835","nonce":2184151593}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/2817614886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2817614886},{"type":"v1","addr":"172.21.15.59:6837","nonce":2817614886}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6839/229873490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":229873490},{"type":"v1","addr":"172.21.15.59:6839","nonce":229873490}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6841/967998755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":967998755},{"type":"v1","addr":"172.21.15.59:6841","nonce":967998755}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/395684042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":395684042},{"type":"v1","addr":"172.21.15.181:6833","nonce":395684042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6833/3214606191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3214606191},{"type":"v1","addr":"172.21.15.115:6833","nonce":3214606191}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-12-11T17:44:20.076660+0000","modified":"2023-12-11T17:44:47.770695+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24521,"mds_1":24484,"mds_2":24482},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/1874159657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1874159657},{"type":"v1","addr":"172.21.15.115:6835","nonce":1874159657}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24457":{"gid":24457,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/4221183620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":4221183620},{"type":"v1","addr":"172.21.15.181:6835","nonce":4221183620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24466":{"gid":24466,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6837/3580309407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":3580309407},{"type":"v1","addr":"172.21.15.181:6837","nonce":3580309407}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24482":{"gid":24482,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.115:6837/1209365460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":1209365460},{"type":"v1","addr":"172.21.15.115:6837","nonce":1209365460}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.181:6839/2214384806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":2214384806},{"type":"v1","addr":"172.21.15.181:6839","nonce":2214384806}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.115:6839/4122922866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":4122922866},{"type":"v1","addr":"172.21.15.115:6839","nonce":4122922866}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-11T17:46:01.529 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:01.526+0000 7f3e7ec98700 1 -- 172.21.15.59:0/1081546782 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f3e680a1510 msgr2=0x7f3e680a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:46:01.530 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:01.526+0000 7f3e7ec98700 1 --2- 172.21.15.59:0/1081546782 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f3e680a1510 0x7f3e680a39d0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f3e781024f0 tx=0x7f3e6c00b410 comp rx=0 tx=0).stop 2023-12-11T17:46:01.530 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:01.526+0000 7f3e7ec98700 1 -- 172.21.15.59:0/1081546782 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e78102690 msgr2=0x7f3e780739c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:46:01.530 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:01.526+0000 7f3e7ec98700 1 --2- 172.21.15.59:0/1081546782 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e78102690 0x7f3e780739c0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f3e6400b580 tx=0x7f3e64009580 comp rx=0 tx=0).stop 2023-12-11T17:46:01.530 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:01.526+0000 7f3e7ec98700 1 -- 172.21.15.59:0/1081546782 shutdown_connections 2023-12-11T17:46:01.530 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:01.526+0000 7f3e7ec98700 1 --2- 172.21.15.59:0/1081546782 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f3e680a1510 0x7f3e680a39d0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:01.530 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:01.526+0000 7f3e7ec98700 1 --2- 172.21.15.59:0/1081546782 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3e78103030 0x7f3e78073f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:01.530 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:01.526+0000 7f3e7ec98700 1 --2- 172.21.15.59:0/1081546782 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3e781014a0 0x7f3e78075330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:01.530 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:01.526+0000 7f3e7ec98700 1 --2- 172.21.15.59:0/1081546782 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e78102690 0x7f3e780739c0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:01.530 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:01.526+0000 7f3e7ec98700 1 -- 172.21.15.59:0/1081546782 >> 172.21.15.59:0/1081546782 conn(0x7f3e780fca30 msgr2=0x7f3e78105a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:46:01.530 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:01.526+0000 7f3e7ec98700 1 -- 172.21.15.59:0/1081546782 shutdown_connections 2023-12-11T17:46:01.530 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:01.526+0000 7f3e7ec98700 1 -- 172.21.15.59:0/1081546782 wait complete. 2023-12-11T17:46:01.531 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 24 2023-12-11T17:46:01.543 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-11T17:46:01.672 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:01.665+0000 7f27a0260700 1 -- 172.21.15.59:0/1017599630 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27981014b0 msgr2=0x7f27981018d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:46:01.672 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:01.666+0000 7f27a0260700 1 --2- 172.21.15.59:0/1017599630 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27981014b0 0x7f27981018d0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f27940099c0 tx=0x7f2794009cd0 comp rx=0 tx=0).stop 2023-12-11T17:46:01.672 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:01.666+0000 7f27a0260700 1 -- 172.21.15.59:0/1017599630 shutdown_connections 2023-12-11T17:46:01.672 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:01.666+0000 7f27a0260700 1 --2- 172.21.15.59:0/1017599630 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2798103040 0x7f27981098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:01.672 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:01.666+0000 7f27a0260700 1 --2- 172.21.15.59:0/1017599630 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27981014b0 0x7f27981018d0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:01.672 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:01.666+0000 7f27a0260700 1 --2- 172.21.15.59:0/1017599630 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f27981026a0 0x7f2798102b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:01.672 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:01.666+0000 7f27a0260700 1 -- 172.21.15.59:0/1017599630 >> 172.21.15.59:0/1017599630 conn(0x7f27980fca20 msgr2=0x7f27980fee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:46:01.672 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:01.666+0000 7f27a0260700 1 -- 172.21.15.59:0/1017599630 shutdown_connections 2023-12-11T17:46:01.673 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:01.666+0000 7f27a0260700 1 -- 172.21.15.59:0/1017599630 wait complete. 2023-12-11T17:46:01.673 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:01.666+0000 7f27a0260700 1 Processor -- start 2023-12-11T17:46:01.673 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:01.666+0000 7f27a0260700 1 -- start start 2023-12-11T17:46:01.673 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:01.666+0000 7f27a0260700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27981014b0 0x7f2798197410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:01.673 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:01.667+0000 7f27a0260700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f27981026a0 0x7f2798197950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:01.673 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:01.667+0000 7f27a0260700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2798103040 0x7f279819e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:01.673 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:01.667+0000 7f27a0260700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f279819ef40 con 0x7f27981014b0 2023-12-11T17:46:01.673 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:01.667+0000 7f27a0260700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f279819f080 con 0x7f27981026a0 2023-12-11T17:46:01.673 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:01.667+0000 7f27a0260700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f279819f210 con 0x7f2798103040 2023-12-11T17:46:01.673 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:01.667+0000 7f279dffc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27981014b0 0x7f2798197410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:46:01.673 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:01.667+0000 7f279dffc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27981014b0 0x7f2798197410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34406/0 (socket says 172.21.15.59:34406) 2023-12-11T17:46:01.673 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:01.667+0000 7f279dffc700 1 -- 172.21.15.59:0/1676080828 learned_addr learned my addr 172.21.15.59:0/1676080828 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:46:01.673 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:01.667+0000 7f279d7fb700 1 --2- 172.21.15.59:0/1676080828 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f27981026a0 0x7f2798197950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:46:01.673 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:01.667+0000 7f279e7fd700 1 --2- 172.21.15.59:0/1676080828 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2798103040 0x7f279819e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:46:01.674 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:01.667+0000 7f279dffc700 1 -- 172.21.15.59:0/1676080828 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2798103040 msgr2=0x7f279819e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:46:01.674 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:01.667+0000 7f279dffc700 1 --2- 172.21.15.59:0/1676080828 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2798103040 0x7f279819e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:01.674 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:01.667+0000 7f279dffc700 1 -- 172.21.15.59:0/1676080828 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f27981026a0 msgr2=0x7f2798197950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:46:01.674 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:01.667+0000 7f279dffc700 1 --2- 172.21.15.59:0/1676080828 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f27981026a0 0x7f2798197950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:01.674 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:01.667+0000 7f279dffc700 1 -- 172.21.15.59:0/1676080828 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27940095f0 con 0x7f27981014b0 2023-12-11T17:46:01.674 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:01.667+0000 7f279d7fb700 1 --2- 172.21.15.59:0/1676080828 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f27981026a0 0x7f2798197950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-11T17:46:01.674 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:01.667+0000 7f279e7fd700 1 --2- 172.21.15.59:0/1676080828 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2798103040 0x7f279819e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-11T17:46:01.674 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:01.667+0000 7f279dffc700 1 --2- 172.21.15.59:0/1676080828 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27981014b0 0x7f2798197410 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f2794003820 tx=0x7f2794005ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:46:01.674 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:01.667+0000 7f278effd700 1 -- 172.21.15.59:0/1676080828 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f279401d070 con 0x7f27981014b0 2023-12-11T17:46:01.674 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:01.667+0000 7f278effd700 1 -- 172.21.15.59:0/1676080828 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2794004110 con 0x7f27981014b0 2023-12-11T17:46:01.674 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:01.667+0000 7f278effd700 1 -- 172.21.15.59:0/1676080828 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2794022e40 con 0x7f27981014b0 2023-12-11T17:46:01.674 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:01.667+0000 7f27a0260700 1 -- 172.21.15.59:0/1676080828 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f279819f490 con 0x7f27981014b0 2023-12-11T17:46:01.674 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:01.667+0000 7f27a0260700 1 -- 172.21.15.59:0/1676080828 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f279819f8f0 con 0x7f27981014b0 2023-12-11T17:46:01.674 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:01.668+0000 7f27a0260700 1 -- 172.21.15.59:0/1676080828 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27981088d0 con 0x7f27981014b0 2023-12-11T17:46:01.675 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:01.669+0000 7f278effd700 1 -- 172.21.15.59:0/1676080828 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f2794022570 con 0x7f27981014b0 2023-12-11T17:46:01.675 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:01.670+0000 7f278effd700 1 --2- 172.21.15.59:0/1676080828 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f27840a1510 0x7f27840a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:01.675 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:01.670+0000 7f278effd700 1 -- 172.21.15.59:0/1676080828 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f27940ce520 con 0x7f27981014b0 2023-12-11T17:46:01.676 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:01.670+0000 7f279d7fb700 1 --2- 172.21.15.59:0/1676080828 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f27840a1510 0x7f27840a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:46:01.676 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:01.670+0000 7f279d7fb700 1 --2- 172.21.15.59:0/1676080828 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f27840a1510 0x7f27840a39d0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f2788005fd0 tx=0x7f2788005ee0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:46:01.677 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:01.674+0000 7f278effd700 1 -- 172.21.15.59:0/1676080828 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f27940977b0 con 0x7f27981014b0 2023-12-11T17:46:01.882 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:01.879+0000 7f27a0260700 1 -- 172.21.15.59:0/1676080828 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2798068230 con 0x7f27981014b0 2023-12-11T17:46:01.883 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:01.880+0000 7f278effd700 1 -- 172.21.15.59:0/1676080828 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+24459 (secure 0 0 0) 0x7f279409add0 con 0x7f27981014b0 2023-12-11T17:46:01.884 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-12-11T17:46:01.884 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":88,"fsid":"d0a07eaa-984b-11ee-95a3-87774f69a715","created":"2023-12-11T17:37:14.526539+0000","modified":"2023-12-11T17:45:15.870614+0000","last_up_change":"2023-12-11T17:43:38.535503+0000","last_in_change":"2023-12-11T17:43:21.014888+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-11T17:40:26.596029+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-11T17:44:18.586290+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-11T17:44:19.432952+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-11T17:44:23.097308+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3c0b3218-4b2b-40f8-8bef-ed8cd5633607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":1040759398},{"type":"v1","addr":"172.21.15.59:6803","nonce":1040759398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":1040759398},{"type":"v1","addr":"172.21.15.59:6805","nonce":1040759398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":1040759398},{"type":"v1","addr":"172.21.15.59:6809","nonce":1040759398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":1040759398},{"type":"v1","addr":"172.21.15.59:6807","nonce":1040759398}]},"public_addr":"172.21.15.59:6803/1040759398","cluster_addr":"172.21.15.59:6805/1040759398","heartbeat_back_addr":"172.21.15.59:6809/1040759398","heartbeat_front_addr":"172.21.15.59:6807/1040759398","state":["exists","up"]},{"osd":1,"uuid":"2ead7578-765b-4f5a-a72c-863731ed69e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":2776354245},{"type":"v1","addr":"172.21.15.115:6801","nonce":2776354245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":2776354245},{"type":"v1","addr":"172.21.15.115:6803","nonce":2776354245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":2776354245},{"type":"v1","addr":"172.21.15.115:6807","nonce":2776354245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":2776354245},{"type":"v1","addr":"172.21.15.115:6805","nonce":2776354245}]},"public_addr":"172.21.15.115:6801/2776354245","cluster_addr":"172.21.15.115:6803/2776354245","heartbeat_back_addr":"172.21.15.115:6807/2776354245","heartbeat_front_addr":"172.21.15.115:6805/2776354245","state":["exists","up"]},{"osd":2,"uuid":"8cb1d7ec-53cc-4672-b0b6-70d73797c44f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4287675481},{"type":"v1","addr":"172.21.15.181:6801","nonce":4287675481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4287675481},{"type":"v1","addr":"172.21.15.181:6803","nonce":4287675481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4287675481},{"type":"v1","addr":"172.21.15.181:6807","nonce":4287675481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4287675481},{"type":"v1","addr":"172.21.15.181:6805","nonce":4287675481}]},"public_addr":"172.21.15.181:6801/4287675481","cluster_addr":"172.21.15.181:6803/4287675481","heartbeat_back_addr":"172.21.15.181:6807/4287675481","heartbeat_front_addr":"172.21.15.181:6805/4287675481","state":["exists","up"]},{"osd":3,"uuid":"5d64d9af-831b-4d0a-8ace-a452169351d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":219909995},{"type":"v1","addr":"172.21.15.59:6811","nonce":219909995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":219909995},{"type":"v1","addr":"172.21.15.59:6813","nonce":219909995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":219909995},{"type":"v1","addr":"172.21.15.59:6817","nonce":219909995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":219909995},{"type":"v1","addr":"172.21.15.59:6815","nonce":219909995}]},"public_addr":"172.21.15.59:6811/219909995","cluster_addr":"172.21.15.59:6813/219909995","heartbeat_back_addr":"172.21.15.59:6817/219909995","heartbeat_front_addr":"172.21.15.59:6815/219909995","state":["exists","up"]},{"osd":4,"uuid":"ef8fad28-1db5-4cba-b5f2-10e54e8e1549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":1021473826},{"type":"v1","addr":"172.21.15.115:6809","nonce":1021473826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":1021473826},{"type":"v1","addr":"172.21.15.115:6811","nonce":1021473826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":1021473826},{"type":"v1","addr":"172.21.15.115:6815","nonce":1021473826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":1021473826},{"type":"v1","addr":"172.21.15.115:6813","nonce":1021473826}]},"public_addr":"172.21.15.115:6809/1021473826","cluster_addr":"172.21.15.115:6811/1021473826","heartbeat_back_addr":"172.21.15.115:6815/1021473826","heartbeat_front_addr":"172.21.15.115:6813/1021473826","state":["exists","up"]},{"osd":5,"uuid":"4233a512-9e8f-4db8-aed2-6c3af706e589","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":2405172521},{"type":"v1","addr":"172.21.15.181:6809","nonce":2405172521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":2405172521},{"type":"v1","addr":"172.21.15.181:6811","nonce":2405172521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2405172521},{"type":"v1","addr":"172.21.15.181:6815","nonce":2405172521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":2405172521},{"type":"v1","addr":"172.21.15.181:6813","nonce":2405172521}]},"public_addr":"172.21.15.181:6809/2405172521","cluster_addr":"172.21.15.181:6811/2405172521","heartbeat_back_addr":"172.21.15.181:6815/2405172521","heartbeat_front_addr":"172.21.15.181:6813/2405172521","state":["exists","up"]},{"osd":6,"uuid":"8faab332-016e-4c77-8370-8bef95fc6f34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":815374853},{"type":"v1","addr":"172.21.15.59:6819","nonce":815374853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":815374853},{"type":"v1","addr":"172.21.15.59:6821","nonce":815374853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":815374853},{"type":"v1","addr":"172.21.15.59:6825","nonce":815374853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":815374853},{"type":"v1","addr":"172.21.15.59:6823","nonce":815374853}]},"public_addr":"172.21.15.59:6819/815374853","cluster_addr":"172.21.15.59:6821/815374853","heartbeat_back_addr":"172.21.15.59:6825/815374853","heartbeat_front_addr":"172.21.15.59:6823/815374853","state":["exists","up"]},{"osd":7,"uuid":"f20f470a-0d40-4ed3-8e0e-fd58675cf143","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":1314971870},{"type":"v1","addr":"172.21.15.115:6817","nonce":1314971870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":1314971870},{"type":"v1","addr":"172.21.15.115:6819","nonce":1314971870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":1314971870},{"type":"v1","addr":"172.21.15.115:6823","nonce":1314971870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":1314971870},{"type":"v1","addr":"172.21.15.115:6821","nonce":1314971870}]},"public_addr":"172.21.15.115:6817/1314971870","cluster_addr":"172.21.15.115:6819/1314971870","heartbeat_back_addr":"172.21.15.115:6823/1314971870","heartbeat_front_addr":"172.21.15.115:6821/1314971870","state":["exists","up"]},{"osd":8,"uuid":"eb657d32-1a95-4fd7-a293-d0a970837fc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":3954509873},{"type":"v1","addr":"172.21.15.181:6817","nonce":3954509873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":3954509873},{"type":"v1","addr":"172.21.15.181:6819","nonce":3954509873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":3954509873},{"type":"v1","addr":"172.21.15.181:6823","nonce":3954509873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":3954509873},{"type":"v1","addr":"172.21.15.181:6821","nonce":3954509873}]},"public_addr":"172.21.15.181:6817/3954509873","cluster_addr":"172.21.15.181:6819/3954509873","heartbeat_back_addr":"172.21.15.181:6823/3954509873","heartbeat_front_addr":"172.21.15.181:6821/3954509873","state":["exists","up"]},{"osd":9,"uuid":"8f7717d4-b6f2-4ec1-9b58-62feb0c9a146","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":1163095677},{"type":"v1","addr":"172.21.15.59:6827","nonce":1163095677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":1163095677},{"type":"v1","addr":"172.21.15.59:6829","nonce":1163095677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":1163095677},{"type":"v1","addr":"172.21.15.59:6833","nonce":1163095677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":1163095677},{"type":"v1","addr":"172.21.15.59:6831","nonce":1163095677}]},"public_addr":"172.21.15.59:6827/1163095677","cluster_addr":"172.21.15.59:6829/1163095677","heartbeat_back_addr":"172.21.15.59:6833/1163095677","heartbeat_front_addr":"172.21.15.59:6831/1163095677","state":["exists","up"]},{"osd":10,"uuid":"b75c179a-875d-43bc-b4c1-e8eb80fb64fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":2199610269},{"type":"v1","addr":"172.21.15.115:6825","nonce":2199610269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":2199610269},{"type":"v1","addr":"172.21.15.115:6827","nonce":2199610269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":2199610269},{"type":"v1","addr":"172.21.15.115:6831","nonce":2199610269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":2199610269},{"type":"v1","addr":"172.21.15.115:6829","nonce":2199610269}]},"public_addr":"172.21.15.115:6825/2199610269","cluster_addr":"172.21.15.115:6827/2199610269","heartbeat_back_addr":"172.21.15.115:6831/2199610269","heartbeat_front_addr":"172.21.15.115:6829/2199610269","state":["exists","up"]},{"osd":11,"uuid":"5e141dfe-8a00-4bbc-8204-2dc9be24e18d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":226506989},{"type":"v1","addr":"172.21.15.181:6825","nonce":226506989}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":226506989},{"type":"v1","addr":"172.21.15.181:6827","nonce":226506989}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":226506989},{"type":"v1","addr":"172.21.15.181:6831","nonce":226506989}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":226506989},{"type":"v1","addr":"172.21.15.181:6829","nonce":226506989}]},"public_addr":"172.21.15.181:6825/226506989","cluster_addr":"172.21.15.181:6827/226506989","heartbeat_back_addr":"172.21.15.181:6831/226506989","heartbeat_front_addr":"172.21.15.181:6829/226506989","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:39:42.687084+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:40:03.903448+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:40:24.899908+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:40:46.600251+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:41:07.690037+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:41:28.488855+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:41:49.459274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:42:10.863881+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:42:32.248391+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:42:54.112265+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:43:15.634872+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:43:36.922534+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":7}]},{"pgid":"3.28","mappings":[{"from":5,"to":6},{"from":3,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.59:6800/1259916482":"2023-12-12T17:38:14.037922+0000","172.21.15.59:0/17457774":"2023-12-12T17:38:14.037922+0000","172.21.15.59:6801/1672530462":"2023-12-12T17:37:44.773925+0000","172.21.15.59:0/1053944547":"2023-12-12T17:38:14.037922+0000","172.21.15.59:6801/1259916482":"2023-12-12T17:38:14.037922+0000","172.21.15.59:6800/1672530462":"2023-12-12T17:37:44.773925+0000","172.21.15.59:0/3570767650":"2023-12-12T17:37:44.773925+0000","172.21.15.59:0/3853462899":"2023-12-12T17:37:44.773925+0000","172.21.15.59:0/2859415380":"2023-12-12T17:38:14.037922+0000","172.21.15.59:0/2192742819":"2023-12-12T17:37:44.773925+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-11T17:46:01.886 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:01.883+0000 7f27a0260700 1 -- 172.21.15.59:0/1676080828 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f27840a1510 msgr2=0x7f27840a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:46:01.886 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:01.883+0000 7f27a0260700 1 --2- 172.21.15.59:0/1676080828 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f27840a1510 0x7f27840a39d0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f2788005fd0 tx=0x7f2788005ee0 comp rx=0 tx=0).stop 2023-12-11T17:46:01.887 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:01.883+0000 7f27a0260700 1 -- 172.21.15.59:0/1676080828 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27981014b0 msgr2=0x7f2798197410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:46:01.887 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:01.883+0000 7f27a0260700 1 --2- 172.21.15.59:0/1676080828 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27981014b0 0x7f2798197410 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f2794003820 tx=0x7f2794005ad0 comp rx=0 tx=0).stop 2023-12-11T17:46:01.887 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:01.883+0000 7f27a0260700 1 -- 172.21.15.59:0/1676080828 shutdown_connections 2023-12-11T17:46:01.887 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:01.883+0000 7f27a0260700 1 --2- 172.21.15.59:0/1676080828 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f27840a1510 0x7f27840a39d0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:01.887 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:01.883+0000 7f27a0260700 1 --2- 172.21.15.59:0/1676080828 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2798103040 0x7f279819e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:01.887 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:01.883+0000 7f27a0260700 1 --2- 172.21.15.59:0/1676080828 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27981014b0 0x7f2798197410 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:01.887 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:01.883+0000 7f27a0260700 1 --2- 172.21.15.59:0/1676080828 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f27981026a0 0x7f2798197950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:01.887 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:01.883+0000 7f27a0260700 1 -- 172.21.15.59:0/1676080828 >> 172.21.15.59:0/1676080828 conn(0x7f27980fca20 msgr2=0x7f2798105a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:46:01.887 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:01.883+0000 7f27a0260700 1 -- 172.21.15.59:0/1676080828 shutdown_connections 2023-12-11T17:46:01.887 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:01.883+0000 7f27a0260700 1 -- 172.21.15.59:0/1676080828 wait complete. 2023-12-11T17:46:01.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:01 smithi181 ceph-mon[103627]: pgmap v311: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-12-11T17:46:01.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:01 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/1525683277' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-11T17:46:01.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:01 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/3492812958' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-11T17:46:01.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:01 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/1081546782' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T17:46:01.897 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:01 smithi059 ceph-mon[98847]: pgmap v311: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-12-11T17:46:01.898 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:01 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/1525683277' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-11T17:46:01.898 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:01 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/3492812958' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-11T17:46:01.898 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:01 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/1081546782' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T17:46:01.899 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-11T17:40:26.596029+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-11T17:44:18.586290+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-11T17:44:19.432952+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-12-11T17:44:23.097308+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-12-11T17:46:01.899 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-11T17:46:02.031 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.025+0000 7fa33e07b700 1 -- 172.21.15.59:0/3559452510 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa338102010 msgr2=0x7fa3381088c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:46:02.031 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.025+0000 7fa33e07b700 1 --2- 172.21.15.59:0/3559452510 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa338102010 0x7fa3381088c0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fa33000b0d0 tx=0x7fa33000b3e0 comp rx=0 tx=0).stop 2023-12-11T17:46:02.031 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.025+0000 7fa33e07b700 1 -- 172.21.15.59:0/3559452510 shutdown_connections 2023-12-11T17:46:02.031 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.025+0000 7fa33e07b700 1 --2- 172.21.15.59:0/3559452510 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa338102010 0x7fa3381088c0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:02.032 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.025+0000 7fa33e07b700 1 --2- 172.21.15.59:0/3559452510 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa338100480 0x7fa3381008a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:02.032 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.025+0000 7fa33e07b700 1 --2- 172.21.15.59:0/3559452510 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa338101670 0x7fa338101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:02.032 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.025+0000 7fa33e07b700 1 -- 172.21.15.59:0/3559452510 >> 172.21.15.59:0/3559452510 conn(0x7fa3380fb9f0 msgr2=0x7fa3380fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:46:02.032 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.026+0000 7fa33e07b700 1 -- 172.21.15.59:0/3559452510 shutdown_connections 2023-12-11T17:46:02.032 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.026+0000 7fa33e07b700 1 -- 172.21.15.59:0/3559452510 wait complete. 2023-12-11T17:46:02.032 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.026+0000 7fa33e07b700 1 Processor -- start 2023-12-11T17:46:02.032 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.026+0000 7fa33e07b700 1 -- start start 2023-12-11T17:46:02.032 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.026+0000 7fa33e07b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa338100480 0x7fa338197600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:02.032 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.027+0000 7fa33e07b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa338101670 0x7fa338197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:02.032 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.027+0000 7fa33e07b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa338102010 0x7fa33819eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:02.032 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.027+0000 7fa33e07b700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa33819f130 con 0x7fa338100480 2023-12-11T17:46:02.032 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.027+0000 7fa33e07b700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa33819f270 con 0x7fa338102010 2023-12-11T17:46:02.032 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.027+0000 7fa33e07b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa33819f400 con 0x7fa338101670 2023-12-11T17:46:02.032 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.027+0000 7fa3377fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa338100480 0x7fa338197600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:46:02.032 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.027+0000 7fa3377fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa338100480 0x7fa338197600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34420/0 (socket says 172.21.15.59:34420) 2023-12-11T17:46:02.033 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.027+0000 7fa3377fe700 1 -- 172.21.15.59:0/788625743 learned_addr learned my addr 172.21.15.59:0/788625743 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:46:02.033 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.027+0000 7fa336ffd700 1 --2- 172.21.15.59:0/788625743 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa338101670 0x7fa338197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:46:02.033 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.027+0000 7fa337fff700 1 --2- 172.21.15.59:0/788625743 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa338102010 0x7fa33819eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:46:02.033 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.027+0000 7fa3377fe700 1 -- 172.21.15.59:0/788625743 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa338101670 msgr2=0x7fa338197b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:46:02.033 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.027+0000 7fa3377fe700 1 --2- 172.21.15.59:0/788625743 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa338101670 0x7fa338197b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:02.033 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.027+0000 7fa3377fe700 1 -- 172.21.15.59:0/788625743 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa338102010 msgr2=0x7fa33819eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:46:02.033 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.027+0000 7fa3377fe700 1 --2- 172.21.15.59:0/788625743 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa338102010 0x7fa33819eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:02.033 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.027+0000 7fa3377fe700 1 -- 172.21.15.59:0/788625743 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa330009d00 con 0x7fa338100480 2023-12-11T17:46:02.033 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.027+0000 7fa336ffd700 1 --2- 172.21.15.59:0/788625743 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa338101670 0x7fa338197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T17:46:02.033 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.027+0000 7fa3377fe700 1 --2- 172.21.15.59:0/788625743 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa338100480 0x7fa338197600 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fa32800e970 tx=0x7fa32800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:46:02.033 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.027+0000 7fa334ff9700 1 -- 172.21.15.59:0/788625743 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa328009790 con 0x7fa338100480 2023-12-11T17:46:02.033 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.027+0000 7fa334ff9700 1 -- 172.21.15.59:0/788625743 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa3280044e0 con 0x7fa338100480 2023-12-11T17:46:02.033 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.027+0000 7fa33e07b700 1 -- 172.21.15.59:0/788625743 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa33819f6e0 con 0x7fa338100480 2023-12-11T17:46:02.033 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.027+0000 7fa334ff9700 1 -- 172.21.15.59:0/788625743 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa328003e30 con 0x7fa338100480 2023-12-11T17:46:02.034 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.027+0000 7fa33e07b700 1 -- 172.21.15.59:0/788625743 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa33819fbf0 con 0x7fa338100480 2023-12-11T17:46:02.034 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:01 smithi115 ceph-mon[102748]: pgmap v311: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-12-11T17:46:02.034 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:01 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/1525683277' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-11T17:46:02.034 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:01 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/3492812958' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-11T17:46:02.034 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:01 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/1081546782' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T17:46:02.035 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.028+0000 7fa33e07b700 1 -- 172.21.15.59:0/788625743 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa338068310 con 0x7fa338100480 2023-12-11T17:46:02.035 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.030+0000 7fa334ff9700 1 -- 172.21.15.59:0/788625743 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7fa328003f90 con 0x7fa338100480 2023-12-11T17:46:02.035 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.030+0000 7fa334ff9700 1 --2- 172.21.15.59:0/788625743 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fa3200a1510 0x7fa3200a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:02.035 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.030+0000 7fa336ffd700 1 --2- 172.21.15.59:0/788625743 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fa3200a1510 0x7fa3200a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:46:02.035 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.030+0000 7fa334ff9700 1 -- 172.21.15.59:0/788625743 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fa328014070 con 0x7fa338100480 2023-12-11T17:46:02.035 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.031+0000 7fa336ffd700 1 --2- 172.21.15.59:0/788625743 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fa3200a1510 0x7fa3200a39d0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fa32c009cb0 tx=0x7fa32c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:46:02.037 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.034+0000 7fa334ff9700 1 -- 172.21.15.59:0/788625743 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fa328096840 con 0x7fa338100480 2023-12-11T17:46:02.239 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.236+0000 7fa33e07b700 1 -- 172.21.15.59:0/788625743 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa33819ff70 con 0x7fa338100480 2023-12-11T17:46:02.239 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.236+0000 7fa334ff9700 1 -- 172.21.15.59:0/788625743 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa328099e60 con 0x7fa338100480 2023-12-11T17:46:02.240 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2023-12-11T17:46:02.242 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.238+0000 7fa33e07b700 1 -- 172.21.15.59:0/788625743 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fa3200a1510 msgr2=0x7fa3200a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:46:02.242 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.238+0000 7fa33e07b700 1 --2- 172.21.15.59:0/788625743 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fa3200a1510 0x7fa3200a39d0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fa32c009cb0 tx=0x7fa32c009260 comp rx=0 tx=0).stop 2023-12-11T17:46:02.242 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.238+0000 7fa33e07b700 1 -- 172.21.15.59:0/788625743 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa338100480 msgr2=0x7fa338197600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:46:02.242 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.238+0000 7fa33e07b700 1 --2- 172.21.15.59:0/788625743 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa338100480 0x7fa338197600 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fa32800e970 tx=0x7fa32800ed30 comp rx=0 tx=0).stop 2023-12-11T17:46:02.242 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.238+0000 7fa33e07b700 1 -- 172.21.15.59:0/788625743 shutdown_connections 2023-12-11T17:46:02.242 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.238+0000 7fa33e07b700 1 --2- 172.21.15.59:0/788625743 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fa3200a1510 0x7fa3200a39d0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:02.242 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.238+0000 7fa33e07b700 1 --2- 172.21.15.59:0/788625743 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa338102010 0x7fa33819eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:02.242 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.238+0000 7fa33e07b700 1 --2- 172.21.15.59:0/788625743 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa338100480 0x7fa338197600 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:02.242 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.238+0000 7fa33e07b700 1 --2- 172.21.15.59:0/788625743 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa338101670 0x7fa338197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:02.242 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.238+0000 7fa33e07b700 1 -- 172.21.15.59:0/788625743 >> 172.21.15.59:0/788625743 conn(0x7fa3380fb9f0 msgr2=0x7fa338068610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:46:02.242 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.238+0000 7fa33e07b700 1 -- 172.21.15.59:0/788625743 shutdown_connections 2023-12-11T17:46:02.242 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.238+0000 7fa33e07b700 1 -- 172.21.15.59:0/788625743 wait complete. 2023-12-11T17:46:02.254 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-11T17:46:02.384 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.379+0000 7efef82f0700 1 -- 172.21.15.59:0/4134482635 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efef0102010 msgr2=0x7efef01088c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:46:02.385 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.379+0000 7efef82f0700 1 --2- 172.21.15.59:0/4134482635 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efef0102010 0x7efef01088c0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7efee800b0d0 tx=0x7efee800b3e0 comp rx=0 tx=0).stop 2023-12-11T17:46:02.385 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.379+0000 7efef82f0700 1 -- 172.21.15.59:0/4134482635 shutdown_connections 2023-12-11T17:46:02.385 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.379+0000 7efef82f0700 1 --2- 172.21.15.59:0/4134482635 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efef0102010 0x7efef01088c0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:02.385 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.379+0000 7efef82f0700 1 --2- 172.21.15.59:0/4134482635 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efef0100480 0x7efef01008a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:02.385 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.379+0000 7efef82f0700 1 --2- 172.21.15.59:0/4134482635 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efef0101670 0x7efef0101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:02.385 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.379+0000 7efef82f0700 1 -- 172.21.15.59:0/4134482635 >> 172.21.15.59:0/4134482635 conn(0x7efef00fb9f0 msgr2=0x7efef00fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:46:02.385 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.379+0000 7efef82f0700 1 -- 172.21.15.59:0/4134482635 shutdown_connections 2023-12-11T17:46:02.385 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.379+0000 7efef82f0700 1 -- 172.21.15.59:0/4134482635 wait complete. 2023-12-11T17:46:02.385 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.380+0000 7efef82f0700 1 Processor -- start 2023-12-11T17:46:02.385 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.380+0000 7efef82f0700 1 -- start start 2023-12-11T17:46:02.385 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.380+0000 7efef82f0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efef0100480 0x7efef01043c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:02.385 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.380+0000 7efef82f0700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efef0101670 0x7efef0102a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:02.386 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.380+0000 7efef82f0700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efef0102010 0x7efef0102f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:02.386 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.380+0000 7efef82f0700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efef0103730 con 0x7efef0101670 2023-12-11T17:46:02.386 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.380+0000 7efef82f0700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efef01038a0 con 0x7efef0102010 2023-12-11T17:46:02.386 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.380+0000 7efef82f0700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efef01a1600 con 0x7efef0100480 2023-12-11T17:46:02.386 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.380+0000 7efef588b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efef0101670 0x7efef0102a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:46:02.386 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.380+0000 7efef588b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efef0101670 0x7efef0102a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34436/0 (socket says 172.21.15.59:34436) 2023-12-11T17:46:02.386 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.380+0000 7efef588b700 1 -- 172.21.15.59:0/943303263 learned_addr learned my addr 172.21.15.59:0/943303263 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:46:02.386 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.380+0000 7efef608c700 1 --2- 172.21.15.59:0/943303263 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efef0100480 0x7efef01043c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:46:02.386 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.380+0000 7efef688d700 1 --2- 172.21.15.59:0/943303263 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efef0102010 0x7efef0102f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:46:02.386 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.380+0000 7efef588b700 1 -- 172.21.15.59:0/943303263 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efef0100480 msgr2=0x7efef01043c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:46:02.386 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.380+0000 7efef588b700 1 --2- 172.21.15.59:0/943303263 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efef0100480 0x7efef01043c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:02.386 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.380+0000 7efef588b700 1 -- 172.21.15.59:0/943303263 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efef0102010 msgr2=0x7efef0102f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:46:02.386 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.380+0000 7efef588b700 1 --2- 172.21.15.59:0/943303263 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efef0102010 0x7efef0102f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:02.386 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.380+0000 7efef588b700 1 -- 172.21.15.59:0/943303263 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efee8009d00 con 0x7efef0101670 2023-12-11T17:46:02.387 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.380+0000 7efef688d700 1 --2- 172.21.15.59:0/943303263 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efef0102010 0x7efef0102f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T17:46:02.387 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.380+0000 7efef588b700 1 --2- 172.21.15.59:0/943303263 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efef0101670 0x7efef0102a50 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7efeec00e9a0 tx=0x7efeec00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:46:02.387 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.381+0000 7efee77fe700 1 -- 172.21.15.59:0/943303263 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeec009790 con 0x7efef0101670 2023-12-11T17:46:02.387 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.381+0000 7efef82f0700 1 -- 172.21.15.59:0/943303263 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efef01a1890 con 0x7efef0101670 2023-12-11T17:46:02.387 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.381+0000 7efee77fe700 1 -- 172.21.15.59:0/943303263 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efeec00cb70 con 0x7efef0101670 2023-12-11T17:46:02.387 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.381+0000 7efee77fe700 1 -- 172.21.15.59:0/943303263 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeec010ea0 con 0x7efef0101670 2023-12-11T17:46:02.387 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.381+0000 7efef82f0700 1 -- 172.21.15.59:0/943303263 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efef01a1da0 con 0x7efef0101670 2023-12-11T17:46:02.388 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.382+0000 7efef82f0700 1 -- 172.21.15.59:0/943303263 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efef0068310 con 0x7efef0101670 2023-12-11T17:46:02.388 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.383+0000 7efee77fe700 1 -- 172.21.15.59:0/943303263 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7efeec00cce0 con 0x7efef0101670 2023-12-11T17:46:02.388 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.383+0000 7efee77fe700 1 --2- 172.21.15.59:0/943303263 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7efedc0a1510 0x7efedc0a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:02.388 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.383+0000 7efef608c700 1 --2- 172.21.15.59:0/943303263 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7efedc0a1510 0x7efedc0a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:46:02.388 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.383+0000 7efee77fe700 1 -- 172.21.15.59:0/943303263 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8999+0+0 (secure 0 0 0) 0x7efeec014070 con 0x7efef0101670 2023-12-11T17:46:02.388 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.384+0000 7efef608c700 1 --2- 172.21.15.59:0/943303263 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7efedc0a1510 0x7efedc0a39d0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7efef01014d0 tx=0x7efee000a3b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:46:02.391 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.388+0000 7efee77fe700 1 -- 172.21.15.59:0/943303263 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7efeec0522f0 con 0x7efef0101670 2023-12-11T17:46:02.594 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.590+0000 7efef82f0700 1 -- 172.21.15.59:0/943303263 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7efef01a20c0 con 0x7efef0101670 2023-12-11T17:46:02.594 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.590+0000 7efee77fe700 1 -- 172.21.15.59:0/943303263 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7efeec0965c0 con 0x7efef0101670 2023-12-11T17:46:02.594 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2023-12-11T17:46:02.596 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.593+0000 7efef82f0700 1 -- 172.21.15.59:0/943303263 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7efedc0a1510 msgr2=0x7efedc0a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:46:02.597 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.593+0000 7efef82f0700 1 --2- 172.21.15.59:0/943303263 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7efedc0a1510 0x7efedc0a39d0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7efef01014d0 tx=0x7efee000a3b0 comp rx=0 tx=0).stop 2023-12-11T17:46:02.597 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.593+0000 7efef82f0700 1 -- 172.21.15.59:0/943303263 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efef0101670 msgr2=0x7efef0102a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:46:02.597 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.593+0000 7efef82f0700 1 --2- 172.21.15.59:0/943303263 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efef0101670 0x7efef0102a50 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7efeec00e9a0 tx=0x7efeec00ecb0 comp rx=0 tx=0).stop 2023-12-11T17:46:02.597 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.593+0000 7efef82f0700 1 -- 172.21.15.59:0/943303263 shutdown_connections 2023-12-11T17:46:02.597 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.593+0000 7efef82f0700 1 --2- 172.21.15.59:0/943303263 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7efedc0a1510 0x7efedc0a39d0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:02.597 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.593+0000 7efef82f0700 1 --2- 172.21.15.59:0/943303263 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efef0102010 0x7efef0102f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:02.597 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.593+0000 7efef82f0700 1 --2- 172.21.15.59:0/943303263 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efef0100480 0x7efef01043c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:02.597 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.593+0000 7efef82f0700 1 --2- 172.21.15.59:0/943303263 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efef0101670 0x7efef0102a50 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:02.597 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.593+0000 7efef82f0700 1 -- 172.21.15.59:0/943303263 >> 172.21.15.59:0/943303263 conn(0x7efef00fb9f0 msgr2=0x7efef00fd410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:46:02.597 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.593+0000 7efef82f0700 1 -- 172.21.15.59:0/943303263 shutdown_connections 2023-12-11T17:46:02.598 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.593+0000 7efef82f0700 1 -- 172.21.15.59:0/943303263 wait complete. 2023-12-11T17:46:02.609 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-11T17:46:02.740 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.734+0000 7fa994edd700 1 -- 172.21.15.59:0/3314786851 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa990100480 msgr2=0x7fa9901008a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:46:02.740 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.734+0000 7fa994edd700 1 --2- 172.21.15.59:0/3314786851 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa990100480 0x7fa9901008a0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fa9840099c0 tx=0x7fa984009cd0 comp rx=0 tx=0).stop 2023-12-11T17:46:02.740 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.734+0000 7fa994edd700 1 -- 172.21.15.59:0/3314786851 shutdown_connections 2023-12-11T17:46:02.740 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.734+0000 7fa994edd700 1 --2- 172.21.15.59:0/3314786851 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa990102010 0x7fa9901088c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:02.740 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.734+0000 7fa994edd700 1 --2- 172.21.15.59:0/3314786851 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa990100480 0x7fa9901008a0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:02.740 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.734+0000 7fa994edd700 1 --2- 172.21.15.59:0/3314786851 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa990101670 0x7fa990101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:02.740 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.734+0000 7fa994edd700 1 -- 172.21.15.59:0/3314786851 >> 172.21.15.59:0/3314786851 conn(0x7fa9900fb9f0 msgr2=0x7fa9900fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:46:02.740 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.734+0000 7fa994edd700 1 -- 172.21.15.59:0/3314786851 shutdown_connections 2023-12-11T17:46:02.741 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.734+0000 7fa994edd700 1 -- 172.21.15.59:0/3314786851 wait complete. 2023-12-11T17:46:02.741 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.735+0000 7fa994edd700 1 Processor -- start 2023-12-11T17:46:02.741 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.735+0000 7fa994edd700 1 -- start start 2023-12-11T17:46:02.741 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.735+0000 7fa994edd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa990100480 0x7fa9901963a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:02.741 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.735+0000 7fa994edd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa990101670 0x7fa9901968e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:02.741 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.735+0000 7fa994edd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa990102010 0x7fa99019d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:02.741 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.735+0000 7fa994edd700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa99019ded0 con 0x7fa990101670 2023-12-11T17:46:02.741 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.735+0000 7fa994edd700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa99019e010 con 0x7fa990102010 2023-12-11T17:46:02.741 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.735+0000 7fa994edd700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa99019e1a0 con 0x7fa990100480 2023-12-11T17:46:02.741 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.735+0000 7fa98dd9b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa990101670 0x7fa9901968e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:46:02.741 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.735+0000 7fa98dd9b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa990101670 0x7fa9901968e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34450/0 (socket says 172.21.15.59:34450) 2023-12-11T17:46:02.741 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.735+0000 7fa98dd9b700 1 -- 172.21.15.59:0/2473727919 learned_addr learned my addr 172.21.15.59:0/2473727919 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:46:02.741 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.735+0000 7fa98e59c700 1 --2- 172.21.15.59:0/2473727919 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa990100480 0x7fa9901963a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:46:02.741 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.736+0000 7fa98ed9d700 1 --2- 172.21.15.59:0/2473727919 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa990102010 0x7fa99019d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:46:02.742 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.736+0000 7fa98dd9b700 1 -- 172.21.15.59:0/2473727919 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa990100480 msgr2=0x7fa9901963a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:46:02.742 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.736+0000 7fa98dd9b700 1 --2- 172.21.15.59:0/2473727919 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa990100480 0x7fa9901963a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:02.742 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.736+0000 7fa98dd9b700 1 -- 172.21.15.59:0/2473727919 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa990102010 msgr2=0x7fa99019d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:46:02.742 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.736+0000 7fa98dd9b700 1 --2- 172.21.15.59:0/2473727919 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa990102010 0x7fa99019d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:02.742 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.736+0000 7fa98dd9b700 1 -- 172.21.15.59:0/2473727919 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9840095f0 con 0x7fa990101670 2023-12-11T17:46:02.742 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.736+0000 7fa98ed9d700 1 --2- 172.21.15.59:0/2473727919 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa990102010 0x7fa99019d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T17:46:02.742 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.736+0000 7fa98dd9b700 1 --2- 172.21.15.59:0/2473727919 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa990101670 0x7fa9901968e0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fa98000d950 tx=0x7fa98000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:46:02.742 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.736+0000 7fa97f7fe700 1 -- 172.21.15.59:0/2473727919 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa980009940 con 0x7fa990101670 2023-12-11T17:46:02.742 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.736+0000 7fa994edd700 1 -- 172.21.15.59:0/2473727919 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa99019e480 con 0x7fa990101670 2023-12-11T17:46:02.742 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.736+0000 7fa97f7fe700 1 -- 172.21.15.59:0/2473727919 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa980010460 con 0x7fa990101670 2023-12-11T17:46:02.742 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.736+0000 7fa97f7fe700 1 -- 172.21.15.59:0/2473727919 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa98000f620 con 0x7fa990101670 2023-12-11T17:46:02.742 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.736+0000 7fa994edd700 1 -- 172.21.15.59:0/2473727919 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa99019e990 con 0x7fa990101670 2023-12-11T17:46:02.743 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.737+0000 7fa994edd700 1 -- 172.21.15.59:0/2473727919 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9901078b0 con 0x7fa990101670 2023-12-11T17:46:02.743 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.738+0000 7fa97f7fe700 1 -- 172.21.15.59:0/2473727919 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7fa9800108d0 con 0x7fa990101670 2023-12-11T17:46:02.744 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.739+0000 7fa97f7fe700 1 --2- 172.21.15.59:0/2473727919 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fa9780a1510 0x7fa9780a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:02.744 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.739+0000 7fa97f7fe700 1 -- 172.21.15.59:0/2473727919 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fa9800cd080 con 0x7fa990101670 2023-12-11T17:46:02.744 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.739+0000 7fa98e59c700 1 --2- 172.21.15.59:0/2473727919 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fa9780a1510 0x7fa9780a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:46:02.744 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.739+0000 7fa98e59c700 1 --2- 172.21.15.59:0/2473727919 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fa9780a1510 0x7fa9780a39d0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fa984003820 tx=0x7fa984005920 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:46:02.745 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.742+0000 7fa97f7fe700 1 -- 172.21.15.59:0/2473727919 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fa980096220 con 0x7fa990101670 2023-12-11T17:46:02.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:02 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/1676080828' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-11T17:46:02.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:02 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/788625743' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-11T17:46:02.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:02 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/1676080828' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-11T17:46:02.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:02 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/788625743' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-11T17:46:02.946 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.942+0000 7fa994edd700 1 -- 172.21.15.59:0/2473727919 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa99010bdb0 con 0x7fa990101670 2023-12-11T17:46:02.946 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.943+0000 7fa97f7fe700 1 -- 172.21.15.59:0/2473727919 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa980016020 con 0x7fa990101670 2023-12-11T17:46:02.947 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2023-12-11T17:46:02.949 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.945+0000 7fa994edd700 1 -- 172.21.15.59:0/2473727919 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fa9780a1510 msgr2=0x7fa9780a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:46:02.949 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.945+0000 7fa994edd700 1 --2- 172.21.15.59:0/2473727919 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fa9780a1510 0x7fa9780a39d0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fa984003820 tx=0x7fa984005920 comp rx=0 tx=0).stop 2023-12-11T17:46:02.949 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.945+0000 7fa994edd700 1 -- 172.21.15.59:0/2473727919 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa990101670 msgr2=0x7fa9901968e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:46:02.949 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.945+0000 7fa994edd700 1 --2- 172.21.15.59:0/2473727919 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa990101670 0x7fa9901968e0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fa98000d950 tx=0x7fa98000dc60 comp rx=0 tx=0).stop 2023-12-11T17:46:02.949 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.945+0000 7fa994edd700 1 -- 172.21.15.59:0/2473727919 shutdown_connections 2023-12-11T17:46:02.949 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.945+0000 7fa994edd700 1 --2- 172.21.15.59:0/2473727919 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fa9780a1510 0x7fa9780a39d0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:02.950 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.945+0000 7fa994edd700 1 --2- 172.21.15.59:0/2473727919 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa990102010 0x7fa99019d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:02.950 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.945+0000 7fa994edd700 1 --2- 172.21.15.59:0/2473727919 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa990100480 0x7fa9901963a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:02.950 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.945+0000 7fa994edd700 1 --2- 172.21.15.59:0/2473727919 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa990101670 0x7fa9901968e0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:02.950 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.945+0000 7fa994edd700 1 -- 172.21.15.59:0/2473727919 >> 172.21.15.59:0/2473727919 conn(0x7fa9900fb9f0 msgr2=0x7fa9901049f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:46:02.950 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.945+0000 7fa994edd700 1 -- 172.21.15.59:0/2473727919 shutdown_connections 2023-12-11T17:46:02.950 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:02.945+0000 7fa994edd700 1 -- 172.21.15.59:0/2473727919 wait complete. 2023-12-11T17:46:02.961 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-12-11T17:46:03.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:02 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/1676080828' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-11T17:46:03.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:02 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/788625743' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-11T17:46:03.092 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.086+0000 7fa2a3414700 1 -- 172.21.15.59:0/2708704232 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa29c102010 msgr2=0x7fa29c1088c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:46:03.092 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.086+0000 7fa2a3414700 1 --2- 172.21.15.59:0/2708704232 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa29c102010 0x7fa29c1088c0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fa29400b0d0 tx=0x7fa29400b3e0 comp rx=0 tx=0).stop 2023-12-11T17:46:03.092 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.086+0000 7fa2a3414700 1 -- 172.21.15.59:0/2708704232 shutdown_connections 2023-12-11T17:46:03.092 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.086+0000 7fa2a3414700 1 --2- 172.21.15.59:0/2708704232 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa29c102010 0x7fa29c1088c0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:03.092 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.086+0000 7fa2a3414700 1 --2- 172.21.15.59:0/2708704232 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa29c100480 0x7fa29c1008a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:03.092 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.086+0000 7fa2a3414700 1 --2- 172.21.15.59:0/2708704232 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa29c101670 0x7fa29c101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:03.092 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.086+0000 7fa2a3414700 1 -- 172.21.15.59:0/2708704232 >> 172.21.15.59:0/2708704232 conn(0x7fa29c0fb9f0 msgr2=0x7fa29c0fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:46:03.092 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.086+0000 7fa2a3414700 1 -- 172.21.15.59:0/2708704232 shutdown_connections 2023-12-11T17:46:03.092 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.086+0000 7fa2a3414700 1 -- 172.21.15.59:0/2708704232 wait complete. 2023-12-11T17:46:03.092 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.087+0000 7fa2a3414700 1 Processor -- start 2023-12-11T17:46:03.092 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.087+0000 7fa2a3414700 1 -- start start 2023-12-11T17:46:03.093 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.087+0000 7fa2a3414700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa29c100480 0x7fa29c1964d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:03.093 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.087+0000 7fa2a3414700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa29c101670 0x7fa29c196a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:03.093 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.087+0000 7fa2a3414700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa29c102010 0x7fa29c19da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:03.093 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.087+0000 7fa2a3414700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa29c19e000 con 0x7fa29c101670 2023-12-11T17:46:03.093 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.087+0000 7fa2a3414700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa29c19e140 con 0x7fa29c102010 2023-12-11T17:46:03.093 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.087+0000 7fa2a3414700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa29c19e2d0 con 0x7fa29c100480 2023-12-11T17:46:03.093 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.087+0000 7fa2a09af700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa29c101670 0x7fa29c196a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:46:03.093 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.087+0000 7fa2a09af700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa29c101670 0x7fa29c196a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34464/0 (socket says 172.21.15.59:34464) 2023-12-11T17:46:03.093 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.087+0000 7fa2a09af700 1 -- 172.21.15.59:0/1373855869 learned_addr learned my addr 172.21.15.59:0/1373855869 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:46:03.093 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.087+0000 7fa2a11b0700 1 --2- 172.21.15.59:0/1373855869 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa29c100480 0x7fa29c1964d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:46:03.093 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.087+0000 7fa2a19b1700 1 --2- 172.21.15.59:0/1373855869 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa29c102010 0x7fa29c19da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:46:03.093 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.087+0000 7fa2a09af700 1 -- 172.21.15.59:0/1373855869 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa29c100480 msgr2=0x7fa29c1964d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:46:03.093 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.087+0000 7fa2a09af700 1 --2- 172.21.15.59:0/1373855869 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa29c100480 0x7fa29c1964d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:03.093 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.087+0000 7fa2a09af700 1 -- 172.21.15.59:0/1373855869 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa29c102010 msgr2=0x7fa29c19da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:46:03.094 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.087+0000 7fa2a09af700 1 --2- 172.21.15.59:0/1373855869 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa29c102010 0x7fa29c19da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:03.094 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.087+0000 7fa2a09af700 1 -- 172.21.15.59:0/1373855869 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa294009d00 con 0x7fa29c101670 2023-12-11T17:46:03.094 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.088+0000 7fa2a09af700 1 --2- 172.21.15.59:0/1373855869 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa29c101670 0x7fa29c196a10 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fa29800d950 tx=0x7fa29800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:46:03.094 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.088+0000 7fa28e7fc700 1 -- 172.21.15.59:0/1373855869 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa29800f890 con 0x7fa29c101670 2023-12-11T17:46:03.094 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.088+0000 7fa28e7fc700 1 -- 172.21.15.59:0/1373855869 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa298010460 con 0x7fa29c101670 2023-12-11T17:46:03.094 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.088+0000 7fa2a3414700 1 -- 172.21.15.59:0/1373855869 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa29c19e5b0 con 0x7fa29c101670 2023-12-11T17:46:03.094 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.088+0000 7fa28e7fc700 1 -- 172.21.15.59:0/1373855869 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa29800e610 con 0x7fa29c101670 2023-12-11T17:46:03.094 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.088+0000 7fa2a3414700 1 -- 172.21.15.59:0/1373855869 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa29c19eb10 con 0x7fa29c101670 2023-12-11T17:46:03.095 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.089+0000 7fa2a3414700 1 -- 172.21.15.59:0/1373855869 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa29c068310 con 0x7fa29c101670 2023-12-11T17:46:03.095 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.090+0000 7fa28e7fc700 1 -- 172.21.15.59:0/1373855869 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7fa2980108d0 con 0x7fa29c101670 2023-12-11T17:46:03.095 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.091+0000 7fa28e7fc700 1 --2- 172.21.15.59:0/1373855869 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fa2880a1590 0x7fa2880a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:03.095 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.091+0000 7fa2a11b0700 1 --2- 172.21.15.59:0/1373855869 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fa2880a1590 0x7fa2880a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:46:03.095 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.091+0000 7fa28e7fc700 1 -- 172.21.15.59:0/1373855869 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fa2980ccee0 con 0x7fa29c101670 2023-12-11T17:46:03.095 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.091+0000 7fa2a11b0700 1 --2- 172.21.15.59:0/1373855869 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fa2880a1590 0x7fa2880a3a50 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fa29c1014d0 tx=0x7fa29000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:46:03.097 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.094+0000 7fa28e7fc700 1 -- 172.21.15.59:0/1373855869 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fa2980960e0 con 0x7fa29c101670 2023-12-11T17:46:03.301 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.297+0000 7fa2a3414700 1 -- 172.21.15.59:0/1373855869 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fa29c107310 con 0x7fa29c101670 2023-12-11T17:46:03.301 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.298+0000 7fa28e7fc700 1 -- 172.21.15.59:0/1373855869 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v88) v1 ==== 103+0+11 (secure 0 0 0) 0x7fa2980200a0 con 0x7fa29c101670 2023-12-11T17:46:03.301 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2023-12-11T17:46:03.303 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.300+0000 7fa2a3414700 1 -- 172.21.15.59:0/1373855869 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fa2880a1590 msgr2=0x7fa2880a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:46:03.303 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.300+0000 7fa2a3414700 1 --2- 172.21.15.59:0/1373855869 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fa2880a1590 0x7fa2880a3a50 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fa29c1014d0 tx=0x7fa29000b410 comp rx=0 tx=0).stop 2023-12-11T17:46:03.304 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.300+0000 7fa2a3414700 1 -- 172.21.15.59:0/1373855869 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa29c101670 msgr2=0x7fa29c196a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:46:03.304 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.300+0000 7fa2a3414700 1 --2- 172.21.15.59:0/1373855869 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa29c101670 0x7fa29c196a10 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fa29800d950 tx=0x7fa29800dc60 comp rx=0 tx=0).stop 2023-12-11T17:46:03.304 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.300+0000 7fa2a3414700 1 -- 172.21.15.59:0/1373855869 shutdown_connections 2023-12-11T17:46:03.304 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.300+0000 7fa2a3414700 1 --2- 172.21.15.59:0/1373855869 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fa2880a1590 0x7fa2880a3a50 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:03.304 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.300+0000 7fa2a3414700 1 --2- 172.21.15.59:0/1373855869 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa29c102010 0x7fa29c19da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:03.304 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.300+0000 7fa2a3414700 1 --2- 172.21.15.59:0/1373855869 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa29c100480 0x7fa29c1964d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:03.304 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.300+0000 7fa2a3414700 1 --2- 172.21.15.59:0/1373855869 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa29c101670 0x7fa29c196a10 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:03.304 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.300+0000 7fa2a3414700 1 -- 172.21.15.59:0/1373855869 >> 172.21.15.59:0/1373855869 conn(0x7fa29c0fb9f0 msgr2=0x7fa29c104970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:46:03.304 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.300+0000 7fa2a3414700 1 -- 172.21.15.59:0/1373855869 shutdown_connections 2023-12-11T17:46:03.304 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.300+0000 7fa2a3414700 1 -- 172.21.15.59:0/1373855869 wait complete. 2023-12-11T17:46:03.316 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-11T17:46:03.444 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.437+0000 7f1ae858a700 1 -- 172.21.15.59:0/3583911880 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ae0101570 msgr2=0x7f1ae01019d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:46:03.444 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.437+0000 7f1ae858a700 1 --2- 172.21.15.59:0/3583911880 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ae0101570 0x7f1ae01019d0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f1ad00099c0 tx=0x7f1ad0009cd0 comp rx=0 tx=0).stop 2023-12-11T17:46:03.444 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.438+0000 7f1ae858a700 1 -- 172.21.15.59:0/3583911880 shutdown_connections 2023-12-11T17:46:03.444 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.438+0000 7f1ae858a700 1 --2- 172.21.15.59:0/3583911880 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ae0101f10 0x7f1ae01087c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:03.444 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.438+0000 7f1ae858a700 1 --2- 172.21.15.59:0/3583911880 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1ae0100380 0x7f1ae01007a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:03.444 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.438+0000 7f1ae858a700 1 --2- 172.21.15.59:0/3583911880 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ae0101570 0x7f1ae01019d0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:03.444 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.438+0000 7f1ae858a700 1 -- 172.21.15.59:0/3583911880 >> 172.21.15.59:0/3583911880 conn(0x7f1ae00fb8f0 msgr2=0x7f1ae00fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:46:03.444 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.438+0000 7f1ae858a700 1 -- 172.21.15.59:0/3583911880 shutdown_connections 2023-12-11T17:46:03.444 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.438+0000 7f1ae858a700 1 -- 172.21.15.59:0/3583911880 wait complete. 2023-12-11T17:46:03.444 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.438+0000 7f1ae858a700 1 Processor -- start 2023-12-11T17:46:03.445 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.438+0000 7f1ae858a700 1 -- start start 2023-12-11T17:46:03.445 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.439+0000 7f1ae858a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ae0100380 0x7f1ae0069190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:03.445 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.439+0000 7f1ae858a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ae0101570 0x7f1ae019c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:03.445 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.439+0000 7f1ae858a700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1ae0101f10 0x7f1ae019e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:03.445 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.439+0000 7f1ae858a700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ae019ee80 con 0x7f1ae0101570 2023-12-11T17:46:03.445 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.439+0000 7f1ae858a700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ae019eff0 con 0x7f1ae0101f10 2023-12-11T17:46:03.445 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.439+0000 7f1ae858a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ae019f1b0 con 0x7f1ae0100380 2023-12-11T17:46:03.445 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.439+0000 7f1ae5b25700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ae0101570 0x7f1ae019c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:46:03.445 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.439+0000 7f1ae6326700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ae0100380 0x7f1ae0069190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:46:03.445 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.439+0000 7f1ae5b25700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ae0101570 0x7f1ae019c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34478/0 (socket says 172.21.15.59:34478) 2023-12-11T17:46:03.445 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.439+0000 7f1ae5b25700 1 -- 172.21.15.59:0/3733923019 learned_addr learned my addr 172.21.15.59:0/3733923019 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:46:03.445 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.439+0000 7f1ae6b27700 1 --2- 172.21.15.59:0/3733923019 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1ae0101f10 0x7f1ae019e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:46:03.445 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.439+0000 7f1ae5b25700 1 -- 172.21.15.59:0/3733923019 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ae0100380 msgr2=0x7f1ae0069190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:46:03.445 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.439+0000 7f1ae5b25700 1 --2- 172.21.15.59:0/3733923019 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ae0100380 0x7f1ae0069190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:03.446 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.439+0000 7f1ae5b25700 1 -- 172.21.15.59:0/3733923019 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1ae0101f10 msgr2=0x7f1ae019e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:46:03.446 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.439+0000 7f1ae5b25700 1 --2- 172.21.15.59:0/3733923019 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1ae0101f10 0x7f1ae019e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:03.446 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.439+0000 7f1ae5b25700 1 -- 172.21.15.59:0/3733923019 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ad00095f0 con 0x7f1ae0101570 2023-12-11T17:46:03.446 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.439+0000 7f1ae6b27700 1 --2- 172.21.15.59:0/3733923019 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1ae0101f10 0x7f1ae019e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T17:46:03.446 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.439+0000 7f1ae5b25700 1 --2- 172.21.15.59:0/3733923019 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ae0101570 0x7f1ae019c090 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f1ad0003730 tx=0x7f1ad000f670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:46:03.446 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.439+0000 7f1ad77fe700 1 -- 172.21.15.59:0/3733923019 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ad001d070 con 0x7f1ae0101570 2023-12-11T17:46:03.446 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.440+0000 7f1ad77fe700 1 -- 172.21.15.59:0/3733923019 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ad00059a0 con 0x7f1ae0101570 2023-12-11T17:46:03.446 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.440+0000 7f1ae858a700 1 -- 172.21.15.59:0/3733923019 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ae019f430 con 0x7f1ae0101570 2023-12-11T17:46:03.446 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.440+0000 7f1ad77fe700 1 -- 172.21.15.59:0/3733923019 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ad0004a70 con 0x7f1ae0101570 2023-12-11T17:46:03.446 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.440+0000 7f1ae858a700 1 -- 172.21.15.59:0/3733923019 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ae019f930 con 0x7f1ae0101570 2023-12-11T17:46:03.446 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.440+0000 7f1ae858a700 1 -- 172.21.15.59:0/3733923019 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ae01077b0 con 0x7f1ae0101570 2023-12-11T17:46:03.447 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.442+0000 7f1ad77fe700 1 -- 172.21.15.59:0/3733923019 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f1ad000fb60 con 0x7f1ae0101570 2023-12-11T17:46:03.447 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.442+0000 7f1ad77fe700 1 --2- 172.21.15.59:0/3733923019 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f1acc0a1590 0x7f1acc0a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:03.447 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.442+0000 7f1ad77fe700 1 -- 172.21.15.59:0/3733923019 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f1ad00ce190 con 0x7f1ae0101570 2023-12-11T17:46:03.447 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.442+0000 7f1ae6326700 1 --2- 172.21.15.59:0/3733923019 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f1acc0a1590 0x7f1acc0a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:46:03.447 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.443+0000 7f1ae6326700 1 --2- 172.21.15.59:0/3733923019 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f1acc0a1590 0x7f1acc0a3a50 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f1ae01013d0 tx=0x7f1adc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:46:03.450 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.446+0000 7f1ad77fe700 1 -- 172.21.15.59:0/3733923019 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f1ad0097540 con 0x7f1ae0101570 2023-12-11T17:46:03.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:03 smithi059 ceph-mon[98847]: pgmap v312: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-12-11T17:46:03.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:03 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/943303263' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-11T17:46:03.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:03 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/2473727919' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-11T17:46:03.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:03 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/1373855869' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-11T17:46:03.701 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.698+0000 7f1ae858a700 1 -- 172.21.15.59:0/3733923019 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1ae0068230 con 0x7f1ae0101570 2023-12-11T17:46:03.702 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.698+0000 7f1ad77fe700 1 -- 172.21.15.59:0/3733923019 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+11325 (secure 0 0 0) 0x7f1ad0026090 con 0x7f1ae0101570 2023-12-11T17:46:03.702 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-12-11T17:46:03.702 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/2184151593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":2184151593},{"type":"v1","addr":"172.21.15.59:6835","nonce":2184151593}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/2817614886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2817614886},{"type":"v1","addr":"172.21.15.59:6837","nonce":2817614886}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6839/229873490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":229873490},{"type":"v1","addr":"172.21.15.59:6839","nonce":229873490}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6841/967998755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":967998755},{"type":"v1","addr":"172.21.15.59:6841","nonce":967998755}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/395684042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":395684042},{"type":"v1","addr":"172.21.15.181:6833","nonce":395684042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6833/3214606191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3214606191},{"type":"v1","addr":"172.21.15.115:6833","nonce":3214606191}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-12-11T17:44:20.076660+0000","modified":"2023-12-11T17:44:47.770695+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24521,"mds_1":24484,"mds_2":24482},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/1874159657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1874159657},{"type":"v1","addr":"172.21.15.115:6835","nonce":1874159657}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24457":{"gid":24457,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6835/4221183620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":4221183620},{"type":"v1","addr":"172.21.15.181:6835","nonce":4221183620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24466":{"gid":24466,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6837/3580309407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":3580309407},{"type":"v1","addr":"172.21.15.181:6837","nonce":3580309407}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24482":{"gid":24482,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.115:6837/1209365460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":1209365460},{"type":"v1","addr":"172.21.15.115:6837","nonce":1209365460}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.181:6839/2214384806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":2214384806},{"type":"v1","addr":"172.21.15.181:6839","nonce":2214384806}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.115:6839/4122922866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":4122922866},{"type":"v1","addr":"172.21.15.115:6839","nonce":4122922866}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-11T17:46:03.704 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.700+0000 7f1ae858a700 1 -- 172.21.15.59:0/3733923019 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f1acc0a1590 msgr2=0x7f1acc0a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:46:03.705 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.700+0000 7f1ae858a700 1 --2- 172.21.15.59:0/3733923019 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f1acc0a1590 0x7f1acc0a3a50 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f1ae01013d0 tx=0x7f1adc009260 comp rx=0 tx=0).stop 2023-12-11T17:46:03.705 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.700+0000 7f1ae858a700 1 -- 172.21.15.59:0/3733923019 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ae0101570 msgr2=0x7f1ae019c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:46:03.705 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.700+0000 7f1ae858a700 1 --2- 172.21.15.59:0/3733923019 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ae0101570 0x7f1ae019c090 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f1ad0003730 tx=0x7f1ad000f670 comp rx=0 tx=0).stop 2023-12-11T17:46:03.705 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.701+0000 7f1ae858a700 1 -- 172.21.15.59:0/3733923019 shutdown_connections 2023-12-11T17:46:03.705 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.701+0000 7f1ae858a700 1 --2- 172.21.15.59:0/3733923019 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f1acc0a1590 0x7f1acc0a3a50 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:03.705 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.701+0000 7f1ae858a700 1 --2- 172.21.15.59:0/3733923019 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1ae0101f10 0x7f1ae019e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:03.705 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.701+0000 7f1ae858a700 1 --2- 172.21.15.59:0/3733923019 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ae0100380 0x7f1ae0069190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:03.705 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.701+0000 7f1ae858a700 1 --2- 172.21.15.59:0/3733923019 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ae0101570 0x7f1ae019c090 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:03.705 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.701+0000 7f1ae858a700 1 -- 172.21.15.59:0/3733923019 >> 172.21.15.59:0/3733923019 conn(0x7f1ae00fb8f0 msgr2=0x7f1ae01048f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:46:03.705 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.701+0000 7f1ae858a700 1 -- 172.21.15.59:0/3733923019 shutdown_connections 2023-12-11T17:46:03.705 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.701+0000 7f1ae858a700 1 -- 172.21.15.59:0/3733923019 wait complete. 2023-12-11T17:46:03.706 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 24 2023-12-11T17:46:03.717 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-11T17:46:03.845 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.840+0000 7fde2acf3700 1 -- 172.21.15.59:0/3481485695 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fde24100380 msgr2=0x7fde241007a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:46:03.845 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.840+0000 7fde2acf3700 1 --2- 172.21.15.59:0/3481485695 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fde24100380 0x7fde241007a0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fde180099c0 tx=0x7fde18009cd0 comp rx=0 tx=0).stop 2023-12-11T17:46:03.846 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.840+0000 7fde2acf3700 1 -- 172.21.15.59:0/3481485695 shutdown_connections 2023-12-11T17:46:03.846 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.840+0000 7fde2acf3700 1 --2- 172.21.15.59:0/3481485695 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fde24101f10 0x7fde241087c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:03.846 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.840+0000 7fde2acf3700 1 --2- 172.21.15.59:0/3481485695 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fde24100380 0x7fde241007a0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:03.846 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.840+0000 7fde2acf3700 1 --2- 172.21.15.59:0/3481485695 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fde24101570 0x7fde241019d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:03.846 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.840+0000 7fde2acf3700 1 -- 172.21.15.59:0/3481485695 >> 172.21.15.59:0/3481485695 conn(0x7fde240fb8f0 msgr2=0x7fde240fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:46:03.846 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.840+0000 7fde2acf3700 1 -- 172.21.15.59:0/3481485695 shutdown_connections 2023-12-11T17:46:03.846 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.840+0000 7fde2acf3700 1 -- 172.21.15.59:0/3481485695 wait complete. 2023-12-11T17:46:03.846 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.840+0000 7fde2acf3700 1 Processor -- start 2023-12-11T17:46:03.846 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.840+0000 7fde2acf3700 1 -- start start 2023-12-11T17:46:03.846 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.841+0000 7fde2acf3700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fde24100380 0x7fde24197420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:03.846 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.841+0000 7fde2acf3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fde24101570 0x7fde24197960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:03.846 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.841+0000 7fde2acf3700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fde24101f10 0x7fde2419e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:03.846 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.841+0000 7fde2acf3700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde2419ef50 con 0x7fde24100380 2023-12-11T17:46:03.847 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.841+0000 7fde2acf3700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde2419f090 con 0x7fde24101f10 2023-12-11T17:46:03.847 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.841+0000 7fde2acf3700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde2419f220 con 0x7fde24101570 2023-12-11T17:46:03.847 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.841+0000 7fde28a8f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fde24100380 0x7fde24197420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:46:03.847 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.841+0000 7fde28a8f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fde24100380 0x7fde24197420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34490/0 (socket says 172.21.15.59:34490) 2023-12-11T17:46:03.847 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.841+0000 7fde28a8f700 1 -- 172.21.15.59:0/184179589 learned_addr learned my addr 172.21.15.59:0/184179589 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:46:03.847 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.841+0000 7fde23fff700 1 --2- 172.21.15.59:0/184179589 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fde24101570 0x7fde24197960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:46:03.847 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.841+0000 7fde29290700 1 --2- 172.21.15.59:0/184179589 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fde24101f10 0x7fde2419e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:46:03.847 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.841+0000 7fde28a8f700 1 -- 172.21.15.59:0/184179589 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fde24101570 msgr2=0x7fde24197960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:46:03.847 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.841+0000 7fde28a8f700 1 --2- 172.21.15.59:0/184179589 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fde24101570 0x7fde24197960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:03.847 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.841+0000 7fde28a8f700 1 -- 172.21.15.59:0/184179589 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fde24101f10 msgr2=0x7fde2419e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:46:03.847 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.841+0000 7fde28a8f700 1 --2- 172.21.15.59:0/184179589 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fde24101f10 0x7fde2419e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:03.847 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.841+0000 7fde28a8f700 1 -- 172.21.15.59:0/184179589 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde180095f0 con 0x7fde24100380 2023-12-11T17:46:03.847 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.841+0000 7fde23fff700 1 --2- 172.21.15.59:0/184179589 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fde24101570 0x7fde24197960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T17:46:03.847 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.841+0000 7fde28a8f700 1 --2- 172.21.15.59:0/184179589 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fde24100380 0x7fde24197420 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fde18003730 tx=0x7fde1800f720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:46:03.848 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.841+0000 7fde21ffb700 1 -- 172.21.15.59:0/184179589 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde1801d070 con 0x7fde24100380 2023-12-11T17:46:03.848 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.841+0000 7fde21ffb700 1 -- 172.21.15.59:0/184179589 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fde180059a0 con 0x7fde24100380 2023-12-11T17:46:03.848 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.841+0000 7fde2acf3700 1 -- 172.21.15.59:0/184179589 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde2419f4a0 con 0x7fde24100380 2023-12-11T17:46:03.848 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.841+0000 7fde21ffb700 1 -- 172.21.15.59:0/184179589 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde18004b30 con 0x7fde24100380 2023-12-11T17:46:03.848 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.842+0000 7fde2acf3700 1 -- 172.21.15.59:0/184179589 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde2419f950 con 0x7fde24100380 2023-12-11T17:46:03.849 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.843+0000 7fde2acf3700 1 -- 172.21.15.59:0/184179589 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde24068230 con 0x7fde24100380 2023-12-11T17:46:03.849 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.844+0000 7fde21ffb700 1 -- 172.21.15.59:0/184179589 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7fde1800fc10 con 0x7fde24100380 2023-12-11T17:46:03.849 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.845+0000 7fde21ffb700 1 --2- 172.21.15.59:0/184179589 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fde140a1510 0x7fde140a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:03.849 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.845+0000 7fde23fff700 1 --2- 172.21.15.59:0/184179589 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fde140a1510 0x7fde140a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:46:03.849 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.845+0000 7fde21ffb700 1 -- 172.21.15.59:0/184179589 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fde180ce2b0 con 0x7fde24100380 2023-12-11T17:46:03.849 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.845+0000 7fde23fff700 1 --2- 172.21.15.59:0/184179589 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fde140a1510 0x7fde140a39d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fde10009cb0 tx=0x7fde10009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:46:03.852 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:03.848+0000 7fde21ffb700 1 -- 172.21.15.59:0/184179589 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fde18097540 con 0x7fde24100380 2023-12-11T17:46:03.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:03 smithi181 ceph-mon[103627]: pgmap v312: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-12-11T17:46:03.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:03 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/943303263' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-11T17:46:03.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:03 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/2473727919' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-11T17:46:03.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:03 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/1373855869' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-11T17:46:04.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:03 smithi115 ceph-mon[102748]: pgmap v312: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-12-11T17:46:04.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:03 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/943303263' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-11T17:46:04.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:03 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/2473727919' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-11T17:46:04.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:03 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/1373855869' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-11T17:46:04.055 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.052+0000 7fde2acf3700 1 -- 172.21.15.59:0/184179589 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fde2410bb90 con 0x7fde24100380 2023-12-11T17:46:04.056 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.053+0000 7fde21ffb700 1 -- 172.21.15.59:0/184179589 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+24459 (secure 0 0 0) 0x7fde1809ab60 con 0x7fde24100380 2023-12-11T17:46:04.057 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-12-11T17:46:04.057 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":88,"fsid":"d0a07eaa-984b-11ee-95a3-87774f69a715","created":"2023-12-11T17:37:14.526539+0000","modified":"2023-12-11T17:45:15.870614+0000","last_up_change":"2023-12-11T17:43:38.535503+0000","last_in_change":"2023-12-11T17:43:21.014888+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-11T17:40:26.596029+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-11T17:44:18.586290+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-11T17:44:19.432952+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-11T17:44:23.097308+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3c0b3218-4b2b-40f8-8bef-ed8cd5633607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":1040759398},{"type":"v1","addr":"172.21.15.59:6803","nonce":1040759398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":1040759398},{"type":"v1","addr":"172.21.15.59:6805","nonce":1040759398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":1040759398},{"type":"v1","addr":"172.21.15.59:6809","nonce":1040759398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":1040759398},{"type":"v1","addr":"172.21.15.59:6807","nonce":1040759398}]},"public_addr":"172.21.15.59:6803/1040759398","cluster_addr":"172.21.15.59:6805/1040759398","heartbeat_back_addr":"172.21.15.59:6809/1040759398","heartbeat_front_addr":"172.21.15.59:6807/1040759398","state":["exists","up"]},{"osd":1,"uuid":"2ead7578-765b-4f5a-a72c-863731ed69e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":2776354245},{"type":"v1","addr":"172.21.15.115:6801","nonce":2776354245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":2776354245},{"type":"v1","addr":"172.21.15.115:6803","nonce":2776354245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":2776354245},{"type":"v1","addr":"172.21.15.115:6807","nonce":2776354245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":2776354245},{"type":"v1","addr":"172.21.15.115:6805","nonce":2776354245}]},"public_addr":"172.21.15.115:6801/2776354245","cluster_addr":"172.21.15.115:6803/2776354245","heartbeat_back_addr":"172.21.15.115:6807/2776354245","heartbeat_front_addr":"172.21.15.115:6805/2776354245","state":["exists","up"]},{"osd":2,"uuid":"8cb1d7ec-53cc-4672-b0b6-70d73797c44f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":4287675481},{"type":"v1","addr":"172.21.15.181:6801","nonce":4287675481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4287675481},{"type":"v1","addr":"172.21.15.181:6803","nonce":4287675481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":4287675481},{"type":"v1","addr":"172.21.15.181:6807","nonce":4287675481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":4287675481},{"type":"v1","addr":"172.21.15.181:6805","nonce":4287675481}]},"public_addr":"172.21.15.181:6801/4287675481","cluster_addr":"172.21.15.181:6803/4287675481","heartbeat_back_addr":"172.21.15.181:6807/4287675481","heartbeat_front_addr":"172.21.15.181:6805/4287675481","state":["exists","up"]},{"osd":3,"uuid":"5d64d9af-831b-4d0a-8ace-a452169351d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":219909995},{"type":"v1","addr":"172.21.15.59:6811","nonce":219909995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":219909995},{"type":"v1","addr":"172.21.15.59:6813","nonce":219909995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":219909995},{"type":"v1","addr":"172.21.15.59:6817","nonce":219909995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":219909995},{"type":"v1","addr":"172.21.15.59:6815","nonce":219909995}]},"public_addr":"172.21.15.59:6811/219909995","cluster_addr":"172.21.15.59:6813/219909995","heartbeat_back_addr":"172.21.15.59:6817/219909995","heartbeat_front_addr":"172.21.15.59:6815/219909995","state":["exists","up"]},{"osd":4,"uuid":"ef8fad28-1db5-4cba-b5f2-10e54e8e1549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":1021473826},{"type":"v1","addr":"172.21.15.115:6809","nonce":1021473826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":1021473826},{"type":"v1","addr":"172.21.15.115:6811","nonce":1021473826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":1021473826},{"type":"v1","addr":"172.21.15.115:6815","nonce":1021473826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":1021473826},{"type":"v1","addr":"172.21.15.115:6813","nonce":1021473826}]},"public_addr":"172.21.15.115:6809/1021473826","cluster_addr":"172.21.15.115:6811/1021473826","heartbeat_back_addr":"172.21.15.115:6815/1021473826","heartbeat_front_addr":"172.21.15.115:6813/1021473826","state":["exists","up"]},{"osd":5,"uuid":"4233a512-9e8f-4db8-aed2-6c3af706e589","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":2405172521},{"type":"v1","addr":"172.21.15.181:6809","nonce":2405172521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":2405172521},{"type":"v1","addr":"172.21.15.181:6811","nonce":2405172521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2405172521},{"type":"v1","addr":"172.21.15.181:6815","nonce":2405172521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":2405172521},{"type":"v1","addr":"172.21.15.181:6813","nonce":2405172521}]},"public_addr":"172.21.15.181:6809/2405172521","cluster_addr":"172.21.15.181:6811/2405172521","heartbeat_back_addr":"172.21.15.181:6815/2405172521","heartbeat_front_addr":"172.21.15.181:6813/2405172521","state":["exists","up"]},{"osd":6,"uuid":"8faab332-016e-4c77-8370-8bef95fc6f34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":815374853},{"type":"v1","addr":"172.21.15.59:6819","nonce":815374853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":815374853},{"type":"v1","addr":"172.21.15.59:6821","nonce":815374853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":815374853},{"type":"v1","addr":"172.21.15.59:6825","nonce":815374853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":815374853},{"type":"v1","addr":"172.21.15.59:6823","nonce":815374853}]},"public_addr":"172.21.15.59:6819/815374853","cluster_addr":"172.21.15.59:6821/815374853","heartbeat_back_addr":"172.21.15.59:6825/815374853","heartbeat_front_addr":"172.21.15.59:6823/815374853","state":["exists","up"]},{"osd":7,"uuid":"f20f470a-0d40-4ed3-8e0e-fd58675cf143","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":1314971870},{"type":"v1","addr":"172.21.15.115:6817","nonce":1314971870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":1314971870},{"type":"v1","addr":"172.21.15.115:6819","nonce":1314971870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":1314971870},{"type":"v1","addr":"172.21.15.115:6823","nonce":1314971870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":1314971870},{"type":"v1","addr":"172.21.15.115:6821","nonce":1314971870}]},"public_addr":"172.21.15.115:6817/1314971870","cluster_addr":"172.21.15.115:6819/1314971870","heartbeat_back_addr":"172.21.15.115:6823/1314971870","heartbeat_front_addr":"172.21.15.115:6821/1314971870","state":["exists","up"]},{"osd":8,"uuid":"eb657d32-1a95-4fd7-a293-d0a970837fc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":3954509873},{"type":"v1","addr":"172.21.15.181:6817","nonce":3954509873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":3954509873},{"type":"v1","addr":"172.21.15.181:6819","nonce":3954509873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":3954509873},{"type":"v1","addr":"172.21.15.181:6823","nonce":3954509873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":3954509873},{"type":"v1","addr":"172.21.15.181:6821","nonce":3954509873}]},"public_addr":"172.21.15.181:6817/3954509873","cluster_addr":"172.21.15.181:6819/3954509873","heartbeat_back_addr":"172.21.15.181:6823/3954509873","heartbeat_front_addr":"172.21.15.181:6821/3954509873","state":["exists","up"]},{"osd":9,"uuid":"8f7717d4-b6f2-4ec1-9b58-62feb0c9a146","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":1163095677},{"type":"v1","addr":"172.21.15.59:6827","nonce":1163095677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":1163095677},{"type":"v1","addr":"172.21.15.59:6829","nonce":1163095677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":1163095677},{"type":"v1","addr":"172.21.15.59:6833","nonce":1163095677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":1163095677},{"type":"v1","addr":"172.21.15.59:6831","nonce":1163095677}]},"public_addr":"172.21.15.59:6827/1163095677","cluster_addr":"172.21.15.59:6829/1163095677","heartbeat_back_addr":"172.21.15.59:6833/1163095677","heartbeat_front_addr":"172.21.15.59:6831/1163095677","state":["exists","up"]},{"osd":10,"uuid":"b75c179a-875d-43bc-b4c1-e8eb80fb64fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":2199610269},{"type":"v1","addr":"172.21.15.115:6825","nonce":2199610269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":2199610269},{"type":"v1","addr":"172.21.15.115:6827","nonce":2199610269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":2199610269},{"type":"v1","addr":"172.21.15.115:6831","nonce":2199610269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":2199610269},{"type":"v1","addr":"172.21.15.115:6829","nonce":2199610269}]},"public_addr":"172.21.15.115:6825/2199610269","cluster_addr":"172.21.15.115:6827/2199610269","heartbeat_back_addr":"172.21.15.115:6831/2199610269","heartbeat_front_addr":"172.21.15.115:6829/2199610269","state":["exists","up"]},{"osd":11,"uuid":"5e141dfe-8a00-4bbc-8204-2dc9be24e18d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":226506989},{"type":"v1","addr":"172.21.15.181:6825","nonce":226506989}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":226506989},{"type":"v1","addr":"172.21.15.181:6827","nonce":226506989}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":226506989},{"type":"v1","addr":"172.21.15.181:6831","nonce":226506989}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":226506989},{"type":"v1","addr":"172.21.15.181:6829","nonce":226506989}]},"public_addr":"172.21.15.181:6825/226506989","cluster_addr":"172.21.15.181:6827/226506989","heartbeat_back_addr":"172.21.15.181:6831/226506989","heartbeat_front_addr":"172.21.15.181:6829/226506989","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:39:42.687084+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:40:03.903448+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:40:24.899908+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:40:46.600251+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:41:07.690037+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:41:28.488855+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:41:49.459274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:42:10.863881+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:42:32.248391+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:42:54.112265+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:43:15.634872+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:43:36.922534+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":7}]},{"pgid":"3.28","mappings":[{"from":5,"to":6},{"from":3,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.59:6800/1259916482":"2023-12-12T17:38:14.037922+0000","172.21.15.59:0/17457774":"2023-12-12T17:38:14.037922+0000","172.21.15.59:6801/1672530462":"2023-12-12T17:37:44.773925+0000","172.21.15.59:0/1053944547":"2023-12-12T17:38:14.037922+0000","172.21.15.59:6801/1259916482":"2023-12-12T17:38:14.037922+0000","172.21.15.59:6800/1672530462":"2023-12-12T17:37:44.773925+0000","172.21.15.59:0/3570767650":"2023-12-12T17:37:44.773925+0000","172.21.15.59:0/3853462899":"2023-12-12T17:37:44.773925+0000","172.21.15.59:0/2859415380":"2023-12-12T17:38:14.037922+0000","172.21.15.59:0/2192742819":"2023-12-12T17:37:44.773925+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-11T17:46:04.059 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.056+0000 7fde2acf3700 1 -- 172.21.15.59:0/184179589 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fde140a1510 msgr2=0x7fde140a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:46:04.059 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.056+0000 7fde2acf3700 1 --2- 172.21.15.59:0/184179589 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fde140a1510 0x7fde140a39d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fde10009cb0 tx=0x7fde10009260 comp rx=0 tx=0).stop 2023-12-11T17:46:04.059 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.056+0000 7fde2acf3700 1 -- 172.21.15.59:0/184179589 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fde24100380 msgr2=0x7fde24197420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:46:04.060 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.056+0000 7fde2acf3700 1 --2- 172.21.15.59:0/184179589 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fde24100380 0x7fde24197420 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fde18003730 tx=0x7fde1800f720 comp rx=0 tx=0).stop 2023-12-11T17:46:04.060 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.056+0000 7fde2acf3700 1 -- 172.21.15.59:0/184179589 shutdown_connections 2023-12-11T17:46:04.060 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.056+0000 7fde2acf3700 1 --2- 172.21.15.59:0/184179589 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fde140a1510 0x7fde140a39d0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:04.060 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.056+0000 7fde2acf3700 1 --2- 172.21.15.59:0/184179589 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fde24101f10 0x7fde2419e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:04.060 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.056+0000 7fde2acf3700 1 --2- 172.21.15.59:0/184179589 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fde24100380 0x7fde24197420 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:04.060 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.056+0000 7fde2acf3700 1 --2- 172.21.15.59:0/184179589 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fde24101570 0x7fde24197960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:04.060 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.056+0000 7fde2acf3700 1 -- 172.21.15.59:0/184179589 >> 172.21.15.59:0/184179589 conn(0x7fde240fb8f0 msgr2=0x7fde24068640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:46:04.060 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.056+0000 7fde2acf3700 1 -- 172.21.15.59:0/184179589 shutdown_connections 2023-12-11T17:46:04.060 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.056+0000 7fde2acf3700 1 -- 172.21.15.59:0/184179589 wait complete. 2023-12-11T17:46:04.072 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2023-12-11T17:44:20.076660+0000', 'modified': '2023-12-11T17:44:47.770695+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24521, 'mds_1': 24484, 'mds_2': 24482}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24449': {'gid': 24449, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.115:6835/1874159657', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6834', 'nonce': 1874159657}, {'type': 'v1', 'addr': '172.21.15.115:6835', 'nonce': 1874159657}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24457': {'gid': 24457, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.181:6835/4221183620', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6834', 'nonce': 4221183620}, {'type': 'v1', 'addr': '172.21.15.181:6835', 'nonce': 4221183620}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24466': {'gid': 24466, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.181:6837/3580309407', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6836', 'nonce': 3580309407}, {'type': 'v1', 'addr': '172.21.15.181:6837', 'nonce': 3580309407}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24482': {'gid': 24482, 'name': 'h', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.115:6837/1209365460', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6836', 'nonce': 1209365460}, {'type': 'v1', 'addr': '172.21.15.115:6837', 'nonce': 1209365460}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24484': {'gid': 24484, 'name': 'i', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.181:6839/2214384806', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6838', 'nonce': 2214384806}, {'type': 'v1', 'addr': '172.21.15.181:6839', 'nonce': 2214384806}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24521': {'gid': 24521, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.115:6839/4122922866', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6838', 'nonce': 4122922866}, {'type': 'v1', 'addr': '172.21.15.115:6839', 'nonce': 4122922866}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-12-11T17:46:04.072 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2023-12-11T17:46:04.073 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2023-12-11T17:46:04.202 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.196+0000 7f04270d8700 1 -- 172.21.15.59:0/3908591892 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0420100480 msgr2=0x7f04201008a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:46:04.202 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.196+0000 7f04270d8700 1 --2- 172.21.15.59:0/3908591892 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0420100480 0x7f04201008a0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f04140099c0 tx=0x7f0414009cd0 comp rx=0 tx=0).stop 2023-12-11T17:46:04.202 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.196+0000 7f04270d8700 1 -- 172.21.15.59:0/3908591892 shutdown_connections 2023-12-11T17:46:04.202 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.196+0000 7f04270d8700 1 --2- 172.21.15.59:0/3908591892 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0420102010 0x7f04201088c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:04.202 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.196+0000 7f04270d8700 1 --2- 172.21.15.59:0/3908591892 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0420100480 0x7f04201008a0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:04.203 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.196+0000 7f04270d8700 1 --2- 172.21.15.59:0/3908591892 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0420101670 0x7f0420101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:04.203 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.196+0000 7f04270d8700 1 -- 172.21.15.59:0/3908591892 >> 172.21.15.59:0/3908591892 conn(0x7f04200fb9f0 msgr2=0x7f04200fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:46:04.203 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.197+0000 7f04270d8700 1 -- 172.21.15.59:0/3908591892 shutdown_connections 2023-12-11T17:46:04.203 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.197+0000 7f04270d8700 1 -- 172.21.15.59:0/3908591892 wait complete. 2023-12-11T17:46:04.203 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.197+0000 7f04270d8700 1 Processor -- start 2023-12-11T17:46:04.203 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.197+0000 7f04270d8700 1 -- start start 2023-12-11T17:46:04.203 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.197+0000 7f04270d8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0420100480 0x7f0420104560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:04.203 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.197+0000 7f04270d8700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0420101670 0x7f0420102bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:04.203 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.197+0000 7f04270d8700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0420102010 0x7f0420103130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:04.203 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.197+0000 7f04270d8700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04201038d0 con 0x7f0420102010 2023-12-11T17:46:04.203 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.197+0000 7f04270d8700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0420103a40 con 0x7f0420101670 2023-12-11T17:46:04.203 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.197+0000 7f04270d8700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04201a1580 con 0x7f0420100480 2023-12-11T17:46:04.203 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.197+0000 7f0425675700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0420102010 0x7f0420103130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:46:04.203 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.197+0000 7f0425675700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0420102010 0x7f0420103130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34508/0 (socket says 172.21.15.59:34508) 2023-12-11T17:46:04.204 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.197+0000 7f0425675700 1 -- 172.21.15.59:0/1752970833 learned_addr learned my addr 172.21.15.59:0/1752970833 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:46:04.204 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.198+0000 7f0425675700 1 -- 172.21.15.59:0/1752970833 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0420100480 msgr2=0x7f0420104560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:46:04.204 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.198+0000 7f0424e74700 1 --2- 172.21.15.59:0/1752970833 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0420100480 0x7f0420104560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:46:04.204 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.198+0000 7f041ffff700 1 --2- 172.21.15.59:0/1752970833 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0420101670 0x7f0420102bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:46:04.204 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.198+0000 7f0425675700 1 --2- 172.21.15.59:0/1752970833 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0420100480 0x7f0420104560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:04.204 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.198+0000 7f0425675700 1 -- 172.21.15.59:0/1752970833 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0420101670 msgr2=0x7f0420102bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:46:04.204 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.198+0000 7f0425675700 1 --2- 172.21.15.59:0/1752970833 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0420101670 0x7f0420102bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:04.204 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.198+0000 7f0425675700 1 -- 172.21.15.59:0/1752970833 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04140095f0 con 0x7f0420102010 2023-12-11T17:46:04.204 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.198+0000 7f0425675700 1 --2- 172.21.15.59:0/1752970833 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0420102010 0x7f0420103130 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f0418007ee0 tx=0x7f041800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:46:04.204 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.198+0000 7f041dffb700 1 -- 172.21.15.59:0/1752970833 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f041800dd70 con 0x7f0420102010 2023-12-11T17:46:04.204 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.198+0000 7f04270d8700 1 -- 172.21.15.59:0/1752970833 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04201a1810 con 0x7f0420102010 2023-12-11T17:46:04.204 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.198+0000 7f041dffb700 1 -- 172.21.15.59:0/1752970833 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f041800f040 con 0x7f0420102010 2023-12-11T17:46:04.204 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.198+0000 7f041dffb700 1 -- 172.21.15.59:0/1752970833 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04180048f0 con 0x7f0420102010 2023-12-11T17:46:04.204 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.198+0000 7f04270d8700 1 -- 172.21.15.59:0/1752970833 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04201a1cd0 con 0x7f0420102010 2023-12-11T17:46:04.207 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.200+0000 7f041dffb700 1 -- 172.21.15.59:0/1752970833 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f0418003a00 con 0x7f0420102010 2023-12-11T17:46:04.207 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.201+0000 7f041dffb700 1 --2- 172.21.15.59:0/1752970833 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f04100a1510 0x7f04100a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:04.207 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.201+0000 7f0424e74700 1 --2- 172.21.15.59:0/1752970833 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f04100a1510 0x7f04100a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:46:04.207 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.201+0000 7f041dffb700 1 -- 172.21.15.59:0/1752970833 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f04180ccb20 con 0x7f0420102010 2023-12-11T17:46:04.207 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.202+0000 7f0424e74700 1 --2- 172.21.15.59:0/1752970833 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f04100a1510 0x7f04100a39d0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f0414009fd0 tx=0x7f041400b540 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:46:04.207 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.202+0000 7f04332d1b80 1 Processor -- start 2023-12-11T17:46:04.207 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.202+0000 7f04332d1b80 1 -- start start 2023-12-11T17:46:04.207 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.202+0000 7f04332d1b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac42e2c380 0x55ac42e2c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:04.207 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.202+0000 7f04332d1b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ac42e2cce0 0x55ac42e2d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:04.207 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.202+0000 7f04332d1b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac42e2dbd0 0x55ac42e2e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:04.207 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.202+0000 7f04332d1b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac42d87b00 con 0x55ac42e2dbd0 2023-12-11T17:46:04.207 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.202+0000 7f04332d1b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac42d85420 con 0x55ac42e2cce0 2023-12-11T17:46:04.208 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.202+0000 7f04332d1b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac42d80d10 con 0x55ac42e2c380 2023-12-11T17:46:04.208 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.202+0000 7f0425675700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac42e2dbd0 0x55ac42e2e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:46:04.208 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.203+0000 7f0425675700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac42e2dbd0 0x55ac42e2e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34516/0 (socket says 172.21.15.59:34516) 2023-12-11T17:46:04.208 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.203+0000 7f0425675700 1 -- 172.21.15.59:0/198076269 learned_addr learned my addr 172.21.15.59:0/198076269 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:46:04.208 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.203+0000 7f0425675700 1 -- 172.21.15.59:0/198076269 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac42e2c380 msgr2=0x55ac42e2c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T17:46:04.208 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.203+0000 7f0425675700 1 --2- 172.21.15.59:0/198076269 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac42e2c380 0x55ac42e2c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:04.208 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.203+0000 7f041ffff700 1 --2- 172.21.15.59:0/198076269 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ac42e2cce0 0x55ac42e2d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:46:04.208 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.203+0000 7f0425675700 1 -- 172.21.15.59:0/198076269 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ac42e2cce0 msgr2=0x55ac42e2d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T17:46:04.208 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.203+0000 7f0425675700 1 --2- 172.21.15.59:0/198076269 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ac42e2cce0 0x55ac42e2d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:04.208 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.203+0000 7f0425675700 1 -- 172.21.15.59:0/198076269 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ac42bfd5e0 con 0x55ac42e2dbd0 2023-12-11T17:46:04.208 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.203+0000 7f041ffff700 1 --2- 172.21.15.59:0/198076269 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ac42e2cce0 0x55ac42e2d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-11T17:46:04.208 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.203+0000 7f0425675700 1 --2- 172.21.15.59:0/198076269 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac42e2dbd0 0x55ac42e2e050 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f0418004b80 tx=0x7f04180094e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6c7203bbb10ce05 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:46:04.208 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.203+0000 7f040affd700 1 -- 172.21.15.59:0/198076269 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04180a1d70 con 0x55ac42e2dbd0 2023-12-11T17:46:04.208 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.203+0000 7f040affd700 1 -- 172.21.15.59:0/198076269 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0418004000 con 0x55ac42e2dbd0 2023-12-11T17:46:04.209 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.203+0000 7f040affd700 1 -- 172.21.15.59:0/198076269 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0418099ea0 con 0x55ac42e2dbd0 2023-12-11T17:46:04.213 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.204+0000 7f04332d1b80 1 -- 172.21.15.59:0/198076269 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac42e2dbd0 msgr2=0x55ac42e2e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:46:04.213 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.204+0000 7f04332d1b80 1 --2- 172.21.15.59:0/198076269 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac42e2dbd0 0x55ac42e2e050 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f0418004b80 tx=0x7f04180094e0 comp rx=0 tx=0).stop 2023-12-11T17:46:04.213 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.204+0000 7f04332d1b80 1 -- 172.21.15.59:0/198076269 shutdown_connections 2023-12-11T17:46:04.213 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.204+0000 7f04332d1b80 1 --2- 172.21.15.59:0/198076269 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac42e2dbd0 0x55ac42e2e050 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:04.213 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.204+0000 7f04332d1b80 1 --2- 172.21.15.59:0/198076269 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac42e2c380 0x55ac42e2c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:04.214 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.204+0000 7f04332d1b80 1 --2- 172.21.15.59:0/198076269 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ac42e2cce0 0x55ac42e2d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:04.214 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.204+0000 7f04332d1b80 1 -- 172.21.15.59:0/198076269 >> 172.21.15.59:0/198076269 conn(0x55ac42d5b790 msgr2=0x55ac42e28d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:46:04.214 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.204+0000 7f04332d1b80 1 -- 172.21.15.59:0/198076269 shutdown_connections 2023-12-11T17:46:04.214 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.204+0000 7f04332d1b80 1 -- 172.21.15.59:0/198076269 wait complete. 2023-12-11T17:46:04.214 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.204+0000 7f04332d1b80 1 -- start start 2023-12-11T17:46:04.214 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.204+0000 7f04332d1b80 1 Processor -- start 2023-12-11T17:46:04.214 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.205+0000 7f04332d1b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac42e2c380 0x55ac42e36220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:04.214 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.205+0000 7f04332d1b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ac42e2cce0 0x55ac42e36760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:04.214 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.205+0000 7f04332d1b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac42e2dbd0 0x55ac42e3d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:04.214 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.205+0000 7f04332d1b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac42d3a510 con 0x55ac42e2c380 2023-12-11T17:46:04.214 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.205+0000 7f04332d1b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac42d3a1b0 con 0x55ac42e2cce0 2023-12-11T17:46:04.214 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.205+0000 7f04332d1b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac42bff8d0 con 0x55ac42e2dbd0 2023-12-11T17:46:04.214 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.205+0000 7f04332d1b80 10 client.0 _ll_register_callbacks cb 0x55ac42c65a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T17:46:04.215 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.205+0000 7f0424e74700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac42e2c380 0x55ac42e36220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:46:04.215 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.205+0000 7f0424e74700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac42e2c380 0x55ac42e36220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34518/0 (socket says 172.21.15.59:34518) 2023-12-11T17:46:04.215 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.205+0000 7f0424e74700 1 -- 172.21.15.59:0/213185980 learned_addr learned my addr 172.21.15.59:0/213185980 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:46:04.215 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.205+0000 7f041ffff700 1 --2- 172.21.15.59:0/213185980 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ac42e2cce0 0x55ac42e36760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:46:04.215 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.205+0000 7f0425675700 1 --2- 172.21.15.59:0/213185980 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac42e2dbd0 0x55ac42e3d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:46:04.215 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.205+0000 7f0424e74700 1 -- 172.21.15.59:0/213185980 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac42e2dbd0 msgr2=0x55ac42e3d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T17:46:04.215 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.205+0000 7f0424e74700 1 --2- 172.21.15.59:0/213185980 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac42e2dbd0 0x55ac42e3d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:04.215 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.205+0000 7f0424e74700 1 -- 172.21.15.59:0/213185980 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ac42e2cce0 msgr2=0x55ac42e36760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T17:46:04.215 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.205+0000 7f0424e74700 1 --2- 172.21.15.59:0/213185980 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ac42e2cce0 0x55ac42e36760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:04.215 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.205+0000 7f0424e74700 1 -- 172.21.15.59:0/213185980 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04201a1cd0 con 0x55ac42e2c380 2023-12-11T17:46:04.215 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.205+0000 7f0424e74700 1 --2- 172.21.15.59:0/213185980 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac42e2c380 0x55ac42e36220 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f04201014d0 tx=0x7f0414000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=159067fe20f56131 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:46:04.215 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.205+0000 7f03fffff700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2023-12-11T17:46:04.215 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.206+0000 7f03fffff700 1 -- 172.21.15.59:0/213185980 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f041400bec0 con 0x55ac42e2c380 2023-12-11T17:46:04.215 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.206+0000 7f03fffff700 1 -- 172.21.15.59:0/213185980 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f041402f480 con 0x55ac42e2c380 2023-12-11T17:46:04.215 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.206+0000 7f04332d1b80 1 -- 172.21.15.59:0/213185980 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ac42e1faf0 con 0x55ac42e2c380 2023-12-11T17:46:04.216 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.206+0000 7f03fffff700 1 -- 172.21.15.59:0/213185980 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f04140056e0 con 0x55ac42e2c380 2023-12-11T17:46:04.216 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.206+0000 7f04332d1b80 10 client.15348 fetch_fsmap learned FSMap version 24 2023-12-11T17:46:04.216 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.206+0000 7f04332d1b80 1 -- 172.21.15.59:0/213185980 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55ac42ba9d90 con 0x55ac42e2c380 2023-12-11T17:46:04.216 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.206+0000 7f03fffff700 1 -- 172.21.15.59:0/213185980 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ac42e1faf0 con 0x55ac42e2c380 2023-12-11T17:46:04.216 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.206+0000 7f03fffff700 1 -- 172.21.15.59:0/213185980 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f0414024880 con 0x55ac42e2c380 2023-12-11T17:46:04.216 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.206+0000 7f04332d1b80 10 client.15348 fetch_fsmap finished waiting for FSMap version 24 2023-12-11T17:46:04.216 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.206+0000 7f04332d1b80 10 client.15348 resolve_mds: resolved name 'h' to daemon mds.h 2023-12-11T17:46:04.216 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.206+0000 7f04332d1b80 20 client.15348 populate_metadata read hostname 'smithi059' 2023-12-11T17:46:04.216 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.206+0000 7f04332d1b80 1 --2- 172.21.15.59:0/213185980 >> [v2:172.21.15.115:6836/1209365460,v1:172.21.15.115:6837/1209365460] conn(0x55ac42e41dd0 0x55ac42e40900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:04.216 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.206+0000 7f04332d1b80 4 client.15348 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-11T17:46:04.216 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.206+0000 7f04332d1b80 1 -- 172.21.15.59:0/213185980 --> [v2:172.21.15.115:6836/1209365460,v1:172.21.15.115:6837/1209365460] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ac42dfbef0 con 0x55ac42e41dd0 2023-12-11T17:46:04.216 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.207+0000 7f041ffff700 1 --2- 172.21.15.59:0/213185980 >> [v2:172.21.15.115:6836/1209365460,v1:172.21.15.115:6837/1209365460] conn(0x55ac42e41dd0 0x55ac42e40900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:46:04.216 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.207+0000 7f041ffff700 1 --2- 172.21.15.59:0/213185980 >> [v2:172.21.15.115:6836/1209365460,v1:172.21.15.115:6837/1209365460] conn(0x55ac42e41dd0 0x55ac42e40900 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=66f45efecf4162ea server_cookie=26ba6acc968a2fa3 in_seq=0 out_seq=0 2023-12-11T17:46:04.216 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.207+0000 7f03fffff700 10 client.15348 ms_handle_connect on v2:172.21.15.115:6836/1209365460 2023-12-11T17:46:04.217 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.208+0000 7f03fffff700 1 -- 172.21.15.59:0/213185980 <== mds.2 v2:172.21.15.115:6836/1209365460 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ac42dfbef0 con 0x55ac42e41dd0 2023-12-11T17:46:04.217 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.208+0000 7f03fffff700 10 client.15348 handle_command_reply: tid=0 2023-12-11T17:46:04.217 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.208+0000 7f04332d1b80 1 client.15348 shutdown 2023-12-11T17:46:04.217 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.208+0000 7f04332d1b80 1 -- 172.21.15.59:0/213185980 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac42e2c380 msgr2=0x55ac42e36220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:46:04.217 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.208+0000 7f04332d1b80 1 --2- 172.21.15.59:0/213185980 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac42e2c380 0x55ac42e36220 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f04201014d0 tx=0x7f0414000bc0 comp rx=0 tx=0).stop 2023-12-11T17:46:04.217 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.208+0000 7f04332d1b80 1 -- 172.21.15.59:0/213185980 shutdown_connections 2023-12-11T17:46:04.217 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.208+0000 7f04332d1b80 1 --2- 172.21.15.59:0/213185980 >> [v2:172.21.15.115:6836/1209365460,v1:172.21.15.115:6837/1209365460] conn(0x55ac42e41dd0 0x55ac42e40900 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:04.217 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.208+0000 7f04332d1b80 1 --2- 172.21.15.59:0/213185980 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac42e2dbd0 0x55ac42e3d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:04.217 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.208+0000 7f03fffff700 0 client.15348 ms_handle_reset on v2:172.21.15.115:6836/1209365460 2023-12-11T17:46:04.217 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.208+0000 7f04332d1b80 1 --2- 172.21.15.59:0/213185980 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac42e2c380 0x55ac42e36220 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:04.217 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.208+0000 7f04332d1b80 1 --2- 172.21.15.59:0/213185980 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ac42e2cce0 0x55ac42e36760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:04.217 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.208+0000 7f04332d1b80 1 -- 172.21.15.59:0/213185980 >> 172.21.15.59:0/213185980 conn(0x55ac42d5b790 msgr2=0x55ac42e279f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:46:04.217 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.208+0000 7f04332d1b80 1 -- 172.21.15.59:0/213185980 shutdown_connections 2023-12-11T17:46:04.218 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.208+0000 7f04332d1b80 1 -- 172.21.15.59:0/213185980 wait complete. 2023-12-11T17:46:04.218 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.208+0000 7f04332d1b80 20 client.15348 trim_cache size 0 max 16384 2023-12-11T17:46:04.235 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.226+0000 7f04332d1b80 1 Processor -- start 2023-12-11T17:46:04.235 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.226+0000 7f04332d1b80 1 -- start start 2023-12-11T17:46:04.235 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.226+0000 7f04332d1b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac42e2c380 0x55ac42e2dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:04.235 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.226+0000 7f04332d1b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac42e2cce0 0x55ac42e3d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:04.235 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.226+0000 7f04332d1b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ac42e2dbd0 0x55ac42e28920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:04.235 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.226+0000 7f04332d1b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac42e1fad0 con 0x55ac42e2c380 2023-12-11T17:46:04.235 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.226+0000 7f04332d1b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac42e3e920 con 0x55ac42e2dbd0 2023-12-11T17:46:04.235 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.226+0000 7f04332d1b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac42e2d0c0 con 0x55ac42e2cce0 2023-12-11T17:46:04.235 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.226+0000 7f0424e74700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac42e2c380 0x55ac42e2dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:46:04.235 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.226+0000 7f0424e74700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac42e2c380 0x55ac42e2dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34526/0 (socket says 172.21.15.59:34526) 2023-12-11T17:46:04.236 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.226+0000 7f0424e74700 1 -- 172.21.15.59:0/1275934430 learned_addr learned my addr 172.21.15.59:0/1275934430 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:46:04.236 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.226+0000 7f0425675700 1 --2- 172.21.15.59:0/1275934430 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ac42e2dbd0 0x55ac42e28920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:46:04.236 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.227+0000 7f041ffff700 1 --2- 172.21.15.59:0/1275934430 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac42e2cce0 0x55ac42e3d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:46:04.236 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.227+0000 7f0424e74700 1 -- 172.21.15.59:0/1275934430 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac42e2cce0 msgr2=0x55ac42e3d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T17:46:04.236 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.227+0000 7f0424e74700 1 --2- 172.21.15.59:0/1275934430 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac42e2cce0 0x55ac42e3d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:04.236 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.227+0000 7f0424e74700 1 -- 172.21.15.59:0/1275934430 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ac42e2dbd0 msgr2=0x55ac42e28920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T17:46:04.236 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.227+0000 7f0424e74700 1 --2- 172.21.15.59:0/1275934430 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ac42e2dbd0 0x55ac42e28920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:04.236 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.227+0000 7f0424e74700 1 -- 172.21.15.59:0/1275934430 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ac42b92c10 con 0x55ac42e2c380 2023-12-11T17:46:04.236 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.227+0000 7f0424e74700 1 --2- 172.21.15.59:0/1275934430 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac42e2c380 0x55ac42e2dfd0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f0414024be0 tx=0x7f0414005f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3b3ceaa34bea82 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:46:04.236 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.227+0000 7f040b7fe700 1 -- 172.21.15.59:0/1275934430 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0414024e50 con 0x55ac42e2c380 2023-12-11T17:46:04.236 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.227+0000 7f040b7fe700 1 -- 172.21.15.59:0/1275934430 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f041401d040 con 0x55ac42e2c380 2023-12-11T17:46:04.236 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.227+0000 7f040b7fe700 1 -- 172.21.15.59:0/1275934430 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f041402fb90 con 0x55ac42e2c380 2023-12-11T17:46:04.236 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.227+0000 7f04332d1b80 1 -- 172.21.15.59:0/1275934430 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac42e2c380 msgr2=0x55ac42e2dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:46:04.236 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.227+0000 7f04332d1b80 1 --2- 172.21.15.59:0/1275934430 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac42e2c380 0x55ac42e2dfd0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f0414024be0 tx=0x7f0414005f00 comp rx=0 tx=0).stop 2023-12-11T17:46:04.236 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.227+0000 7f04332d1b80 1 -- 172.21.15.59:0/1275934430 shutdown_connections 2023-12-11T17:46:04.237 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.227+0000 7f04332d1b80 1 --2- 172.21.15.59:0/1275934430 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ac42e2dbd0 0x55ac42e28920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:04.237 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.227+0000 7f04332d1b80 1 --2- 172.21.15.59:0/1275934430 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac42e2c380 0x55ac42e2dfd0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:04.237 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.227+0000 7f04332d1b80 1 --2- 172.21.15.59:0/1275934430 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac42e2cce0 0x55ac42e3d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:04.237 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.227+0000 7f04332d1b80 1 -- 172.21.15.59:0/1275934430 >> 172.21.15.59:0/1275934430 conn(0x55ac42d5b790 msgr2=0x55ac42e30b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:46:04.237 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.228+0000 7f04332d1b80 1 -- 172.21.15.59:0/1275934430 shutdown_connections 2023-12-11T17:46:04.237 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.228+0000 7f04332d1b80 1 -- 172.21.15.59:0/1275934430 wait complete. 2023-12-11T17:46:04.237 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.228+0000 7f04332d1b80 1 -- start start 2023-12-11T17:46:04.237 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.228+0000 7f04332d1b80 1 Processor -- start 2023-12-11T17:46:04.237 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.228+0000 7f04332d1b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac42e2c380 0x55ac42e3d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:04.237 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.228+0000 7f04332d1b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac42e2cce0 0x55ac42e3d750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:04.237 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.228+0000 7f04332d1b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ac42e2dbd0 0x55ac42e39da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:04.237 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.228+0000 7f04332d1b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac42e3a480 con 0x55ac42e2c380 2023-12-11T17:46:04.237 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.228+0000 7f04332d1b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac42e3a5f0 con 0x55ac42e2dbd0 2023-12-11T17:46:04.237 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.228+0000 7f04332d1b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac42e3a7b0 con 0x55ac42e2cce0 2023-12-11T17:46:04.237 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.228+0000 7f04332d1b80 10 client.0 _ll_register_callbacks cb 0x55ac42c65a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T17:46:04.238 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.228+0000 7f0424e74700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac42e2c380 0x55ac42e3d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:46:04.238 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.228+0000 7f0424e74700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac42e2c380 0x55ac42e3d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34532/0 (socket says 172.21.15.59:34532) 2023-12-11T17:46:04.238 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.228+0000 7f0424e74700 1 -- 172.21.15.59:0/2747572555 learned_addr learned my addr 172.21.15.59:0/2747572555 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:46:04.238 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.229+0000 7f041ffff700 1 --2- 172.21.15.59:0/2747572555 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac42e2cce0 0x55ac42e3d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:46:04.238 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.229+0000 7f0425675700 1 --2- 172.21.15.59:0/2747572555 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ac42e2dbd0 0x55ac42e39da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:46:04.238 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.229+0000 7f0424e74700 1 -- 172.21.15.59:0/2747572555 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac42e2cce0 msgr2=0x55ac42e3d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T17:46:04.238 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.229+0000 7f0424e74700 1 --2- 172.21.15.59:0/2747572555 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac42e2cce0 0x55ac42e3d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:04.238 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.229+0000 7f0424e74700 1 -- 172.21.15.59:0/2747572555 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ac42e2dbd0 msgr2=0x55ac42e39da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T17:46:04.238 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.229+0000 7f0424e74700 1 --2- 172.21.15.59:0/2747572555 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ac42e2dbd0 0x55ac42e39da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:04.238 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.229+0000 7f0424e74700 1 -- 172.21.15.59:0/2747572555 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0414034040 con 0x55ac42e2c380 2023-12-11T17:46:04.238 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.229+0000 7f0424e74700 1 --2- 172.21.15.59:0/2747572555 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac42e2c380 0x55ac42e3d210 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f04140098e0 tx=0x7f04140092f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7af05a3a87d5703 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:46:04.238 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.229+0000 7f03fffff700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2023-12-11T17:46:04.238 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.229+0000 7f03fffff700 1 -- 172.21.15.59:0/2747572555 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f041402faa0 con 0x55ac42e2c380 2023-12-11T17:46:04.239 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.229+0000 7f03fffff700 1 -- 172.21.15.59:0/2747572555 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f041402fc00 con 0x55ac42e2c380 2023-12-11T17:46:04.239 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.229+0000 7f04332d1b80 1 -- 172.21.15.59:0/2747572555 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0418004000 con 0x55ac42e2c380 2023-12-11T17:46:04.239 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.229+0000 7f03fffff700 1 -- 172.21.15.59:0/2747572555 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f041402fe00 con 0x55ac42e2c380 2023-12-11T17:46:04.239 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.229+0000 7f04332d1b80 10 client.15354 fetch_fsmap learned FSMap version 24 2023-12-11T17:46:04.239 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.229+0000 7f04332d1b80 1 -- 172.21.15.59:0/2747572555 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55ac42bd0200 con 0x55ac42e2c380 2023-12-11T17:46:04.239 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.229+0000 7f03fffff700 1 -- 172.21.15.59:0/2747572555 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0418004000 con 0x55ac42e2c380 2023-12-11T17:46:04.239 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.229+0000 7f03fffff700 1 -- 172.21.15.59:0/2747572555 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f041404d070 con 0x55ac42e2c380 2023-12-11T17:46:04.239 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.230+0000 7f04332d1b80 10 client.15354 fetch_fsmap finished waiting for FSMap version 24 2023-12-11T17:46:04.239 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.230+0000 7f04332d1b80 10 client.15354 resolve_mds: resolved name 'h' to daemon mds.h 2023-12-11T17:46:04.239 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.230+0000 7f04332d1b80 20 client.15354 populate_metadata read hostname 'smithi059' 2023-12-11T17:46:04.239 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.230+0000 7f04332d1b80 1 --2- 172.21.15.59:0/2747572555 >> [v2:172.21.15.115:6836/1209365460,v1:172.21.15.115:6837/1209365460] conn(0x55ac42e9cb50 0x55ac42e99010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:04.239 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.230+0000 7f04332d1b80 4 client.15354 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "status"}] 2023-12-11T17:46:04.239 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.230+0000 7f04332d1b80 1 -- 172.21.15.59:0/2747572555 --> [v2:172.21.15.115:6836/1209365460,v1:172.21.15.115:6837/1209365460] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55ac42d46f80 con 0x55ac42e9cb50 2023-12-11T17:46:04.239 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.230+0000 7f041ffff700 1 --2- 172.21.15.59:0/2747572555 >> [v2:172.21.15.115:6836/1209365460,v1:172.21.15.115:6837/1209365460] conn(0x55ac42e9cb50 0x55ac42e99010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:46:04.240 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.230+0000 7f03fffff700 10 client.15354 ms_handle_connect on v2:172.21.15.115:6836/1209365460 2023-12-11T17:46:04.240 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.230+0000 7f041ffff700 1 --2- 172.21.15.59:0/2747572555 >> [v2:172.21.15.115:6836/1209365460,v1:172.21.15.115:6837/1209365460] conn(0x55ac42e9cb50 0x55ac42e99010 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4c7b377ea9051b1d server_cookie=59f07d1a0f89da5e in_seq=0 out_seq=0 2023-12-11T17:46:04.240 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.230+0000 7f03fffff700 1 -- 172.21.15.59:0/2747572555 <== mds.2 v2:172.21.15.115:6836/1209365460 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55ac42d46f80 con 0x55ac42e9cb50 2023-12-11T17:46:04.240 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.230+0000 7f03fffff700 10 client.15354 handle_command_reply: tid=0 2023-12-11T17:46:04.240 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.230+0000 7f04332d1b80 1 client.15354 shutdown 2023-12-11T17:46:04.240 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.231+0000 7f04332d1b80 1 -- 172.21.15.59:0/2747572555 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac42e2c380 msgr2=0x55ac42e3d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:46:04.240 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.231+0000 7f04332d1b80 1 --2- 172.21.15.59:0/2747572555 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac42e2c380 0x55ac42e3d210 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f04140098e0 tx=0x7f04140092f0 comp rx=0 tx=0).stop 2023-12-11T17:46:04.240 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.231+0000 7f04332d1b80 1 -- 172.21.15.59:0/2747572555 shutdown_connections 2023-12-11T17:46:04.240 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.231+0000 7f04332d1b80 1 --2- 172.21.15.59:0/2747572555 >> [v2:172.21.15.115:6836/1209365460,v1:172.21.15.115:6837/1209365460] conn(0x55ac42e9cb50 0x55ac42e99010 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:04.240 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.231+0000 7f04332d1b80 1 --2- 172.21.15.59:0/2747572555 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ac42e2dbd0 0x55ac42e39da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:04.240 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.231+0000 7f04332d1b80 1 --2- 172.21.15.59:0/2747572555 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac42e2c380 0x55ac42e3d210 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:04.240 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.231+0000 7f03fffff700 0 client.15354 ms_handle_reset on v2:172.21.15.115:6836/1209365460 2023-12-11T17:46:04.240 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.231+0000 7f04332d1b80 1 --2- 172.21.15.59:0/2747572555 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac42e2cce0 0x55ac42e3d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:04.240 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.231+0000 7f04332d1b80 1 -- 172.21.15.59:0/2747572555 >> 172.21.15.59:0/2747572555 conn(0x55ac42d5b790 msgr2=0x55ac42e28920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:46:04.240 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.231+0000 7f04332d1b80 1 -- 172.21.15.59:0/2747572555 shutdown_connections 2023-12-11T17:46:04.241 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.231+0000 7f04332d1b80 1 -- 172.21.15.59:0/2747572555 wait complete. 2023-12-11T17:46:04.241 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.231+0000 7f04332d1b80 20 client.15354 trim_cache size 0 max 16384 2023-12-11T17:46:04.241 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2023-12-11T17:46:04.241 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_fsid": "d0a07eaa-984b-11ee-95a3-87774f69a715", 2023-12-11T17:46:04.241 INFO:teuthology.orchestra.run.smithi059.stdout: "whoami": 2, 2023-12-11T17:46:04.241 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 24482, 2023-12-11T17:46:04.241 INFO:teuthology.orchestra.run.smithi059.stdout: "want_state": "up:active", 2023-12-11T17:46:04.241 INFO:teuthology.orchestra.run.smithi059.stdout: "state": "up:active", 2023-12-11T17:46:04.241 INFO:teuthology.orchestra.run.smithi059.stdout: "fs_name": "cephfs", 2023-12-11T17:46:04.241 INFO:teuthology.orchestra.run.smithi059.stdout: "rank_uptime": 77.466232396999999, 2023-12-11T17:46:04.241 INFO:teuthology.orchestra.run.smithi059.stdout: "mdsmap_epoch": 24, 2023-12-11T17:46:04.242 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch": 88, 2023-12-11T17:46:04.242 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch_barrier": 86, 2023-12-11T17:46:04.242 INFO:teuthology.orchestra.run.smithi059.stdout: "uptime": 118.270432359 2023-12-11T17:46:04.242 INFO:teuthology.orchestra.run.smithi059.stdout:} 2023-12-11T17:46:04.243 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.231+0000 7f04270d8700 1 -- 172.21.15.59:0/1752970833 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f04100a1510 msgr2=0x7f04100a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:46:04.243 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.231+0000 7f04270d8700 1 --2- 172.21.15.59:0/1752970833 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f04100a1510 0x7f04100a39d0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f0414009fd0 tx=0x7f041400b540 comp rx=0 tx=0).stop 2023-12-11T17:46:04.243 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.231+0000 7f04270d8700 1 -- 172.21.15.59:0/1752970833 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0420102010 msgr2=0x7f0420103130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:46:04.243 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.231+0000 7f04270d8700 1 --2- 172.21.15.59:0/1752970833 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0420102010 0x7f0420103130 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f0418007ee0 tx=0x7f041800d3b0 comp rx=0 tx=0).stop 2023-12-11T17:46:04.243 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.231+0000 7f04270d8700 1 -- 172.21.15.59:0/1752970833 shutdown_connections 2023-12-11T17:46:04.243 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.231+0000 7f04270d8700 1 --2- 172.21.15.59:0/1752970833 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f04100a1510 0x7f04100a39d0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:04.243 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.231+0000 7f04270d8700 1 --2- 172.21.15.59:0/1752970833 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0420102010 0x7f0420103130 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:04.243 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.231+0000 7f04270d8700 1 --2- 172.21.15.59:0/1752970833 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0420100480 0x7f0420104560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:04.243 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.231+0000 7f04270d8700 1 --2- 172.21.15.59:0/1752970833 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0420101670 0x7f0420102bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:04.243 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.231+0000 7f04270d8700 1 -- 172.21.15.59:0/1752970833 >> 172.21.15.59:0/1752970833 conn(0x7f04200fb9f0 msgr2=0x7f0420068610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:46:04.243 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.232+0000 7f04270d8700 1 -- 172.21.15.59:0/1752970833 shutdown_connections 2023-12-11T17:46:04.243 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.232+0000 7f04270d8700 1 -- 172.21.15.59:0/1752970833 wait complete. 2023-12-11T17:46:04.247 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2023-12-11T17:46:04.385 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.370+0000 7f74ca01c700 1 -- 172.21.15.59:0/2783953349 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f74c4100480 msgr2=0x7f74c41008a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:46:04.385 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.370+0000 7f74ca01c700 1 --2- 172.21.15.59:0/2783953349 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f74c4100480 0x7f74c41008a0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f74bc0099c0 tx=0x7f74bc009cd0 comp rx=0 tx=0).stop 2023-12-11T17:46:04.385 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.370+0000 7f74ca01c700 1 -- 172.21.15.59:0/2783953349 shutdown_connections 2023-12-11T17:46:04.385 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.370+0000 7f74ca01c700 1 --2- 172.21.15.59:0/2783953349 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74c4102010 0x7f74c41088c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:04.385 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.370+0000 7f74ca01c700 1 --2- 172.21.15.59:0/2783953349 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f74c4100480 0x7f74c41008a0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:04.385 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.370+0000 7f74ca01c700 1 --2- 172.21.15.59:0/2783953349 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f74c4101670 0x7f74c4101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:04.385 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.370+0000 7f74ca01c700 1 -- 172.21.15.59:0/2783953349 >> 172.21.15.59:0/2783953349 conn(0x7f74c40fb9f0 msgr2=0x7f74c40fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:46:04.385 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.370+0000 7f74ca01c700 1 -- 172.21.15.59:0/2783953349 shutdown_connections 2023-12-11T17:46:04.385 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.370+0000 7f74ca01c700 1 -- 172.21.15.59:0/2783953349 wait complete. 2023-12-11T17:46:04.385 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.371+0000 7f74ca01c700 1 Processor -- start 2023-12-11T17:46:04.385 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.371+0000 7f74ca01c700 1 -- start start 2023-12-11T17:46:04.385 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.371+0000 7f74ca01c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f74c4100480 0x7f74c4069150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:04.386 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.371+0000 7f74ca01c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f74c4101670 0x7f74c419c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:04.386 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.371+0000 7f74ca01c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74c4102010 0x7f74c419e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:04.386 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.371+0000 7f74ca01c700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74c419f060 con 0x7f74c4101670 2023-12-11T17:46:04.386 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.371+0000 7f74ca01c700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74c419f1d0 con 0x7f74c4100480 2023-12-11T17:46:04.386 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.371+0000 7f74ca01c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74c419f390 con 0x7f74c4102010 2023-12-11T17:46:04.386 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.372+0000 7f74c2ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f74c4101670 0x7f74c419c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:46:04.386 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.372+0000 7f74c2ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f74c4101670 0x7f74c419c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34548/0 (socket says 172.21.15.59:34548) 2023-12-11T17:46:04.386 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.372+0000 7f74c2ffd700 1 -- 172.21.15.59:0/3791494321 learned_addr learned my addr 172.21.15.59:0/3791494321 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:46:04.387 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.372+0000 7f74c3fff700 1 --2- 172.21.15.59:0/3791494321 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74c4102010 0x7f74c419e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:46:04.387 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.372+0000 7f74c37fe700 1 --2- 172.21.15.59:0/3791494321 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f74c4100480 0x7f74c4069150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:46:04.387 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.372+0000 7f74c2ffd700 1 -- 172.21.15.59:0/3791494321 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74c4102010 msgr2=0x7f74c419e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:46:04.387 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.372+0000 7f74c2ffd700 1 --2- 172.21.15.59:0/3791494321 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74c4102010 0x7f74c419e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:04.387 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.372+0000 7f74c2ffd700 1 -- 172.21.15.59:0/3791494321 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f74c4100480 msgr2=0x7f74c4069150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:46:04.387 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.372+0000 7f74c2ffd700 1 --2- 172.21.15.59:0/3791494321 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f74c4100480 0x7f74c4069150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:04.387 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.372+0000 7f74c2ffd700 1 -- 172.21.15.59:0/3791494321 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74bc0095f0 con 0x7f74c4101670 2023-12-11T17:46:04.388 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.372+0000 7f74c2ffd700 1 --2- 172.21.15.59:0/3791494321 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f74c4101670 0x7f74c419c250 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f74ac00d950 tx=0x7f74ac00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:46:04.388 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.372+0000 7f74c0ff9700 1 -- 172.21.15.59:0/3791494321 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74ac009940 con 0x7f74c4101670 2023-12-11T17:46:04.388 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.372+0000 7f74c0ff9700 1 -- 172.21.15.59:0/3791494321 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f74ac010460 con 0x7f74c4101670 2023-12-11T17:46:04.388 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.372+0000 7f74ca01c700 1 -- 172.21.15.59:0/3791494321 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74c419f610 con 0x7f74c4101670 2023-12-11T17:46:04.388 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.372+0000 7f74c0ff9700 1 -- 172.21.15.59:0/3791494321 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74ac00f620 con 0x7f74c4101670 2023-12-11T17:46:04.388 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.372+0000 7f74ca01c700 1 -- 172.21.15.59:0/3791494321 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74c419fb20 con 0x7f74c4101670 2023-12-11T17:46:04.389 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.375+0000 7f74c0ff9700 1 -- 172.21.15.59:0/3791494321 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f74ac0108d0 con 0x7f74c4101670 2023-12-11T17:46:04.389 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.375+0000 7f74c0ff9700 1 --2- 172.21.15.59:0/3791494321 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f74b00a1510 0x7f74b00a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:04.389 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.376+0000 7f74c37fe700 1 --2- 172.21.15.59:0/3791494321 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f74b00a1510 0x7f74b00a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:46:04.389 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.376+0000 7f74c0ff9700 1 -- 172.21.15.59:0/3791494321 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f74ac0cd000 con 0x7f74c4101670 2023-12-11T17:46:04.389 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.376+0000 7f74c37fe700 1 --2- 172.21.15.59:0/3791494321 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f74b00a1510 0x7f74b00a39d0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f74bc00b580 tx=0x7f74bc005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:46:04.390 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.376+0000 7f74d6215b80 1 Processor -- start 2023-12-11T17:46:04.390 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.376+0000 7f74d6215b80 1 -- start start 2023-12-11T17:46:04.390 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.377+0000 7f74d6215b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ab02ed6380 0x55ab02ed67a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:04.390 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.377+0000 7f74d6215b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ab02ed6ce0 0x55ab02ed75f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:04.390 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.377+0000 7f74d6215b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ab02ed7bd0 0x55ab02ed8050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:04.390 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.377+0000 7f74d6215b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab02e31b00 con 0x55ab02ed6ce0 2023-12-11T17:46:04.390 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.377+0000 7f74d6215b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab02e2f420 con 0x55ab02ed6380 2023-12-11T17:46:04.390 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.377+0000 7f74d6215b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab02e2ad10 con 0x55ab02ed7bd0 2023-12-11T17:46:04.390 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.377+0000 7f74c2ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ab02ed6ce0 0x55ab02ed75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:46:04.391 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.377+0000 7f74c2ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ab02ed6ce0 0x55ab02ed75f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34556/0 (socket says 172.21.15.59:34556) 2023-12-11T17:46:04.391 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.377+0000 7f74c2ffd700 1 -- 172.21.15.59:0/143682908 learned_addr learned my addr 172.21.15.59:0/143682908 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:46:04.391 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.377+0000 7f74c3fff700 1 --2- 172.21.15.59:0/143682908 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ab02ed7bd0 0x55ab02ed8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:46:04.391 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.377+0000 7f74c37fe700 1 --2- 172.21.15.59:0/143682908 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ab02ed6380 0x55ab02ed67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:46:04.391 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.377+0000 7f74c2ffd700 1 -- 172.21.15.59:0/143682908 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ab02ed7bd0 msgr2=0x55ab02ed8050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T17:46:04.392 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.377+0000 7f74c2ffd700 1 --2- 172.21.15.59:0/143682908 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ab02ed7bd0 0x55ab02ed8050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:04.392 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.377+0000 7f74c2ffd700 1 -- 172.21.15.59:0/143682908 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ab02ed6380 msgr2=0x55ab02ed67a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T17:46:04.392 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.377+0000 7f74c2ffd700 1 --2- 172.21.15.59:0/143682908 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ab02ed6380 0x55ab02ed67a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:04.392 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.377+0000 7f74c2ffd700 1 -- 172.21.15.59:0/143682908 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ab02ca75e0 con 0x55ab02ed6ce0 2023-12-11T17:46:04.392 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.377+0000 7f74c37fe700 1 --2- 172.21.15.59:0/143682908 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ab02ed6380 0x55ab02ed67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-11T17:46:04.393 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.377+0000 7f74c2ffd700 1 --2- 172.21.15.59:0/143682908 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ab02ed6ce0 0x55ab02ed75f0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f74ac096930 tx=0x7f74ac099920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dba654d31b80316f server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:46:04.393 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.377+0000 7f74b9ffb700 1 -- 172.21.15.59:0/143682908 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74ac0a2250 con 0x55ab02ed6ce0 2023-12-11T17:46:04.393 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.377+0000 7f74b9ffb700 1 -- 172.21.15.59:0/143682908 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f74ac0ca900 con 0x55ab02ed6ce0 2023-12-11T17:46:04.393 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.377+0000 7f74b9ffb700 1 -- 172.21.15.59:0/143682908 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74ac0a1dc0 con 0x55ab02ed6ce0 2023-12-11T17:46:04.393 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.378+0000 7f74d6215b80 1 -- 172.21.15.59:0/143682908 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ab02ed6ce0 msgr2=0x55ab02ed75f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:46:04.393 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.378+0000 7f74d6215b80 1 --2- 172.21.15.59:0/143682908 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ab02ed6ce0 0x55ab02ed75f0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f74ac096930 tx=0x7f74ac099920 comp rx=0 tx=0).stop 2023-12-11T17:46:04.394 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.378+0000 7f74d6215b80 1 -- 172.21.15.59:0/143682908 shutdown_connections 2023-12-11T17:46:04.394 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.378+0000 7f74d6215b80 1 --2- 172.21.15.59:0/143682908 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ab02ed7bd0 0x55ab02ed8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:04.394 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.378+0000 7f74d6215b80 1 --2- 172.21.15.59:0/143682908 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ab02ed6380 0x55ab02ed67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:04.394 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.378+0000 7f74d6215b80 1 --2- 172.21.15.59:0/143682908 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ab02ed6ce0 0x55ab02ed75f0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:04.394 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.378+0000 7f74d6215b80 1 -- 172.21.15.59:0/143682908 >> 172.21.15.59:0/143682908 conn(0x55ab02e05790 msgr2=0x55ab02ed2d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:46:04.394 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.378+0000 7f74d6215b80 1 -- 172.21.15.59:0/143682908 shutdown_connections 2023-12-11T17:46:04.394 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.378+0000 7f74d6215b80 1 -- 172.21.15.59:0/143682908 wait complete. 2023-12-11T17:46:04.394 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.378+0000 7f74d6215b80 1 -- start start 2023-12-11T17:46:04.394 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.379+0000 7f74d6215b80 1 Processor -- start 2023-12-11T17:46:04.395 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.379+0000 7f74d6215b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ab02ed6380 0x55ab02ee0190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:04.395 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.379+0000 7f74d6215b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ab02ed6ce0 0x55ab02ee06d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:04.395 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.379+0000 7f74d6215b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ab02ed7bd0 0x55ab02ee7710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:04.395 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.379+0000 7f74d6215b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab02de4510 con 0x55ab02ed7bd0 2023-12-11T17:46:04.395 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.379+0000 7f74d6215b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab02de41b0 con 0x55ab02ed6380 2023-12-11T17:46:04.395 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.379+0000 7f74d6215b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab02ca98d0 con 0x55ab02ed6ce0 2023-12-11T17:46:04.395 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.379+0000 7f74d6215b80 10 client.0 _ll_register_callbacks cb 0x55ab02d0fa30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T17:46:04.395 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.379+0000 7f74c3fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ab02ed7bd0 0x55ab02ee7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:46:04.395 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.379+0000 7f74c3fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ab02ed7bd0 0x55ab02ee7710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34564/0 (socket says 172.21.15.59:34564) 2023-12-11T17:46:04.395 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.379+0000 7f74c3fff700 1 -- 172.21.15.59:0/365910016 learned_addr learned my addr 172.21.15.59:0/365910016 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:46:04.395 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.379+0000 7f74c37fe700 1 --2- 172.21.15.59:0/365910016 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ab02ed6380 0x55ab02ee0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:46:04.395 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.379+0000 7f74c2ffd700 1 --2- 172.21.15.59:0/365910016 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ab02ed6ce0 0x55ab02ee06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:46:04.395 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.379+0000 7f74c3fff700 1 -- 172.21.15.59:0/365910016 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ab02ed6ce0 msgr2=0x55ab02ee06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T17:46:04.395 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.379+0000 7f74c3fff700 1 --2- 172.21.15.59:0/365910016 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ab02ed6ce0 0x55ab02ee06d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:04.396 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.379+0000 7f74c3fff700 1 -- 172.21.15.59:0/365910016 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ab02ed6380 msgr2=0x55ab02ee0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T17:46:04.396 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.379+0000 7f74c3fff700 1 --2- 172.21.15.59:0/365910016 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ab02ed6380 0x55ab02ee0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:04.396 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.379+0000 7f74c3fff700 1 -- 172.21.15.59:0/365910016 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74c419fb20 con 0x55ab02ed7bd0 2023-12-11T17:46:04.396 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.380+0000 7f74c3fff700 1 --2- 172.21.15.59:0/365910016 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ab02ed7bd0 0x55ab02ee7710 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f74b4008e60 tx=0x7f74b40114a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc5c4017c779ef67 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:46:04.396 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.380+0000 7f74a2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2023-12-11T17:46:04.396 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.380+0000 7f74a2ffd700 1 -- 172.21.15.59:0/365910016 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b400f050 con 0x55ab02ed7bd0 2023-12-11T17:46:04.396 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.380+0000 7f74a2ffd700 1 -- 172.21.15.59:0/365910016 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f74b4011e40 con 0x55ab02ed7bd0 2023-12-11T17:46:04.396 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.380+0000 7f74d6215b80 1 -- 172.21.15.59:0/365910016 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ab02ec9af0 con 0x55ab02ed7bd0 2023-12-11T17:46:04.396 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.380+0000 7f74a2ffd700 1 -- 172.21.15.59:0/365910016 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f74b4008e90 con 0x55ab02ed7bd0 2023-12-11T17:46:04.396 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.380+0000 7f74d6215b80 10 client.15366 fetch_fsmap learned FSMap version 24 2023-12-11T17:46:04.396 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.380+0000 7f74d6215b80 1 -- 172.21.15.59:0/365910016 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55ab02c53d90 con 0x55ab02ed7bd0 2023-12-11T17:46:04.396 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.380+0000 7f74a2ffd700 1 -- 172.21.15.59:0/365910016 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ab02ec9af0 con 0x55ab02ed7bd0 2023-12-11T17:46:04.396 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.380+0000 7f74a2ffd700 1 -- 172.21.15.59:0/365910016 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f74b4015590 con 0x55ab02ed7bd0 2023-12-11T17:46:04.396 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.380+0000 7f74d6215b80 10 client.15366 fetch_fsmap finished waiting for FSMap version 24 2023-12-11T17:46:04.396 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.380+0000 7f74d6215b80 10 client.15366 resolve_mds: resolved name 'i' to daemon mds.i 2023-12-11T17:46:04.397 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.380+0000 7f74d6215b80 20 client.15366 populate_metadata read hostname 'smithi059' 2023-12-11T17:46:04.397 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.380+0000 7f74d6215b80 1 --2- 172.21.15.59:0/365910016 >> [v2:172.21.15.181:6838/2214384806,v1:172.21.15.181:6839/2214384806] conn(0x55ab02eebe70 0x55ab02eea8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:04.397 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.380+0000 7f74d6215b80 4 client.15366 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-11T17:46:04.397 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.380+0000 7f74d6215b80 1 -- 172.21.15.59:0/365910016 --> [v2:172.21.15.181:6838/2214384806,v1:172.21.15.181:6839/2214384806] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ab02ea5ef0 con 0x55ab02eebe70 2023-12-11T17:46:04.397 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.381+0000 7f74c37fe700 1 --2- 172.21.15.59:0/365910016 >> [v2:172.21.15.181:6838/2214384806,v1:172.21.15.181:6839/2214384806] conn(0x55ab02eebe70 0x55ab02eea8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:46:04.397 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.381+0000 7f74c37fe700 1 --2- 172.21.15.59:0/365910016 >> [v2:172.21.15.181:6838/2214384806,v1:172.21.15.181:6839/2214384806] conn(0x55ab02eebe70 0x55ab02eea8d0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cec23e4c2a0a4a8a server_cookie=40074c47be25eb71 in_seq=0 out_seq=0 2023-12-11T17:46:04.397 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.381+0000 7f74a2ffd700 10 client.15366 ms_handle_connect on v2:172.21.15.181:6838/2214384806 2023-12-11T17:46:04.398 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.382+0000 7f74a2ffd700 1 -- 172.21.15.59:0/365910016 <== mds.1 v2:172.21.15.181:6838/2214384806 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ab02ea5ef0 con 0x55ab02eebe70 2023-12-11T17:46:04.398 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.382+0000 7f74a2ffd700 10 client.15366 handle_command_reply: tid=0 2023-12-11T17:46:04.398 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.382+0000 7f74d6215b80 1 client.15366 shutdown 2023-12-11T17:46:04.398 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.382+0000 7f74d6215b80 1 -- 172.21.15.59:0/365910016 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ab02ed7bd0 msgr2=0x55ab02ee7710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:46:04.399 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.382+0000 7f74d6215b80 1 --2- 172.21.15.59:0/365910016 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ab02ed7bd0 0x55ab02ee7710 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f74b4008e60 tx=0x7f74b40114a0 comp rx=0 tx=0).stop 2023-12-11T17:46:04.399 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.382+0000 7f74d6215b80 1 -- 172.21.15.59:0/365910016 shutdown_connections 2023-12-11T17:46:04.399 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.382+0000 7f74d6215b80 1 --2- 172.21.15.59:0/365910016 >> [v2:172.21.15.181:6838/2214384806,v1:172.21.15.181:6839/2214384806] conn(0x55ab02eebe70 0x55ab02eea8d0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:04.399 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.382+0000 7f74d6215b80 1 --2- 172.21.15.59:0/365910016 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ab02ed7bd0 0x55ab02ee7710 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:04.399 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.382+0000 7f74d6215b80 1 --2- 172.21.15.59:0/365910016 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ab02ed6380 0x55ab02ee0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:04.399 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.382+0000 7f74a2ffd700 0 client.15366 ms_handle_reset on v2:172.21.15.181:6838/2214384806 2023-12-11T17:46:04.399 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.382+0000 7f74d6215b80 1 --2- 172.21.15.59:0/365910016 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ab02ed6ce0 0x55ab02ee06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:04.399 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.382+0000 7f74d6215b80 1 -- 172.21.15.59:0/365910016 >> 172.21.15.59:0/365910016 conn(0x55ab02e05790 msgr2=0x55ab02ed19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:46:04.399 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.382+0000 7f74d6215b80 1 -- 172.21.15.59:0/365910016 shutdown_connections 2023-12-11T17:46:04.399 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.382+0000 7f74d6215b80 1 -- 172.21.15.59:0/365910016 wait complete. 2023-12-11T17:46:04.399 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.382+0000 7f74d6215b80 20 client.15366 trim_cache size 0 max 16384 2023-12-11T17:46:04.410 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.400+0000 7f74d6215b80 1 Processor -- start 2023-12-11T17:46:04.410 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.400+0000 7f74d6215b80 1 -- start start 2023-12-11T17:46:04.410 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.400+0000 7f74d6215b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ab02ed6380 0x55ab02eea8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:04.410 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.400+0000 7f74d6215b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ab02ed6ce0 0x55ab02ee7630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:04.410 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.400+0000 7f74d6215b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ab02ed7bd0 0x55ab02eeae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:04.410 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.400+0000 7f74d6215b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab02ed5f20 con 0x55ab02ed7bd0 2023-12-11T17:46:04.411 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.400+0000 7f74d6215b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab02ee1090 con 0x55ab02ed6ce0 2023-12-11T17:46:04.411 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.400+0000 7f74d6215b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab02edfd00 con 0x55ab02ed6380 2023-12-11T17:46:04.411 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.400+0000 7f74c3fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ab02ed7bd0 0x55ab02eeae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:46:04.411 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.400+0000 7f74c3fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ab02ed7bd0 0x55ab02eeae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34570/0 (socket says 172.21.15.59:34570) 2023-12-11T17:46:04.411 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.400+0000 7f74c3fff700 1 -- 172.21.15.59:0/3936327662 learned_addr learned my addr 172.21.15.59:0/3936327662 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:46:04.412 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.401+0000 7f74c37fe700 1 --2- 172.21.15.59:0/3936327662 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ab02ed6380 0x55ab02eea8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:46:04.412 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.401+0000 7f74c3fff700 1 -- 172.21.15.59:0/3936327662 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ab02ed6380 msgr2=0x55ab02eea8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T17:46:04.412 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.401+0000 7f74c2ffd700 1 --2- 172.21.15.59:0/3936327662 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ab02ed6ce0 0x55ab02ee7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:46:04.412 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.401+0000 7f74c3fff700 1 --2- 172.21.15.59:0/3936327662 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ab02ed6380 0x55ab02eea8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:04.412 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.401+0000 7f74c3fff700 1 -- 172.21.15.59:0/3936327662 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ab02ed6ce0 msgr2=0x55ab02ee7630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T17:46:04.412 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.401+0000 7f74c3fff700 1 --2- 172.21.15.59:0/3936327662 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ab02ed6ce0 0x55ab02ee7630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:04.412 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.401+0000 7f74c3fff700 1 -- 172.21.15.59:0/3936327662 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ab02c3cc10 con 0x55ab02ed7bd0 2023-12-11T17:46:04.413 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.401+0000 7f74c37fe700 1 --2- 172.21.15.59:0/3936327662 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ab02ed6380 0x55ab02eea8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-11T17:46:04.413 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.401+0000 7f74c2ffd700 1 --2- 172.21.15.59:0/3936327662 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ab02ed6ce0 0x55ab02ee7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-11T17:46:04.413 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.401+0000 7f74c3fff700 1 --2- 172.21.15.59:0/3936327662 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ab02ed7bd0 0x55ab02eeae10 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f74b40158f0 tx=0x7f74b4007400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7902c4d665886de0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:46:04.413 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.401+0000 7f74ba7fc700 1 -- 172.21.15.59:0/3936327662 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b400f070 con 0x55ab02ed7bd0 2023-12-11T17:46:04.413 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.401+0000 7f74ba7fc700 1 -- 172.21.15.59:0/3936327662 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f74b4008e90 con 0x55ab02ed7bd0 2023-12-11T17:46:04.413 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.401+0000 7f74ba7fc700 1 -- 172.21.15.59:0/3936327662 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b4004e90 con 0x55ab02ed7bd0 2023-12-11T17:46:04.414 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.402+0000 7f74d6215b80 1 -- 172.21.15.59:0/3936327662 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ab02ed7bd0 msgr2=0x55ab02eeae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:46:04.414 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.402+0000 7f74d6215b80 1 --2- 172.21.15.59:0/3936327662 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ab02ed7bd0 0x55ab02eeae10 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f74b40158f0 tx=0x7f74b4007400 comp rx=0 tx=0).stop 2023-12-11T17:46:04.414 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.402+0000 7f74d6215b80 1 -- 172.21.15.59:0/3936327662 shutdown_connections 2023-12-11T17:46:04.414 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.402+0000 7f74d6215b80 1 --2- 172.21.15.59:0/3936327662 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ab02ed7bd0 0x55ab02eeae10 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:04.414 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.402+0000 7f74d6215b80 1 --2- 172.21.15.59:0/3936327662 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ab02ed6380 0x55ab02eea8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:04.414 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.402+0000 7f74d6215b80 1 --2- 172.21.15.59:0/3936327662 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ab02ed6ce0 0x55ab02ee7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:04.414 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.402+0000 7f74d6215b80 1 -- 172.21.15.59:0/3936327662 >> 172.21.15.59:0/3936327662 conn(0x55ab02e05790 msgr2=0x55ab02ed2920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:46:04.415 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.402+0000 7f74d6215b80 1 -- 172.21.15.59:0/3936327662 shutdown_connections 2023-12-11T17:46:04.415 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.402+0000 7f74d6215b80 1 -- 172.21.15.59:0/3936327662 wait complete. 2023-12-11T17:46:04.415 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.402+0000 7f74d6215b80 1 -- start start 2023-12-11T17:46:04.415 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.402+0000 7f74d6215b80 1 Processor -- start 2023-12-11T17:46:04.415 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.402+0000 7f74d6215b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ab02ed6380 0x55ab02ee44f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:04.415 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.402+0000 7f74d6215b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ab02ed6ce0 0x55ab02ee4a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:04.415 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.402+0000 7f74d6215b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ab02ed7bd0 0x55ab02f41f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:04.415 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.402+0000 7f74d6215b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab02ee5110 con 0x55ab02ed7bd0 2023-12-11T17:46:04.415 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.402+0000 7f74d6215b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab02f42470 con 0x55ab02ed6ce0 2023-12-11T17:46:04.415 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.402+0000 7f74d6215b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab02f425b0 con 0x55ab02ed6380 2023-12-11T17:46:04.415 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.402+0000 7f74d6215b80 10 client.0 _ll_register_callbacks cb 0x55ab02d0fa30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T17:46:04.415 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.403+0000 7f74c3fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ab02ed7bd0 0x55ab02f41f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:46:04.415 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.403+0000 7f74c3fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ab02ed7bd0 0x55ab02f41f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34576/0 (socket says 172.21.15.59:34576) 2023-12-11T17:46:04.415 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.403+0000 7f74c3fff700 1 -- 172.21.15.59:0/1572510666 learned_addr learned my addr 172.21.15.59:0/1572510666 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:46:04.415 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.403+0000 7f74c37fe700 1 --2- 172.21.15.59:0/1572510666 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ab02ed6380 0x55ab02ee44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:46:04.416 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.403+0000 7f74c2ffd700 1 --2- 172.21.15.59:0/1572510666 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ab02ed6ce0 0x55ab02ee4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:46:04.416 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.403+0000 7f74c3fff700 1 -- 172.21.15.59:0/1572510666 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ab02ed6380 msgr2=0x55ab02ee44f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T17:46:04.416 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.403+0000 7f74c3fff700 1 --2- 172.21.15.59:0/1572510666 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ab02ed6380 0x55ab02ee44f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:04.416 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.403+0000 7f74c3fff700 1 -- 172.21.15.59:0/1572510666 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ab02ed6ce0 msgr2=0x55ab02ee4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T17:46:04.416 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.403+0000 7f74c3fff700 1 --2- 172.21.15.59:0/1572510666 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ab02ed6ce0 0x55ab02ee4a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:04.416 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.403+0000 7f74c3fff700 1 -- 172.21.15.59:0/1572510666 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ab02c53d90 con 0x55ab02ed7bd0 2023-12-11T17:46:04.416 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.403+0000 7f74c3fff700 1 --2- 172.21.15.59:0/1572510666 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ab02ed7bd0 0x55ab02f41f30 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f74b40079b0 tx=0x7f74b40078d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77ee54f3205eec27 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:46:04.416 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.403+0000 7f74a2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2023-12-11T17:46:04.416 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.403+0000 7f74a2ffd700 1 -- 172.21.15.59:0/1572510666 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b400f040 con 0x55ab02ed7bd0 2023-12-11T17:46:04.416 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.403+0000 7f74a2ffd700 1 -- 172.21.15.59:0/1572510666 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f74b4015b10 con 0x55ab02ed7bd0 2023-12-11T17:46:04.416 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.403+0000 7f74d6215b80 1 -- 172.21.15.59:0/1572510666 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f74ac0ca900 con 0x55ab02ed7bd0 2023-12-11T17:46:04.416 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.403+0000 7f74a2ffd700 1 -- 172.21.15.59:0/1572510666 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f74b4015cd0 con 0x55ab02ed7bd0 2023-12-11T17:46:04.416 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.403+0000 7f74d6215b80 10 client.15372 fetch_fsmap learned FSMap version 24 2023-12-11T17:46:04.417 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.403+0000 7f74d6215b80 1 -- 172.21.15.59:0/1572510666 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55ab02c7a200 con 0x55ab02ed7bd0 2023-12-11T17:46:04.417 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.404+0000 7f74a2ffd700 1 -- 172.21.15.59:0/1572510666 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f74ac0ca900 con 0x55ab02ed7bd0 2023-12-11T17:46:04.417 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.404+0000 7f74a2ffd700 1 -- 172.21.15.59:0/1572510666 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f74b4016440 con 0x55ab02ed7bd0 2023-12-11T17:46:04.417 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.404+0000 7f74d6215b80 10 client.15372 fetch_fsmap finished waiting for FSMap version 24 2023-12-11T17:46:04.417 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.404+0000 7f74d6215b80 10 client.15372 resolve_mds: resolved name 'i' to daemon mds.i 2023-12-11T17:46:04.417 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.404+0000 7f74d6215b80 20 client.15372 populate_metadata read hostname 'smithi059' 2023-12-11T17:46:04.417 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.404+0000 7f74d6215b80 1 --2- 172.21.15.59:0/1572510666 >> [v2:172.21.15.181:6838/2214384806,v1:172.21.15.181:6839/2214384806] conn(0x55ab02f473f0 0x55ab02f438d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:04.417 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.404+0000 7f74d6215b80 4 client.15372 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "status"}] 2023-12-11T17:46:04.417 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.404+0000 7f74d6215b80 1 -- 172.21.15.59:0/1572510666 --> [v2:172.21.15.181:6838/2214384806,v1:172.21.15.181:6839/2214384806] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55ab02df0f80 con 0x55ab02f473f0 2023-12-11T17:46:04.417 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.404+0000 7f74c37fe700 1 --2- 172.21.15.59:0/1572510666 >> [v2:172.21.15.181:6838/2214384806,v1:172.21.15.181:6839/2214384806] conn(0x55ab02f473f0 0x55ab02f438d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:46:04.417 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.405+0000 7f74a2ffd700 10 client.15372 ms_handle_connect on v2:172.21.15.181:6838/2214384806 2023-12-11T17:46:04.417 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.405+0000 7f74c37fe700 1 --2- 172.21.15.59:0/1572510666 >> [v2:172.21.15.181:6838/2214384806,v1:172.21.15.181:6839/2214384806] conn(0x55ab02f473f0 0x55ab02f438d0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b9bc26df784a66d0 server_cookie=765ce666c0c3106e in_seq=0 out_seq=0 2023-12-11T17:46:04.417 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.405+0000 7f74a2ffd700 1 -- 172.21.15.59:0/1572510666 <== mds.1 v2:172.21.15.181:6838/2214384806 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55ab02df0f80 con 0x55ab02f473f0 2023-12-11T17:46:04.417 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.405+0000 7f74a2ffd700 10 client.15372 handle_command_reply: tid=0 2023-12-11T17:46:04.417 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.405+0000 7f74d6215b80 1 client.15372 shutdown 2023-12-11T17:46:04.418 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.405+0000 7f74d6215b80 1 -- 172.21.15.59:0/1572510666 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ab02ed7bd0 msgr2=0x55ab02f41f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:46:04.418 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.405+0000 7f74d6215b80 1 --2- 172.21.15.59:0/1572510666 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ab02ed7bd0 0x55ab02f41f30 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f74b40079b0 tx=0x7f74b40078d0 comp rx=0 tx=0).stop 2023-12-11T17:46:04.418 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.405+0000 7f74d6215b80 1 -- 172.21.15.59:0/1572510666 shutdown_connections 2023-12-11T17:46:04.418 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.405+0000 7f74d6215b80 1 --2- 172.21.15.59:0/1572510666 >> [v2:172.21.15.181:6838/2214384806,v1:172.21.15.181:6839/2214384806] conn(0x55ab02f473f0 0x55ab02f438d0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:04.418 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.405+0000 7f74d6215b80 1 --2- 172.21.15.59:0/1572510666 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ab02ed7bd0 0x55ab02f41f30 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:04.418 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.405+0000 7f74a2ffd700 0 client.15372 ms_handle_reset on v2:172.21.15.181:6838/2214384806 2023-12-11T17:46:04.418 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.405+0000 7f74d6215b80 1 --2- 172.21.15.59:0/1572510666 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ab02ed6380 0x55ab02ee44f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:04.418 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.405+0000 7f74d6215b80 1 --2- 172.21.15.59:0/1572510666 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55ab02ed6ce0 0x55ab02ee4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:04.418 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.405+0000 7f74d6215b80 1 -- 172.21.15.59:0/1572510666 >> 172.21.15.59:0/1572510666 conn(0x55ab02e05790 msgr2=0x55ab02eea8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:46:04.419 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.405+0000 7f74d6215b80 1 -- 172.21.15.59:0/1572510666 shutdown_connections 2023-12-11T17:46:04.419 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.405+0000 7f74d6215b80 1 -- 172.21.15.59:0/1572510666 wait complete. 2023-12-11T17:46:04.419 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.405+0000 7f74d6215b80 20 client.15372 trim_cache size 0 max 16384 2023-12-11T17:46:04.419 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2023-12-11T17:46:04.420 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_fsid": "d0a07eaa-984b-11ee-95a3-87774f69a715", 2023-12-11T17:46:04.420 INFO:teuthology.orchestra.run.smithi059.stdout: "whoami": 1, 2023-12-11T17:46:04.420 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 24484, 2023-12-11T17:46:04.420 INFO:teuthology.orchestra.run.smithi059.stdout: "want_state": "up:active", 2023-12-11T17:46:04.420 INFO:teuthology.orchestra.run.smithi059.stdout: "state": "up:active", 2023-12-11T17:46:04.420 INFO:teuthology.orchestra.run.smithi059.stdout: "fs_name": "cephfs", 2023-12-11T17:46:04.420 INFO:teuthology.orchestra.run.smithi059.stdout: "rank_uptime": 78.652548519999996, 2023-12-11T17:46:04.420 INFO:teuthology.orchestra.run.smithi059.stdout: "mdsmap_epoch": 24, 2023-12-11T17:46:04.420 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch": 88, 2023-12-11T17:46:04.420 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch_barrier": 86, 2023-12-11T17:46:04.420 INFO:teuthology.orchestra.run.smithi059.stdout: "uptime": 116.15607043999999 2023-12-11T17:46:04.420 INFO:teuthology.orchestra.run.smithi059.stdout:} 2023-12-11T17:46:04.422 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.406+0000 7f74ca01c700 1 -- 172.21.15.59:0/3791494321 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f74b00a1510 msgr2=0x7f74b00a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:46:04.422 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.406+0000 7f74ca01c700 1 --2- 172.21.15.59:0/3791494321 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f74b00a1510 0x7f74b00a39d0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f74bc00b580 tx=0x7f74bc005fb0 comp rx=0 tx=0).stop 2023-12-11T17:46:04.422 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.406+0000 7f74ca01c700 1 -- 172.21.15.59:0/3791494321 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f74c4101670 msgr2=0x7f74c419c250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:46:04.422 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.406+0000 7f74ca01c700 1 --2- 172.21.15.59:0/3791494321 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f74c4101670 0x7f74c419c250 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f74ac00d950 tx=0x7f74ac00dc60 comp rx=0 tx=0).stop 2023-12-11T17:46:04.422 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.406+0000 7f74ca01c700 1 -- 172.21.15.59:0/3791494321 shutdown_connections 2023-12-11T17:46:04.422 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.406+0000 7f74ca01c700 1 --2- 172.21.15.59:0/3791494321 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f74b00a1510 0x7f74b00a39d0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:04.422 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.406+0000 7f74ca01c700 1 --2- 172.21.15.59:0/3791494321 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74c4102010 0x7f74c419e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:04.422 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.406+0000 7f74ca01c700 1 --2- 172.21.15.59:0/3791494321 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f74c4100480 0x7f74c4069150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:04.422 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.406+0000 7f74ca01c700 1 --2- 172.21.15.59:0/3791494321 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f74c4101670 0x7f74c419c250 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:04.422 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.406+0000 7f74ca01c700 1 -- 172.21.15.59:0/3791494321 >> 172.21.15.59:0/3791494321 conn(0x7f74c40fb9f0 msgr2=0x7f74c4104a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:46:04.422 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.406+0000 7f74ca01c700 1 -- 172.21.15.59:0/3791494321 shutdown_connections 2023-12-11T17:46:04.422 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.406+0000 7f74ca01c700 1 -- 172.21.15.59:0/3791494321 wait complete. 2023-12-11T17:46:04.423 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-12-11T17:46:04.553 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.546+0000 7f3649a5a700 1 -- 172.21.15.59:0/2617685874 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f36441015a0 msgr2=0x7f36441019c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:46:04.553 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.546+0000 7f3649a5a700 1 --2- 172.21.15.59:0/2617685874 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f36441015a0 0x7f36441019c0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f3634009a40 tx=0x7f3634009d50 comp rx=0 tx=0).stop 2023-12-11T17:46:04.553 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.547+0000 7f3649a5a700 1 -- 172.21.15.59:0/2617685874 shutdown_connections 2023-12-11T17:46:04.553 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.547+0000 7f3649a5a700 1 --2- 172.21.15.59:0/2617685874 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3644103130 0x7f36441099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:04.553 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.547+0000 7f3649a5a700 1 --2- 172.21.15.59:0/2617685874 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f36441015a0 0x7f36441019c0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:04.553 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.547+0000 7f3649a5a700 1 --2- 172.21.15.59:0/2617685874 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3644102790 0x7f3644102bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:04.553 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.547+0000 7f3649a5a700 1 -- 172.21.15.59:0/2617685874 >> 172.21.15.59:0/2617685874 conn(0x7f36440fcb10 msgr2=0x7f36440fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:46:04.553 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.547+0000 7f3649a5a700 1 -- 172.21.15.59:0/2617685874 shutdown_connections 2023-12-11T17:46:04.553 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.547+0000 7f3649a5a700 1 -- 172.21.15.59:0/2617685874 wait complete. 2023-12-11T17:46:04.553 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.547+0000 7f3649a5a700 1 Processor -- start 2023-12-11T17:46:04.554 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.547+0000 7f3649a5a700 1 -- start start 2023-12-11T17:46:04.554 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.548+0000 7f3649a5a700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f36441015a0 0x7f3644195390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:04.554 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.548+0000 7f3649a5a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3644102790 0x7f36441958d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:04.554 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.548+0000 7f3649a5a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3644103130 0x7f364419c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:04.554 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.548+0000 7f3649a5a700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f364419cec0 con 0x7f3644103130 2023-12-11T17:46:04.554 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.548+0000 7f3649a5a700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f364419d000 con 0x7f36441015a0 2023-12-11T17:46:04.554 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.548+0000 7f3649a5a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f364419d190 con 0x7f3644102790 2023-12-11T17:46:04.554 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.548+0000 7f36437fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3644103130 0x7f364419c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:46:04.554 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.548+0000 7f36437fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3644103130 0x7f364419c8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34590/0 (socket says 172.21.15.59:34590) 2023-12-11T17:46:04.554 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.548+0000 7f36437fe700 1 -- 172.21.15.59:0/985433070 learned_addr learned my addr 172.21.15.59:0/985433070 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:46:04.554 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.548+0000 7f36427fc700 1 --2- 172.21.15.59:0/985433070 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3644102790 0x7f36441958d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:46:04.554 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.548+0000 7f3642ffd700 1 --2- 172.21.15.59:0/985433070 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f36441015a0 0x7f3644195390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:46:04.554 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.548+0000 7f36437fe700 1 -- 172.21.15.59:0/985433070 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3644102790 msgr2=0x7f36441958d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:46:04.555 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.548+0000 7f36437fe700 1 --2- 172.21.15.59:0/985433070 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3644102790 0x7f36441958d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:04.555 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.548+0000 7f36437fe700 1 -- 172.21.15.59:0/985433070 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f36441015a0 msgr2=0x7f3644195390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:46:04.555 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.548+0000 7f36437fe700 1 --2- 172.21.15.59:0/985433070 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f36441015a0 0x7f3644195390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:04.555 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.548+0000 7f36437fe700 1 -- 172.21.15.59:0/985433070 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36340095f0 con 0x7f3644103130 2023-12-11T17:46:04.555 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.548+0000 7f3642ffd700 1 --2- 172.21.15.59:0/985433070 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f36441015a0 0x7f3644195390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T17:46:04.555 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.548+0000 7f36437fe700 1 --2- 172.21.15.59:0/985433070 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3644103130 0x7f364419c8f0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f363c007ee0 tx=0x7f363c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:46:04.555 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.548+0000 7f3648a58700 1 -- 172.21.15.59:0/985433070 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f363c00ddd0 con 0x7f3644103130 2023-12-11T17:46:04.555 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.548+0000 7f3648a58700 1 -- 172.21.15.59:0/985433070 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f363c00f040 con 0x7f3644103130 2023-12-11T17:46:04.555 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.548+0000 7f3649a5a700 1 -- 172.21.15.59:0/985433070 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f364419d470 con 0x7f3644103130 2023-12-11T17:46:04.555 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.548+0000 7f3648a58700 1 -- 172.21.15.59:0/985433070 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f363c0048e0 con 0x7f3644103130 2023-12-11T17:46:04.556 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.548+0000 7f3649a5a700 1 -- 172.21.15.59:0/985433070 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f364419d930 con 0x7f3644103130 2023-12-11T17:46:04.563 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.551+0000 7f3648a58700 1 -- 172.21.15.59:0/985433070 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f363c003a00 con 0x7f3644103130 2023-12-11T17:46:04.563 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.551+0000 7f3648a58700 1 --2- 172.21.15.59:0/985433070 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f362c0a1510 0x7f362c0a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:04.563 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.551+0000 7f3648a58700 1 -- 172.21.15.59:0/985433070 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f363c0cca80 con 0x7f3644103130 2023-12-11T17:46:04.563 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.551+0000 7f3642ffd700 1 --2- 172.21.15.59:0/985433070 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f362c0a1510 0x7f362c0a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:46:04.563 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.552+0000 7f3642ffd700 1 --2- 172.21.15.59:0/985433070 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f362c0a1510 0x7f362c0a39d0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f363400b540 tx=0x7f3634005c00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:46:04.563 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.552+0000 7f3655c53b80 1 Processor -- start 2023-12-11T17:46:04.563 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.552+0000 7f3655c53b80 1 -- start start 2023-12-11T17:46:04.563 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.552+0000 7f3655c53b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e61eca380 0x558e61eca7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:04.563 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.552+0000 7f3655c53b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558e61ecace0 0x558e61ecb5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:04.563 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.552+0000 7f3655c53b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558e61ecbbd0 0x558e61ecc050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:04.563 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.552+0000 7f3655c53b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e61e25b00 con 0x558e61ecbbd0 2023-12-11T17:46:04.563 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.552+0000 7f3655c53b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e61e23420 con 0x558e61eca380 2023-12-11T17:46:04.564 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.552+0000 7f3655c53b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e61e1ed10 con 0x558e61ecace0 2023-12-11T17:46:04.564 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.552+0000 7f36437fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558e61ecbbd0 0x558e61ecc050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:46:04.564 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.552+0000 7f36437fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558e61ecbbd0 0x558e61ecc050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34598/0 (socket says 172.21.15.59:34598) 2023-12-11T17:46:04.564 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.552+0000 7f36437fe700 1 -- 172.21.15.59:0/1638942185 learned_addr learned my addr 172.21.15.59:0/1638942185 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:46:04.564 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.552+0000 7f36427fc700 1 --2- 172.21.15.59:0/1638942185 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558e61ecace0 0x558e61ecb5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:46:04.564 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.553+0000 7f3642ffd700 1 --2- 172.21.15.59:0/1638942185 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e61eca380 0x558e61eca7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:46:04.564 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.553+0000 7f36437fe700 1 -- 172.21.15.59:0/1638942185 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558e61ecace0 msgr2=0x558e61ecb5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T17:46:04.564 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.553+0000 7f36437fe700 1 --2- 172.21.15.59:0/1638942185 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558e61ecace0 0x558e61ecb5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:04.564 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.553+0000 7f36437fe700 1 -- 172.21.15.59:0/1638942185 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e61eca380 msgr2=0x558e61eca7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T17:46:04.564 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.553+0000 7f36437fe700 1 --2- 172.21.15.59:0/1638942185 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e61eca380 0x558e61eca7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:04.564 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.553+0000 7f36437fe700 1 -- 172.21.15.59:0/1638942185 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558e61c9b5e0 con 0x558e61ecbbd0 2023-12-11T17:46:04.564 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.553+0000 7f36437fe700 1 --2- 172.21.15.59:0/1638942185 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558e61ecbbd0 0x558e61ecc050 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f363c0a17a0 tx=0x7f363c099710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fc64150a2bf8f7 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:46:04.564 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.553+0000 7f3629ffb700 1 -- 172.21.15.59:0/1638942185 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f363c09a040 con 0x558e61ecbbd0 2023-12-11T17:46:04.565 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.553+0000 7f3629ffb700 1 -- 172.21.15.59:0/1638942185 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f363c099940 con 0x558e61ecbbd0 2023-12-11T17:46:04.565 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.553+0000 7f3629ffb700 1 -- 172.21.15.59:0/1638942185 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f363c003d90 con 0x558e61ecbbd0 2023-12-11T17:46:04.565 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.554+0000 7f3655c53b80 1 -- 172.21.15.59:0/1638942185 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558e61ecbbd0 msgr2=0x558e61ecc050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:46:04.565 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.554+0000 7f3655c53b80 1 --2- 172.21.15.59:0/1638942185 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558e61ecbbd0 0x558e61ecc050 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f363c0a17a0 tx=0x7f363c099710 comp rx=0 tx=0).stop 2023-12-11T17:46:04.565 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.554+0000 7f3655c53b80 1 -- 172.21.15.59:0/1638942185 shutdown_connections 2023-12-11T17:46:04.565 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.554+0000 7f3655c53b80 1 --2- 172.21.15.59:0/1638942185 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558e61ecbbd0 0x558e61ecc050 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:04.565 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.554+0000 7f3655c53b80 1 --2- 172.21.15.59:0/1638942185 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e61eca380 0x558e61eca7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:04.565 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.554+0000 7f3655c53b80 1 --2- 172.21.15.59:0/1638942185 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558e61ecace0 0x558e61ecb5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:04.565 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.554+0000 7f3655c53b80 1 -- 172.21.15.59:0/1638942185 >> 172.21.15.59:0/1638942185 conn(0x558e61df9790 msgr2=0x558e61ec6d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:46:04.565 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.554+0000 7f3655c53b80 1 -- 172.21.15.59:0/1638942185 shutdown_connections 2023-12-11T17:46:04.565 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.554+0000 7f3655c53b80 1 -- 172.21.15.59:0/1638942185 wait complete. 2023-12-11T17:46:04.566 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.554+0000 7f3655c53b80 1 -- start start 2023-12-11T17:46:04.566 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.554+0000 7f3655c53b80 1 Processor -- start 2023-12-11T17:46:04.566 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.555+0000 7f3655c53b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e61eca380 0x558e61ed4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:04.566 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.555+0000 7f3655c53b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558e61ecace0 0x558e61ed4690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:04.566 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.555+0000 7f3655c53b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558e61ecbbd0 0x558e61edb6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:04.566 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.555+0000 7f3655c53b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e61dd8510 con 0x558e61ecace0 2023-12-11T17:46:04.566 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.555+0000 7f3655c53b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e61dd81b0 con 0x558e61eca380 2023-12-11T17:46:04.566 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.555+0000 7f3655c53b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e61c9d8d0 con 0x558e61ecbbd0 2023-12-11T17:46:04.566 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.555+0000 7f3655c53b80 10 client.0 _ll_register_callbacks cb 0x558e61d03a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T17:46:04.566 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.555+0000 7f36427fc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558e61ecace0 0x558e61ed4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:46:04.566 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.555+0000 7f36427fc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558e61ecace0 0x558e61ed4690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34602/0 (socket says 172.21.15.59:34602) 2023-12-11T17:46:04.567 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.555+0000 7f36427fc700 1 -- 172.21.15.59:0/2624922578 learned_addr learned my addr 172.21.15.59:0/2624922578 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:46:04.567 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.555+0000 7f36437fe700 1 --2- 172.21.15.59:0/2624922578 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558e61ecbbd0 0x558e61edb6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:46:04.567 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.555+0000 7f3642ffd700 1 --2- 172.21.15.59:0/2624922578 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e61eca380 0x558e61ed4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:46:04.567 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.555+0000 7f36427fc700 1 -- 172.21.15.59:0/2624922578 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558e61ecbbd0 msgr2=0x558e61edb6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T17:46:04.567 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.555+0000 7f36427fc700 1 --2- 172.21.15.59:0/2624922578 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558e61ecbbd0 0x558e61edb6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:04.567 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.555+0000 7f36427fc700 1 -- 172.21.15.59:0/2624922578 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e61eca380 msgr2=0x558e61ed4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T17:46:04.567 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.555+0000 7f36427fc700 1 --2- 172.21.15.59:0/2624922578 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e61eca380 0x558e61ed4150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:04.567 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.555+0000 7f36427fc700 1 -- 172.21.15.59:0/2624922578 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f364419d930 con 0x558e61ecace0 2023-12-11T17:46:04.567 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.555+0000 7f36427fc700 1 --2- 172.21.15.59:0/2624922578 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558e61ecace0 0x558e61ed4690 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f3638011440 tx=0x7f3638011750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=715686f30a1317be server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:46:04.567 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.555+0000 7f3622ffd700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2023-12-11T17:46:04.567 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.555+0000 7f3622ffd700 1 -- 172.21.15.59:0/2624922578 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3638016070 con 0x558e61ecace0 2023-12-11T17:46:04.567 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.555+0000 7f3622ffd700 1 -- 172.21.15.59:0/2624922578 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f3638010640 con 0x558e61ecace0 2023-12-11T17:46:04.567 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.555+0000 7f3655c53b80 1 -- 172.21.15.59:0/2624922578 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558e61ebdaf0 con 0x558e61ecace0 2023-12-11T17:46:04.567 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.556+0000 7f3622ffd700 1 -- 172.21.15.59:0/2624922578 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3638010800 con 0x558e61ecace0 2023-12-11T17:46:04.568 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.556+0000 7f3655c53b80 10 client.15384 fetch_fsmap learned FSMap version 24 2023-12-11T17:46:04.568 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.556+0000 7f3655c53b80 1 -- 172.21.15.59:0/2624922578 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x558e61c47d90 con 0x558e61ecace0 2023-12-11T17:46:04.568 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.556+0000 7f3622ffd700 1 -- 172.21.15.59:0/2624922578 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558e61ebdaf0 con 0x558e61ecace0 2023-12-11T17:46:04.568 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.556+0000 7f3622ffd700 1 -- 172.21.15.59:0/2624922578 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f363801bb50 con 0x558e61ecace0 2023-12-11T17:46:04.568 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.556+0000 7f3655c53b80 10 client.15384 fetch_fsmap finished waiting for FSMap version 24 2023-12-11T17:46:04.568 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.556+0000 7f3655c53b80 10 client.15384 resolve_mds: resolved name 'b' to daemon mds.b 2023-12-11T17:46:04.568 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.556+0000 7f3655c53b80 20 client.15384 populate_metadata read hostname 'smithi059' 2023-12-11T17:46:04.568 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.556+0000 7f3655c53b80 1 --2- 172.21.15.59:0/2624922578 >> [v2:172.21.15.115:6838/4122922866,v1:172.21.15.115:6839/4122922866] conn(0x558e61edfea0 0x558e61ede920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:04.568 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.556+0000 7f3655c53b80 4 client.15384 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-11T17:46:04.568 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.556+0000 7f3655c53b80 1 -- 172.21.15.59:0/2624922578 --> [v2:172.21.15.115:6838/4122922866,v1:172.21.15.115:6839/4122922866] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558e61e99ef0 con 0x558e61edfea0 2023-12-11T17:46:04.568 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.556+0000 7f3642ffd700 1 --2- 172.21.15.59:0/2624922578 >> [v2:172.21.15.115:6838/4122922866,v1:172.21.15.115:6839/4122922866] conn(0x558e61edfea0 0x558e61ede920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:46:04.568 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.557+0000 7f3622ffd700 10 client.15384 ms_handle_connect on v2:172.21.15.115:6838/4122922866 2023-12-11T17:46:04.568 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.557+0000 7f3642ffd700 1 --2- 172.21.15.59:0/2624922578 >> [v2:172.21.15.115:6838/4122922866,v1:172.21.15.115:6839/4122922866] conn(0x558e61edfea0 0x558e61ede920 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54a70d0b73f80541 server_cookie=e711cc50e6b64192 in_seq=0 out_seq=0 2023-12-11T17:46:04.568 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.558+0000 7f3622ffd700 1 -- 172.21.15.59:0/2624922578 <== mds.0 v2:172.21.15.115:6838/4122922866 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558e61e99ef0 con 0x558e61edfea0 2023-12-11T17:46:04.568 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.558+0000 7f3622ffd700 10 client.15384 handle_command_reply: tid=0 2023-12-11T17:46:04.569 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.558+0000 7f3655c53b80 1 client.15384 shutdown 2023-12-11T17:46:04.569 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.558+0000 7f3655c53b80 1 -- 172.21.15.59:0/2624922578 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558e61ecace0 msgr2=0x558e61ed4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:46:04.569 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.558+0000 7f3655c53b80 1 --2- 172.21.15.59:0/2624922578 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558e61ecace0 0x558e61ed4690 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f3638011440 tx=0x7f3638011750 comp rx=0 tx=0).stop 2023-12-11T17:46:04.569 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.558+0000 7f3655c53b80 1 -- 172.21.15.59:0/2624922578 shutdown_connections 2023-12-11T17:46:04.569 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.558+0000 7f3655c53b80 1 --2- 172.21.15.59:0/2624922578 >> [v2:172.21.15.115:6838/4122922866,v1:172.21.15.115:6839/4122922866] conn(0x558e61edfea0 0x558e61ede920 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:04.569 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.558+0000 7f3655c53b80 1 --2- 172.21.15.59:0/2624922578 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558e61ecbbd0 0x558e61edb6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:04.569 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.558+0000 7f3655c53b80 1 --2- 172.21.15.59:0/2624922578 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e61eca380 0x558e61ed4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:04.569 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.558+0000 7f3622ffd700 0 client.15384 ms_handle_reset on v2:172.21.15.115:6838/4122922866 2023-12-11T17:46:04.569 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.558+0000 7f3655c53b80 1 --2- 172.21.15.59:0/2624922578 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558e61ecace0 0x558e61ed4690 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:04.569 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.558+0000 7f3655c53b80 1 -- 172.21.15.59:0/2624922578 >> 172.21.15.59:0/2624922578 conn(0x558e61df9790 msgr2=0x558e61ec59f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:46:04.569 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.558+0000 7f3655c53b80 1 -- 172.21.15.59:0/2624922578 shutdown_connections 2023-12-11T17:46:04.569 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.558+0000 7f3655c53b80 1 -- 172.21.15.59:0/2624922578 wait complete. 2023-12-11T17:46:04.569 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.558+0000 7f3655c53b80 20 client.15384 trim_cache size 0 max 16384 2023-12-11T17:46:04.587 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.576+0000 7f3655c53b80 1 Processor -- start 2023-12-11T17:46:04.587 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.576+0000 7f3655c53b80 1 -- start start 2023-12-11T17:46:04.587 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.576+0000 7f3655c53b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558e61eca380 0x558e61ed2e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:04.587 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.576+0000 7f3655c53b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e61ecace0 0x558e61edb690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:04.587 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.576+0000 7f3655c53b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558e61ecbbd0 0x558e61eceb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:04.588 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.576+0000 7f3655c53b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e61dc1830 con 0x558e61ecbbd0 2023-12-11T17:46:04.588 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.576+0000 7f3655c53b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e61ec1740 con 0x558e61ecace0 2023-12-11T17:46:04.588 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.576+0000 7f3655c53b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e61ecba50 con 0x558e61eca380 2023-12-11T17:46:04.588 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.576+0000 7f36437fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558e61ecbbd0 0x558e61eceb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:46:04.588 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.576+0000 7f36437fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558e61ecbbd0 0x558e61eceb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34612/0 (socket says 172.21.15.59:34612) 2023-12-11T17:46:04.588 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.576+0000 7f36437fe700 1 -- 172.21.15.59:0/4193235423 learned_addr learned my addr 172.21.15.59:0/4193235423 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:46:04.588 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.576+0000 7f36427fc700 1 --2- 172.21.15.59:0/4193235423 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e61ecace0 0x558e61edb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:46:04.588 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.576+0000 7f3642ffd700 1 --2- 172.21.15.59:0/4193235423 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558e61eca380 0x558e61ed2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:46:04.588 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.576+0000 7f36437fe700 1 -- 172.21.15.59:0/4193235423 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558e61eca380 msgr2=0x558e61ed2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T17:46:04.588 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.576+0000 7f36437fe700 1 --2- 172.21.15.59:0/4193235423 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558e61eca380 0x558e61ed2e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:04.588 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.576+0000 7f36437fe700 1 -- 172.21.15.59:0/4193235423 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e61ecace0 msgr2=0x558e61edb690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T17:46:04.588 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.576+0000 7f36437fe700 1 --2- 172.21.15.59:0/4193235423 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e61ecace0 0x558e61edb690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:04.588 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.576+0000 7f36437fe700 1 -- 172.21.15.59:0/4193235423 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558e61c30c10 con 0x558e61ecbbd0 2023-12-11T17:46:04.588 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.576+0000 7f36427fc700 1 --2- 172.21.15.59:0/4193235423 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e61ecace0 0x558e61edb690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T17:46:04.589 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.576+0000 7f36437fe700 1 --2- 172.21.15.59:0/4193235423 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558e61ecbbd0 0x558e61eceb40 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f363c0963b0 tx=0x7f363c003f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=739f303161af01a7 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:46:04.589 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.577+0000 7f362a7fc700 1 -- 172.21.15.59:0/4193235423 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f363c09da90 con 0x558e61ecbbd0 2023-12-11T17:46:04.589 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.577+0000 7f362a7fc700 1 -- 172.21.15.59:0/4193235423 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f363c0a1cd0 con 0x558e61ecbbd0 2023-12-11T17:46:04.589 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.577+0000 7f362a7fc700 1 -- 172.21.15.59:0/4193235423 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f363c0b90c0 con 0x558e61ecbbd0 2023-12-11T17:46:04.589 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.577+0000 7f3655c53b80 1 -- 172.21.15.59:0/4193235423 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558e61ecbbd0 msgr2=0x558e61eceb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:46:04.589 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.577+0000 7f3655c53b80 1 --2- 172.21.15.59:0/4193235423 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558e61ecbbd0 0x558e61eceb40 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f363c0963b0 tx=0x7f363c003f50 comp rx=0 tx=0).stop 2023-12-11T17:46:04.589 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.577+0000 7f3655c53b80 1 -- 172.21.15.59:0/4193235423 shutdown_connections 2023-12-11T17:46:04.589 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.577+0000 7f3655c53b80 1 --2- 172.21.15.59:0/4193235423 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558e61ecbbd0 0x558e61eceb40 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:04.589 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.577+0000 7f3655c53b80 1 --2- 172.21.15.59:0/4193235423 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558e61eca380 0x558e61ed2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:04.589 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.577+0000 7f3655c53b80 1 --2- 172.21.15.59:0/4193235423 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e61ecace0 0x558e61edb690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:04.590 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.577+0000 7f3655c53b80 1 -- 172.21.15.59:0/4193235423 >> 172.21.15.59:0/4193235423 conn(0x558e61df9790 msgr2=0x558e61ed3660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:46:04.590 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.577+0000 7f3655c53b80 1 -- 172.21.15.59:0/4193235423 shutdown_connections 2023-12-11T17:46:04.590 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.577+0000 7f3655c53b80 1 -- 172.21.15.59:0/4193235423 wait complete. 2023-12-11T17:46:04.590 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.578+0000 7f3655c53b80 1 -- start start 2023-12-11T17:46:04.590 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.578+0000 7f3655c53b80 1 Processor -- start 2023-12-11T17:46:04.590 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.578+0000 7f3655c53b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558e61eca380 0x558e61ed7820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:04.590 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.578+0000 7f3655c53b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558e61ecace0 0x558e61ed7d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:04.590 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.578+0000 7f3655c53b80 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e61ecbbd0 0x558e61edaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:04.590 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.578+0000 7f3655c53b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e61edb1e0 con 0x558e61eca380 2023-12-11T17:46:04.590 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.578+0000 7f3655c53b80 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e61edb350 con 0x558e61ecbbd0 2023-12-11T17:46:04.590 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.578+0000 7f3655c53b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e61edb510 con 0x558e61ecace0 2023-12-11T17:46:04.590 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.578+0000 7f3655c53b80 10 client.0 _ll_register_callbacks cb 0x558e61d03a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-11T17:46:04.590 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.578+0000 7f3642ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558e61eca380 0x558e61ed7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:46:04.591 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.578+0000 7f3642ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558e61eca380 0x558e61ed7820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34614/0 (socket says 172.21.15.59:34614) 2023-12-11T17:46:04.591 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.578+0000 7f3642ffd700 1 -- 172.21.15.59:0/3368812315 learned_addr learned my addr 172.21.15.59:0/3368812315 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:46:04.591 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.579+0000 7f36427fc700 1 --2- 172.21.15.59:0/3368812315 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558e61ecace0 0x558e61ed7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:46:04.591 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.579+0000 7f36437fe700 1 --2- 172.21.15.59:0/3368812315 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e61ecbbd0 0x558e61edaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:46:04.591 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.579+0000 7f3642ffd700 1 -- 172.21.15.59:0/3368812315 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558e61ecace0 msgr2=0x558e61ed7d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T17:46:04.591 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.579+0000 7f3642ffd700 1 --2- 172.21.15.59:0/3368812315 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558e61ecace0 0x558e61ed7d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:04.591 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.579+0000 7f3642ffd700 1 -- 172.21.15.59:0/3368812315 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e61ecbbd0 msgr2=0x558e61edaca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-11T17:46:04.591 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.579+0000 7f3642ffd700 1 --2- 172.21.15.59:0/3368812315 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e61ecbbd0 0x558e61edaca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:04.591 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.579+0000 7f3642ffd700 1 -- 172.21.15.59:0/3368812315 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558e61c9b5e0 con 0x558e61eca380 2023-12-11T17:46:04.591 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.579+0000 7f36427fc700 1 --2- 172.21.15.59:0/3368812315 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558e61ecace0 0x558e61ed7d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-11T17:46:04.591 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.579+0000 7f3642ffd700 1 --2- 172.21.15.59:0/3368812315 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558e61eca380 0x558e61ed7820 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f3634000c00 tx=0x7f3634009d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe79c4e4def7f05c server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:46:04.591 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.579+0000 7f3622ffd700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2023-12-11T17:46:04.591 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.579+0000 7f3622ffd700 1 -- 172.21.15.59:0/3368812315 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3634036050 con 0x558e61eca380 2023-12-11T17:46:04.592 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.579+0000 7f3622ffd700 1 -- 172.21.15.59:0/3368812315 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f3634004500 con 0x558e61eca380 2023-12-11T17:46:04.592 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.579+0000 7f3655c53b80 1 -- 172.21.15.59:0/3368812315 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f363c099940 con 0x558e61eca380 2023-12-11T17:46:04.592 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.579+0000 7f3622ffd700 1 -- 172.21.15.59:0/3368812315 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f363402a040 con 0x558e61eca380 2023-12-11T17:46:04.592 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.579+0000 7f3655c53b80 10 client.15390 fetch_fsmap learned FSMap version 24 2023-12-11T17:46:04.592 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.579+0000 7f3655c53b80 1 -- 172.21.15.59:0/3368812315 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x558e61c6e200 con 0x558e61eca380 2023-12-11T17:46:04.592 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.579+0000 7f3622ffd700 1 -- 172.21.15.59:0/3368812315 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f363c099940 con 0x558e61eca380 2023-12-11T17:46:04.592 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.579+0000 7f3622ffd700 1 -- 172.21.15.59:0/3368812315 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f3634023680 con 0x558e61eca380 2023-12-11T17:46:04.592 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.579+0000 7f3655c53b80 10 client.15390 fetch_fsmap finished waiting for FSMap version 24 2023-12-11T17:46:04.592 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.580+0000 7f3655c53b80 10 client.15390 resolve_mds: resolved name 'b' to daemon mds.b 2023-12-11T17:46:04.592 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.580+0000 7f3655c53b80 20 client.15390 populate_metadata read hostname 'smithi059' 2023-12-11T17:46:04.592 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.580+0000 7f3655c53b80 1 --2- 172.21.15.59:0/3368812315 >> [v2:172.21.15.115:6838/4122922866,v1:172.21.15.115:6839/4122922866] conn(0x558e61f3ab30 0x558e61f36f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:04.592 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.580+0000 7f3655c53b80 4 client.15390 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "status"}] 2023-12-11T17:46:04.593 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.580+0000 7f3655c53b80 1 -- 172.21.15.59:0/3368812315 --> [v2:172.21.15.115:6838/4122922866,v1:172.21.15.115:6839/4122922866] -- command(tid 0: {"prefix": "status"}) v1 -- 0x558e61de4f80 con 0x558e61f3ab30 2023-12-11T17:46:04.593 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.580+0000 7f36427fc700 1 --2- 172.21.15.59:0/3368812315 >> [v2:172.21.15.115:6838/4122922866,v1:172.21.15.115:6839/4122922866] conn(0x558e61f3ab30 0x558e61f36f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:46:04.593 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.580+0000 7f36427fc700 1 --2- 172.21.15.59:0/3368812315 >> [v2:172.21.15.115:6838/4122922866,v1:172.21.15.115:6839/4122922866] conn(0x558e61f3ab30 0x558e61f36f90 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14276012f98bd4d1 server_cookie=c3cec008660af3a6 in_seq=0 out_seq=0 2023-12-11T17:46:04.593 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.580+0000 7f3622ffd700 10 client.15390 ms_handle_connect on v2:172.21.15.115:6838/4122922866 2023-12-11T17:46:04.593 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.580+0000 7f3622ffd700 1 -- 172.21.15.59:0/3368812315 <== mds.0 v2:172.21.15.115:6838/4122922866 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x558e61de4f80 con 0x558e61f3ab30 2023-12-11T17:46:04.593 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.580+0000 7f3622ffd700 10 client.15390 handle_command_reply: tid=0 2023-12-11T17:46:04.593 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.580+0000 7f3655c53b80 1 client.15390 shutdown 2023-12-11T17:46:04.593 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.581+0000 7f3655c53b80 1 -- 172.21.15.59:0/3368812315 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558e61eca380 msgr2=0x558e61ed7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:46:04.593 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.581+0000 7f3655c53b80 1 --2- 172.21.15.59:0/3368812315 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558e61eca380 0x558e61ed7820 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f3634000c00 tx=0x7f3634009d50 comp rx=0 tx=0).stop 2023-12-11T17:46:04.593 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.581+0000 7f3655c53b80 1 -- 172.21.15.59:0/3368812315 shutdown_connections 2023-12-11T17:46:04.593 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.581+0000 7f3655c53b80 1 --2- 172.21.15.59:0/3368812315 >> [v2:172.21.15.115:6838/4122922866,v1:172.21.15.115:6839/4122922866] conn(0x558e61f3ab30 0x558e61f36f90 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:04.593 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.581+0000 7f3655c53b80 1 --2- 172.21.15.59:0/3368812315 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558e61ecbbd0 0x558e61edaca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:04.593 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.581+0000 7f3655c53b80 1 --2- 172.21.15.59:0/3368812315 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558e61eca380 0x558e61ed7820 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:04.593 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.581+0000 7f3622ffd700 0 client.15390 ms_handle_reset on v2:172.21.15.115:6838/4122922866 2023-12-11T17:46:04.594 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.581+0000 7f3655c53b80 1 --2- 172.21.15.59:0/3368812315 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558e61ecace0 0x558e61ed7d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:04.594 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.581+0000 7f3655c53b80 1 -- 172.21.15.59:0/3368812315 >> 172.21.15.59:0/3368812315 conn(0x558e61df9790 msgr2=0x558e61ed2e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:46:04.594 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.581+0000 7f3655c53b80 1 -- 172.21.15.59:0/3368812315 shutdown_connections 2023-12-11T17:46:04.594 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.581+0000 7f3655c53b80 1 -- 172.21.15.59:0/3368812315 wait complete. 2023-12-11T17:46:04.594 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.581+0000 7f3655c53b80 20 client.15390 trim_cache size 0 max 16384 2023-12-11T17:46:04.594 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2023-12-11T17:46:04.594 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_fsid": "d0a07eaa-984b-11ee-95a3-87774f69a715", 2023-12-11T17:46:04.595 INFO:teuthology.orchestra.run.smithi059.stdout: "whoami": 0, 2023-12-11T17:46:04.595 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 24521, 2023-12-11T17:46:04.595 INFO:teuthology.orchestra.run.smithi059.stdout: "want_state": "up:active", 2023-12-11T17:46:04.595 INFO:teuthology.orchestra.run.smithi059.stdout: "state": "up:active", 2023-12-11T17:46:04.595 INFO:teuthology.orchestra.run.smithi059.stdout: "fs_name": "cephfs", 2023-12-11T17:46:04.595 INFO:teuthology.orchestra.run.smithi059.stdout: "rank_uptime": 103.93497486699999, 2023-12-11T17:46:04.595 INFO:teuthology.orchestra.run.smithi059.stdout: "mdsmap_epoch": 24, 2023-12-11T17:46:04.595 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch": 88, 2023-12-11T17:46:04.595 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch_barrier": 86, 2023-12-11T17:46:04.595 INFO:teuthology.orchestra.run.smithi059.stdout: "uptime": 109.414430795 2023-12-11T17:46:04.595 INFO:teuthology.orchestra.run.smithi059.stdout:} 2023-12-11T17:46:04.596 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.581+0000 7f3649a5a700 1 -- 172.21.15.59:0/985433070 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f362c0a1510 msgr2=0x7f362c0a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:46:04.596 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.581+0000 7f3649a5a700 1 --2- 172.21.15.59:0/985433070 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f362c0a1510 0x7f362c0a39d0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f363400b540 tx=0x7f3634005c00 comp rx=0 tx=0).stop 2023-12-11T17:46:04.596 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.581+0000 7f3649a5a700 1 -- 172.21.15.59:0/985433070 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3644103130 msgr2=0x7f364419c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:46:04.596 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.581+0000 7f3649a5a700 1 --2- 172.21.15.59:0/985433070 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3644103130 0x7f364419c8f0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f363c007ee0 tx=0x7f363c00d3b0 comp rx=0 tx=0).stop 2023-12-11T17:46:04.596 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.581+0000 7f3649a5a700 1 -- 172.21.15.59:0/985433070 shutdown_connections 2023-12-11T17:46:04.596 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.581+0000 7f3649a5a700 1 --2- 172.21.15.59:0/985433070 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f362c0a1510 0x7f362c0a39d0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:04.597 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.581+0000 7f3649a5a700 1 --2- 172.21.15.59:0/985433070 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3644103130 0x7f364419c8f0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:04.597 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.581+0000 7f3649a5a700 1 --2- 172.21.15.59:0/985433070 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f36441015a0 0x7f3644195390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:04.597 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.581+0000 7f3649a5a700 1 --2- 172.21.15.59:0/985433070 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3644102790 0x7f36441958d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:04.597 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.581+0000 7f3649a5a700 1 -- 172.21.15.59:0/985433070 >> 172.21.15.59:0/985433070 conn(0x7f36440fcb10 msgr2=0x7f3644105b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:46:04.597 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.581+0000 7f3649a5a700 1 -- 172.21.15.59:0/985433070 shutdown_connections 2023-12-11T17:46:04.597 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:04.581+0000 7f3649a5a700 1 -- 172.21.15.59:0/985433070 wait complete. 2023-12-11T17:46:04.598 INFO:teuthology.run_tasks:Running task kclient... 2023-12-11T17:46:04.612 INFO:tasks.kclient:Mounting kernel clients... 2023-12-11T17:46:04.612 INFO:tasks.kclient:config is {'client.0': {}} 2023-12-11T17:46:04.613 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-11T17:46:04.613 DEBUG:teuthology.orchestra.run.smithi059:> ip netns list 2023-12-11T17:46:04.632 INFO:teuthology.orchestra.run.smithi059.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2023-12-11T17:46:04.632 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-11T17:46:04.633 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-12-11T17:46:04.696 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:04 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/3733923019' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T17:46:04.696 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:04 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/184179589' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-11T17:46:04.717 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-11T17:46:04.717 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link delete ceph-brx 2023-12-11T17:46:04.809 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=secure', 'wsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2023-12-11T17:46:04.809 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-12-11T17:46:04.809 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-12-11T17:46:04.809 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell --fsid d0a07eaa-984b-11ee-95a3-87774f69a715 -- ceph fs ls 2023-12-11T17:46:04.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:04 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/3733923019' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T17:46:04.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:04 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/184179589' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-11T17:46:05.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:04 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/3733923019' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T17:46:05.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:04 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/184179589' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-11T17:46:05.416 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/mon.a/config 2023-12-11T17:46:05.831 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:05 smithi059 ceph-mon[98847]: pgmap v313: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-11T17:46:05.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:05 smithi181 ceph-mon[103627]: pgmap v313: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-11T17:46:06.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:05 smithi115 ceph-mon[102748]: pgmap v313: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-11T17:46:06.771 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:06.767+0000 7f6378b47700 1 -- 172.21.15.59:0/768004056 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6374100e80 msgr2=0x7f637410d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:46:06.771 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:06.767+0000 7f6378b47700 1 --2- 172.21.15.59:0/768004056 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6374100e80 0x7f637410d370 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f636c00b0d0 tx=0x7f636c00b3e0 comp rx=0 tx=0).stop 2023-12-11T17:46:06.771 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:06.767+0000 7f6378b47700 1 -- 172.21.15.59:0/768004056 shutdown_connections 2023-12-11T17:46:06.771 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:06.767+0000 7f6378b47700 1 --2- 172.21.15.59:0/768004056 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6374100e80 0x7f637410d370 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:06.771 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:06.767+0000 7f6378b47700 1 --2- 172.21.15.59:0/768004056 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6374105470 0x7f6374105850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:06.772 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:06.767+0000 7f6378b47700 1 --2- 172.21.15.59:0/768004056 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f63741004c0 0x7f6374100940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:06.772 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:06.767+0000 7f6378b47700 1 -- 172.21.15.59:0/768004056 >> 172.21.15.59:0/768004056 conn(0x7f63740774a0 msgr2=0x7f63740778b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:46:06.772 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:06.767+0000 7f6378b47700 1 -- 172.21.15.59:0/768004056 shutdown_connections 2023-12-11T17:46:06.772 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:06.767+0000 7f6378b47700 1 -- 172.21.15.59:0/768004056 wait complete. 2023-12-11T17:46:06.774 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:06.768+0000 7f6378b47700 1 Processor -- start 2023-12-11T17:46:06.774 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:06.768+0000 7f6378b47700 1 -- start start 2023-12-11T17:46:06.774 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:06.768+0000 7f6378b47700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f63741004c0 0x7f6374102100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:06.774 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:06.768+0000 7f6378b47700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6374100e80 0x7f6374102640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:06.774 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:06.768+0000 7f6378b47700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6374105470 0x7f6374102b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:06.775 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:06.768+0000 7f6378b47700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63741a5e40 con 0x7f6374105470 2023-12-11T17:46:06.775 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:06.768+0000 7f6378b47700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63741a5fb0 con 0x7f63741004c0 2023-12-11T17:46:06.775 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:06.768+0000 7f6378b47700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63741a6170 con 0x7f6374100e80 2023-12-11T17:46:06.775 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:06.768+0000 7f6372d9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6374105470 0x7f6374102b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:46:06.775 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:06.769+0000 7f6372d9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6374105470 0x7f6374102b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34632/0 (socket says 172.21.15.59:34632) 2023-12-11T17:46:06.775 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:06.769+0000 7f6372d9d700 1 -- 172.21.15.59:0/1293854250 learned_addr learned my addr 172.21.15.59:0/1293854250 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:46:06.775 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:06.769+0000 7f637259c700 1 --2- 172.21.15.59:0/1293854250 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f63741004c0 0x7f6374102100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:46:06.775 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:06.769+0000 7f6371d9b700 1 --2- 172.21.15.59:0/1293854250 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6374100e80 0x7f6374102640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:46:06.775 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:06.769+0000 7f6372d9d700 1 -- 172.21.15.59:0/1293854250 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6374100e80 msgr2=0x7f6374102640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:46:06.775 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:06.769+0000 7f6372d9d700 1 --2- 172.21.15.59:0/1293854250 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6374100e80 0x7f6374102640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:06.775 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:06.769+0000 7f6372d9d700 1 -- 172.21.15.59:0/1293854250 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f63741004c0 msgr2=0x7f6374102100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:46:06.775 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:06.769+0000 7f6372d9d700 1 --2- 172.21.15.59:0/1293854250 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f63741004c0 0x7f6374102100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:06.776 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:06.769+0000 7f6372d9d700 1 -- 172.21.15.59:0/1293854250 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f636c009d00 con 0x7f6374105470 2023-12-11T17:46:06.776 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:06.769+0000 7f637259c700 1 --2- 172.21.15.59:0/1293854250 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f63741004c0 0x7f6374102100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T17:46:06.776 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:06.769+0000 7f6372d9d700 1 --2- 172.21.15.59:0/1293854250 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6374105470 0x7f6374102b80 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f636c0094c0 tx=0x7f636c0095a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:46:06.776 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:06.769+0000 7f63637fe700 1 -- 172.21.15.59:0/1293854250 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f636c00e040 con 0x7f6374105470 2023-12-11T17:46:06.776 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:06.769+0000 7f6378b47700 1 -- 172.21.15.59:0/1293854250 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63741a63f0 con 0x7f6374105470 2023-12-11T17:46:06.776 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:06.769+0000 7f63637fe700 1 -- 172.21.15.59:0/1293854250 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f636c01db40 con 0x7f6374105470 2023-12-11T17:46:06.776 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:06.769+0000 7f63637fe700 1 -- 172.21.15.59:0/1293854250 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f636c01baf0 con 0x7f6374105470 2023-12-11T17:46:06.776 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:06.769+0000 7f6378b47700 1 -- 172.21.15.59:0/1293854250 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63741a6940 con 0x7f6374105470 2023-12-11T17:46:06.776 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:06.770+0000 7f6378b47700 1 -- 172.21.15.59:0/1293854250 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6374109a60 con 0x7f6374105470 2023-12-11T17:46:06.777 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:06.772+0000 7f63637fe700 1 -- 172.21.15.59:0/1293854250 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f636c01dcb0 con 0x7f6374105470 2023-12-11T17:46:06.777 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:06.772+0000 7f63637fe700 1 --2- 172.21.15.59:0/1293854250 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f635c0a1510 0x7f635c0a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:06.777 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:06.773+0000 7f637259c700 1 --2- 172.21.15.59:0/1293854250 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f635c0a1510 0x7f635c0a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:46:06.777 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:06.773+0000 7f63637fe700 1 -- 172.21.15.59:0/1293854250 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f636c0cee40 con 0x7f6374105470 2023-12-11T17:46:06.777 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:06.773+0000 7f637259c700 1 --2- 172.21.15.59:0/1293854250 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f635c0a1510 0x7f635c0a39d0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f63741016d0 tx=0x7f636800b5f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:46:06.780 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:06.776+0000 7f63637fe700 1 -- 172.21.15.59:0/1293854250 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f636c0980d0 con 0x7f6374105470 2023-12-11T17:46:07.024 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:07.020+0000 7f6378b47700 1 -- 172.21.15.59:0/1293854250 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f63741035e0 con 0x7f6374105470 2023-12-11T17:46:07.024 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:07.020+0000 7f63637fe700 1 -- 172.21.15.59:0/1293854250 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v24) v1 ==== 53+0+88 (secure 0 0 0) 0x7f636c09b6f0 con 0x7f6374105470 2023-12-11T17:46:07.024 INFO:teuthology.orchestra.run.smithi059.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-12-11T17:46:07.026 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:07.022+0000 7f6378b47700 1 -- 172.21.15.59:0/1293854250 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f635c0a1510 msgr2=0x7f635c0a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:46:07.026 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:07.022+0000 7f6378b47700 1 --2- 172.21.15.59:0/1293854250 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f635c0a1510 0x7f635c0a39d0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f63741016d0 tx=0x7f636800b5f0 comp rx=0 tx=0).stop 2023-12-11T17:46:07.026 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:07.022+0000 7f6378b47700 1 -- 172.21.15.59:0/1293854250 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6374105470 msgr2=0x7f6374102b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:46:07.026 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:07.022+0000 7f6378b47700 1 --2- 172.21.15.59:0/1293854250 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6374105470 0x7f6374102b80 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f636c0094c0 tx=0x7f636c0095a0 comp rx=0 tx=0).stop 2023-12-11T17:46:07.026 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:07.022+0000 7f6378b47700 1 -- 172.21.15.59:0/1293854250 shutdown_connections 2023-12-11T17:46:07.026 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:07.022+0000 7f6378b47700 1 --2- 172.21.15.59:0/1293854250 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f635c0a1510 0x7f635c0a39d0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:07.026 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:07.022+0000 7f6378b47700 1 --2- 172.21.15.59:0/1293854250 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6374105470 0x7f6374102b80 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:07.026 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:07.022+0000 7f6378b47700 1 --2- 172.21.15.59:0/1293854250 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f63741004c0 0x7f6374102100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:07.026 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:07.022+0000 7f6378b47700 1 --2- 172.21.15.59:0/1293854250 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6374100e80 0x7f6374102640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:07.026 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:07.022+0000 7f6378b47700 1 -- 172.21.15.59:0/1293854250 >> 172.21.15.59:0/1293854250 conn(0x7f63740774a0 msgr2=0x7f6374069b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:46:07.027 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:07.023+0000 7f6378b47700 1 -- 172.21.15.59:0/1293854250 shutdown_connections 2023-12-11T17:46:07.027 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:07.023+0000 7f6378b47700 1 -- 172.21.15.59:0/1293854250 wait complete. 2023-12-11T17:46:07.384 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-12-11T17:46:07.384 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-11T17:46:07.384 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi059.front.sepia.ceph.com 2023-12-11T17:46:07.384 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-12-11T17:46:07.384 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-12-11T17:46:07.384 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-12-11T17:46:07.385 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-12-11T17:46:07.385 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-11T17:46:07.385 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-11T17:46:07.385 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-11T17:46:07.385 DEBUG:teuthology.orchestra.run.smithi059:> ip addr 2023-12-11T17:46:07.404 INFO:teuthology.orchestra.run.smithi059.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-11T17:46:07.404 INFO:teuthology.orchestra.run.smithi059.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-11T17:46:07.404 INFO:teuthology.orchestra.run.smithi059.stdout: inet 127.0.0.1/8 scope host lo 2023-12-11T17:46:07.405 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2023-12-11T17:46:07.405 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 ::1/128 scope host 2023-12-11T17:46:07.405 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2023-12-11T17:46:07.405 INFO:teuthology.orchestra.run.smithi059.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-11T17:46:07.405 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:6b:fe:ca brd ff:ff:ff:ff:ff:ff 2023-12-11T17:46:07.405 INFO:teuthology.orchestra.run.smithi059.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-11T17:46:07.405 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:6b:fe:cb brd ff:ff:ff:ff:ff:ff 2023-12-11T17:46:07.405 INFO:teuthology.orchestra.run.smithi059.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-11T17:46:07.405 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:bd:19:2a brd ff:ff:ff:ff:ff:ff 2023-12-11T17:46:07.405 INFO:teuthology.orchestra.run.smithi059.stdout: inet 172.21.15.59/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-12-11T17:46:07.405 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft 41795sec preferred_lft 41795sec 2023-12-11T17:46:07.405 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::ec4:7aff:febd:192a/64 scope link 2023-12-11T17:46:07.405 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2023-12-11T17:46:07.405 INFO:teuthology.orchestra.run.smithi059.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-11T17:46:07.406 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:bd:19:2b brd ff:ff:ff:ff:ff:ff 2023-12-11T17:46:07.406 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-12-11T17:46:07.406 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-11T17:46:07.406 DEBUG:teuthology.orchestra.run.smithi059:> (cd / && exec bash -c ' 2023-12-11T17:46:07.406 DEBUG:teuthology.orchestra.run.smithi059:> set -e 2023-12-11T17:46:07.406 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link add name ceph-brx type bridge 2023-12-11T17:46:07.406 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip addr flush dev ceph-brx 2023-12-11T17:46:07.406 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link set ceph-brx up 2023-12-11T17:46:07.407 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-12-11T17:46:07.407 DEBUG:teuthology.orchestra.run.smithi059:> ') 2023-12-11T17:46:07.529 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-11T17:46:07.529 DEBUG:teuthology.orchestra.run.smithi059:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-11T17:46:07.557 INFO:teuthology.orchestra.run.smithi059.stdout:1 2023-12-11T17:46:07.559 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-11T17:46:07.559 DEBUG:teuthology.orchestra.run.smithi059:> route 2023-12-11T17:46:07.619 INFO:teuthology.orchestra.run.smithi059.stdout:Kernel IP routing table 2023-12-11T17:46:07.619 INFO:teuthology.orchestra.run.smithi059.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-11T17:46:07.619 INFO:teuthology.orchestra.run.smithi059.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-12-11T17:46:07.619 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-12-11T17:46:07.620 INFO:teuthology.orchestra.run.smithi059.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-11T17:46:07.620 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-11T17:46:07.620 DEBUG:teuthology.orchestra.run.smithi059:> (cd / && exec bash -c ' 2023-12-11T17:46:07.620 DEBUG:teuthology.orchestra.run.smithi059:> set -e 2023-12-11T17:46:07.620 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-12-11T17:46:07.620 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-12-11T17:46:07.620 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-12-11T17:46:07.620 DEBUG:teuthology.orchestra.run.smithi059:> ') 2023-12-11T17:46:07.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:07 smithi059 ceph-mon[98847]: pgmap v314: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-12-11T17:46:07.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:07 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/1293854250' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-12-11T17:46:07.722 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-11T17:46:07.722 DEBUG:teuthology.orchestra.run.smithi059:> ip netns list 2023-12-11T17:46:07.739 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-11T17:46:07.740 DEBUG:teuthology.orchestra.run.smithi059:> ip netns list-id 2023-12-11T17:46:07.796 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-11T17:46:07.796 DEBUG:teuthology.orchestra.run.smithi059:> (cd / && exec bash -c ' 2023-12-11T17:46:07.796 DEBUG:teuthology.orchestra.run.smithi059:> set -e 2023-12-11T17:46:07.797 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-11T17:46:07.797 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-12-11T17:46:07.797 DEBUG:teuthology.orchestra.run.smithi059:> ') 2023-12-11T17:46:07.893 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-12-11T17:46:07.893 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-11T17:46:07.893 DEBUG:teuthology.orchestra.run.smithi059:> (cd / && exec bash -c ' 2023-12-11T17:46:07.893 DEBUG:teuthology.orchestra.run.smithi059:> set -e 2023-12-11T17:46:07.893 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-12-11T17:46:07.893 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-12-11T17:46:07.893 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-12-11T17:46:07.894 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-12-11T17:46:07.894 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-12-11T17:46:07.894 DEBUG:teuthology.orchestra.run.smithi059:> ') 2023-12-11T17:46:07.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:07 smithi181 ceph-mon[103627]: pgmap v314: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-12-11T17:46:07.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:07 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/1293854250' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-12-11T17:46:08.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:07 smithi115 ceph-mon[102748]: pgmap v314: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-12-11T17:46:08.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:07 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/1293854250' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-12-11T17:46:08.092 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-11T17:46:08.092 DEBUG:teuthology.orchestra.run.smithi059:> (cd / && exec bash -c ' 2023-12-11T17:46:08.092 DEBUG:teuthology.orchestra.run.smithi059:> set -e 2023-12-11T17:46:08.092 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link set brx.0 up 2023-12-11T17:46:08.093 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link set dev brx.0 master ceph-brx 2023-12-11T17:46:08.093 DEBUG:teuthology.orchestra.run.smithi059:> ') 2023-12-11T17:46:08.242 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-11T17:46:08.242 DEBUG:teuthology.orchestra.run.smithi059:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-12-11T17:46:08.260 INFO:teuthology.orchestra.run.smithi059.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-12-11T17:46:08.260 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-11T17:46:08.260 DEBUG:teuthology.orchestra.run.smithi059:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-12-11T17:46:08.317 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2023-12-11T17:46:08.317 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2023-12-11T17:46:08.317 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/_nogroup/sv_1/b5d81e8c-41e7-4beb-9cc4-c365f0e4a5a8 2023-12-11T17:46:08.317 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-11T17:46:08.317 DEBUG:teuthology.orchestra.run.smithi059:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/_nogroup/sv_1/b5d81e8c-41e7-4beb-9cc4-c365f0e4a5a8 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync 2023-12-11T17:46:08.403 INFO:teuthology.orchestra.run.smithi059.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync 2023-12-11T17:46:08.404 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync". 2023-12-11T17:46:08.404 INFO:teuthology.orchestra.run.smithi059.stdout:invalid new device string format 2023-12-11T17:46:08.591 INFO:teuthology.orchestra.run.smithi059.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync 2023-12-11T17:46:08.591 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync". 2023-12-11T17:46:08.591 INFO:teuthology.orchestra.run.smithi059.stdout:invalid new device string format 2023-12-11T17:46:08.591 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: resolved to: "172.21.15.59:3300,172.21.15.115:3300,172.21.15.181:3300" 2023-12-11T17:46:08.592 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.59:3300,172.21.15.115:3300,172.21.15.181:3300:/volumes/_nogroup/sv_1/b5d81e8c-41e7-4beb-9cc4-c365f0e4a5a8 2023-12-11T17:46:08.592 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync,key=0,fsid=d0a07eaa-984b-11ee-95a3-87774f69a715" will pass to kernel 2023-12-11T17:46:08.594 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-12-11T17:46:08.595 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-11T17:46:08.595 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-12-11T17:46:08.650 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-11T17:46:08.650 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-12-11T17:46:08.704 INFO:teuthology.orchestra.run.smithi059.stdout:client25132 2023-12-11T17:46:08.705 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2023-12-11T17:46:08.705 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-11T17:46:08.723 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-11T17:46:08.723 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-12-11T17:46:08.822 INFO:teuthology.orchestra.run.smithi059.stdout:client25132 2023-12-11T17:46:08.823 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-11T17:46:08.823 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-12-11T17:46:08.882 INFO:teuthology.orchestra.run.smithi059.stdout:client25132 2023-12-11T17:46:08.883 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-11T17:46:08.883 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715.client25132/status') 2023-12-11T17:46:08.947 INFO:teuthology.orchestra.run.smithi059.stdout:instance: client.25132 (3)172.21.15.59:0/1653421755 2023-12-11T17:46:08.947 INFO:teuthology.orchestra.run.smithi059.stdout:blocklisted: false 2023-12-11T17:46:08.947 INFO:teuthology.orchestra.run.smithi059.stderr:0+1 records in 2023-12-11T17:46:08.947 INFO:teuthology.orchestra.run.smithi059.stderr:0+1 records out 2023-12-11T17:46:08.948 INFO:teuthology.orchestra.run.smithi059.stderr:71 bytes copied, 2.4607e-05 s, 2.9 MB/s 2023-12-11T17:46:08.949 INFO:teuthology.run_tasks:Running task check-counter... 2023-12-11T17:46:08.962 INFO:teuthology.run_tasks:Running task exec... 2023-12-11T17:46:08.987 INFO:teuthology.task.exec:Executing custom commands... 2023-12-11T17:46:08.987 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi059.front.sepia.ceph.com 2023-12-11T17:46:08.987 DEBUG:teuthology.orchestra.run.smithi059:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2023-12-11T17:46:09.115 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:09.109+0000 7fec8a5df700 1 -- 172.21.15.59:0/241210299 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fec84101620 msgr2=0x7fec84101a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:46:09.116 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:09.109+0000 7fec8a5df700 1 --2- 172.21.15.59:0/241210299 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fec84101620 0x7fec84101a80 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fec780099c0 tx=0x7fec78009cd0 comp rx=0 tx=0).stop 2023-12-11T17:46:09.116 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:09.109+0000 7fec8a5df700 1 -- 172.21.15.59:0/241210299 shutdown_connections 2023-12-11T17:46:09.116 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:09.109+0000 7fec8a5df700 1 --2- 172.21.15.59:0/241210299 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fec84101fc0 0x7fec84108870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:09.116 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:09.109+0000 7fec8a5df700 1 --2- 172.21.15.59:0/241210299 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec84100430 0x7fec84100850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:09.116 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:09.109+0000 7fec8a5df700 1 --2- 172.21.15.59:0/241210299 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fec84101620 0x7fec84101a80 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:09.116 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:09.109+0000 7fec8a5df700 1 -- 172.21.15.59:0/241210299 >> 172.21.15.59:0/241210299 conn(0x7fec840fb9c0 msgr2=0x7fec840fdde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:46:09.116 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:09.110+0000 7fec8a5df700 1 -- 172.21.15.59:0/241210299 shutdown_connections 2023-12-11T17:46:09.116 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:09.110+0000 7fec8a5df700 1 -- 172.21.15.59:0/241210299 wait complete. 2023-12-11T17:46:09.116 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:09.110+0000 7fec8a5df700 1 Processor -- start 2023-12-11T17:46:09.116 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:09.110+0000 7fec8a5df700 1 -- start start 2023-12-11T17:46:09.116 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:09.110+0000 7fec8a5df700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fec84100430 0x7fec8406a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:09.116 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:09.110+0000 7fec8a5df700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fec84101620 0x7fec8419c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:09.117 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:09.110+0000 7fec8a5df700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec84101fc0 0x7fec8419e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:09.117 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:09.110+0000 7fec8a5df700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec8419ef30 con 0x7fec84101620 2023-12-11T17:46:09.117 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:09.110+0000 7fec8a5df700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec8419f0a0 con 0x7fec84100430 2023-12-11T17:46:09.117 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:09.110+0000 7fec8a5df700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec8419f260 con 0x7fec84101fc0 2023-12-11T17:46:09.117 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:09.111+0000 7fec837fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fec84101620 0x7fec8419c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:46:09.117 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:09.111+0000 7fec837fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fec84101620 0x7fec8419c120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34648/0 (socket says 172.21.15.59:34648) 2023-12-11T17:46:09.117 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:09.111+0000 7fec837fe700 1 -- 172.21.15.59:0/2488983732 learned_addr learned my addr 172.21.15.59:0/2488983732 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:46:09.117 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:09.111+0000 7fec83fff700 1 --2- 172.21.15.59:0/2488983732 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fec84100430 0x7fec8406a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:46:09.117 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:09.111+0000 7fec88b7c700 1 --2- 172.21.15.59:0/2488983732 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec84101fc0 0x7fec8419e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:46:09.117 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:09.111+0000 7fec837fe700 1 -- 172.21.15.59:0/2488983732 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec84101fc0 msgr2=0x7fec8419e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:46:09.117 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:09.111+0000 7fec837fe700 1 --2- 172.21.15.59:0/2488983732 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec84101fc0 0x7fec8419e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:09.117 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:09.111+0000 7fec837fe700 1 -- 172.21.15.59:0/2488983732 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fec84100430 msgr2=0x7fec8406a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:46:09.118 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:09.111+0000 7fec837fe700 1 --2- 172.21.15.59:0/2488983732 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fec84100430 0x7fec8406a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:09.118 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:09.111+0000 7fec837fe700 1 -- 172.21.15.59:0/2488983732 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec780095f0 con 0x7fec84101620 2023-12-11T17:46:09.118 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:09.111+0000 7fec837fe700 1 --2- 172.21.15.59:0/2488983732 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fec84101620 0x7fec8419c120 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fec7800b580 tx=0x7fec78005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:46:09.118 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:09.111+0000 7fec817fa700 1 -- 172.21.15.59:0/2488983732 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec7801d070 con 0x7fec84101620 2023-12-11T17:46:09.118 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:09.111+0000 7fec817fa700 1 -- 172.21.15.59:0/2488983732 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fec78004200 con 0x7fec84101620 2023-12-11T17:46:09.118 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:09.111+0000 7fec8a5df700 1 -- 172.21.15.59:0/2488983732 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec8419f480 con 0x7fec84101620 2023-12-11T17:46:09.118 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:09.111+0000 7fec817fa700 1 -- 172.21.15.59:0/2488983732 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec78022e90 con 0x7fec84101620 2023-12-11T17:46:09.118 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:09.111+0000 7fec8a5df700 1 -- 172.21.15.59:0/2488983732 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec8419f980 con 0x7fec84101620 2023-12-11T17:46:09.119 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:09.112+0000 7fec8a5df700 1 -- 172.21.15.59:0/2488983732 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec841077e0 con 0x7fec84101620 2023-12-11T17:46:09.119 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:09.113+0000 7fec817fa700 1 -- 172.21.15.59:0/2488983732 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7fec78003d40 con 0x7fec84101620 2023-12-11T17:46:09.120 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:09.114+0000 7fec817fa700 1 --2- 172.21.15.59:0/2488983732 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fec6c0a1590 0x7fec6c0a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:09.120 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:09.114+0000 7fec817fa700 1 -- 172.21.15.59:0/2488983732 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fec780ce3f0 con 0x7fec84101620 2023-12-11T17:46:09.120 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:09.114+0000 7fec83fff700 1 --2- 172.21.15.59:0/2488983732 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fec6c0a1590 0x7fec6c0a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:46:09.120 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:09.114+0000 7fec83fff700 1 --2- 172.21.15.59:0/2488983732 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fec6c0a1590 0x7fec6c0a3a50 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fec74005fd0 tx=0x7fec74005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:46:09.122 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:09.118+0000 7fec817fa700 1 -- 172.21.15.59:0/2488983732 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fec7805d2f0 con 0x7fec84101620 2023-12-11T17:46:09.390 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:09.386+0000 7fec8a5df700 1 -- 172.21.15.59:0/2488983732 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7fec8410bc50 con 0x7fec84101620 2023-12-11T17:46:09.556 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:09.553+0000 7fec817fa700 1 -- 172.21.15.59:0/2488983732 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v18) v1 ==== 92+0+0 (secure 0 0 0) 0x7fec780a1070 con 0x7fec84101620 2023-12-11T17:46:09.563 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:09.559+0000 7fec8a5df700 1 -- 172.21.15.59:0/2488983732 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fec6c0a1590 msgr2=0x7fec6c0a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:46:09.563 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:09.559+0000 7fec8a5df700 1 --2- 172.21.15.59:0/2488983732 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fec6c0a1590 0x7fec6c0a3a50 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fec74005fd0 tx=0x7fec74005e50 comp rx=0 tx=0).stop 2023-12-11T17:46:09.563 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:09.559+0000 7fec8a5df700 1 -- 172.21.15.59:0/2488983732 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fec84101620 msgr2=0x7fec8419c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:46:09.563 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:09.559+0000 7fec8a5df700 1 --2- 172.21.15.59:0/2488983732 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fec84101620 0x7fec8419c120 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fec7800b580 tx=0x7fec78005ef0 comp rx=0 tx=0).stop 2023-12-11T17:46:09.564 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:09.560+0000 7fec8a5df700 1 -- 172.21.15.59:0/2488983732 shutdown_connections 2023-12-11T17:46:09.564 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:09.560+0000 7fec8a5df700 1 --2- 172.21.15.59:0/2488983732 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fec6c0a1590 0x7fec6c0a3a50 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:09.564 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:09.560+0000 7fec8a5df700 1 --2- 172.21.15.59:0/2488983732 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec84101fc0 0x7fec8419e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:09.564 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:09.560+0000 7fec8a5df700 1 --2- 172.21.15.59:0/2488983732 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fec84100430 0x7fec8406a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:09.564 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:09.560+0000 7fec8a5df700 1 --2- 172.21.15.59:0/2488983732 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fec84101620 0x7fec8419c120 secure :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fec7800b580 tx=0x7fec78005ef0 comp rx=0 tx=0).stop 2023-12-11T17:46:09.564 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:09.560+0000 7fec8a5df700 1 -- 172.21.15.59:0/2488983732 >> 172.21.15.59:0/2488983732 conn(0x7fec840fb9c0 msgr2=0x7fec84104920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:46:09.565 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:09.562+0000 7fec8a5df700 1 -- 172.21.15.59:0/2488983732 shutdown_connections 2023-12-11T17:46:09.566 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:09.563+0000 7fec8a5df700 1 -- 172.21.15.59:0/2488983732 wait complete. 2023-12-11T17:46:09.580 DEBUG:teuthology.orchestra.run.smithi059:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2023-12-11T17:46:09.716 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:09.710+0000 7f82a1a14700 1 -- 172.21.15.59:0/2063385354 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f829c1030c0 msgr2=0x7f829c109970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:46:09.716 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:09.710+0000 7f82a1a14700 1 --2- 172.21.15.59:0/2063385354 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f829c1030c0 0x7f829c109970 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f828c0099c0 tx=0x7f828c009cd0 comp rx=0 tx=0).stop 2023-12-11T17:46:09.716 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:09.710+0000 7f82a1a14700 1 -- 172.21.15.59:0/2063385354 shutdown_connections 2023-12-11T17:46:09.716 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:09.710+0000 7f82a1a14700 1 --2- 172.21.15.59:0/2063385354 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f829c1030c0 0x7f829c109970 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:09.716 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:09.710+0000 7f82a1a14700 1 --2- 172.21.15.59:0/2063385354 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f829c101530 0x7f829c101950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:09.716 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:09.710+0000 7f82a1a14700 1 --2- 172.21.15.59:0/2063385354 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f829c102720 0x7f829c102b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:09.717 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:09.710+0000 7f82a1a14700 1 -- 172.21.15.59:0/2063385354 >> 172.21.15.59:0/2063385354 conn(0x7f829c0fcaa0 msgr2=0x7f829c0feee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:46:09.717 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:09.710+0000 7f82a1a14700 1 -- 172.21.15.59:0/2063385354 shutdown_connections 2023-12-11T17:46:09.717 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:09.710+0000 7f82a1a14700 1 -- 172.21.15.59:0/2063385354 wait complete. 2023-12-11T17:46:09.717 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:09.711+0000 7f82a1a14700 1 Processor -- start 2023-12-11T17:46:09.717 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:09.711+0000 7f82a1a14700 1 -- start start 2023-12-11T17:46:09.717 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:09.711+0000 7f82a1a14700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f829c101530 0x7f829c197440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:09.717 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:09.711+0000 7f82a1a14700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f829c102720 0x7f829c197980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:09.717 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:09.711+0000 7f82a1a14700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f829c1030c0 0x7f829c19e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:09.717 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:09.711+0000 7f82a1a14700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f829c19ef70 con 0x7f829c102720 2023-12-11T17:46:09.717 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:09.711+0000 7f82a1a14700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f829c19f0b0 con 0x7f829c1030c0 2023-12-11T17:46:09.717 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:09.711+0000 7f82a1a14700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f829c19f240 con 0x7f829c101530 2023-12-11T17:46:09.717 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:09.711+0000 7f829a7fc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f829c102720 0x7f829c197980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:46:09.717 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:09.712+0000 7f829a7fc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f829c102720 0x7f829c197980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34692/0 (socket says 172.21.15.59:34692) 2023-12-11T17:46:09.717 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:09.712+0000 7f829a7fc700 1 -- 172.21.15.59:0/2050560681 learned_addr learned my addr 172.21.15.59:0/2050560681 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:46:09.718 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:09.712+0000 7f829affd700 1 --2- 172.21.15.59:0/2050560681 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f829c101530 0x7f829c197440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:46:09.718 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:09.712+0000 7f829b7fe700 1 --2- 172.21.15.59:0/2050560681 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f829c1030c0 0x7f829c19e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:46:09.718 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:09.712+0000 7f829a7fc700 1 -- 172.21.15.59:0/2050560681 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f829c101530 msgr2=0x7f829c197440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:46:09.718 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:09.712+0000 7f829a7fc700 1 --2- 172.21.15.59:0/2050560681 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f829c101530 0x7f829c197440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:09.718 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:09.712+0000 7f829a7fc700 1 -- 172.21.15.59:0/2050560681 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f829c1030c0 msgr2=0x7f829c19e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:46:09.718 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:09.712+0000 7f829a7fc700 1 --2- 172.21.15.59:0/2050560681 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f829c1030c0 0x7f829c19e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:09.718 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:09.712+0000 7f829a7fc700 1 -- 172.21.15.59:0/2050560681 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f828c0095f0 con 0x7f829c102720 2023-12-11T17:46:09.718 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:09.712+0000 7f829a7fc700 1 --2- 172.21.15.59:0/2050560681 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f829c102720 0x7f829c197980 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f828400d980 tx=0x7f828400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:46:09.718 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:09.712+0000 7f82a0a12700 1 -- 172.21.15.59:0/2050560681 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8284004220 con 0x7f829c102720 2023-12-11T17:46:09.718 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:09.712+0000 7f82a1a14700 1 -- 172.21.15.59:0/2050560681 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f829c19f520 con 0x7f829c102720 2023-12-11T17:46:09.719 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:09.712+0000 7f82a0a12700 1 -- 172.21.15.59:0/2050560681 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8284007640 con 0x7f829c102720 2023-12-11T17:46:09.719 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:09.712+0000 7f82a0a12700 1 -- 172.21.15.59:0/2050560681 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8284005690 con 0x7f829c102720 2023-12-11T17:46:09.719 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:09.712+0000 7f82a1a14700 1 -- 172.21.15.59:0/2050560681 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f829c19fa30 con 0x7f829c102720 2023-12-11T17:46:09.719 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:09.713+0000 7f82a1a14700 1 -- 172.21.15.59:0/2050560681 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f829c1088e0 con 0x7f829c102720 2023-12-11T17:46:09.720 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:09.715+0000 7f82a0a12700 1 -- 172.21.15.59:0/2050560681 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) v1 ==== 136074+0+0 (secure 0 0 0) 0x7f8284004750 con 0x7f829c102720 2023-12-11T17:46:09.720 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:09.715+0000 7f82a0a12700 1 --2- 172.21.15.59:0/2050560681 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f82880a1560 0x7f82880a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:09.720 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:09.715+0000 7f829affd700 1 -- 172.21.15.59:0/2050560681 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f82880a1560 msgr2=0x7f82880a3a20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.59:6800/3627728353 2023-12-11T17:46:09.720 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:09.715+0000 7f82a0a12700 1 -- 172.21.15.59:0/2050560681 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f82840cd470 con 0x7f829c102720 2023-12-11T17:46:09.720 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:09.715+0000 7f829affd700 1 --2- 172.21.15.59:0/2050560681 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f82880a1560 0x7f82880a3a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-11T17:46:09.727 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:09.724+0000 7f82a0a12700 1 -- 172.21.15.59:0/2050560681 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f8284096670 con 0x7f829c102720 2023-12-11T17:46:09.812 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Dec 11 17:46:09 smithi181 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-z[105741]: ignoring --setuser ceph since I am not root 2023-12-11T17:46:09.812 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Dec 11 17:46:09 smithi181 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-z[105741]: ignoring --setgroup ceph since I am not root 2023-12-11T17:46:09.812 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Dec 11 17:46:09 smithi181 ceph-mgr[105765]: -- 172.21.15.181:0/2553137918 <== mon.1 v2:172.21.15.181:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55e1b9803520 con 0x55e1b97d9000 2023-12-11T17:46:09.812 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Dec 11 17:46:09 smithi181 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-z[105741]: 2023-12-11T17:46:09.729+0000 7fa00c47d200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-11T17:46:09.812 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Dec 11 17:46:09 smithi181 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-z[105741]: 2023-12-11T17:46:09.809+0000 7fa00c47d200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-11T17:46:09.812 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:09 smithi181 ceph-mon[103627]: pgmap v315: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-12-11T17:46:09.812 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:09 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/2488983732' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2023-12-11T17:46:09.816 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:09 smithi115 ceph-mon[102748]: pgmap v315: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-12-11T17:46:09.816 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:09 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/2488983732' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2023-12-11T17:46:09.816 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Dec 11 17:46:09 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-y[104201]: ignoring --setuser ceph since I am not root 2023-12-11T17:46:09.816 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Dec 11 17:46:09 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-y[104201]: ignoring --setgroup ceph since I am not root 2023-12-11T17:46:09.816 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Dec 11 17:46:09 smithi115 ceph-mgr[104226]: -- 172.21.15.115:0/3652039973 <== mon.2 v2:172.21.15.115:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55b270269520 con 0x55b27023ec00 2023-12-11T17:46:09.816 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Dec 11 17:46:09 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-y[104201]: 2023-12-11T17:46:09.734+0000 7fa60f175200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-11T17:46:09.816 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Dec 11 17:46:09 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-y[104201]: 2023-12-11T17:46:09.814+0000 7fa60f175200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-11T17:46:09.860 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:09 smithi059 ceph-mon[98847]: pgmap v315: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-12-11T17:46:09.860 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:09 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/2488983732' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2023-12-11T17:46:09.860 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:46:09 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: ignoring --setuser ceph since I am not root 2023-12-11T17:46:09.860 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:46:09 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: ignoring --setgroup ceph since I am not root 2023-12-11T17:46:09.861 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:46:09 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T17:46:09.758+0000 7f46b2ba6200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-11T17:46:09.919 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:09.916+0000 7f829affd700 1 -- 172.21.15.59:0/2050560681 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f82880a1560 msgr2=0x7f82880a3a20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.59:6800/3627728353 2023-12-11T17:46:09.919 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:09.916+0000 7f829affd700 1 --2- 172.21.15.59:0/2050560681 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f82880a1560 0x7f82880a3a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-11T17:46:09.931 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:09.928+0000 7f82a1a14700 1 -- 172.21.15.59:0/2050560681 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f829c068450 con 0x7f829c102720 2023-12-11T17:46:09.935 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:09.932+0000 7f82a0a12700 1 -- 172.21.15.59:0/2050560681 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v23)=0 v23) v1 ==== 138+0+0 (secure 0 0 0) 0x7f82840160d0 con 0x7f829c102720 2023-12-11T17:46:09.937 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:09.934+0000 7f82a1a14700 1 -- 172.21.15.59:0/2050560681 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f82880a1560 msgr2=0x7f82880a3a20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-11T17:46:09.937 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:09.934+0000 7f82a1a14700 1 --2- 172.21.15.59:0/2050560681 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f82880a1560 0x7f82880a3a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:09.937 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:09.934+0000 7f82a1a14700 1 -- 172.21.15.59:0/2050560681 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f829c102720 msgr2=0x7f829c197980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:46:09.938 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:09.934+0000 7f82a1a14700 1 --2- 172.21.15.59:0/2050560681 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f829c102720 0x7f829c197980 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f828400d980 tx=0x7f828400dc90 comp rx=0 tx=0).stop 2023-12-11T17:46:09.938 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:09.934+0000 7f82a1a14700 1 -- 172.21.15.59:0/2050560681 shutdown_connections 2023-12-11T17:46:09.938 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:09.934+0000 7f82a1a14700 1 --2- 172.21.15.59:0/2050560681 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f82880a1560 0x7f82880a3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:09.938 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:09.934+0000 7f82a1a14700 1 --2- 172.21.15.59:0/2050560681 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f829c1030c0 0x7f829c19e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:09.938 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:09.934+0000 7f82a1a14700 1 --2- 172.21.15.59:0/2050560681 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f829c101530 0x7f829c197440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:09.938 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:09.934+0000 7f82a1a14700 1 --2- 172.21.15.59:0/2050560681 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f829c102720 0x7f829c197980 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:09.938 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:09.934+0000 7f82a1a14700 1 -- 172.21.15.59:0/2050560681 >> 172.21.15.59:0/2050560681 conn(0x7f829c0fcaa0 msgr2=0x7f829c105a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:46:09.938 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:09.934+0000 7f82a1a14700 1 -- 172.21.15.59:0/2050560681 shutdown_connections 2023-12-11T17:46:09.938 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:09.934+0000 7f82a1a14700 1 -- 172.21.15.59:0/2050560681 wait complete. 2023-12-11T17:46:09.950 DEBUG:teuthology.orchestra.run.smithi059:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2023-12-11T17:46:10.081 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:10.075+0000 7f07777c7700 1 -- 172.21.15.59:0/2351435245 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f07701048d0 msgr2=0x7f0770106cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:46:10.081 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:10.075+0000 7f07777c7700 1 --2- 172.21.15.59:0/2351435245 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f07701048d0 0x7f0770106cc0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f0760009990 tx=0x7f0760009ca0 comp rx=0 tx=0).stop 2023-12-11T17:46:10.081 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:10.075+0000 7f07777c7700 1 -- 172.21.15.59:0/2351435245 shutdown_connections 2023-12-11T17:46:10.081 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:10.075+0000 7f07777c7700 1 --2- 172.21.15.59:0/2351435245 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0770107200 0x7f0770109670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:10.081 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:10.075+0000 7f07777c7700 1 --2- 172.21.15.59:0/2351435245 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0770101fa0 0x7f0770104390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:10.081 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:10.075+0000 7f07777c7700 1 --2- 172.21.15.59:0/2351435245 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f07701048d0 0x7f0770106cc0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:10.081 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:10.075+0000 7f07777c7700 1 -- 172.21.15.59:0/2351435245 >> 172.21.15.59:0/2351435245 conn(0x7f07700fb990 msgr2=0x7f07700fddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:46:10.081 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:10.075+0000 7f07777c7700 1 -- 172.21.15.59:0/2351435245 shutdown_connections 2023-12-11T17:46:10.082 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:10.075+0000 7f07777c7700 1 -- 172.21.15.59:0/2351435245 wait complete. 2023-12-11T17:46:10.082 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:10.076+0000 7f07777c7700 1 Processor -- start 2023-12-11T17:46:10.082 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:10.076+0000 7f07777c7700 1 -- start start 2023-12-11T17:46:10.082 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:10.076+0000 7f07777c7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0770101fa0 0x7f07701974a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:10.082 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:10.076+0000 7f07777c7700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f07701048d0 0x7f07701979e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:10.082 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:10.076+0000 7f07777c7700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0770107200 0x7f077019ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:10.082 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:10.076+0000 7f07777c7700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f077019efd0 con 0x7f07701048d0 2023-12-11T17:46:10.082 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:10.076+0000 7f07777c7700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f077019f110 con 0x7f0770107200 2023-12-11T17:46:10.082 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:10.076+0000 7f07777c7700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f077019f2a0 con 0x7f0770101fa0 2023-12-11T17:46:10.082 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:10.076+0000 7f0774d62700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f07701048d0 0x7f07701979e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:46:10.082 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:10.076+0000 7f0774d62700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f07701048d0 0x7f07701979e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34756/0 (socket says 172.21.15.59:34756) 2023-12-11T17:46:10.082 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:10.076+0000 7f0774d62700 1 -- 172.21.15.59:0/2372078066 learned_addr learned my addr 172.21.15.59:0/2372078066 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:46:10.082 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:10.076+0000 7f0775563700 1 --2- 172.21.15.59:0/2372078066 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0770101fa0 0x7f07701974a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:46:10.082 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:10.076+0000 7f0775d64700 1 --2- 172.21.15.59:0/2372078066 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0770107200 0x7f077019ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:46:10.083 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:10.076+0000 7f0774d62700 1 -- 172.21.15.59:0/2372078066 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0770101fa0 msgr2=0x7f07701974a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:46:10.083 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:10.076+0000 7f0774d62700 1 --2- 172.21.15.59:0/2372078066 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0770101fa0 0x7f07701974a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:10.083 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:10.076+0000 7f0774d62700 1 -- 172.21.15.59:0/2372078066 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0770107200 msgr2=0x7f077019ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:46:10.083 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:10.076+0000 7f0774d62700 1 --2- 172.21.15.59:0/2372078066 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0770107200 0x7f077019ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:10.083 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:10.076+0000 7f0774d62700 1 -- 172.21.15.59:0/2372078066 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07600095f0 con 0x7f07701048d0 2023-12-11T17:46:10.083 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:10.077+0000 7f0774d62700 1 --2- 172.21.15.59:0/2372078066 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f07701048d0 0x7f07701979e0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f07600095c0 tx=0x7f076000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:46:10.083 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:10.077+0000 7f07667fc700 1 -- 172.21.15.59:0/2372078066 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f076001d070 con 0x7f07701048d0 2023-12-11T17:46:10.083 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:10.077+0000 7f07667fc700 1 -- 172.21.15.59:0/2372078066 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0760005030 con 0x7f07701048d0 2023-12-11T17:46:10.083 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:10.077+0000 7f07777c7700 1 -- 172.21.15.59:0/2372078066 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f077019f520 con 0x7f07701048d0 2023-12-11T17:46:10.083 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:10.077+0000 7f07667fc700 1 -- 172.21.15.59:0/2372078066 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0760004b00 con 0x7f07701048d0 2023-12-11T17:46:10.083 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:10.077+0000 7f07777c7700 1 -- 172.21.15.59:0/2372078066 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f077019f9a0 con 0x7f07701048d0 2023-12-11T17:46:10.084 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:10.078+0000 7f07777c7700 1 -- 172.21.15.59:0/2372078066 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07700fd510 con 0x7f07701048d0 2023-12-11T17:46:10.084 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:10.080+0000 7f07667fc700 1 -- 172.21.15.59:0/2372078066 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) v1 ==== 136074+0+0 (secure 0 0 0) 0x7f0760005410 con 0x7f07701048d0 2023-12-11T17:46:10.084 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:10.081+0000 7f07667fc700 1 --2- 172.21.15.59:0/2372078066 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f075c0a15e0 0x7f075c0a3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:10.084 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:10.081+0000 7f0775563700 1 -- 172.21.15.59:0/2372078066 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f075c0a15e0 msgr2=0x7f075c0a3aa0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.59:6800/3627728353 2023-12-11T17:46:10.085 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:10.081+0000 7f07667fc700 1 -- 172.21.15.59:0/2372078066 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f07600ce420 con 0x7f07701048d0 2023-12-11T17:46:10.085 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:10.081+0000 7f0775563700 1 --2- 172.21.15.59:0/2372078066 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f075c0a15e0 0x7f075c0a3aa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-11T17:46:10.087 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:10.084+0000 7f07667fc700 1 -- 172.21.15.59:0/2372078066 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f07600976b0 con 0x7f07701048d0 2023-12-11T17:46:10.145 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Dec 11 17:46:09 smithi181 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-z[105741]: 2023-12-11T17:46:09.898+0000 7fa00c47d200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-11T17:46:10.145 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Dec 11 17:46:09 smithi181 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-z[105741]: 2023-12-11T17:46:09.992+0000 7fa00c47d200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-11T17:46:10.151 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:46:09 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T17:46:09.856+0000 7f46b2ba6200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-11T17:46:10.151 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:46:09 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T17:46:09.969+0000 7f46b2ba6200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-11T17:46:10.152 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:46:10 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T17:46:10.086+0000 7f46b2ba6200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-11T17:46:10.170 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Dec 11 17:46:09 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-y[104201]: 2023-12-11T17:46:09.903+0000 7fa60f175200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-11T17:46:10.170 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Dec 11 17:46:09 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-y[104201]: 2023-12-11T17:46:09.997+0000 7fa60f175200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-11T17:46:10.285 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:10.282+0000 7f0775563700 1 -- 172.21.15.59:0/2372078066 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f075c0a15e0 msgr2=0x7f075c0a3aa0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.59:6800/3627728353 2023-12-11T17:46:10.285 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:10.282+0000 7f0775563700 1 --2- 172.21.15.59:0/2372078066 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f075c0a15e0 0x7f075c0a3aa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-11T17:46:10.292 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:10.288+0000 7f07777c7700 1 -- 172.21.15.59:0/2372078066 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f077010cd80 con 0x7f07701048d0 2023-12-11T17:46:10.295 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:10.292+0000 7f07667fc700 1 -- 172.21.15.59:0/2372078066 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v24)=0 v24) v1 ==== 133+0+0 (secure 0 0 0) 0x7f076009acd0 con 0x7f07701048d0 2023-12-11T17:46:10.298 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:10.294+0000 7f07777c7700 1 -- 172.21.15.59:0/2372078066 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f075c0a15e0 msgr2=0x7f075c0a3aa0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-11T17:46:10.298 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:10.294+0000 7f07777c7700 1 --2- 172.21.15.59:0/2372078066 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f075c0a15e0 0x7f075c0a3aa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:10.298 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:10.294+0000 7f07777c7700 1 -- 172.21.15.59:0/2372078066 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f07701048d0 msgr2=0x7f07701979e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:46:10.298 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:10.294+0000 7f07777c7700 1 --2- 172.21.15.59:0/2372078066 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f07701048d0 0x7f07701979e0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f07600095c0 tx=0x7f076000f690 comp rx=0 tx=0).stop 2023-12-11T17:46:10.298 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:10.294+0000 7f07777c7700 1 -- 172.21.15.59:0/2372078066 shutdown_connections 2023-12-11T17:46:10.298 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:10.294+0000 7f07777c7700 1 --2- 172.21.15.59:0/2372078066 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7f075c0a15e0 0x7f075c0a3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:10.299 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:10.294+0000 7f07777c7700 1 --2- 172.21.15.59:0/2372078066 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0770107200 0x7f077019ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:10.299 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:10.294+0000 7f07777c7700 1 --2- 172.21.15.59:0/2372078066 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0770101fa0 0x7f07701974a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:10.299 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:10.294+0000 7f07777c7700 1 --2- 172.21.15.59:0/2372078066 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f07701048d0 0x7f07701979e0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:10.299 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:10.294+0000 7f07777c7700 1 -- 172.21.15.59:0/2372078066 >> 172.21.15.59:0/2372078066 conn(0x7f07700fb990 msgr2=0x7f0770102a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:46:10.299 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:10.294+0000 7f07777c7700 1 -- 172.21.15.59:0/2372078066 shutdown_connections 2023-12-11T17:46:10.299 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:10.294+0000 7f07777c7700 1 -- 172.21.15.59:0/2372078066 wait complete. 2023-12-11T17:46:10.311 DEBUG:teuthology.orchestra.run.smithi059:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2023-12-11T17:46:10.404 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:46:10 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T17:46:10.297+0000 7f46b2ba6200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-11T17:46:10.448 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:10.439+0000 7fd6c6485700 1 -- 172.21.15.59:0/612551770 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd6c0102720 msgr2=0x7fd6c0102b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:46:10.448 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:10.439+0000 7fd6c6485700 1 --2- 172.21.15.59:0/612551770 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd6c0102720 0x7fd6c0102b80 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fd6b40099c0 tx=0x7fd6b4009cd0 comp rx=0 tx=0).stop 2023-12-11T17:46:10.449 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:10.439+0000 7fd6c6485700 1 -- 172.21.15.59:0/612551770 shutdown_connections 2023-12-11T17:46:10.449 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:10.439+0000 7fd6c6485700 1 --2- 172.21.15.59:0/612551770 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd6c01030c0 0x7fd6c0109970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:10.449 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:10.439+0000 7fd6c6485700 1 --2- 172.21.15.59:0/612551770 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd6c0101530 0x7fd6c0101950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:10.449 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:10.439+0000 7fd6c6485700 1 --2- 172.21.15.59:0/612551770 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd6c0102720 0x7fd6c0102b80 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:10.449 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:10.439+0000 7fd6c6485700 1 -- 172.21.15.59:0/612551770 >> 172.21.15.59:0/612551770 conn(0x7fd6c00fcaa0 msgr2=0x7fd6c00feee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:46:10.449 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:10.439+0000 7fd6c6485700 1 -- 172.21.15.59:0/612551770 shutdown_connections 2023-12-11T17:46:10.449 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:10.439+0000 7fd6c6485700 1 -- 172.21.15.59:0/612551770 wait complete. 2023-12-11T17:46:10.450 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:10.440+0000 7fd6c6485700 1 Processor -- start 2023-12-11T17:46:10.450 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:10.440+0000 7fd6c6485700 1 -- start start 2023-12-11T17:46:10.450 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:10.440+0000 7fd6c6485700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd6c0101530 0x7fd6c0075330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:10.450 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:10.440+0000 7fd6c6485700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd6c0102720 0x7fd6c00739c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:10.450 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:10.440+0000 7fd6c6485700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd6c01030c0 0x7fd6c0073f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:10.450 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:10.440+0000 7fd6c6485700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6c00746a0 con 0x7fd6c01030c0 2023-12-11T17:46:10.450 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:10.440+0000 7fd6c6485700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6c0074810 con 0x7fd6c0102720 2023-12-11T17:46:10.450 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:10.440+0000 7fd6c6485700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6c01a1510 con 0x7fd6c0101530 2023-12-11T17:46:10.451 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:10.440+0000 7fd6c4a22700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd6c01030c0 0x7fd6c0073f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:46:10.451 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:10.440+0000 7fd6c4a22700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd6c01030c0 0x7fd6c0073f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34820/0 (socket says 172.21.15.59:34820) 2023-12-11T17:46:10.451 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:10.440+0000 7fd6c4a22700 1 -- 172.21.15.59:0/3178124056 learned_addr learned my addr 172.21.15.59:0/3178124056 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:46:10.451 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:10.440+0000 7fd6bf7fe700 1 --2- 172.21.15.59:0/3178124056 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd6c0102720 0x7fd6c00739c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:46:10.451 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:10.440+0000 7fd6bffff700 1 --2- 172.21.15.59:0/3178124056 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd6c0101530 0x7fd6c0075330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:46:10.451 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:10.441+0000 7fd6c4a22700 1 -- 172.21.15.59:0/3178124056 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd6c0101530 msgr2=0x7fd6c0075330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:46:10.451 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:10.441+0000 7fd6c4a22700 1 --2- 172.21.15.59:0/3178124056 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd6c0101530 0x7fd6c0075330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:10.451 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:10.441+0000 7fd6c4a22700 1 -- 172.21.15.59:0/3178124056 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd6c0102720 msgr2=0x7fd6c00739c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:46:10.451 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:10.441+0000 7fd6c4a22700 1 --2- 172.21.15.59:0/3178124056 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd6c0102720 0x7fd6c00739c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:10.451 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:10.441+0000 7fd6c4a22700 1 -- 172.21.15.59:0/3178124056 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6b40095f0 con 0x7fd6c01030c0 2023-12-11T17:46:10.451 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:10.441+0000 7fd6c4a22700 1 --2- 172.21.15.59:0/3178124056 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd6c01030c0 0x7fd6c0073f00 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fd6b800ef30 tx=0x7fd6b800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:46:10.452 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:10.441+0000 7fd6bd7fa700 1 -- 172.21.15.59:0/3178124056 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6b8018070 con 0x7fd6c01030c0 2023-12-11T17:46:10.452 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:10.441+0000 7fd6bd7fa700 1 -- 172.21.15.59:0/3178124056 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd6b800f040 con 0x7fd6c01030c0 2023-12-11T17:46:10.452 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:10.441+0000 7fd6c6485700 1 -- 172.21.15.59:0/3178124056 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6c01a1740 con 0x7fd6c01030c0 2023-12-11T17:46:10.452 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:10.441+0000 7fd6bd7fa700 1 -- 172.21.15.59:0/3178124056 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6b800cca0 con 0x7fd6c01030c0 2023-12-11T17:46:10.452 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:10.441+0000 7fd6c6485700 1 -- 172.21.15.59:0/3178124056 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6c01a1c00 con 0x7fd6c01030c0 2023-12-11T17:46:10.452 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:10.442+0000 7fd6c6485700 1 -- 172.21.15.59:0/3178124056 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6c0108950 con 0x7fd6c01030c0 2023-12-11T17:46:10.452 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:10.443+0000 7fd6bd7fa700 1 -- 172.21.15.59:0/3178124056 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) v1 ==== 136074+0+0 (secure 0 0 0) 0x7fd6b8003a00 con 0x7fd6c01030c0 2023-12-11T17:46:10.452 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:10.444+0000 7fd6bd7fa700 1 --2- 172.21.15.59:0/3178124056 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fd6a80a1560 0x7fd6a80a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:10.452 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:10.444+0000 7fd6bffff700 1 -- 172.21.15.59:0/3178124056 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fd6a80a1560 msgr2=0x7fd6a80a3a20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.59:6800/3627728353 2023-12-11T17:46:10.453 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:10.444+0000 7fd6bd7fa700 1 -- 172.21.15.59:0/3178124056 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fd6b80cd6f0 con 0x7fd6c01030c0 2023-12-11T17:46:10.453 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:10.444+0000 7fd6bffff700 1 --2- 172.21.15.59:0/3178124056 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fd6a80a1560 0x7fd6a80a3a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-11T17:46:10.453 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:10.447+0000 7fd6bd7fa700 1 -- 172.21.15.59:0/3178124056 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fd6b8096920 con 0x7fd6c01030c0 2023-12-11T17:46:10.529 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Dec 11 17:46:10 smithi181 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-z[105741]: 2023-12-11T17:46:10.162+0000 7fa00c47d200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-11T17:46:10.532 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Dec 11 17:46:10 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-y[104201]: 2023-12-11T17:46:10.168+0000 7fa60f175200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-11T17:46:10.648 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:10.645+0000 7fd6bffff700 1 -- 172.21.15.59:0/3178124056 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fd6a80a1560 msgr2=0x7fd6a80a3a20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.59:6800/3627728353 2023-12-11T17:46:10.648 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:10.645+0000 7fd6bffff700 1 --2- 172.21.15.59:0/3178124056 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fd6a80a1560 0x7fd6a80a3a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-11T17:46:10.681 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:10.678+0000 7fd6c6485700 1 -- 172.21.15.59:0/3178124056 --> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7fd6c0061a10 con 0x7fd6a80a1560 2023-12-11T17:46:10.840 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:10 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/2488983732' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2023-12-11T17:46:10.840 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:10 smithi181 ceph-mon[103627]: mgrmap e18: x(active, since 7m), standbys: z, y 2023-12-11T17:46:10.840 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:10 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/2050560681' entity='client.admin' 2023-12-11T17:46:10.840 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:10 smithi181 ceph-mon[103627]: from='client.? 172.21.15.59:0/2372078066' entity='client.admin' 2023-12-11T17:46:10.841 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Dec 11 17:46:10 smithi181 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-z[105741]: 2023-12-11T17:46:10.526+0000 7fa00c47d200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-11T17:46:10.841 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Dec 11 17:46:10 smithi181 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-z[105741]: 2023-12-11T17:46:10.656+0000 7fa00c47d200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-11T17:46:10.841 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Dec 11 17:46:10 smithi181 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-z[105741]: 2023-12-11T17:46:10.743+0000 7fa00c47d200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-11T17:46:10.852 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:10 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/2488983732' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2023-12-11T17:46:10.852 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:10 smithi115 ceph-mon[102748]: mgrmap e18: x(active, since 7m), standbys: z, y 2023-12-11T17:46:10.852 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:10 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/2050560681' entity='client.admin' 2023-12-11T17:46:10.852 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:10 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/2372078066' entity='client.admin' 2023-12-11T17:46:10.853 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Dec 11 17:46:10 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-y[104201]: 2023-12-11T17:46:10.535+0000 7fa60f175200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-11T17:46:10.853 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Dec 11 17:46:10 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-y[104201]: 2023-12-11T17:46:10.666+0000 7fa60f175200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-11T17:46:10.853 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Dec 11 17:46:10 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-y[104201]: 2023-12-11T17:46:10.754+0000 7fa60f175200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-11T17:46:10.904 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:46:10 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T17:46:10.783+0000 7f46b2ba6200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-11T17:46:10.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:10 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/2488983732' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2023-12-11T17:46:10.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:10 smithi059 ceph-mon[98847]: mgrmap e18: x(active, since 7m), standbys: z, y 2023-12-11T17:46:10.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:10 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/2050560681' entity='client.admin' 2023-12-11T17:46:10.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:10 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/2372078066' entity='client.admin' 2023-12-11T17:46:11.049 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:11.046+0000 7fd6bffff700 1 -- 172.21.15.59:0/3178124056 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fd6a80a1560 msgr2=0x7fd6a80a3a20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.59:6800/3627728353 2023-12-11T17:46:11.049 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:11.046+0000 7fd6bffff700 1 --2- 172.21.15.59:0/3178124056 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fd6a80a1560 0x7fd6a80a3a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-11T17:46:11.274 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Dec 11 17:46:10 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-y[104201]: 2023-12-11T17:46:10.849+0000 7fa60f175200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-11T17:46:11.404 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:46:10 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T17:46:10.936+0000 7f46b2ba6200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-11T17:46:11.404 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:46:11 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T17:46:11.046+0000 7f46b2ba6200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-11T17:46:11.404 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:46:11 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T17:46:11.161+0000 7f46b2ba6200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-11T17:46:11.532 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Dec 11 17:46:11 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-y[104201]: 2023-12-11T17:46:11.272+0000 7fa60f175200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-11T17:46:11.645 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Dec 11 17:46:10 smithi181 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-z[105741]: 2023-12-11T17:46:10.837+0000 7fa00c47d200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-11T17:46:11.645 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Dec 11 17:46:11 smithi181 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-z[105741]: 2023-12-11T17:46:11.257+0000 7fa00c47d200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-11T17:46:11.849 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:11.847+0000 7fd6bffff700 1 -- 172.21.15.59:0/3178124056 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fd6a80a1560 msgr2=0x7fd6a80a3a20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.59:6800/3627728353 2023-12-11T17:46:11.850 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:11.847+0000 7fd6bffff700 1 --2- 172.21.15.59:0/3178124056 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fd6a80a1560 0x7fd6a80a3a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-11T17:46:12.155 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:46:11 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T17:46:11.672+0000 7f46b2ba6200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-11T17:46:12.283 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Dec 11 17:46:11 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-y[104201]: 2023-12-11T17:46:11.929+0000 7fa60f175200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-11T17:46:12.283 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Dec 11 17:46:12 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-y[104201]: 2023-12-11T17:46:12.011+0000 7fa60f175200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-11T17:46:12.395 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Dec 11 17:46:11 smithi181 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-z[105741]: 2023-12-11T17:46:11.913+0000 7fa00c47d200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-11T17:46:12.395 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Dec 11 17:46:11 smithi181 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-z[105741]: 2023-12-11T17:46:11.995+0000 7fa00c47d200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-11T17:46:12.904 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:46:12 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T17:46:12.457+0000 7f46b2ba6200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-11T17:46:12.904 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:46:12 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T17:46:12.557+0000 7f46b2ba6200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-11T17:46:13.040 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Dec 11 17:46:12 smithi181 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-z[105741]: 2023-12-11T17:46:12.771+0000 7fa00c47d200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-11T17:46:13.040 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Dec 11 17:46:12 smithi181 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-z[105741]: 2023-12-11T17:46:12.863+0000 7fa00c47d200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-11T17:46:13.056 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Dec 11 17:46:12 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-y[104201]: 2023-12-11T17:46:12.789+0000 7fa60f175200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-11T17:46:13.057 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Dec 11 17:46:12 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-y[104201]: 2023-12-11T17:46:12.881+0000 7fa60f175200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-11T17:46:13.395 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Dec 11 17:46:13 smithi181 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-z[105741]: 2023-12-11T17:46:13.037+0000 7fa00c47d200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-11T17:46:13.451 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:13.449+0000 7fd6bffff700 1 -- 172.21.15.59:0/3178124056 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fd6a80a1560 msgr2=0x7fd6a80a3a20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.59:6800/3627728353 2023-12-11T17:46:13.452 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:13.449+0000 7fd6bffff700 1 --2- 172.21.15.59:0/3178124056 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fd6a80a1560 0x7fd6a80a3a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-12-11T17:46:13.473 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Dec 11 17:46:13 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-y[104201]: 2023-12-11T17:46:13.054+0000 7fa60f175200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-11T17:46:13.782 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Dec 11 17:46:13 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-y[104201]: 2023-12-11T17:46:13.471+0000 7fa60f175200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-11T17:46:13.783 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Dec 11 17:46:13 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-y[104201]: 2023-12-11T17:46:13.558+0000 7fa60f175200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-11T17:46:13.783 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Dec 11 17:46:13 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-y[104201]: 2023-12-11T17:46:13.707+0000 7fa60f175200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-11T17:46:13.883 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:46:13 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T17:46:13.542+0000 7f46b2ba6200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-11T17:46:13.883 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:46:13 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T17:46:13.655+0000 7f46b2ba6200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-11T17:46:13.895 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Dec 11 17:46:13 smithi181 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-z[105741]: 2023-12-11T17:46:13.455+0000 7fa00c47d200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-11T17:46:13.895 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Dec 11 17:46:13 smithi181 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-z[105741]: 2023-12-11T17:46:13.543+0000 7fa00c47d200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-11T17:46:13.895 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Dec 11 17:46:13 smithi181 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-z[105741]: 2023-12-11T17:46:13.690+0000 7fa00c47d200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-11T17:46:14.154 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:46:13 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T17:46:13.880+0000 7f46b2ba6200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-11T17:46:14.283 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Dec 11 17:46:13 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-y[104201]: 2023-12-11T17:46:13.974+0000 7fa60f175200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-11T17:46:14.395 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Dec 11 17:46:13 smithi181 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-z[105741]: 2023-12-11T17:46:13.954+0000 7fa00c47d200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-11T17:46:14.654 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:46:14 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T17:46:14.382+0000 7f46b2ba6200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-11T17:46:14.654 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:46:14 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T17:46:14.489+0000 7f46b2ba6200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-11T17:46:14.979 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Dec 11 17:46:14 smithi181 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-z[105741]: 2023-12-11T17:46:14.652+0000 7fa00c47d200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-11T17:46:14.980 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Dec 11 17:46:14 smithi181 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-z[105741]: 2023-12-11T17:46:14.782+0000 7fa00c47d200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-11T17:46:15.000 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Dec 11 17:46:14 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-y[104201]: 2023-12-11T17:46:14.672+0000 7fa60f175200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-11T17:46:15.000 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Dec 11 17:46:14 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-y[104201]: 2023-12-11T17:46:14.803+0000 7fa60f175200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-11T17:46:15.013 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:46:14 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T17:46:14.673+0000 7f46b2ba6200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-11T17:46:15.282 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Dec 11 17:46:14 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-y[104201]: 2023-12-11T17:46:14.998+0000 7fa60f175200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-11T17:46:15.283 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Dec 11 17:46:15 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-y[104201]: 2023-12-11T17:46:15.111+0000 7fa60f175200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-11T17:46:15.308 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Dec 11 17:46:14 smithi181 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-z[105741]: 2023-12-11T17:46:14.976+0000 7fa00c47d200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-11T17:46:15.308 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Dec 11 17:46:15 smithi181 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-z[105741]: 2023-12-11T17:46:15.088+0000 7fa00c47d200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-11T17:46:15.404 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:46:15 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T17:46:15.010+0000 7f46b2ba6200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-11T17:46:15.645 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Dec 11 17:46:15 smithi181 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-z[105741]: 2023-12-11T17:46:15.305+0000 7fa00c47d200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-11T17:46:15.783 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Dec 11 17:46:15 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-y[104201]: 2023-12-11T17:46:15.329+0000 7fa60f175200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-11T17:46:16.154 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:46:15 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T17:46:15.839+0000 7f46b2ba6200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-11T17:46:16.154 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:46:16 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T17:46:15.999+0000 7f46b2ba6200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-11T17:46:16.240 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:16.238+0000 7fd6bd7fa700 1 -- 172.21.15.59:0/3178124056 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mgrmap(e 19) v1 ==== 136074+0+0 (secure 0 0 0) 0x7fd6b8003cb0 con 0x7fd6c01030c0 2023-12-11T17:46:16.533 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Dec 11 17:46:16 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-y[104201]: 2023-12-11T17:46:16.188+0000 7fa60f175200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-11T17:46:16.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:16 smithi115 ceph-mon[102748]: Standby manager daemon z restarted 2023-12-11T17:46:16.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:16 smithi115 ceph-mon[102748]: Standby manager daemon z started 2023-12-11T17:46:16.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:16 smithi115 ceph-mon[102748]: from='mgr.? 172.21.15.181:0/1728008140' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-12-11T17:46:16.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:16 smithi115 ceph-mon[102748]: from='mgr.? 172.21.15.181:0/1728008140' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-12-11T17:46:16.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:16 smithi115 ceph-mon[102748]: from='mgr.? 172.21.15.181:0/1728008140' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-12-11T17:46:16.534 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:16 smithi115 ceph-mon[102748]: from='mgr.? 172.21.15.181:0/1728008140' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-12-11T17:46:16.534 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:16 smithi115 ceph-mon[102748]: Standby manager daemon y restarted 2023-12-11T17:46:16.534 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:16 smithi115 ceph-mon[102748]: Standby manager daemon y started 2023-12-11T17:46:16.534 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:16 smithi115 ceph-mon[102748]: from='mgr.? 172.21.15.115:0/435840665' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-12-11T17:46:16.534 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:16 smithi115 ceph-mon[102748]: from='mgr.? 172.21.15.115:0/435840665' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-12-11T17:46:16.534 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:16 smithi115 ceph-mon[102748]: from='mgr.? 172.21.15.115:0/435840665' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-12-11T17:46:16.534 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:16 smithi115 ceph-mon[102748]: from='mgr.? 172.21.15.115:0/435840665' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-12-11T17:46:16.645 INFO:journalctl@ceph.mgr.z.smithi181.stdout:Dec 11 17:46:16 smithi181 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-z[105741]: 2023-12-11T17:46:16.164+0000 7fa00c47d200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-11T17:46:16.645 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:16 smithi181 ceph-mon[103627]: Standby manager daemon z restarted 2023-12-11T17:46:16.645 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:16 smithi181 ceph-mon[103627]: Standby manager daemon z started 2023-12-11T17:46:16.645 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:16 smithi181 ceph-mon[103627]: from='mgr.? 172.21.15.181:0/1728008140' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-12-11T17:46:16.645 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:16 smithi181 ceph-mon[103627]: from='mgr.? 172.21.15.181:0/1728008140' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-12-11T17:46:16.645 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:16 smithi181 ceph-mon[103627]: from='mgr.? 172.21.15.181:0/1728008140' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-12-11T17:46:16.645 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:16 smithi181 ceph-mon[103627]: from='mgr.? 172.21.15.181:0/1728008140' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-12-11T17:46:16.646 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:16 smithi181 ceph-mon[103627]: Standby manager daemon y restarted 2023-12-11T17:46:16.646 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:16 smithi181 ceph-mon[103627]: Standby manager daemon y started 2023-12-11T17:46:16.646 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:16 smithi181 ceph-mon[103627]: from='mgr.? 172.21.15.115:0/435840665' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-12-11T17:46:16.646 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:16 smithi181 ceph-mon[103627]: from='mgr.? 172.21.15.115:0/435840665' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-12-11T17:46:16.646 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:16 smithi181 ceph-mon[103627]: from='mgr.? 172.21.15.115:0/435840665' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-12-11T17:46:16.646 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:16 smithi181 ceph-mon[103627]: from='mgr.? 172.21.15.115:0/435840665' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-12-11T17:46:16.654 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:16.651+0000 7fd6bffff700 1 -- 172.21.15.59:0/3178124056 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fd6a80a1560 msgr2=0x7fd6a80a3a20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.59:6800/3627728353 2023-12-11T17:46:16.654 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:16.652+0000 7fd6bffff700 1 --2- 172.21.15.59:0/3178124056 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fd6a80a1560 0x7fd6a80a3a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2023-12-11T17:46:16.654 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:46:16 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T17:46:16.256+0000 7f46b2ba6200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-11T17:46:16.654 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:46:16 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T17:46:16.395+0000 7f46b2ba6200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-11T17:46:16.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:16 smithi059 ceph-mon[98847]: Standby manager daemon z restarted 2023-12-11T17:46:16.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:16 smithi059 ceph-mon[98847]: Standby manager daemon z started 2023-12-11T17:46:16.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:16 smithi059 ceph-mon[98847]: from='mgr.? 172.21.15.181:0/1728008140' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-12-11T17:46:16.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:16 smithi059 ceph-mon[98847]: from='mgr.? 172.21.15.181:0/1728008140' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-12-11T17:46:16.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:16 smithi059 ceph-mon[98847]: from='mgr.? 172.21.15.181:0/1728008140' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-12-11T17:46:16.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:16 smithi059 ceph-mon[98847]: from='mgr.? 172.21.15.181:0/1728008140' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-12-11T17:46:16.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:16 smithi059 ceph-mon[98847]: Standby manager daemon y restarted 2023-12-11T17:46:16.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:16 smithi059 ceph-mon[98847]: Standby manager daemon y started 2023-12-11T17:46:16.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:16 smithi059 ceph-mon[98847]: from='mgr.? 172.21.15.115:0/435840665' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-12-11T17:46:16.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:16 smithi059 ceph-mon[98847]: from='mgr.? 172.21.15.115:0/435840665' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-12-11T17:46:16.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:16 smithi059 ceph-mon[98847]: from='mgr.? 172.21.15.115:0/435840665' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-12-11T17:46:16.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:16 smithi059 ceph-mon[98847]: from='mgr.? 172.21.15.115:0/435840665' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-12-11T17:46:17.154 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:46:16 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T17:46:16.675+0000 7f46b2ba6200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-11T17:46:17.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:17 smithi115 ceph-mon[102748]: mgrmap e19: x(active, since 8m), standbys: z, y 2023-12-11T17:46:17.558 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:17 smithi059 ceph-mon[98847]: mgrmap e19: x(active, since 8m), standbys: z, y 2023-12-11T17:46:17.645 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:17 smithi181 ceph-mon[103627]: mgrmap e19: x(active, since 8m), standbys: z, y 2023-12-11T17:46:17.766 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:17.764+0000 7fd6bd7fa700 1 -- 172.21.15.59:0/3178124056 <== mon.0 v2:172.21.15.59:3300/0 8 ==== mgrmap(e 20) v1 ==== 135691+0+0 (secure 0 0 0) 0x7fd6b8096cf0 con 0x7fd6c01030c0 2023-12-11T17:46:17.767 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:17.764+0000 7fd6bd7fa700 1 -- 172.21.15.59:0/3178124056 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fd6a80a1560 msgr2=0x7fd6a80a3a20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-11T17:46:17.767 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:17.764+0000 7fd6bd7fa700 1 --2- 172.21.15.59:0/3178124056 >> [v2:172.21.15.59:6800/3627728353,v1:172.21.15.59:6801/3627728353] conn(0x7fd6a80a1560 0x7fd6a80a3a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:17.905 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:46:17 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T17:46:17.738+0000 7f46b2ba6200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-11T17:46:18.242 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:46:17 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T17:46:17.981+0000 7f4615473700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-11T17:46:18.242 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:46:17 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T17:46:17.981+0000 7f4615473700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-11T17:46:18.242 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:46:17 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T17:46:17.981+0000 7f4615473700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-11T17:46:18.242 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:46:17 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T17:46:17.981+0000 7f4615473700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-11T17:46:18.242 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:46:17 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T17:46:17.981+0000 7f4615473700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-11T17:46:18.242 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:46:17 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T17:46:17.982+0000 7f4612c6e700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-11T17:46:18.242 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:46:17 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T17:46:17.982+0000 7f4612c6e700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-11T17:46:18.243 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:46:17 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T17:46:17.982+0000 7f4612c6e700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-11T17:46:18.243 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:46:17 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T17:46:17.982+0000 7f4612c6e700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-11T17:46:18.243 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:46:17 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T17:46:17.982+0000 7f4612c6e700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-11T17:46:18.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:18 smithi115 ceph-mon[102748]: Active manager daemon x restarted 2023-12-11T17:46:18.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:18 smithi115 ceph-mon[102748]: Activating manager daemon x 2023-12-11T17:46:18.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:18 smithi115 ceph-mon[102748]: osdmap e89: 12 total, 12 up, 12 in 2023-12-11T17:46:18.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:18 smithi115 ceph-mon[102748]: mgrmap e20: x(active, starting, since 0.0132475s), standbys: z, y 2023-12-11T17:46:18.534 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:18 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-11T17:46:18.534 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:18 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-11T17:46:18.534 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:18 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-11T17:46:18.534 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:18 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-12-11T17:46:18.534 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:18 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-12-11T17:46:18.534 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:18 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-12-11T17:46:18.534 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:18 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-12-11T17:46:18.534 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:18 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-12-11T17:46:18.534 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:18 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-12-11T17:46:18.534 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:18 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-12-11T17:46:18.534 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:18 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-12-11T17:46:18.534 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:18 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-12-11T17:46:18.534 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:18 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-12-11T17:46:18.534 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:18 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-12-11T17:46:18.535 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:18 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-12-11T17:46:18.535 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:18 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-12-11T17:46:18.535 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:18 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-12-11T17:46:18.535 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:18 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-12-11T17:46:18.535 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:18 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-12-11T17:46:18.535 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:18 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-12-11T17:46:18.535 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:18 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-12-11T17:46:18.535 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:18 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-11T17:46:18.535 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:18 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-11T17:46:18.535 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:18 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-11T17:46:18.535 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:18 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-11T17:46:18.535 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:18 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-11T17:46:18.535 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:18 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-11T17:46:18.535 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:18 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-11T17:46:18.536 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:18 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-11T17:46:18.536 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:18 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-11T17:46:18.536 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:18 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-11T17:46:18.536 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:18 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-11T17:46:18.536 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:18 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-11T17:46:18.536 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:18 smithi115 ceph-mon[102748]: Manager daemon x is now available 2023-12-11T17:46:18.536 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:18 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:46:18.536 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:18 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:46:18.536 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:18 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-12-11T17:46:18.536 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:18 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-12-11T17:46:18.646 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:18 smithi181 ceph-mon[103627]: Active manager daemon x restarted 2023-12-11T17:46:18.646 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:18 smithi181 ceph-mon[103627]: Activating manager daemon x 2023-12-11T17:46:18.646 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:18 smithi181 ceph-mon[103627]: osdmap e89: 12 total, 12 up, 12 in 2023-12-11T17:46:18.646 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:18 smithi181 ceph-mon[103627]: mgrmap e20: x(active, starting, since 0.0132475s), standbys: z, y 2023-12-11T17:46:18.646 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:18 smithi181 ceph-mon[103627]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-11T17:46:18.646 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:18 smithi181 ceph-mon[103627]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-11T17:46:18.646 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:18 smithi181 ceph-mon[103627]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-11T17:46:18.646 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:18 smithi181 ceph-mon[103627]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-12-11T17:46:18.646 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:18 smithi181 ceph-mon[103627]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-12-11T17:46:18.646 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:18 smithi181 ceph-mon[103627]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-12-11T17:46:18.647 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:18 smithi181 ceph-mon[103627]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-12-11T17:46:18.647 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:18 smithi181 ceph-mon[103627]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-12-11T17:46:18.647 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:18 smithi181 ceph-mon[103627]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-12-11T17:46:18.647 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:18 smithi181 ceph-mon[103627]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-12-11T17:46:18.647 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:18 smithi181 ceph-mon[103627]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-12-11T17:46:18.647 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:18 smithi181 ceph-mon[103627]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-12-11T17:46:18.647 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:18 smithi181 ceph-mon[103627]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-12-11T17:46:18.647 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:18 smithi181 ceph-mon[103627]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-12-11T17:46:18.647 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:18 smithi181 ceph-mon[103627]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-12-11T17:46:18.647 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:18 smithi181 ceph-mon[103627]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-12-11T17:46:18.647 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:18 smithi181 ceph-mon[103627]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-12-11T17:46:18.647 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:18 smithi181 ceph-mon[103627]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-12-11T17:46:18.648 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:18 smithi181 ceph-mon[103627]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-12-11T17:46:18.648 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:18 smithi181 ceph-mon[103627]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-12-11T17:46:18.648 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:18 smithi181 ceph-mon[103627]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-12-11T17:46:18.648 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:18 smithi181 ceph-mon[103627]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-11T17:46:18.648 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:18 smithi181 ceph-mon[103627]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-11T17:46:18.648 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:18 smithi181 ceph-mon[103627]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-11T17:46:18.648 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:18 smithi181 ceph-mon[103627]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-11T17:46:18.648 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:18 smithi181 ceph-mon[103627]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-11T17:46:18.648 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:18 smithi181 ceph-mon[103627]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-11T17:46:18.648 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:18 smithi181 ceph-mon[103627]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-11T17:46:18.648 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:18 smithi181 ceph-mon[103627]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-11T17:46:18.648 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:18 smithi181 ceph-mon[103627]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-11T17:46:18.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:18 smithi181 ceph-mon[103627]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-11T17:46:18.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:18 smithi181 ceph-mon[103627]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-11T17:46:18.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:18 smithi181 ceph-mon[103627]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-11T17:46:18.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:18 smithi181 ceph-mon[103627]: Manager daemon x is now available 2023-12-11T17:46:18.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:18 smithi181 ceph-mon[103627]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:46:18.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:18 smithi181 ceph-mon[103627]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:46:18.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:18 smithi181 ceph-mon[103627]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-12-11T17:46:18.649 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:18 smithi181 ceph-mon[103627]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-12-11T17:46:18.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:18 smithi059 ceph-mon[98847]: Active manager daemon x restarted 2023-12-11T17:46:18.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:18 smithi059 ceph-mon[98847]: Activating manager daemon x 2023-12-11T17:46:18.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:18 smithi059 ceph-mon[98847]: osdmap e89: 12 total, 12 up, 12 in 2023-12-11T17:46:18.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:18 smithi059 ceph-mon[98847]: mgrmap e20: x(active, starting, since 0.0132475s), standbys: z, y 2023-12-11T17:46:18.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:18 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-11T17:46:18.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:18 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-11T17:46:18.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:18 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-11T17:46:18.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:18 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-12-11T17:46:18.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:18 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-12-11T17:46:18.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:18 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-12-11T17:46:18.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:18 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-12-11T17:46:18.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:18 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-12-11T17:46:18.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:18 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-12-11T17:46:18.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:18 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-12-11T17:46:18.656 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:18 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-12-11T17:46:18.656 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:18 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-12-11T17:46:18.656 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:18 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-12-11T17:46:18.656 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:18 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-12-11T17:46:18.656 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:18 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-12-11T17:46:18.656 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:18 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-12-11T17:46:18.656 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:18 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-12-11T17:46:18.656 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:18 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-12-11T17:46:18.656 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:18 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-12-11T17:46:18.656 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:18 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-12-11T17:46:18.656 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:18 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-12-11T17:46:18.656 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:18 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-11T17:46:18.656 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:18 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-11T17:46:18.656 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:18 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-11T17:46:18.657 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:18 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-11T17:46:18.657 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:18 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-11T17:46:18.657 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:18 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-11T17:46:18.657 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:18 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-11T17:46:18.657 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:18 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-11T17:46:18.657 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:18 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-11T17:46:18.657 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:18 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-11T17:46:18.657 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:18 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-11T17:46:18.657 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:18 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-11T17:46:18.657 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:18 smithi059 ceph-mon[98847]: Manager daemon x is now available 2023-12-11T17:46:18.657 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:18 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:46:18.657 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:18 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:46:18.657 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:18 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-12-11T17:46:18.657 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:18 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-12-11T17:46:18.771 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:18.769+0000 7fd6bd7fa700 1 -- 172.21.15.59:0/3178124056 <== mon.0 v2:172.21.15.59:3300/0 9 ==== mgrmap(e 21) v1 ==== 135818+0+0 (secure 0 0 0) 0x7fd6b80caf90 con 0x7fd6c01030c0 2023-12-11T17:46:18.772 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:18.769+0000 7fd6bd7fa700 1 --2- 172.21.15.59:0/3178124056 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7fd6a801b200 0x7fd6a801b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:18.772 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:18.769+0000 7fd6bd7fa700 1 -- 172.21.15.59:0/3178124056 --> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7fd6c0061a10 con 0x7fd6a801b200 2023-12-11T17:46:18.774 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:18.772+0000 7fd6bffff700 1 --2- 172.21.15.59:0/3178124056 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7fd6a801b200 0x7fd6a801b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:46:18.776 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:18.774+0000 7fd6bffff700 1 --2- 172.21.15.59:0/3178124056 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7fd6a801b200 0x7fd6a801b5e0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd6c0102580 tx=0x7fd6b0007400 comp rx=0 tx=0).ready entity=mgr.15414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:46:18.834 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:18.832+0000 7fd6bd7fa700 1 -- 172.21.15.59:0/3178124056 <== mgr.15414 v2:172.21.15.59:6800/4126604994 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fd6c0061a10 con 0x7fd6a801b200 2023-12-11T17:46:18.835 INFO:teuthology.orchestra.run.smithi059.stdout:Schedule set for path / 2023-12-11T17:46:18.837 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:18.834+0000 7fd6c6485700 1 -- 172.21.15.59:0/3178124056 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7fd6a801b200 msgr2=0x7fd6a801b5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:46:18.837 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:18.834+0000 7fd6c6485700 1 --2- 172.21.15.59:0/3178124056 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7fd6a801b200 0x7fd6a801b5e0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd6c0102580 tx=0x7fd6b0007400 comp rx=0 tx=0).stop 2023-12-11T17:46:18.837 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:18.834+0000 7fd6c6485700 1 -- 172.21.15.59:0/3178124056 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd6c01030c0 msgr2=0x7fd6c0073f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:46:18.837 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:18.834+0000 7fd6c6485700 1 --2- 172.21.15.59:0/3178124056 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd6c01030c0 0x7fd6c0073f00 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fd6b800ef30 tx=0x7fd6b800c3a0 comp rx=0 tx=0).stop 2023-12-11T17:46:18.837 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:18.834+0000 7fd6c4a22700 1 -- 172.21.15.59:0/3178124056 reap_dead start 2023-12-11T17:46:18.837 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:18.834+0000 7fd6c6485700 1 -- 172.21.15.59:0/3178124056 shutdown_connections 2023-12-11T17:46:18.837 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:18.834+0000 7fd6c6485700 1 -- 172.21.15.59:0/3178124056 >> 172.21.15.59:0/3178124056 conn(0x7fd6c00fcaa0 msgr2=0x7fd6c0105a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:46:18.837 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:18.834+0000 7fd6c6485700 1 -- 172.21.15.59:0/3178124056 shutdown_connections 2023-12-11T17:46:18.837 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:18.834+0000 7fd6c6485700 1 -- 172.21.15.59:0/3178124056 wait complete. 2023-12-11T17:46:18.851 DEBUG:teuthology.orchestra.run.smithi059:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2023-12-11T17:46:18.989 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:18.986+0000 7f4211e41700 1 -- 172.21.15.59:0/1360782064 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f420c100430 msgr2=0x7f420c100850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:46:18.989 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:18.986+0000 7f4211e41700 1 --2- 172.21.15.59:0/1360782064 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f420c100430 0x7f420c100850 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f41fc0099c0 tx=0x7f41fc009cd0 comp rx=0 tx=0).stop 2023-12-11T17:46:18.989 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:18.986+0000 7f4211e41700 1 -- 172.21.15.59:0/1360782064 shutdown_connections 2023-12-11T17:46:18.989 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:18.986+0000 7f4211e41700 1 --2- 172.21.15.59:0/1360782064 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f420c101fc0 0x7f420c108870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:18.989 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:18.986+0000 7f4211e41700 1 --2- 172.21.15.59:0/1360782064 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f420c100430 0x7f420c100850 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:18.989 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:18.986+0000 7f4211e41700 1 --2- 172.21.15.59:0/1360782064 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f420c101620 0x7f420c101a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:18.990 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:18.986+0000 7f4211e41700 1 -- 172.21.15.59:0/1360782064 >> 172.21.15.59:0/1360782064 conn(0x7f420c0fb9c0 msgr2=0x7f420c0fdde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:46:18.990 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:18.986+0000 7f4211e41700 1 -- 172.21.15.59:0/1360782064 shutdown_connections 2023-12-11T17:46:18.990 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:18.986+0000 7f4211e41700 1 -- 172.21.15.59:0/1360782064 wait complete. 2023-12-11T17:46:18.990 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:18.987+0000 7f4211e41700 1 Processor -- start 2023-12-11T17:46:18.990 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:18.987+0000 7f4211e41700 1 -- start start 2023-12-11T17:46:18.992 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:18.987+0000 7f4211e41700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f420c100430 0x7f420c195290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:18.992 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:18.987+0000 7f4211e41700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f420c101620 0x7f420c1957d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:18.992 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:18.987+0000 7f4211e41700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f420c101fc0 0x7f420c19c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:18.992 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:18.987+0000 7f4211e41700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f420c19cdc0 con 0x7f420c101fc0 2023-12-11T17:46:18.992 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:18.987+0000 7f4211e41700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f420c19cf00 con 0x7f420c101620 2023-12-11T17:46:18.992 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:18.987+0000 7f4211e41700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f420c19d090 con 0x7f420c100430 2023-12-11T17:46:18.992 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:18.987+0000 7f420bfff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f420c101fc0 0x7f420c19c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:46:18.992 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:18.987+0000 7f420bfff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f420c101fc0 0x7f420c19c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35048/0 (socket says 172.21.15.59:35048) 2023-12-11T17:46:18.992 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:18.987+0000 7f420bfff700 1 -- 172.21.15.59:0/2850715431 learned_addr learned my addr 172.21.15.59:0/2850715431 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:46:18.992 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:18.987+0000 7f420affd700 1 --2- 172.21.15.59:0/2850715431 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f420c101620 0x7f420c1957d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:46:18.993 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:18.987+0000 7f420bfff700 1 -- 172.21.15.59:0/2850715431 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f420c100430 msgr2=0x7f420c195290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:46:18.993 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:18.987+0000 7f420bfff700 1 --2- 172.21.15.59:0/2850715431 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f420c100430 0x7f420c195290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:18.993 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:18.987+0000 7f420bfff700 1 -- 172.21.15.59:0/2850715431 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f420c101620 msgr2=0x7f420c1957d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:46:18.993 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:18.987+0000 7f420bfff700 1 --2- 172.21.15.59:0/2850715431 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f420c101620 0x7f420c1957d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:18.993 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:18.987+0000 7f420bfff700 1 -- 172.21.15.59:0/2850715431 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41fc0095f0 con 0x7f420c101fc0 2023-12-11T17:46:18.993 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:18.988+0000 7f420bfff700 1 --2- 172.21.15.59:0/2850715431 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f420c101fc0 0x7f420c19c7f0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f420400ef30 tx=0x7f420400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:46:18.993 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:18.988+0000 7f4208ff9700 1 -- 172.21.15.59:0/2850715431 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4204018070 con 0x7f420c101fc0 2023-12-11T17:46:18.993 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:18.988+0000 7f4208ff9700 1 -- 172.21.15.59:0/2850715431 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f420400f040 con 0x7f420c101fc0 2023-12-11T17:46:18.993 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:18.988+0000 7f4211e41700 1 -- 172.21.15.59:0/2850715431 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f420c19d310 con 0x7f420c101fc0 2023-12-11T17:46:18.993 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:18.988+0000 7f4208ff9700 1 -- 172.21.15.59:0/2850715431 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f420400cdb0 con 0x7f420c101fc0 2023-12-11T17:46:18.993 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:18.988+0000 7f4211e41700 1 -- 172.21.15.59:0/2850715431 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f420c19d7d0 con 0x7f420c101fc0 2023-12-11T17:46:18.993 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:18.989+0000 7f4211e41700 1 -- 172.21.15.59:0/2850715431 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f420c1078f0 con 0x7f420c101fc0 2023-12-11T17:46:18.994 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:18.990+0000 7f4208ff9700 1 -- 172.21.15.59:0/2850715431 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 21) v1 ==== 135818+0+0 (secure 0 0 0) 0x7f4204003a00 con 0x7f420c101fc0 2023-12-11T17:46:18.994 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:18.991+0000 7f4208ff9700 1 --2- 172.21.15.59:0/2850715431 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7f41f40a1280 0x7f41f40a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:18.994 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:18.991+0000 7f4208ff9700 1 -- 172.21.15.59:0/2850715431 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9300+0+0 (secure 0 0 0) 0x7f42040cd350 con 0x7f420c101fc0 2023-12-11T17:46:18.994 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:18.991+0000 7f420b7fe700 1 --2- 172.21.15.59:0/2850715431 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7f41f40a1280 0x7f41f40a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:46:18.994 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:18.992+0000 7f420b7fe700 1 --2- 172.21.15.59:0/2850715431 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7f41f40a1280 0x7f41f40a3740 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f41fc009fd0 tx=0x7f41fc0036f0 comp rx=0 tx=0).ready entity=mgr.15414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:46:18.998 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:18.996+0000 7f4208ff9700 1 -- 172.21.15.59:0/2850715431 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f42040968c0 con 0x7f420c101fc0 2023-12-11T17:46:19.234 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:19.232+0000 7f4211e41700 1 -- 172.21.15.59:0/2850715431 --> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) v1 -- 0x7f420c02d450 con 0x7f41f40a1280 2023-12-11T17:46:19.294 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:19.291+0000 7f4208ff9700 1 -- 172.21.15.59:0/2850715431 <== mgr.15414 v2:172.21.15.59:6800/4126604994 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+26 (secure 0 0 0) 0x7f420c02d450 con 0x7f41f40a1280 2023-12-11T17:46:19.294 INFO:teuthology.orchestra.run.smithi059.stdout:Retention added to path / 2023-12-11T17:46:19.296 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:19.294+0000 7f4211e41700 1 -- 172.21.15.59:0/2850715431 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7f41f40a1280 msgr2=0x7f41f40a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:46:19.297 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:19.294+0000 7f4211e41700 1 --2- 172.21.15.59:0/2850715431 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7f41f40a1280 0x7f41f40a3740 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f41fc009fd0 tx=0x7f41fc0036f0 comp rx=0 tx=0).stop 2023-12-11T17:46:19.297 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:19.294+0000 7f4211e41700 1 -- 172.21.15.59:0/2850715431 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f420c101fc0 msgr2=0x7f420c19c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:46:19.297 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:19.294+0000 7f4211e41700 1 --2- 172.21.15.59:0/2850715431 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f420c101fc0 0x7f420c19c7f0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f420400ef30 tx=0x7f420400c3a0 comp rx=0 tx=0).stop 2023-12-11T17:46:19.297 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:19.294+0000 7f4211e41700 1 -- 172.21.15.59:0/2850715431 shutdown_connections 2023-12-11T17:46:19.297 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:19.294+0000 7f4211e41700 1 --2- 172.21.15.59:0/2850715431 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7f41f40a1280 0x7f41f40a3740 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:19.297 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:19.294+0000 7f4211e41700 1 --2- 172.21.15.59:0/2850715431 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f420c101fc0 0x7f420c19c7f0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:19.297 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:19.294+0000 7f4211e41700 1 --2- 172.21.15.59:0/2850715431 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f420c100430 0x7f420c195290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:19.297 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:19.294+0000 7f4211e41700 1 --2- 172.21.15.59:0/2850715431 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f420c101620 0x7f420c1957d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:19.298 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:19.294+0000 7f4211e41700 1 -- 172.21.15.59:0/2850715431 >> 172.21.15.59:0/2850715431 conn(0x7f420c0fb9c0 msgr2=0x7f420c104a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:46:19.298 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:19.294+0000 7f4211e41700 1 -- 172.21.15.59:0/2850715431 shutdown_connections 2023-12-11T17:46:19.298 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:19.294+0000 7f4211e41700 1 -- 172.21.15.59:0/2850715431 wait complete. 2023-12-11T17:46:19.309 DEBUG:teuthology.orchestra.run.smithi059:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2023-12-11T17:46:19.445 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:19.440+0000 7fe98f254700 1 -- 172.21.15.59:0/2875349953 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9881048e0 msgr2=0x7fe988106cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:46:19.445 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:19.440+0000 7fe98f254700 1 --2- 172.21.15.59:0/2875349953 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9881048e0 0x7fe988106cd0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fe974009990 tx=0x7fe974009ca0 comp rx=0 tx=0).stop 2023-12-11T17:46:19.445 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:19.440+0000 7fe98f254700 1 -- 172.21.15.59:0/2875349953 shutdown_connections 2023-12-11T17:46:19.445 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:19.440+0000 7fe98f254700 1 --2- 172.21.15.59:0/2875349953 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe988107210 0x7fe988109680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:19.446 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:19.440+0000 7fe98f254700 1 --2- 172.21.15.59:0/2875349953 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe988101fb0 0x7fe9881043a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:19.446 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:19.440+0000 7fe98f254700 1 --2- 172.21.15.59:0/2875349953 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9881048e0 0x7fe988106cd0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:19.446 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:19.440+0000 7fe98f254700 1 -- 172.21.15.59:0/2875349953 >> 172.21.15.59:0/2875349953 conn(0x7fe9880fb9c0 msgr2=0x7fe9880fdde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:46:19.446 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:19.440+0000 7fe98f254700 1 -- 172.21.15.59:0/2875349953 shutdown_connections 2023-12-11T17:46:19.446 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:19.440+0000 7fe98f254700 1 -- 172.21.15.59:0/2875349953 wait complete. 2023-12-11T17:46:19.446 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:19.441+0000 7fe98f254700 1 Processor -- start 2023-12-11T17:46:19.446 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:19.441+0000 7fe98f254700 1 -- start start 2023-12-11T17:46:19.446 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:19.441+0000 7fe98f254700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe988101fb0 0x7fe988197430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:19.446 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:19.441+0000 7fe98f254700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe9881048e0 0x7fe988197970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:19.446 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:19.441+0000 7fe98f254700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe988107210 0x7fe98819e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:19.446 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:19.441+0000 7fe98f254700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe98819ef60 con 0x7fe988107210 2023-12-11T17:46:19.446 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:19.441+0000 7fe98f254700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe98819f0a0 con 0x7fe9881048e0 2023-12-11T17:46:19.446 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:19.441+0000 7fe98f254700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe98819f230 con 0x7fe988101fb0 2023-12-11T17:46:19.446 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:19.441+0000 7fe98d7f1700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe988107210 0x7fe98819e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:46:19.447 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:19.441+0000 7fe98cff0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe988101fb0 0x7fe988197430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:46:19.447 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:19.441+0000 7fe98d7f1700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe988107210 0x7fe98819e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35062/0 (socket says 172.21.15.59:35062) 2023-12-11T17:46:19.447 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:19.441+0000 7fe98d7f1700 1 -- 172.21.15.59:0/223307004 learned_addr learned my addr 172.21.15.59:0/223307004 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:46:19.447 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:19.441+0000 7fe987fff700 1 --2- 172.21.15.59:0/223307004 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe9881048e0 0x7fe988197970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:46:19.447 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:19.442+0000 7fe98cff0700 1 -- 172.21.15.59:0/223307004 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe9881048e0 msgr2=0x7fe988197970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:46:19.447 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:19.442+0000 7fe98cff0700 1 --2- 172.21.15.59:0/223307004 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe9881048e0 0x7fe988197970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:19.447 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:19.442+0000 7fe98cff0700 1 -- 172.21.15.59:0/223307004 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe988107210 msgr2=0x7fe98819e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:46:19.447 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:19.442+0000 7fe98cff0700 1 --2- 172.21.15.59:0/223307004 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe988107210 0x7fe98819e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:19.447 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:19.442+0000 7fe98cff0700 1 -- 172.21.15.59:0/223307004 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe97c009520 con 0x7fe988101fb0 2023-12-11T17:46:19.447 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:19.442+0000 7fe98d7f1700 1 --2- 172.21.15.59:0/223307004 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe988107210 0x7fe98819e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T17:46:19.447 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:19.442+0000 7fe98cff0700 1 --2- 172.21.15.59:0/223307004 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe988101fb0 0x7fe988197430 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe97c00e9f0 tx=0x7fe97c00ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:46:19.447 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:19.442+0000 7fe985ffb700 1 -- 172.21.15.59:0/223307004 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe97c00cb70 con 0x7fe988101fb0 2023-12-11T17:46:19.447 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:19.442+0000 7fe98f254700 1 -- 172.21.15.59:0/223307004 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9740095f0 con 0x7fe988101fb0 2023-12-11T17:46:19.447 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:19.442+0000 7fe985ffb700 1 -- 172.21.15.59:0/223307004 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe97c00ee90 con 0x7fe988101fb0 2023-12-11T17:46:19.447 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:19.442+0000 7fe985ffb700 1 -- 172.21.15.59:0/223307004 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe97c003cf0 con 0x7fe988101fb0 2023-12-11T17:46:19.448 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:19.442+0000 7fe98f254700 1 -- 172.21.15.59:0/223307004 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe98819f8e0 con 0x7fe988101fb0 2023-12-11T17:46:19.448 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:19.444+0000 7fe98f254700 1 -- 172.21.15.59:0/223307004 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9880fd540 con 0x7fe988101fb0 2023-12-11T17:46:19.448 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:19.444+0000 7fe985ffb700 1 -- 172.21.15.59:0/223307004 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 21) v1 ==== 135818+0+0 (secure 0 0 0) 0x7fe97c00ccd0 con 0x7fe988101fb0 2023-12-11T17:46:19.449 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:19.445+0000 7fe985ffb700 1 --2- 172.21.15.59:0/223307004 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7fe9780a1280 0x7fe9780a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:19.449 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:19.445+0000 7fe987fff700 1 --2- 172.21.15.59:0/223307004 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7fe9780a1280 0x7fe9780a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:46:19.449 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:19.445+0000 7fe985ffb700 1 -- 172.21.15.59:0/223307004 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9300+0+0 (secure 0 0 0) 0x7fe97c014070 con 0x7fe988101fb0 2023-12-11T17:46:19.449 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:19.445+0000 7fe987fff700 1 --2- 172.21.15.59:0/223307004 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7fe9780a1280 0x7fe9780a3740 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe97400b2e0 tx=0x7fe974005fd0 comp rx=0 tx=0).ready entity=mgr.15414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:46:19.451 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:19.449+0000 7fe985ffb700 1 -- 172.21.15.59:0/223307004 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fe97c096520 con 0x7fe988101fb0 2023-12-11T17:46:19.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:19 smithi115 ceph-mon[102748]: [11/Dec/2023:17:46:18] ENGINE Bus STARTING 2023-12-11T17:46:19.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:19 smithi115 ceph-mon[102748]: [11/Dec/2023:17:46:18] ENGINE Serving on https://172.21.15.59:7150 2023-12-11T17:46:19.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:19 smithi115 ceph-mon[102748]: [11/Dec/2023:17:46:18] ENGINE Serving on http://172.21.15.59:8765 2023-12-11T17:46:19.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:19 smithi115 ceph-mon[102748]: [11/Dec/2023:17:46:18] ENGINE Bus STARTED 2023-12-11T17:46:19.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:19 smithi115 ceph-mon[102748]: mgrmap e21: x(active, since 1.01851s), standbys: z, y 2023-12-11T17:46:19.645 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:19 smithi181 ceph-mon[103627]: [11/Dec/2023:17:46:18] ENGINE Bus STARTING 2023-12-11T17:46:19.645 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:19 smithi181 ceph-mon[103627]: [11/Dec/2023:17:46:18] ENGINE Serving on https://172.21.15.59:7150 2023-12-11T17:46:19.645 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:19 smithi181 ceph-mon[103627]: [11/Dec/2023:17:46:18] ENGINE Serving on http://172.21.15.59:8765 2023-12-11T17:46:19.645 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:19 smithi181 ceph-mon[103627]: [11/Dec/2023:17:46:18] ENGINE Bus STARTED 2023-12-11T17:46:19.645 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:19 smithi181 ceph-mon[103627]: mgrmap e21: x(active, since 1.01851s), standbys: z, y 2023-12-11T17:46:19.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:19 smithi059 ceph-mon[98847]: [11/Dec/2023:17:46:18] ENGINE Bus STARTING 2023-12-11T17:46:19.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:19 smithi059 ceph-mon[98847]: [11/Dec/2023:17:46:18] ENGINE Serving on https://172.21.15.59:7150 2023-12-11T17:46:19.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:19 smithi059 ceph-mon[98847]: [11/Dec/2023:17:46:18] ENGINE Serving on http://172.21.15.59:8765 2023-12-11T17:46:19.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:19 smithi059 ceph-mon[98847]: [11/Dec/2023:17:46:18] ENGINE Bus STARTED 2023-12-11T17:46:19.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:19 smithi059 ceph-mon[98847]: mgrmap e21: x(active, since 1.01851s), standbys: z, y 2023-12-11T17:46:19.702 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:19.700+0000 7fe98f254700 1 -- 172.21.15.59:0/223307004 --> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) v1 -- 0x7fe988061a10 con 0x7fe9780a1280 2023-12-11T17:46:19.711 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:19.709+0000 7fe985ffb700 1 -- 172.21.15.59:0/223307004 <== mgr.15414 v2:172.21.15.59:6800/4126604994 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+283 (secure 0 0 0) 0x7fe988061a10 con 0x7fe9780a1280 2023-12-11T17:46:19.711 INFO:teuthology.orchestra.run.smithi059.stdout:{"fs": "cephfs", "subvol": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2023-12-11T00:00:00", "created": "2023-12-11T17:46:18", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2023-12-11T17:46:19.714 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:19.711+0000 7fe98f254700 1 -- 172.21.15.59:0/223307004 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7fe9780a1280 msgr2=0x7fe9780a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:46:19.714 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:19.711+0000 7fe98f254700 1 --2- 172.21.15.59:0/223307004 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7fe9780a1280 0x7fe9780a3740 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe97400b2e0 tx=0x7fe974005fd0 comp rx=0 tx=0).stop 2023-12-11T17:46:19.714 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:19.711+0000 7fe98f254700 1 -- 172.21.15.59:0/223307004 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe988101fb0 msgr2=0x7fe988197430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:46:19.714 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:19.711+0000 7fe98f254700 1 --2- 172.21.15.59:0/223307004 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe988101fb0 0x7fe988197430 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe97c00e9f0 tx=0x7fe97c00ed00 comp rx=0 tx=0).stop 2023-12-11T17:46:19.714 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:19.711+0000 7fe98f254700 1 -- 172.21.15.59:0/223307004 shutdown_connections 2023-12-11T17:46:19.714 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:19.711+0000 7fe98f254700 1 --2- 172.21.15.59:0/223307004 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7fe9780a1280 0x7fe9780a3740 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:19.714 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:19.711+0000 7fe98f254700 1 --2- 172.21.15.59:0/223307004 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe988107210 0x7fe98819e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:19.715 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:19.711+0000 7fe98f254700 1 --2- 172.21.15.59:0/223307004 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe988101fb0 0x7fe988197430 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:19.715 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:19.711+0000 7fe98f254700 1 --2- 172.21.15.59:0/223307004 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe9881048e0 0x7fe988197970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:19.715 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:19.711+0000 7fe98f254700 1 -- 172.21.15.59:0/223307004 >> 172.21.15.59:0/223307004 conn(0x7fe9880fb9c0 msgr2=0x7fe988102a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:46:19.715 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:19.711+0000 7fe98f254700 1 -- 172.21.15.59:0/223307004 shutdown_connections 2023-12-11T17:46:19.715 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:19.711+0000 7fe98f254700 1 -- 172.21.15.59:0/223307004 wait complete. 2023-12-11T17:46:19.727 DEBUG:teuthology.orchestra.run.smithi059:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2023-12-11T17:46:19.860 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:19.857+0000 7fd2f9f08700 1 -- 172.21.15.59:0/742226700 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd2f4101650 msgr2=0x7fd2f4101ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:46:19.860 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:19.857+0000 7fd2f9f08700 1 --2- 172.21.15.59:0/742226700 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd2f4101650 0x7fd2f4101ab0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fd2e80099c0 tx=0x7fd2e8009cd0 comp rx=0 tx=0).stop 2023-12-11T17:46:19.860 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:19.857+0000 7fd2f9f08700 1 -- 172.21.15.59:0/742226700 shutdown_connections 2023-12-11T17:46:19.861 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:19.857+0000 7fd2f9f08700 1 --2- 172.21.15.59:0/742226700 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd2f4101ff0 0x7fd2f41088a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:19.861 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:19.857+0000 7fd2f9f08700 1 --2- 172.21.15.59:0/742226700 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd2f4100460 0x7fd2f4100880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:19.861 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:19.857+0000 7fd2f9f08700 1 --2- 172.21.15.59:0/742226700 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd2f4101650 0x7fd2f4101ab0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:19.861 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:19.857+0000 7fd2f9f08700 1 -- 172.21.15.59:0/742226700 >> 172.21.15.59:0/742226700 conn(0x7fd2f40fb9d0 msgr2=0x7fd2f40fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:46:19.861 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:19.857+0000 7fd2f9f08700 1 -- 172.21.15.59:0/742226700 shutdown_connections 2023-12-11T17:46:19.861 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:19.857+0000 7fd2f9f08700 1 -- 172.21.15.59:0/742226700 wait complete. 2023-12-11T17:46:19.861 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:19.857+0000 7fd2f9f08700 1 Processor -- start 2023-12-11T17:46:19.861 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:19.858+0000 7fd2f9f08700 1 -- start start 2023-12-11T17:46:19.861 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:19.858+0000 7fd2f9f08700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd2f4100460 0x7fd2f41974c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:19.861 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:19.858+0000 7fd2f9f08700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd2f4101650 0x7fd2f4197a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:19.861 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:19.858+0000 7fd2f9f08700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd2f4101ff0 0x7fd2f419ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:19.861 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:19.858+0000 7fd2f9f08700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2f419eff0 con 0x7fd2f4101ff0 2023-12-11T17:46:19.861 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:19.858+0000 7fd2f9f08700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2f419f130 con 0x7fd2f4101650 2023-12-11T17:46:19.861 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:19.858+0000 7fd2f9f08700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2f419f2c0 con 0x7fd2f4100460 2023-12-11T17:46:19.863 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:19.858+0000 7fd2f3fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd2f4101ff0 0x7fd2f419ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:46:19.863 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:19.858+0000 7fd2f3fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd2f4101ff0 0x7fd2f419ea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35072/0 (socket says 172.21.15.59:35072) 2023-12-11T17:46:19.864 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:19.858+0000 7fd2f3fff700 1 -- 172.21.15.59:0/3282300564 learned_addr learned my addr 172.21.15.59:0/3282300564 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T17:46:19.864 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:19.858+0000 7fd2f2ffd700 1 --2- 172.21.15.59:0/3282300564 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd2f4101650 0x7fd2f4197a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:46:19.864 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:19.858+0000 7fd2f37fe700 1 --2- 172.21.15.59:0/3282300564 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd2f4100460 0x7fd2f41974c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:46:19.864 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:19.859+0000 7fd2f3fff700 1 -- 172.21.15.59:0/3282300564 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd2f4100460 msgr2=0x7fd2f41974c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:46:19.864 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:19.859+0000 7fd2f3fff700 1 --2- 172.21.15.59:0/3282300564 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd2f4100460 0x7fd2f41974c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:19.864 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:19.859+0000 7fd2f3fff700 1 -- 172.21.15.59:0/3282300564 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd2f4101650 msgr2=0x7fd2f4197a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:46:19.864 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:19.859+0000 7fd2f3fff700 1 --2- 172.21.15.59:0/3282300564 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd2f4101650 0x7fd2f4197a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:19.864 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:19.859+0000 7fd2f3fff700 1 -- 172.21.15.59:0/3282300564 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2e80095f0 con 0x7fd2f4101ff0 2023-12-11T17:46:19.864 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:19.859+0000 7fd2f2ffd700 1 --2- 172.21.15.59:0/3282300564 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd2f4101650 0x7fd2f4197a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T17:46:19.864 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:19.859+0000 7fd2f3fff700 1 --2- 172.21.15.59:0/3282300564 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd2f4101ff0 0x7fd2f419ea20 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec00ef30 tx=0x7fd2ec00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:46:19.865 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:19.859+0000 7fd2f0ff9700 1 -- 172.21.15.59:0/3282300564 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2ec018070 con 0x7fd2f4101ff0 2023-12-11T17:46:19.865 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:19.859+0000 7fd2f0ff9700 1 -- 172.21.15.59:0/3282300564 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd2ec00f040 con 0x7fd2f4101ff0 2023-12-11T17:46:19.865 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:19.859+0000 7fd2f9f08700 1 -- 172.21.15.59:0/3282300564 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2f419f540 con 0x7fd2f4101ff0 2023-12-11T17:46:19.865 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:19.859+0000 7fd2f0ff9700 1 -- 172.21.15.59:0/3282300564 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2ec00cd00 con 0x7fd2f4101ff0 2023-12-11T17:46:19.865 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:19.859+0000 7fd2f9f08700 1 -- 172.21.15.59:0/3282300564 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2f419fa00 con 0x7fd2f4101ff0 2023-12-11T17:46:19.865 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:19.860+0000 7fd2f9f08700 1 -- 172.21.15.59:0/3282300564 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2f4067bc0 con 0x7fd2f4101ff0 2023-12-11T17:46:19.866 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:19.862+0000 7fd2f0ff9700 1 -- 172.21.15.59:0/3282300564 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 21) v1 ==== 135818+0+0 (secure 0 0 0) 0x7fd2ec003a00 con 0x7fd2f4101ff0 2023-12-11T17:46:19.866 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:19.862+0000 7fd2f0ff9700 1 --2- 172.21.15.59:0/3282300564 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7fd2dc0a1280 0x7fd2dc0a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T17:46:19.866 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:19.863+0000 7fd2f0ff9700 1 -- 172.21.15.59:0/3282300564 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9300+0+0 (secure 0 0 0) 0x7fd2ec0cd230 con 0x7fd2f4101ff0 2023-12-11T17:46:19.866 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:19.863+0000 7fd2f37fe700 1 --2- 172.21.15.59:0/3282300564 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7fd2dc0a1280 0x7fd2dc0a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T17:46:19.866 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:19.863+0000 7fd2f37fe700 1 --2- 172.21.15.59:0/3282300564 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7fd2dc0a1280 0x7fd2dc0a3740 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd2f41014b0 tx=0x7fd2e4009260 comp rx=0 tx=0).ready entity=mgr.15414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T17:46:19.870 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:19.868+0000 7fd2f0ff9700 1 -- 172.21.15.59:0/3282300564 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fd2ec0967e0 con 0x7fd2f4101ff0 2023-12-11T17:46:20.126 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:20.123+0000 7fd2f9f08700 1 -- 172.21.15.59:0/3282300564 --> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fd2f4069e60 con 0x7fd2dc0a1280 2023-12-11T17:46:20.132 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:20.130+0000 7fd2f0ff9700 1 -- 172.21.15.59:0/3282300564 <== mgr.15414 v2:172.21.15.59:6800/4126604994 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+10 (secure 0 0 0) 0x7fd2f4069e60 con 0x7fd2dc0a1280 2023-12-11T17:46:20.133 INFO:teuthology.orchestra.run.smithi059.stdout:/ 1m 6m3h 2023-12-11T17:46:20.135 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:20.132+0000 7fd2f9f08700 1 -- 172.21.15.59:0/3282300564 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7fd2dc0a1280 msgr2=0x7fd2dc0a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:46:20.190 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:20.132+0000 7fd2f9f08700 1 --2- 172.21.15.59:0/3282300564 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7fd2dc0a1280 0x7fd2dc0a3740 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd2f41014b0 tx=0x7fd2e4009260 comp rx=0 tx=0).stop 2023-12-11T17:46:20.190 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:20.132+0000 7fd2f9f08700 1 -- 172.21.15.59:0/3282300564 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd2f4101ff0 msgr2=0x7fd2f419ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T17:46:20.190 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:20.132+0000 7fd2f9f08700 1 --2- 172.21.15.59:0/3282300564 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd2f4101ff0 0x7fd2f419ea20 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec00ef30 tx=0x7fd2ec00c3a0 comp rx=0 tx=0).stop 2023-12-11T17:46:20.190 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:20.132+0000 7fd2f9f08700 1 -- 172.21.15.59:0/3282300564 shutdown_connections 2023-12-11T17:46:20.190 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:20.132+0000 7fd2f9f08700 1 --2- 172.21.15.59:0/3282300564 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7fd2dc0a1280 0x7fd2dc0a3740 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:20.190 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:20.132+0000 7fd2f9f08700 1 --2- 172.21.15.59:0/3282300564 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd2f4101ff0 0x7fd2f419ea20 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:20.190 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:20.132+0000 7fd2f9f08700 1 --2- 172.21.15.59:0/3282300564 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd2f4100460 0x7fd2f41974c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:20.190 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:20.132+0000 7fd2f9f08700 1 --2- 172.21.15.59:0/3282300564 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd2f4101650 0x7fd2f4197a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T17:46:20.190 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:20.132+0000 7fd2f9f08700 1 -- 172.21.15.59:0/3282300564 >> 172.21.15.59:0/3282300564 conn(0x7fd2f40fb9d0 msgr2=0x7fd2f4069750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T17:46:20.191 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:20.133+0000 7fd2f9f08700 1 -- 172.21.15.59:0/3282300564 shutdown_connections 2023-12-11T17:46:20.191 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T17:46:20.133+0000 7fd2f9f08700 1 -- 172.21.15.59:0/3282300564 wait complete. 2023-12-11T17:46:20.192 DEBUG:teuthology.orchestra.run.smithi059:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2023-12-11T17:46:20.225 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2023-12-11T17:46:20.243 INFO:teuthology.run_tasks:Running task workunit... 2023-12-11T17:46:20.261 INFO:tasks.workunit:Pulling workunits from ref a62fda73efba5520d409bb5a21bd8373289dda5d 2023-12-11T17:46:20.261 INFO:tasks.workunit:Making a separate scratch dir for every client... 2023-12-11T17:46:20.261 INFO:tasks.workunit:timeout=3h 2023-12-11T17:46:20.261 INFO:tasks.workunit:cleanup=True 2023-12-11T17:46:20.261 DEBUG:teuthology.orchestra.run.smithi059:> stat -- /home/ubuntu/cephtest/mnt.0 2023-12-11T17:46:20.284 INFO:teuthology.orchestra.run.smithi059.stdout: File: /home/ubuntu/cephtest/mnt.0 2023-12-11T17:46:20.284 INFO:teuthology.orchestra.run.smithi059.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2023-12-11T17:46:20.284 INFO:teuthology.orchestra.run.smithi059.stdout:Device: 7ch/124d Inode: 1099511628283 Links: 2 2023-12-11T17:46:20.284 INFO:teuthology.orchestra.run.smithi059.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-11T17:46:20.284 INFO:teuthology.orchestra.run.smithi059.stdout:Context: system_u:object_r:cephfs_t:s0 2023-12-11T17:46:20.284 INFO:teuthology.orchestra.run.smithi059.stdout:Access: 2023-12-11 17:44:48.999969097 +0000 2023-12-11T17:46:20.284 INFO:teuthology.orchestra.run.smithi059.stdout:Modify: 2023-12-11 17:44:48.999969097 +0000 2023-12-11T17:46:20.285 INFO:teuthology.orchestra.run.smithi059.stdout:Change: 2023-12-11 17:46:08.620842969 +0000 2023-12-11T17:46:20.285 INFO:teuthology.orchestra.run.smithi059.stdout: Birth: 2023-12-11 17:44:48.999969097 +0000 2023-12-11T17:46:20.285 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2023-12-11T17:46:20.285 DEBUG:teuthology.orchestra.run.smithi059:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2023-12-11T17:46:20.384 DEBUG:teuthology.orchestra.run.smithi059:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout a62fda73efba5520d409bb5a21bd8373289dda5d 2023-12-11T17:46:20.403 INFO:tasks.workunit.client.0.smithi059.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2023-12-11T17:46:20.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:20 smithi115 ceph-mon[102748]: from='client.15432 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T17:46:20.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:20 smithi115 ceph-mon[102748]: pgmap v3: 129 pgs: 129 active+clean; 1.0 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T17:46:20.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:20 smithi115 ceph-mon[102748]: from='client.15474 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T17:46:20.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:20 smithi181 ceph-mon[103627]: from='client.15432 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T17:46:20.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:20 smithi181 ceph-mon[103627]: pgmap v3: 129 pgs: 129 active+clean; 1.0 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T17:46:20.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:20 smithi181 ceph-mon[103627]: from='client.15474 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T17:46:20.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:20 smithi059 ceph-mon[98847]: from='client.15432 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T17:46:20.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:20 smithi059 ceph-mon[98847]: pgmap v3: 129 pgs: 129 active+clean; 1.0 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T17:46:20.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:20 smithi059 ceph-mon[98847]: from='client.15474 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T17:46:21.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:21 smithi115 ceph-mon[102748]: from='client.25204 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T17:46:21.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:21 smithi115 ceph-mon[102748]: pgmap v4: 129 pgs: 129 active+clean; 1.0 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T17:46:21.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:21 smithi115 ceph-mon[102748]: from='client.15486 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T17:46:21.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:21 smithi115 ceph-mon[102748]: mgrmap e22: x(active, since 3s), standbys: z, y 2023-12-11T17:46:21.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:21 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:46:21.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:21 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:46:21.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:21 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:46:21.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:21 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:46:21.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:21 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:46:21.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:21 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:46:21.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:21 smithi181 ceph-mon[103627]: from='client.25204 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T17:46:21.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:21 smithi181 ceph-mon[103627]: pgmap v4: 129 pgs: 129 active+clean; 1.0 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T17:46:21.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:21 smithi181 ceph-mon[103627]: from='client.15486 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T17:46:21.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:21 smithi181 ceph-mon[103627]: mgrmap e22: x(active, since 3s), standbys: z, y 2023-12-11T17:46:21.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:21 smithi181 ceph-mon[103627]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:46:21.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:21 smithi181 ceph-mon[103627]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:46:21.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:21 smithi181 ceph-mon[103627]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:46:21.896 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:21 smithi181 ceph-mon[103627]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:46:21.896 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:21 smithi181 ceph-mon[103627]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:46:21.896 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:21 smithi181 ceph-mon[103627]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:46:21.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:21 smithi059 ceph-mon[98847]: from='client.25204 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T17:46:21.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:21 smithi059 ceph-mon[98847]: pgmap v4: 129 pgs: 129 active+clean; 1.0 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T17:46:21.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:21 smithi059 ceph-mon[98847]: from='client.15486 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T17:46:21.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:21 smithi059 ceph-mon[98847]: mgrmap e22: x(active, since 3s), standbys: z, y 2023-12-11T17:46:21.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:21 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:46:21.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:21 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:46:21.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:21 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:46:21.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:21 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:46:21.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:21 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:46:21.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:21 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:46:23.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:22 smithi115 ceph-mon[102748]: pgmap v5: 129 pgs: 129 active+clean; 1.0 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T17:46:23.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:22 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:46:23.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:22 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:46:23.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:22 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:46:23.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:22 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:46:23.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:22 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:46:23.284 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:22 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:46:23.284 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:22 smithi115 ceph-mon[102748]: Adjusting osd_memory_target on smithi181 to 196.2M 2023-12-11T17:46:23.284 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:22 smithi115 ceph-mon[102748]: Unable to set osd_memory_target on smithi181 to 205735936: error parsing value: Value '205735936' is below minimum 939524096 2023-12-11T17:46:23.284 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:22 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:46:23.284 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:22 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:46:23.284 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:22 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:46:23.284 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:22 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:46:23.284 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:22 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:46:23.284 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:22 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:46:23.284 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:22 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:46:23.284 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:22 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:46:23.284 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:22 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:46:23.284 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:22 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:46:23.284 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:22 smithi115 ceph-mon[102748]: Adjusting osd_memory_target on smithi059 to 197.2M 2023-12-11T17:46:23.285 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:22 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:46:23.285 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:22 smithi115 ceph-mon[102748]: Unable to set osd_memory_target on smithi059 to 206875648: error parsing value: Value '206875648' is below minimum 939524096 2023-12-11T17:46:23.285 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:22 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:46:23.285 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:22 smithi115 ceph-mon[102748]: Adjusting osd_memory_target on smithi115 to 196.2M 2023-12-11T17:46:23.285 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:22 smithi115 ceph-mon[102748]: Unable to set osd_memory_target on smithi115 to 205751705: error parsing value: Value '205751705' is below minimum 939524096 2023-12-11T17:46:23.285 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:22 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:46:23.285 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:22 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:46:23.285 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:22 smithi115 ceph-mon[102748]: Updating smithi059:/etc/ceph/ceph.conf 2023-12-11T17:46:23.285 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:22 smithi115 ceph-mon[102748]: Updating smithi115:/etc/ceph/ceph.conf 2023-12-11T17:46:23.285 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:22 smithi115 ceph-mon[102748]: Updating smithi181:/etc/ceph/ceph.conf 2023-12-11T17:46:23.285 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:22 smithi115 ceph-mon[102748]: Updating smithi059:/var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/config/ceph.conf 2023-12-11T17:46:23.285 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:22 smithi115 ceph-mon[102748]: Updating smithi181:/var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/config/ceph.conf 2023-12-11T17:46:23.285 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:22 smithi115 ceph-mon[102748]: Updating smithi115:/var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/config/ceph.conf 2023-12-11T17:46:23.285 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:22 smithi115 ceph-mon[102748]: mgrmap e23: x(active, since 4s), standbys: z, y 2023-12-11T17:46:23.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:22 smithi181 ceph-mon[103627]: pgmap v5: 129 pgs: 129 active+clean; 1.0 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T17:46:23.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:22 smithi181 ceph-mon[103627]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:46:23.396 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:22 smithi181 ceph-mon[103627]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:46:23.396 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:22 smithi181 ceph-mon[103627]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:46:23.396 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:22 smithi181 ceph-mon[103627]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:46:23.396 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:22 smithi181 ceph-mon[103627]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:46:23.396 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:22 smithi181 ceph-mon[103627]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:46:23.396 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:22 smithi181 ceph-mon[103627]: Adjusting osd_memory_target on smithi181 to 196.2M 2023-12-11T17:46:23.396 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:22 smithi181 ceph-mon[103627]: Unable to set osd_memory_target on smithi181 to 205735936: error parsing value: Value '205735936' is below minimum 939524096 2023-12-11T17:46:23.396 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:22 smithi181 ceph-mon[103627]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:46:23.396 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:22 smithi181 ceph-mon[103627]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:46:23.396 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:22 smithi181 ceph-mon[103627]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:46:23.396 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:22 smithi181 ceph-mon[103627]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:46:23.396 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:22 smithi181 ceph-mon[103627]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:46:23.396 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:22 smithi181 ceph-mon[103627]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:46:23.396 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:22 smithi181 ceph-mon[103627]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:46:23.397 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:22 smithi181 ceph-mon[103627]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:46:23.397 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:22 smithi181 ceph-mon[103627]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:46:23.397 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:22 smithi181 ceph-mon[103627]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:46:23.397 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:22 smithi181 ceph-mon[103627]: Adjusting osd_memory_target on smithi059 to 197.2M 2023-12-11T17:46:23.397 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:22 smithi181 ceph-mon[103627]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:46:23.397 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:22 smithi181 ceph-mon[103627]: Unable to set osd_memory_target on smithi059 to 206875648: error parsing value: Value '206875648' is below minimum 939524096 2023-12-11T17:46:23.397 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:22 smithi181 ceph-mon[103627]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:46:23.397 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:22 smithi181 ceph-mon[103627]: Adjusting osd_memory_target on smithi115 to 196.2M 2023-12-11T17:46:23.397 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:22 smithi181 ceph-mon[103627]: Unable to set osd_memory_target on smithi115 to 205751705: error parsing value: Value '205751705' is below minimum 939524096 2023-12-11T17:46:23.397 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:22 smithi181 ceph-mon[103627]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:46:23.397 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:22 smithi181 ceph-mon[103627]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:46:23.397 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:22 smithi181 ceph-mon[103627]: Updating smithi059:/etc/ceph/ceph.conf 2023-12-11T17:46:23.397 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:22 smithi181 ceph-mon[103627]: Updating smithi115:/etc/ceph/ceph.conf 2023-12-11T17:46:23.397 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:22 smithi181 ceph-mon[103627]: Updating smithi181:/etc/ceph/ceph.conf 2023-12-11T17:46:23.398 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:22 smithi181 ceph-mon[103627]: Updating smithi059:/var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/config/ceph.conf 2023-12-11T17:46:23.398 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:22 smithi181 ceph-mon[103627]: Updating smithi181:/var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/config/ceph.conf 2023-12-11T17:46:23.398 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:22 smithi181 ceph-mon[103627]: Updating smithi115:/var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/config/ceph.conf 2023-12-11T17:46:23.398 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:22 smithi181 ceph-mon[103627]: mgrmap e23: x(active, since 4s), standbys: z, y 2023-12-11T17:46:23.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:22 smithi059 ceph-mon[98847]: pgmap v5: 129 pgs: 129 active+clean; 1.0 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T17:46:23.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:22 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:46:23.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:22 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:46:23.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:22 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:46:23.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:22 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:46:23.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:22 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:46:23.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:22 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:46:23.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:22 smithi059 ceph-mon[98847]: Adjusting osd_memory_target on smithi181 to 196.2M 2023-12-11T17:46:23.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:22 smithi059 ceph-mon[98847]: Unable to set osd_memory_target on smithi181 to 205735936: error parsing value: Value '205735936' is below minimum 939524096 2023-12-11T17:46:23.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:22 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:46:23.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:22 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:46:23.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:22 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:46:23.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:22 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:46:23.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:22 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:46:23.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:22 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:46:23.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:22 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:46:23.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:22 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:46:23.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:22 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:46:23.406 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:22 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:46:23.406 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:22 smithi059 ceph-mon[98847]: Adjusting osd_memory_target on smithi059 to 197.2M 2023-12-11T17:46:23.406 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:22 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:46:23.406 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:22 smithi059 ceph-mon[98847]: Unable to set osd_memory_target on smithi059 to 206875648: error parsing value: Value '206875648' is below minimum 939524096 2023-12-11T17:46:23.406 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:22 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:46:23.406 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:22 smithi059 ceph-mon[98847]: Adjusting osd_memory_target on smithi115 to 196.2M 2023-12-11T17:46:23.406 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:22 smithi059 ceph-mon[98847]: Unable to set osd_memory_target on smithi115 to 205751705: error parsing value: Value '205751705' is below minimum 939524096 2023-12-11T17:46:23.406 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:22 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:46:23.406 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:22 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:46:23.406 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:22 smithi059 ceph-mon[98847]: Updating smithi059:/etc/ceph/ceph.conf 2023-12-11T17:46:23.406 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:22 smithi059 ceph-mon[98847]: Updating smithi115:/etc/ceph/ceph.conf 2023-12-11T17:46:23.406 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:22 smithi059 ceph-mon[98847]: Updating smithi181:/etc/ceph/ceph.conf 2023-12-11T17:46:23.406 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:22 smithi059 ceph-mon[98847]: Updating smithi059:/var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/config/ceph.conf 2023-12-11T17:46:23.406 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:22 smithi059 ceph-mon[98847]: Updating smithi181:/var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/config/ceph.conf 2023-12-11T17:46:23.406 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:22 smithi059 ceph-mon[98847]: Updating smithi115:/var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/config/ceph.conf 2023-12-11T17:46:23.407 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:22 smithi059 ceph-mon[98847]: mgrmap e23: x(active, since 4s), standbys: z, y 2023-12-11T17:46:24.645 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:24 smithi181 ceph-mon[103627]: Updating smithi059:/etc/ceph/ceph.client.admin.keyring 2023-12-11T17:46:24.645 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:24 smithi181 ceph-mon[103627]: Updating smithi181:/etc/ceph/ceph.client.admin.keyring 2023-12-11T17:46:24.645 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:24 smithi181 ceph-mon[103627]: Updating smithi115:/etc/ceph/ceph.client.admin.keyring 2023-12-11T17:46:24.645 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:24 smithi181 ceph-mon[103627]: Updating smithi059:/var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/config/ceph.client.admin.keyring 2023-12-11T17:46:24.645 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:24 smithi181 ceph-mon[103627]: Updating smithi181:/var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/config/ceph.client.admin.keyring 2023-12-11T17:46:24.646 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:24 smithi181 ceph-mon[103627]: Updating smithi115:/var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/config/ceph.client.admin.keyring 2023-12-11T17:46:24.646 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:24 smithi181 ceph-mon[103627]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:46:24.646 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:24 smithi181 ceph-mon[103627]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:46:24.646 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:24 smithi181 ceph-mon[103627]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:46:24.646 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:24 smithi181 ceph-mon[103627]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:46:24.646 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:24 smithi181 ceph-mon[103627]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:46:24.646 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:24 smithi181 ceph-mon[103627]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:46:24.646 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:24 smithi181 ceph-mon[103627]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:46:24.646 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:24 smithi181 ceph-mon[103627]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:46:24.646 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:24 smithi181 ceph-mon[103627]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:46:24.646 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:24 smithi181 ceph-mon[103627]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:46:24.646 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:24 smithi181 ceph-mon[103627]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:46:24.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:24 smithi059 ceph-mon[98847]: Updating smithi059:/etc/ceph/ceph.client.admin.keyring 2023-12-11T17:46:24.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:24 smithi059 ceph-mon[98847]: Updating smithi181:/etc/ceph/ceph.client.admin.keyring 2023-12-11T17:46:24.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:24 smithi059 ceph-mon[98847]: Updating smithi115:/etc/ceph/ceph.client.admin.keyring 2023-12-11T17:46:24.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:24 smithi059 ceph-mon[98847]: Updating smithi059:/var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/config/ceph.client.admin.keyring 2023-12-11T17:46:24.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:24 smithi059 ceph-mon[98847]: Updating smithi181:/var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/config/ceph.client.admin.keyring 2023-12-11T17:46:24.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:24 smithi059 ceph-mon[98847]: Updating smithi115:/var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/config/ceph.client.admin.keyring 2023-12-11T17:46:24.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:24 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:46:24.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:24 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:46:24.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:24 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:46:24.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:24 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:46:24.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:24 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:46:24.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:24 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:46:24.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:24 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:46:24.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:24 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:46:24.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:24 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:46:24.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:24 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:46:24.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:24 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:46:24.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:24 smithi115 ceph-mon[102748]: Updating smithi059:/etc/ceph/ceph.client.admin.keyring 2023-12-11T17:46:24.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:24 smithi115 ceph-mon[102748]: Updating smithi181:/etc/ceph/ceph.client.admin.keyring 2023-12-11T17:46:24.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:24 smithi115 ceph-mon[102748]: Updating smithi115:/etc/ceph/ceph.client.admin.keyring 2023-12-11T17:46:24.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:24 smithi115 ceph-mon[102748]: Updating smithi059:/var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/config/ceph.client.admin.keyring 2023-12-11T17:46:24.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:24 smithi115 ceph-mon[102748]: Updating smithi181:/var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/config/ceph.client.admin.keyring 2023-12-11T17:46:24.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:24 smithi115 ceph-mon[102748]: Updating smithi115:/var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/config/ceph.client.admin.keyring 2023-12-11T17:46:24.784 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:24 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:46:24.784 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:24 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:46:24.784 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:24 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:46:24.784 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:24 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:46:24.784 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:24 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:46:24.784 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:24 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:46:24.784 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:24 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:46:24.784 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:24 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:46:24.784 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:24 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:46:24.784 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:24 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:46:24.784 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:24 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:46:25.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:25 smithi115 ceph-mon[102748]: pgmap v6: 129 pgs: 129 active+clean; 1.0 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 205 B/s wr, 26 op/s 2023-12-11T17:46:25.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:25 smithi181 ceph-mon[103627]: pgmap v6: 129 pgs: 129 active+clean; 1.0 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 205 B/s wr, 26 op/s 2023-12-11T17:46:25.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:25 smithi059 ceph-mon[98847]: pgmap v6: 129 pgs: 129 active+clean; 1.0 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 205 B/s wr, 26 op/s 2023-12-11T17:46:27.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:27 smithi115 ceph-mon[102748]: pgmap v7: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 132 KiB/s wr, 68 op/s 2023-12-11T17:46:27.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:27 smithi181 ceph-mon[103627]: pgmap v7: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 132 KiB/s wr, 68 op/s 2023-12-11T17:46:27.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:27 smithi059 ceph-mon[98847]: pgmap v7: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 132 KiB/s wr, 68 op/s 2023-12-11T17:46:29.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:29 smithi115 ceph-mon[102748]: pgmap v8: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 103 KiB/s wr, 53 op/s 2023-12-11T17:46:29.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:29 smithi181 ceph-mon[103627]: pgmap v8: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 103 KiB/s wr, 53 op/s 2023-12-11T17:46:29.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:29 smithi059 ceph-mon[98847]: pgmap v8: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 103 KiB/s wr, 53 op/s 2023-12-11T17:46:31.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:31 smithi115 ceph-mon[102748]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 84 KiB/s wr, 46 op/s 2023-12-11T17:46:31.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:31 smithi181 ceph-mon[103627]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 84 KiB/s wr, 46 op/s 2023-12-11T17:46:31.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:31 smithi059 ceph-mon[98847]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 84 KiB/s wr, 46 op/s 2023-12-11T17:46:33.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:33 smithi115 ceph-mon[102748]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 77 KiB/s wr, 42 op/s 2023-12-11T17:46:33.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:33 smithi181 ceph-mon[103627]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 77 KiB/s wr, 42 op/s 2023-12-11T17:46:33.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:33 smithi059 ceph-mon[98847]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 77 KiB/s wr, 42 op/s 2023-12-11T17:46:35.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:35 smithi115 ceph-mon[102748]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 77 KiB/s wr, 44 op/s 2023-12-11T17:46:35.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:35 smithi181 ceph-mon[103627]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 77 KiB/s wr, 44 op/s 2023-12-11T17:46:35.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:35 smithi059 ceph-mon[98847]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 77 KiB/s wr, 44 op/s 2023-12-11T17:46:37.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:37 smithi115 ceph-mon[102748]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 77 KiB/s wr, 33 op/s 2023-12-11T17:46:37.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:37 smithi181 ceph-mon[103627]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 77 KiB/s wr, 33 op/s 2023-12-11T17:46:37.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:37 smithi059 ceph-mon[98847]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 77 KiB/s wr, 33 op/s 2023-12-11T17:46:39.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:39 smithi115 ceph-mon[102748]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2023-12-11T17:46:39.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:39 smithi181 ceph-mon[103627]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2023-12-11T17:46:39.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:39 smithi059 ceph-mon[98847]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2023-12-11T17:46:41.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:41 smithi115 ceph-mon[102748]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 85 B/s wr, 7 op/s 2023-12-11T17:46:41.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:41 smithi181 ceph-mon[103627]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 85 B/s wr, 7 op/s 2023-12-11T17:46:41.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:41 smithi059 ceph-mon[98847]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 85 B/s wr, 7 op/s 2023-12-11T17:46:43.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:43 smithi115 ceph-mon[102748]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-11T17:46:43.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:43 smithi181 ceph-mon[103627]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-11T17:46:43.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:43 smithi059 ceph-mon[98847]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-11T17:46:45.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:45 smithi115 ceph-mon[102748]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-12-11T17:46:45.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:45 smithi181 ceph-mon[103627]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-12-11T17:46:45.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:45 smithi059 ceph-mon[98847]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-12-11T17:46:47.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:47 smithi115 ceph-mon[102748]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-12-11T17:46:47.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:47 smithi181 ceph-mon[103627]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-12-11T17:46:47.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:47 smithi059 ceph-mon[98847]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-12-11T17:46:49.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:49 smithi115 ceph-mon[102748]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-11T17:46:49.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:49 smithi181 ceph-mon[103627]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-11T17:46:49.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:49 smithi059 ceph-mon[98847]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-11T17:46:51.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:51 smithi115 ceph-mon[102748]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-12-11T17:46:51.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:51 smithi181 ceph-mon[103627]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-12-11T17:46:51.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:51 smithi059 ceph-mon[98847]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-12-11T17:46:53.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:53 smithi181 ceph-mon[103627]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-11T17:46:53.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:53 smithi059 ceph-mon[98847]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-11T17:46:54.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:53 smithi115 ceph-mon[102748]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-11T17:46:55.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:55 smithi181 ceph-mon[103627]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 597 B/s wr, 7 op/s 2023-12-11T17:46:55.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:55 smithi059 ceph-mon[98847]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 597 B/s wr, 7 op/s 2023-12-11T17:46:56.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:55 smithi115 ceph-mon[102748]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 597 B/s wr, 7 op/s 2023-12-11T17:46:57.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:57 smithi181 ceph-mon[103627]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 597 B/s wr, 5 op/s 2023-12-11T17:46:57.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:57 smithi059 ceph-mon[98847]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 597 B/s wr, 5 op/s 2023-12-11T17:46:58.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:57 smithi115 ceph-mon[102748]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 597 B/s wr, 5 op/s 2023-12-11T17:46:59.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:46:59 smithi181 ceph-mon[103627]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 597 B/s wr, 5 op/s 2023-12-11T17:46:59.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:46:59 smithi059 ceph-mon[98847]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 597 B/s wr, 5 op/s 2023-12-11T17:47:00.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:46:59 smithi115 ceph-mon[102748]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 597 B/s wr, 5 op/s 2023-12-11T17:47:00.654 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:47:00 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T17:47:00.304+0000 7f45f3870700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-11T17:47:00.654 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:47:00 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T17:47:00.304+0000 7f45f3870700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-11T17:47:00.654 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:47:00 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T17:47:00.304+0000 7f45f3870700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-11T17:47:00.654 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:47:00 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T17:47:00.304+0000 7f45f3870700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-11T17:47:00.655 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:47:00 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T17:47:00.304+0000 7f45f3870700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-11T17:47:01.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:47:01 smithi181 ceph-mon[103627]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 682 B/s wr, 7 op/s 2023-12-11T17:47:01.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:47:01 smithi059 ceph-mon[98847]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 682 B/s wr, 7 op/s 2023-12-11T17:47:02.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:47:01 smithi115 ceph-mon[102748]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 682 B/s wr, 7 op/s 2023-12-11T17:47:03.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:47:03 smithi181 ceph-mon[103627]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 5 op/s 2023-12-11T17:47:03.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:47:03 smithi181 ceph-mon[103627]: mgrmap e24: x(active, since 44s), standbys: z, y 2023-12-11T17:47:03.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:47:03 smithi059 ceph-mon[98847]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 5 op/s 2023-12-11T17:47:03.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:47:03 smithi059 ceph-mon[98847]: mgrmap e24: x(active, since 44s), standbys: z, y 2023-12-11T17:47:04.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:47:03 smithi115 ceph-mon[102748]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 5 op/s 2023-12-11T17:47:04.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:47:03 smithi115 ceph-mon[102748]: mgrmap e24: x(active, since 44s), standbys: z, y 2023-12-11T17:47:05.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:47:05 smithi181 ceph-mon[103627]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2023-12-11T17:47:05.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:47:05 smithi059 ceph-mon[98847]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2023-12-11T17:47:06.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:47:05 smithi115 ceph-mon[102748]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2023-12-11T17:47:07.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:47:07 smithi181 ceph-mon[103627]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2023-12-11T17:47:07.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:47:07 smithi059 ceph-mon[98847]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2023-12-11T17:47:08.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:47:07 smithi115 ceph-mon[102748]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2023-12-11T17:47:09.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:47:09 smithi181 ceph-mon[103627]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2023-12-11T17:47:09.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:47:09 smithi059 ceph-mon[98847]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2023-12-11T17:47:10.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:47:09 smithi115 ceph-mon[102748]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2023-12-11T17:47:11.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:47:11 smithi181 ceph-mon[103627]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2023-12-11T17:47:11.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:47:11 smithi059 ceph-mon[98847]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2023-12-11T17:47:12.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:47:11 smithi115 ceph-mon[102748]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2023-12-11T17:47:13.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:47:13 smithi181 ceph-mon[103627]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 17 op/s 2023-12-11T17:47:13.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:47:13 smithi059 ceph-mon[98847]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 17 op/s 2023-12-11T17:47:14.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:47:13 smithi115 ceph-mon[102748]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 17 op/s 2023-12-11T17:47:15.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:47:15 smithi181 ceph-mon[103627]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2023-12-11T17:47:15.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:47:15 smithi059 ceph-mon[98847]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2023-12-11T17:47:16.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:47:15 smithi115 ceph-mon[102748]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2023-12-11T17:47:17.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:47:17 smithi181 ceph-mon[103627]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2023-12-11T17:47:17.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:47:17 smithi059 ceph-mon[98847]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2023-12-11T17:47:18.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:47:17 smithi115 ceph-mon[102748]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2023-12-11T17:47:18.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:47:18 smithi181 ceph-mon[103627]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [1, 9]}]: dispatch 2023-12-11T17:47:18.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:47:18 smithi181 ceph-mon[103627]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 8]}]: dispatch 2023-12-11T17:47:18.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:47:18 smithi181 ceph-mon[103627]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]: dispatch 2023-12-11T17:47:18.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:47:18 smithi181 ceph-mon[103627]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]: dispatch 2023-12-11T17:47:18.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:47:18 smithi181 ceph-mon[103627]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]: dispatch 2023-12-11T17:47:18.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:47:18 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [1, 9]}]: dispatch 2023-12-11T17:47:18.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:47:18 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 8]}]: dispatch 2023-12-11T17:47:18.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:47:18 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]: dispatch 2023-12-11T17:47:18.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:47:18 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]: dispatch 2023-12-11T17:47:18.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:47:18 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]: dispatch 2023-12-11T17:47:19.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:47:18 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [1, 9]}]: dispatch 2023-12-11T17:47:19.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:47:18 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 8]}]: dispatch 2023-12-11T17:47:19.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:47:18 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]: dispatch 2023-12-11T17:47:19.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:47:18 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]: dispatch 2023-12-11T17:47:19.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:47:18 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]: dispatch 2023-12-11T17:47:19.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:47:19 smithi181 ceph-mon[103627]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2023-12-11T17:47:19.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:47:19 smithi181 ceph-mon[103627]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [1, 9]}]': finished 2023-12-11T17:47:19.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:47:19 smithi181 ceph-mon[103627]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 8]}]': finished 2023-12-11T17:47:19.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:47:19 smithi181 ceph-mon[103627]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]': finished 2023-12-11T17:47:19.896 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:47:19 smithi181 ceph-mon[103627]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]': finished 2023-12-11T17:47:19.896 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:47:19 smithi181 ceph-mon[103627]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]': finished 2023-12-11T17:47:19.896 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:47:19 smithi181 ceph-mon[103627]: osdmap e90: 12 total, 12 up, 12 in 2023-12-11T17:47:19.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:47:19 smithi059 ceph-mon[98847]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2023-12-11T17:47:19.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:47:19 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [1, 9]}]': finished 2023-12-11T17:47:19.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:47:19 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 8]}]': finished 2023-12-11T17:47:19.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:47:19 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]': finished 2023-12-11T17:47:19.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:47:19 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]': finished 2023-12-11T17:47:19.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:47:19 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]': finished 2023-12-11T17:47:19.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:47:19 smithi059 ceph-mon[98847]: osdmap e90: 12 total, 12 up, 12 in 2023-12-11T17:47:20.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:47:19 smithi115 ceph-mon[102748]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2023-12-11T17:47:20.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:47:19 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [1, 9]}]': finished 2023-12-11T17:47:20.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:47:19 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 8]}]': finished 2023-12-11T17:47:20.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:47:19 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]': finished 2023-12-11T17:47:20.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:47:19 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]': finished 2023-12-11T17:47:20.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:47:19 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]': finished 2023-12-11T17:47:20.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:47:19 smithi115 ceph-mon[102748]: osdmap e90: 12 total, 12 up, 12 in 2023-12-11T17:47:20.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:47:20 smithi181 ceph-mon[103627]: osdmap e91: 12 total, 12 up, 12 in 2023-12-11T17:47:20.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:47:20 smithi059 ceph-mon[98847]: osdmap e91: 12 total, 12 up, 12 in 2023-12-11T17:47:21.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:47:20 smithi115 ceph-mon[102748]: osdmap e91: 12 total, 12 up, 12 in 2023-12-11T17:47:21.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:47:21 smithi181 ceph-mon[103627]: pgmap v36: 129 pgs: 5 peering, 124 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 7 op/s 2023-12-11T17:47:21.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:47:21 smithi059 ceph-mon[98847]: pgmap v36: 129 pgs: 5 peering, 124 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 7 op/s 2023-12-11T17:47:22.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:47:21 smithi115 ceph-mon[102748]: pgmap v36: 129 pgs: 5 peering, 124 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 7 op/s 2023-12-11T17:47:23.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:47:23 smithi181 ceph-mon[103627]: pgmap v37: 129 pgs: 5 peering, 124 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 3 op/s 2023-12-11T17:47:23.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:47:23 smithi059 ceph-mon[98847]: pgmap v37: 129 pgs: 5 peering, 124 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 3 op/s 2023-12-11T17:47:24.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:47:23 smithi115 ceph-mon[102748]: pgmap v37: 129 pgs: 5 peering, 124 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 3 op/s 2023-12-11T17:47:24.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:47:24 smithi181 ceph-mon[103627]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:47:24.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:47:24 smithi181 ceph-mon[103627]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:47:24.896 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:47:24 smithi181 ceph-mon[103627]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:47:24.896 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:47:24 smithi181 ceph-mon[103627]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:47:24.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:47:24 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:47:24.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:47:24 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:47:24.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:47:24 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:47:24.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:47:24 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:47:25.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:47:24 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:47:25.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:47:24 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:47:25.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:47:24 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:47:25.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:47:24 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:47:25.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:47:25 smithi181 ceph-mon[103627]: pgmap v38: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-12-11T17:47:25.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:47:25 smithi059 ceph-mon[98847]: pgmap v38: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-12-11T17:47:26.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:47:25 smithi115 ceph-mon[102748]: pgmap v38: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-12-11T17:47:27.206 INFO:tasks.workunit.client.0.smithi059.stderr:Updating files: 94% (11475/12157) Updating files: 95% (11550/12157) Updating files: 96% (11671/12157) Updating files: 97% (11793/12157) Updating files: 98% (11914/12157) Updating files: 99% (12036/12157) Updating files: 100% (12157/12157) Updating files: 100% (12157/12157), done. 2023-12-11T17:47:27.411 INFO:tasks.workunit.client.0.smithi059.stderr:Note: switching to 'a62fda73efba5520d409bb5a21bd8373289dda5d'. 2023-12-11T17:47:27.412 INFO:tasks.workunit.client.0.smithi059.stderr: 2023-12-11T17:47:27.412 INFO:tasks.workunit.client.0.smithi059.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2023-12-11T17:47:27.412 INFO:tasks.workunit.client.0.smithi059.stderr:changes and commit them, and you can discard any commits you make in this 2023-12-11T17:47:27.412 INFO:tasks.workunit.client.0.smithi059.stderr:state without impacting any branches by switching back to a branch. 2023-12-11T17:47:27.412 INFO:tasks.workunit.client.0.smithi059.stderr: 2023-12-11T17:47:27.412 INFO:tasks.workunit.client.0.smithi059.stderr:If you want to create a new branch to retain commits you create, you may 2023-12-11T17:47:27.412 INFO:tasks.workunit.client.0.smithi059.stderr:do so (now or later) by using -c with the switch command. Example: 2023-12-11T17:47:27.412 INFO:tasks.workunit.client.0.smithi059.stderr: 2023-12-11T17:47:27.412 INFO:tasks.workunit.client.0.smithi059.stderr: git switch -c 2023-12-11T17:47:27.412 INFO:tasks.workunit.client.0.smithi059.stderr: 2023-12-11T17:47:27.412 INFO:tasks.workunit.client.0.smithi059.stderr:Or undo this operation with: 2023-12-11T17:47:27.412 INFO:tasks.workunit.client.0.smithi059.stderr: 2023-12-11T17:47:27.412 INFO:tasks.workunit.client.0.smithi059.stderr: git switch - 2023-12-11T17:47:27.413 INFO:tasks.workunit.client.0.smithi059.stderr: 2023-12-11T17:47:27.413 INFO:tasks.workunit.client.0.smithi059.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2023-12-11T17:47:27.413 INFO:tasks.workunit.client.0.smithi059.stderr: 2023-12-11T17:47:27.413 INFO:tasks.workunit.client.0.smithi059.stderr:HEAD is now at a62fda73efb Merge remote-tracking branch 'milind/mds-enqueue-all-child-frags-for-scrub-for-a-given-fragset' into rishabh-2023dec11 2023-12-11T17:47:27.414 DEBUG:teuthology.orchestra.run.smithi059:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2023-12-11T17:47:27.545 INFO:tasks.workunit.client.0.smithi059.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2023-12-11T17:47:27.547 INFO:tasks.workunit.client.0.smithi059.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2023-12-11T17:47:27.547 INFO:tasks.workunit.client.0.smithi059.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2023-12-11T17:47:27.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:47:27 smithi181 ceph-mon[103627]: pgmap v39: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-12-11T17:47:27.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:47:27 smithi059 ceph-mon[98847]: pgmap v39: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-12-11T17:47:28.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:47:27 smithi115 ceph-mon[102748]: pgmap v39: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-12-11T17:47:28.061 INFO:tasks.workunit.client.0.smithi059.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2023-12-11T17:47:28.124 INFO:tasks.workunit.client.0.smithi059.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2023-12-11T17:47:28.164 INFO:tasks.workunit.client.0.smithi059.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2023-12-11T17:47:28.165 INFO:tasks.workunit.client.0.smithi059.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2023-12-11T17:47:28.165 INFO:tasks.workunit.client.0.smithi059.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2023-12-11T17:47:28.203 INFO:tasks.workunit.client.0.smithi059.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2023-12-11T17:47:28.206 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2023-12-11T17:47:28.206 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2023-12-11T17:47:28.224 INFO:tasks.workunit:Running workunits matching suites/blogbench.sh on client.0... 2023-12-11T17:47:28.225 INFO:tasks.workunit:Running workunit suites/blogbench.sh... 2023-12-11T17:47:28.225 DEBUG:teuthology.orchestra.run.smithi059:workunit test suites/blogbench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a62fda73efba5520d409bb5a21bd8373289dda5d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/blogbench.sh 2023-12-11T17:47:28.287 INFO:tasks.workunit.client.0.smithi059.stdout:getting blogbench 2023-12-11T17:47:28.287 INFO:tasks.workunit.client.0.smithi059.stderr:+ echo 'getting blogbench' 2023-12-11T17:47:28.287 INFO:tasks.workunit.client.0.smithi059.stderr:+ wget http://download.ceph.com/qa/blogbench-1.0.tar.bz2 2023-12-11T17:47:28.311 INFO:tasks.workunit.client.0.smithi059.stderr:--2023-12-11 17:47:28-- http://download.ceph.com/qa/blogbench-1.0.tar.bz2 2023-12-11T17:47:28.312 INFO:tasks.workunit.client.0.smithi059.stderr:Resolving download.ceph.com (download.ceph.com)... 158.69.68.124, 2607:5300:201:2000::3:58a1 2023-12-11T17:47:28.333 INFO:tasks.workunit.client.0.smithi059.stderr:Connecting to download.ceph.com (download.ceph.com)|158.69.68.124|:80... connected. 2023-12-11T17:47:28.354 INFO:tasks.workunit.client.0.smithi059.stderr:HTTP request sent, awaiting response... 200 OK 2023-12-11T17:47:28.354 INFO:tasks.workunit.client.0.smithi059.stderr:Length: 103355 (101K) [application/octet-stream] 2023-12-11T17:47:28.355 INFO:tasks.workunit.client.0.smithi059.stderr:Saving to: ‘blogbench-1.0.tar.bz2’ 2023-12-11T17:47:28.358 INFO:tasks.workunit.client.0.smithi059.stderr: 2023-12-11T17:47:28.395 INFO:tasks.workunit.client.0.smithi059.stderr: 0K .......... .......... .......... .......... .......... 49% 1.23M 0s 2023-12-11T17:47:28.416 INFO:tasks.workunit.client.0.smithi059.stderr: 50K .......... .......... .......... .......... .......... 99% 2.40M 0s 2023-12-11T17:47:28.416 INFO:tasks.workunit.client.0.smithi059.stderr: 100K 100% 1779G=0.06s 2023-12-11T17:47:28.416 INFO:tasks.workunit.client.0.smithi059.stderr: 2023-12-11T17:47:28.416 INFO:tasks.workunit.client.0.smithi059.stderr:2023-12-11 17:47:28 (1.64 MB/s) - ‘blogbench-1.0.tar.bz2’ saved [103355/103355] 2023-12-11T17:47:28.416 INFO:tasks.workunit.client.0.smithi059.stderr: 2023-12-11T17:47:28.416 INFO:tasks.workunit.client.0.smithi059.stderr:+ tar -xvf blogbench-1.0.tar.bz2 2023-12-11T17:47:28.450 INFO:tasks.workunit.client.0.smithi059.stdout:blogbench-1.0/ 2023-12-11T17:47:28.451 INFO:tasks.workunit.client.0.smithi059.stdout:blogbench-1.0/m4/ 2023-12-11T17:47:28.452 INFO:tasks.workunit.client.0.smithi059.stdout:blogbench-1.0/m4/acx_pthread.m4 2023-12-11T17:47:28.453 INFO:tasks.workunit.client.0.smithi059.stdout:blogbench-1.0/man/ 2023-12-11T17:47:28.454 INFO:tasks.workunit.client.0.smithi059.stdout:blogbench-1.0/man/blogbench.8 2023-12-11T17:47:28.455 INFO:tasks.workunit.client.0.smithi059.stdout:blogbench-1.0/man/Makefile.am 2023-12-11T17:47:28.456 INFO:tasks.workunit.client.0.smithi059.stdout:blogbench-1.0/man/Makefile.in 2023-12-11T17:47:28.457 INFO:tasks.workunit.client.0.smithi059.stdout:blogbench-1.0/src/ 2023-12-11T17:47:28.458 INFO:tasks.workunit.client.0.smithi059.stdout:blogbench-1.0/src/mysnprintf.c 2023-12-11T17:47:28.462 INFO:tasks.workunit.client.0.smithi059.stdout:blogbench-1.0/src/mysnprintf.h 2023-12-11T17:47:28.462 INFO:tasks.workunit.client.0.smithi059.stdout:blogbench-1.0/src/blogbench.c 2023-12-11T17:47:28.462 INFO:tasks.workunit.client.0.smithi059.stdout:blogbench-1.0/src/blogbench.h 2023-12-11T17:47:28.462 INFO:tasks.workunit.client.0.smithi059.stdout:blogbench-1.0/src/fakesnprintf.c 2023-12-11T17:47:28.462 INFO:tasks.workunit.client.0.smithi059.stdout:blogbench-1.0/src/fakesnprintf.h 2023-12-11T17:47:28.463 INFO:tasks.workunit.client.0.smithi059.stdout:blogbench-1.0/src/blogbench_p.h 2023-12-11T17:47:28.464 INFO:tasks.workunit.client.0.smithi059.stdout:blogbench-1.0/src/bsd-getopt_long.c 2023-12-11T17:47:28.465 INFO:tasks.workunit.client.0.smithi059.stdout:blogbench-1.0/src/bsd-getopt_long.h 2023-12-11T17:47:28.466 INFO:tasks.workunit.client.0.smithi059.stdout:blogbench-1.0/src/Makefile.am 2023-12-11T17:47:28.467 INFO:tasks.workunit.client.0.smithi059.stdout:blogbench-1.0/src/Makefile.in 2023-12-11T17:47:28.468 INFO:tasks.workunit.client.0.smithi059.stdout:blogbench-1.0/src/process.c 2023-12-11T17:47:28.469 INFO:tasks.workunit.client.0.smithi059.stdout:blogbench-1.0/src/globals.h 2023-12-11T17:47:28.470 INFO:tasks.workunit.client.0.smithi059.stdout:blogbench-1.0/src/commenter.c 2023-12-11T17:47:28.470 INFO:tasks.workunit.client.0.smithi059.stdout:blogbench-1.0/src/rewriter.c 2023-12-11T17:47:28.471 INFO:tasks.workunit.client.0.smithi059.stdout:blogbench-1.0/src/reader.c 2023-12-11T17:47:28.472 INFO:tasks.workunit.client.0.smithi059.stdout:blogbench-1.0/src/writer.c 2023-12-11T17:47:28.473 INFO:tasks.workunit.client.0.smithi059.stdout:blogbench-1.0/src/helpers.c 2023-12-11T17:47:28.473 INFO:tasks.workunit.client.0.smithi059.stdout:blogbench-1.0/NEWS 2023-12-11T17:47:28.474 INFO:tasks.workunit.client.0.smithi059.stdout:blogbench-1.0/compile 2023-12-11T17:47:28.475 INFO:tasks.workunit.client.0.smithi059.stdout:blogbench-1.0/depcomp 2023-12-11T17:47:28.476 INFO:tasks.workunit.client.0.smithi059.stdout:blogbench-1.0/aclocal.m4 2023-12-11T17:47:28.477 INFO:tasks.workunit.client.0.smithi059.stdout:blogbench-1.0/README 2023-12-11T17:47:28.477 INFO:tasks.workunit.client.0.smithi059.stdout:blogbench-1.0/configure 2023-12-11T17:47:28.481 INFO:tasks.workunit.client.0.smithi059.stdout:blogbench-1.0/configure.ac 2023-12-11T17:47:28.482 INFO:tasks.workunit.client.0.smithi059.stdout:blogbench-1.0/config.guess 2023-12-11T17:47:28.483 INFO:tasks.workunit.client.0.smithi059.stdout:blogbench-1.0/install-sh 2023-12-11T17:47:28.483 INFO:tasks.workunit.client.0.smithi059.stdout:blogbench-1.0/config.sub 2023-12-11T17:47:28.484 INFO:tasks.workunit.client.0.smithi059.stdout:blogbench-1.0/missing 2023-12-11T17:47:28.485 INFO:tasks.workunit.client.0.smithi059.stdout:blogbench-1.0/mkinstalldirs 2023-12-11T17:47:28.486 INFO:tasks.workunit.client.0.smithi059.stdout:blogbench-1.0/Makefile.am 2023-12-11T17:47:28.486 INFO:tasks.workunit.client.0.smithi059.stdout:blogbench-1.0/Makefile.in 2023-12-11T17:47:28.487 INFO:tasks.workunit.client.0.smithi059.stdout:blogbench-1.0/config.h.in 2023-12-11T17:47:28.488 INFO:tasks.workunit.client.0.smithi059.stdout:blogbench-1.0/AUTHORS 2023-12-11T17:47:28.489 INFO:tasks.workunit.client.0.smithi059.stdout:blogbench-1.0/INSTALL 2023-12-11T17:47:28.490 INFO:tasks.workunit.client.0.smithi059.stdout:blogbench-1.0/ChangeLog 2023-12-11T17:47:28.491 INFO:tasks.workunit.client.0.smithi059.stdout:blogbench-1.0/COPYING 2023-12-11T17:47:28.492 INFO:tasks.workunit.client.0.smithi059.stderr:+ cd blogbench-1.0/ 2023-12-11T17:47:28.492 INFO:tasks.workunit.client.0.smithi059.stderr:+ echo 'making blogbench' 2023-12-11T17:47:28.492 INFO:tasks.workunit.client.0.smithi059.stdout:making blogbench 2023-12-11T17:47:28.493 INFO:tasks.workunit.client.0.smithi059.stderr:+ ./configure 2023-12-11T17:47:28.595 INFO:tasks.workunit.client.0.smithi059.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2023-12-11T17:47:29.603 INFO:tasks.workunit.client.0.smithi059.stdout:checking whether build environment is sane... yes 2023-12-11T17:47:29.613 INFO:tasks.workunit.client.0.smithi059.stdout:checking for gawk... gawk 2023-12-11T17:47:29.653 INFO:tasks.workunit.client.0.smithi059.stdout:checking whether make sets $(MAKE)... yes 2023-12-11T17:47:29.663 INFO:tasks.workunit.client.0.smithi059.stdout:checking for gcc... gcc 2023-12-11T17:47:29.717 INFO:tasks.workunit.client.0.smithi059.stdout:checking for C compiler default output file name... a.out 2023-12-11T17:47:29.719 INFO:tasks.workunit.client.0.smithi059.stdout:checking whether the C compiler works... yes 2023-12-11T17:47:29.722 INFO:tasks.workunit.client.0.smithi059.stdout:checking whether we are cross compiling... no 2023-12-11T17:47:29.759 INFO:tasks.workunit.client.0.smithi059.stdout:checking for suffix of executables... 2023-12-11T17:47:29.795 INFO:tasks.workunit.client.0.smithi059.stdout:checking for suffix of object files... o 2023-12-11T17:47:29.827 INFO:tasks.workunit.client.0.smithi059.stdout:checking whether we are using the GNU C compiler... yes 2023-12-11T17:47:29.860 INFO:tasks.workunit.client.0.smithi059.stdout:checking whether gcc accepts -g... yes 2023-12-11T17:47:29.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:47:29 smithi181 ceph-mon[103627]: pgmap v40: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-12-11T17:47:29.906 INFO:tasks.workunit.client.0.smithi059.stdout:checking for gcc option to accept ANSI C... none needed 2023-12-11T17:47:29.906 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:47:29 smithi059 ceph-mon[98847]: pgmap v40: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-12-11T17:47:29.942 INFO:tasks.workunit.client.0.smithi059.stdout:checking for style of include used by make... GNU 2023-12-11T17:47:30.026 INFO:tasks.workunit.client.0.smithi059.stdout:checking dependency style of gcc... gcc3 2023-12-11T17:47:30.026 INFO:tasks.workunit.client.0.smithi059.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2023-12-11T17:47:30.032 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:47:29 smithi115 ceph-mon[102748]: pgmap v40: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-12-11T17:47:30.071 INFO:tasks.workunit.client.0.smithi059.stdout:checking how to run the C preprocessor... gcc -E 2023-12-11T17:47:30.106 INFO:tasks.workunit.client.0.smithi059.stdout:checking for egrep... grep -E 2023-12-11T17:47:30.113 INFO:tasks.workunit.client.0.smithi059.stdout:checking for AIX... no 2023-12-11T17:47:30.174 INFO:tasks.workunit.client.0.smithi059.stdout:checking for library containing strerror... none required 2023-12-11T17:47:30.295 INFO:tasks.workunit.client.0.smithi059.stdout:checking for ANSI C header files... yes 2023-12-11T17:47:30.332 INFO:tasks.workunit.client.0.smithi059.stdout:checking whether time.h and sys/time.h may both be included... yes 2023-12-11T17:47:30.377 INFO:tasks.workunit.client.0.smithi059.stdout:checking for sys/types.h... yes 2023-12-11T17:47:30.422 INFO:tasks.workunit.client.0.smithi059.stdout:checking for sys/stat.h... yes 2023-12-11T17:47:30.468 INFO:tasks.workunit.client.0.smithi059.stdout:checking for stdlib.h... yes 2023-12-11T17:47:30.514 INFO:tasks.workunit.client.0.smithi059.stdout:checking for string.h... yes 2023-12-11T17:47:30.562 INFO:tasks.workunit.client.0.smithi059.stdout:checking for memory.h... yes 2023-12-11T17:47:30.609 INFO:tasks.workunit.client.0.smithi059.stdout:checking for strings.h... yes 2023-12-11T17:47:30.657 INFO:tasks.workunit.client.0.smithi059.stdout:checking for inttypes.h... yes 2023-12-11T17:47:30.704 INFO:tasks.workunit.client.0.smithi059.stdout:checking for stdint.h... yes 2023-12-11T17:47:30.753 INFO:tasks.workunit.client.0.smithi059.stdout:checking for unistd.h... yes 2023-12-11T17:47:30.765 INFO:tasks.workunit.client.0.smithi059.stdout:checking for memory.h... (cached) yes 2023-12-11T17:47:30.770 INFO:tasks.workunit.client.0.smithi059.stdout:checking for strings.h... (cached) yes 2023-12-11T17:47:30.775 INFO:tasks.workunit.client.0.smithi059.stdout:checking for unistd.h... (cached) yes 2023-12-11T17:47:30.818 INFO:tasks.workunit.client.0.smithi059.stdout:checking locale.h usability... yes 2023-12-11T17:47:30.835 INFO:tasks.workunit.client.0.smithi059.stdout:checking locale.h presence... yes 2023-12-11T17:47:30.836 INFO:tasks.workunit.client.0.smithi059.stdout:checking for locale.h... yes 2023-12-11T17:47:30.850 INFO:tasks.workunit.client.0.smithi059.stdout:checking for stdlib.h... (cached) yes 2023-12-11T17:47:30.855 INFO:tasks.workunit.client.0.smithi059.stdout:checking for string.h... (cached) yes 2023-12-11T17:47:30.901 INFO:tasks.workunit.client.0.smithi059.stdout:checking sys/socket.h usability... yes 2023-12-11T17:47:30.920 INFO:tasks.workunit.client.0.smithi059.stdout:checking sys/socket.h presence... yes 2023-12-11T17:47:30.921 INFO:tasks.workunit.client.0.smithi059.stdout:checking for sys/socket.h... yes 2023-12-11T17:47:30.972 INFO:tasks.workunit.client.0.smithi059.stdout:checking sys/time.h usability... yes 2023-12-11T17:47:30.989 INFO:tasks.workunit.client.0.smithi059.stdout:checking sys/time.h presence... yes 2023-12-11T17:47:30.990 INFO:tasks.workunit.client.0.smithi059.stdout:checking for sys/time.h... yes 2023-12-11T17:47:31.040 INFO:tasks.workunit.client.0.smithi059.stdout:checking fcntl.h usability... yes 2023-12-11T17:47:31.058 INFO:tasks.workunit.client.0.smithi059.stdout:checking fcntl.h presence... yes 2023-12-11T17:47:31.059 INFO:tasks.workunit.client.0.smithi059.stdout:checking for fcntl.h... yes 2023-12-11T17:47:31.120 INFO:tasks.workunit.client.0.smithi059.stdout:checking sys/fcntl.h usability... yes 2023-12-11T17:47:31.137 INFO:tasks.workunit.client.0.smithi059.stdout:checking sys/fcntl.h presence... yes 2023-12-11T17:47:31.138 INFO:tasks.workunit.client.0.smithi059.stdout:checking for sys/fcntl.h... yes 2023-12-11T17:47:31.198 INFO:tasks.workunit.client.0.smithi059.stdout:checking sys/param.h usability... yes 2023-12-11T17:47:31.217 INFO:tasks.workunit.client.0.smithi059.stdout:checking sys/param.h presence... yes 2023-12-11T17:47:31.218 INFO:tasks.workunit.client.0.smithi059.stdout:checking for sys/param.h... yes 2023-12-11T17:47:31.261 INFO:tasks.workunit.client.0.smithi059.stdout:checking ioctl.h usability... no 2023-12-11T17:47:31.279 INFO:tasks.workunit.client.0.smithi059.stdout:checking ioctl.h presence... no 2023-12-11T17:47:31.279 INFO:tasks.workunit.client.0.smithi059.stdout:checking for ioctl.h... no 2023-12-11T17:47:31.327 INFO:tasks.workunit.client.0.smithi059.stdout:checking sys/ioctl.h usability... yes 2023-12-11T17:47:31.343 INFO:tasks.workunit.client.0.smithi059.stdout:checking sys/ioctl.h presence... yes 2023-12-11T17:47:31.344 INFO:tasks.workunit.client.0.smithi059.stdout:checking for sys/ioctl.h... yes 2023-12-11T17:47:31.392 INFO:tasks.workunit.client.0.smithi059.stdout:checking return type of signal handlers... void 2023-12-11T17:47:31.435 INFO:tasks.workunit.client.0.smithi059.stdout:checking for size_t... yes 2023-12-11T17:47:31.471 INFO:tasks.workunit.client.0.smithi059.stdout:checking for dev_t... yes 2023-12-11T17:47:31.513 INFO:tasks.workunit.client.0.smithi059.stdout:checking for short... yes 2023-12-11T17:47:31.586 INFO:tasks.workunit.client.0.smithi059.stdout:checking size of short... 2 2023-12-11T17:47:31.630 INFO:tasks.workunit.client.0.smithi059.stdout:checking for int... yes 2023-12-11T17:47:31.695 INFO:tasks.workunit.client.0.smithi059.stdout:checking size of int... 4 2023-12-11T17:47:31.739 INFO:tasks.workunit.client.0.smithi059.stdout:checking for long... yes 2023-12-11T17:47:31.802 INFO:tasks.workunit.client.0.smithi059.stdout:checking size of long... 8 2023-12-11T17:47:31.845 INFO:tasks.workunit.client.0.smithi059.stdout:checking for long long... yes 2023-12-11T17:47:31.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:47:31 smithi181 ceph-mon[103627]: pgmap v41: 129 pgs: 129 active+clean; 1.4 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 13 KiB/s wr, 6 op/s 2023-12-11T17:47:31.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:47:31 smithi059 ceph-mon[98847]: pgmap v41: 129 pgs: 129 active+clean; 1.4 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 13 KiB/s wr, 6 op/s 2023-12-11T17:47:31.909 INFO:tasks.workunit.client.0.smithi059.stdout:checking size of long long... 8 2023-12-11T17:47:31.950 INFO:tasks.workunit.client.0.smithi059.stdout:checking for ssize_t... yes 2023-12-11T17:47:31.988 INFO:tasks.workunit.client.0.smithi059.stdout:checking for sig_atomic_t... yes 2023-12-11T17:47:32.027 INFO:tasks.workunit.client.0.smithi059.stdout:checking whether gcc needs -traditional... no 2023-12-11T17:47:32.032 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:47:31 smithi115 ceph-mon[102748]: pgmap v41: 129 pgs: 129 active+clean; 1.4 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 13 KiB/s wr, 6 op/s 2023-12-11T17:47:32.062 INFO:tasks.workunit.client.0.smithi059.stdout:checking for an ANSI C-conforming const... yes 2023-12-11T17:47:32.085 INFO:tasks.workunit.client.0.smithi059.stdout:checking for inline... inline 2023-12-11T17:47:32.141 INFO:tasks.workunit.client.0.smithi059.stdout:checking for working alloca.h... yes 2023-12-11T17:47:32.188 INFO:tasks.workunit.client.0.smithi059.stdout:checking for alloca... yes 2023-12-11T17:47:32.251 INFO:tasks.workunit.client.0.smithi059.stdout:checking for working memcmp... yes 2023-12-11T17:47:32.304 INFO:tasks.workunit.client.0.smithi059.stdout:checking for strftime... yes 2023-12-11T17:47:32.426 INFO:tasks.workunit.client.0.smithi059.stdout:checking whether lstat dereferences a symlink specified with a trailing slash... yes 2023-12-11T17:47:32.496 INFO:tasks.workunit.client.0.smithi059.stdout:checking whether stat accepts an empty string... no 2023-12-11T17:47:32.548 INFO:tasks.workunit.client.0.smithi059.stdout:checking for vprintf... yes 2023-12-11T17:47:32.602 INFO:tasks.workunit.client.0.smithi059.stdout:checking for _doprnt... no 2023-12-11T17:47:32.629 INFO:tasks.workunit.client.0.smithi059.stdout:checking build system type... x86_64-unknown-linux-gnu 2023-12-11T17:47:32.642 INFO:tasks.workunit.client.0.smithi059.stdout:checking host system type... x86_64-unknown-linux-gnu 2023-12-11T17:47:32.692 INFO:tasks.workunit.client.0.smithi059.stdout:checking for the pthreads library -lpthreads... no 2023-12-11T17:47:32.744 INFO:tasks.workunit.client.0.smithi059.stdout:checking whether pthreads work without any flags... no 2023-12-11T17:47:32.768 INFO:tasks.workunit.client.0.smithi059.stdout:checking whether pthreads work with -Kthread... no 2023-12-11T17:47:32.790 INFO:tasks.workunit.client.0.smithi059.stdout:checking whether pthreads work with -kthread... no 2023-12-11T17:47:32.830 INFO:tasks.workunit.client.0.smithi059.stdout:checking for the pthreads library -llthread... no 2023-12-11T17:47:32.881 INFO:tasks.workunit.client.0.smithi059.stdout:checking whether pthreads work with -pthread... yes 2023-12-11T17:47:32.927 INFO:tasks.workunit.client.0.smithi059.stdout:checking for joinable pthread attribute... PTHREAD_CREATE_JOINABLE 2023-12-11T17:47:32.927 INFO:tasks.workunit.client.0.smithi059.stdout:checking if more special flags are required for pthreads... no 2023-12-11T17:47:32.927 INFO:tasks.workunit.client.0.smithi059.stdout:checking for cc_r... gcc 2023-12-11T17:47:33.014 INFO:tasks.workunit.client.0.smithi059.stdout:checking for setrlimit... yes 2023-12-11T17:47:33.072 INFO:tasks.workunit.client.0.smithi059.stdout:checking for setproctitle... no 2023-12-11T17:47:33.128 INFO:tasks.workunit.client.0.smithi059.stdout:checking for getopt_long... yes 2023-12-11T17:47:33.186 INFO:tasks.workunit.client.0.smithi059.stdout:checking for strtoull... yes 2023-12-11T17:47:33.244 INFO:tasks.workunit.client.0.smithi059.stdout:checking for strtoq... yes 2023-12-11T17:47:33.302 INFO:tasks.workunit.client.0.smithi059.stdout:checking for snprintf... yes 2023-12-11T17:47:33.360 INFO:tasks.workunit.client.0.smithi059.stdout:checking for vsnprintf... yes 2023-12-11T17:47:33.418 INFO:tasks.workunit.client.0.smithi059.stdout:checking for vfprintf... yes 2023-12-11T17:47:33.475 INFO:tasks.workunit.client.0.smithi059.stdout:checking for gethostname... yes 2023-12-11T17:47:33.533 INFO:tasks.workunit.client.0.smithi059.stdout:checking for setlocale... yes 2023-12-11T17:47:33.592 INFO:tasks.workunit.client.0.smithi059.stdout:checking for sync... yes 2023-12-11T17:47:33.657 INFO:tasks.workunit.client.0.smithi059.stdout:checking whether snprintf is C99 conformant... done 2023-12-11T17:47:33.667 INFO:tasks.workunit.client.0.smithi059.stdout:configure: creating ./config.status 2023-12-11T17:47:33.818 INFO:tasks.workunit.client.0.smithi059.stdout:config.status: creating Makefile 2023-12-11T17:47:33.834 INFO:tasks.workunit.client.0.smithi059.stdout:config.status: creating src/Makefile 2023-12-11T17:47:33.852 INFO:tasks.workunit.client.0.smithi059.stdout:config.status: creating man/Makefile 2023-12-11T17:47:33.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:47:33 smithi181 ceph-mon[103627]: pgmap v42: 129 pgs: 129 active+clean; 1.4 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 11 KiB/s wr, 5 op/s 2023-12-11T17:47:33.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:47:33 smithi059 ceph-mon[98847]: pgmap v42: 129 pgs: 129 active+clean; 1.4 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 11 KiB/s wr, 5 op/s 2023-12-11T17:47:33.909 INFO:tasks.workunit.client.0.smithi059.stdout:config.status: creating config.h 2023-12-11T17:47:34.022 INFO:tasks.workunit.client.0.smithi059.stdout:config.status: executing depfiles commands 2023-12-11T17:47:34.032 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:47:33 smithi115 ceph-mon[102748]: pgmap v42: 129 pgs: 129 active+clean; 1.4 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 11 KiB/s wr, 5 op/s 2023-12-11T17:47:34.144 INFO:tasks.workunit.client.0.smithi059.stderr:+ make 2023-12-11T17:47:34.154 INFO:tasks.workunit.client.0.smithi059.stdout:make all-recursive 2023-12-11T17:47:34.155 INFO:tasks.workunit.client.0.smithi059.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2023-12-11T17:47:34.157 INFO:tasks.workunit.client.0.smithi059.stdout:Making all in src 2023-12-11T17:47:34.163 INFO:tasks.workunit.client.0.smithi059.stdout:make[2]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/src' 2023-12-11T17:47:34.163 INFO:tasks.workunit.client.0.smithi059.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -pthread -MT blogbench.o -MD -MP -MF ".deps/blogbench.Tpo" -c -o blogbench.o blogbench.c; \ 2023-12-11T17:47:34.163 INFO:tasks.workunit.client.0.smithi059.stdout:then mv -f ".deps/blogbench.Tpo" ".deps/blogbench.Po"; else rm -f ".deps/blogbench.Tpo"; exit 1; fi 2023-12-11T17:47:34.232 INFO:tasks.workunit.client.0.smithi059.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -pthread -MT process.o -MD -MP -MF ".deps/process.Tpo" -c -o process.o process.c; \ 2023-12-11T17:47:34.232 INFO:tasks.workunit.client.0.smithi059.stdout:then mv -f ".deps/process.Tpo" ".deps/process.Po"; else rm -f ".deps/process.Tpo"; exit 1; fi 2023-12-11T17:47:34.295 INFO:tasks.workunit.client.0.smithi059.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -pthread -MT writer.o -MD -MP -MF ".deps/writer.Tpo" -c -o writer.o writer.c; \ 2023-12-11T17:47:34.295 INFO:tasks.workunit.client.0.smithi059.stdout:then mv -f ".deps/writer.Tpo" ".deps/writer.Po"; else rm -f ".deps/writer.Tpo"; exit 1; fi 2023-12-11T17:47:34.346 INFO:tasks.workunit.client.0.smithi059.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -pthread -MT rewriter.o -MD -MP -MF ".deps/rewriter.Tpo" -c -o rewriter.o rewriter.c; \ 2023-12-11T17:47:34.346 INFO:tasks.workunit.client.0.smithi059.stdout:then mv -f ".deps/rewriter.Tpo" ".deps/rewriter.Po"; else rm -f ".deps/rewriter.Tpo"; exit 1; fi 2023-12-11T17:47:34.396 INFO:tasks.workunit.client.0.smithi059.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -pthread -MT reader.o -MD -MP -MF ".deps/reader.Tpo" -c -o reader.o reader.c; \ 2023-12-11T17:47:34.396 INFO:tasks.workunit.client.0.smithi059.stdout:then mv -f ".deps/reader.Tpo" ".deps/reader.Po"; else rm -f ".deps/reader.Tpo"; exit 1; fi 2023-12-11T17:47:34.444 INFO:tasks.workunit.client.0.smithi059.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -pthread -MT commenter.o -MD -MP -MF ".deps/commenter.Tpo" -c -o commenter.o commenter.c; \ 2023-12-11T17:47:34.444 INFO:tasks.workunit.client.0.smithi059.stdout:then mv -f ".deps/commenter.Tpo" ".deps/commenter.Po"; else rm -f ".deps/commenter.Tpo"; exit 1; fi 2023-12-11T17:47:34.490 INFO:tasks.workunit.client.0.smithi059.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -pthread -MT helpers.o -MD -MP -MF ".deps/helpers.Tpo" -c -o helpers.o helpers.c; \ 2023-12-11T17:47:34.490 INFO:tasks.workunit.client.0.smithi059.stdout:then mv -f ".deps/helpers.Tpo" ".deps/helpers.Po"; else rm -f ".deps/helpers.Tpo"; exit 1; fi 2023-12-11T17:47:34.560 INFO:tasks.workunit.client.0.smithi059.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -pthread -MT bsd-getopt_long.o -MD -MP -MF ".deps/bsd-getopt_long.Tpo" -c -o bsd-getopt_long.o bsd-getopt_long.c; \ 2023-12-11T17:47:34.561 INFO:tasks.workunit.client.0.smithi059.stdout:then mv -f ".deps/bsd-getopt_long.Tpo" ".deps/bsd-getopt_long.Po"; else rm -f ".deps/bsd-getopt_long.Tpo"; exit 1; fi 2023-12-11T17:47:34.593 INFO:tasks.workunit.client.0.smithi059.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -pthread -MT fakesnprintf.o -MD -MP -MF ".deps/fakesnprintf.Tpo" -c -o fakesnprintf.o fakesnprintf.c; \ 2023-12-11T17:47:34.593 INFO:tasks.workunit.client.0.smithi059.stdout:then mv -f ".deps/fakesnprintf.Tpo" ".deps/fakesnprintf.Po"; else rm -f ".deps/fakesnprintf.Tpo"; exit 1; fi 2023-12-11T17:47:34.626 INFO:tasks.workunit.client.0.smithi059.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -pthread -MT mysnprintf.o -MD -MP -MF ".deps/mysnprintf.Tpo" -c -o mysnprintf.o mysnprintf.c; \ 2023-12-11T17:47:34.626 INFO:tasks.workunit.client.0.smithi059.stdout:then mv -f ".deps/mysnprintf.Tpo" ".deps/mysnprintf.Po"; else rm -f ".deps/mysnprintf.Tpo"; exit 1; fi 2023-12-11T17:47:34.661 INFO:tasks.workunit.client.0.smithi059.stdout:gcc -g -O2 -pthread -o blogbench blogbench.o process.o writer.o rewriter.o reader.o commenter.o helpers.o bsd-getopt_long.o fakesnprintf.o mysnprintf.o 2023-12-11T17:47:34.681 INFO:tasks.workunit.client.0.smithi059.stdout:make[2]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/src' 2023-12-11T17:47:34.681 INFO:tasks.workunit.client.0.smithi059.stdout:Making all in man 2023-12-11T17:47:34.699 INFO:tasks.workunit.client.0.smithi059.stdout:make[2]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/man' 2023-12-11T17:47:34.699 INFO:tasks.workunit.client.0.smithi059.stdout:make[2]: Nothing to be done for 'all'. 2023-12-11T17:47:34.700 INFO:tasks.workunit.client.0.smithi059.stdout:make[2]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/man' 2023-12-11T17:47:34.701 INFO:tasks.workunit.client.0.smithi059.stdout:make[2]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2023-12-11T17:47:34.702 INFO:tasks.workunit.client.0.smithi059.stdout:make[2]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2023-12-11T17:47:34.702 INFO:tasks.workunit.client.0.smithi059.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2023-12-11T17:47:34.703 INFO:tasks.workunit.client.0.smithi059.stderr:+ cd src 2023-12-11T17:47:34.703 INFO:tasks.workunit.client.0.smithi059.stderr:+ mkdir blogtest_in 2023-12-11T17:47:34.704 INFO:tasks.workunit.client.0.smithi059.stdout:running blogbench 2023-12-11T17:47:34.705 INFO:tasks.workunit.client.0.smithi059.stderr:+ echo 'running blogbench' 2023-12-11T17:47:34.705 INFO:tasks.workunit.client.0.smithi059.stderr:+ ./blogbench -d blogtest_in 2023-12-11T17:47:34.706 INFO:tasks.workunit.client.0.smithi059.stdout: 2023-12-11T17:47:34.706 INFO:tasks.workunit.client.0.smithi059.stdout:Frequency = 10 secs 2023-12-11T17:47:34.706 INFO:tasks.workunit.client.0.smithi059.stdout:Scratch dir = [blogtest_in] 2023-12-11T17:47:35.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:47:35 smithi181 ceph-mon[103627]: pgmap v43: 129 pgs: 129 active+clean; 3.7 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 213 KiB/s wr, 14 op/s 2023-12-11T17:47:35.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:47:35 smithi059 ceph-mon[98847]: pgmap v43: 129 pgs: 129 active+clean; 3.7 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 213 KiB/s wr, 14 op/s 2023-12-11T17:47:36.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:47:35 smithi115 ceph-mon[102748]: pgmap v43: 129 pgs: 129 active+clean; 3.7 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 213 KiB/s wr, 14 op/s 2023-12-11T17:47:37.184 INFO:tasks.workunit.client.0.smithi059.stdout:Spawning 3 writers... 2023-12-11T17:47:37.185 INFO:tasks.workunit.client.0.smithi059.stdout:Spawning 1 rewriters... 2023-12-11T17:47:37.185 INFO:tasks.workunit.client.0.smithi059.stdout:Spawning 5 commenters... 2023-12-11T17:47:37.185 INFO:tasks.workunit.client.0.smithi059.stdout:Spawning 100 readers... 2023-12-11T17:47:37.187 INFO:tasks.workunit.client.0.smithi059.stdout:Benchmarking for 30 iterations. 2023-12-11T17:47:37.187 INFO:tasks.workunit.client.0.smithi059.stdout:The test will run during 5 minutes. 2023-12-11T17:47:37.187 INFO:tasks.workunit.client.0.smithi059.stdout: 2023-12-11T17:47:37.187 INFO:tasks.workunit.client.0.smithi059.stdout: Nb blogs R articles W articles R pictures W pictures R comments W comments 2023-12-11T17:47:37.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:47:37 smithi181 ceph-mon[103627]: pgmap v44: 129 pgs: 129 active+clean; 3.7 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 213 KiB/s wr, 12 op/s 2023-12-11T17:47:37.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:47:37 smithi059 ceph-mon[98847]: pgmap v44: 129 pgs: 129 active+clean; 3.7 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 213 KiB/s wr, 12 op/s 2023-12-11T17:47:38.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:47:37 smithi115 ceph-mon[102748]: pgmap v44: 129 pgs: 129 active+clean; 3.7 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 213 KiB/s wr, 12 op/s 2023-12-11T17:47:39.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:47:39 smithi181 ceph-mon[103627]: pgmap v45: 129 pgs: 129 active+clean; 3.7 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 213 KiB/s wr, 12 op/s 2023-12-11T17:47:39.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:47:39 smithi059 ceph-mon[98847]: pgmap v45: 129 pgs: 129 active+clean; 3.7 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 213 KiB/s wr, 12 op/s 2023-12-11T17:47:40.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:47:39 smithi115 ceph-mon[102748]: pgmap v45: 129 pgs: 129 active+clean; 3.7 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 213 KiB/s wr, 12 op/s 2023-12-11T17:47:41.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:47:41 smithi181 ceph-mon[103627]: pgmap v46: 129 pgs: 129 active+clean; 9.0 MiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 456 KiB/s rd, 763 KiB/s wr, 63 op/s 2023-12-11T17:47:41.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:47:41 smithi181 ceph-mon[103627]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-12-11T17:47:41.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:47:41 smithi059 ceph-mon[98847]: pgmap v46: 129 pgs: 129 active+clean; 9.0 MiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 456 KiB/s rd, 763 KiB/s wr, 63 op/s 2023-12-11T17:47:41.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:47:41 smithi059 ceph-mon[98847]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-12-11T17:47:42.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:47:41 smithi115 ceph-mon[102748]: pgmap v46: 129 pgs: 129 active+clean; 9.0 MiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 456 KiB/s rd, 763 KiB/s wr, 63 op/s 2023-12-11T17:47:42.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:47:41 smithi115 ceph-mon[102748]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-12-11T17:47:43.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:47:42 smithi115 ceph-mon[102748]: pgmap v47: 129 pgs: 129 active+clean; 9.0 MiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 455 KiB/s rd, 752 KiB/s wr, 60 op/s 2023-12-11T17:47:43.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:47:42 smithi181 ceph-mon[103627]: pgmap v47: 129 pgs: 129 active+clean; 9.0 MiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 455 KiB/s rd, 752 KiB/s wr, 60 op/s 2023-12-11T17:47:43.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:47:42 smithi059 ceph-mon[98847]: pgmap v47: 129 pgs: 129 active+clean; 9.0 MiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 455 KiB/s rd, 752 KiB/s wr, 60 op/s 2023-12-11T17:47:45.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:47:45 smithi115 ceph-mon[102748]: pgmap v48: 129 pgs: 129 active+clean; 58 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.9 MiB/s wr, 233 op/s 2023-12-11T17:47:45.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:47:45 smithi181 ceph-mon[103627]: pgmap v48: 129 pgs: 129 active+clean; 58 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.9 MiB/s wr, 233 op/s 2023-12-11T17:47:45.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:47:45 smithi059 ceph-mon[98847]: pgmap v48: 129 pgs: 129 active+clean; 58 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.9 MiB/s wr, 233 op/s 2023-12-11T17:47:47.187 INFO:tasks.workunit.client.0.smithi059.stdout: 13 48230 710 32771 795 3990 399 2023-12-11T17:47:47.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:47:47 smithi115 ceph-mon[102748]: pgmap v49: 129 pgs: 129 active+clean; 58 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.7 MiB/s wr, 224 op/s 2023-12-11T17:47:47.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:47:47 smithi181 ceph-mon[103627]: pgmap v49: 129 pgs: 129 active+clean; 58 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.7 MiB/s wr, 224 op/s 2023-12-11T17:47:47.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:47:47 smithi059 ceph-mon[98847]: pgmap v49: 129 pgs: 129 active+clean; 58 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.7 MiB/s wr, 224 op/s 2023-12-11T17:47:49.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:47:49 smithi115 ceph-mon[102748]: pgmap v50: 129 pgs: 129 active+clean; 58 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.7 MiB/s wr, 224 op/s 2023-12-11T17:47:49.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:47:49 smithi115 ceph-mon[102748]: mgrmap e25: x(active, since 90s), standbys: z, y 2023-12-11T17:47:49.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:47:49 smithi181 ceph-mon[103627]: pgmap v50: 129 pgs: 129 active+clean; 58 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.7 MiB/s wr, 224 op/s 2023-12-11T17:47:49.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:47:49 smithi181 ceph-mon[103627]: mgrmap e25: x(active, since 90s), standbys: z, y 2023-12-11T17:47:49.907 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:47:49 smithi059 ceph-mon[98847]: pgmap v50: 129 pgs: 129 active+clean; 58 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.7 MiB/s wr, 224 op/s 2023-12-11T17:47:49.907 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:47:49 smithi059 ceph-mon[98847]: mgrmap e25: x(active, since 90s), standbys: z, y 2023-12-11T17:47:51.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:47:51 smithi115 ceph-mon[102748]: pgmap v51: 129 pgs: 129 active+clean; 59 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.0 MiB/s wr, 263 op/s 2023-12-11T17:47:51.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:47:51 smithi181 ceph-mon[103627]: pgmap v51: 129 pgs: 129 active+clean; 59 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.0 MiB/s wr, 263 op/s 2023-12-11T17:47:51.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:47:51 smithi059 ceph-mon[98847]: pgmap v51: 129 pgs: 129 active+clean; 59 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.0 MiB/s wr, 263 op/s 2023-12-11T17:47:53.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:47:53 smithi115 ceph-mon[102748]: pgmap v52: 129 pgs: 129 active+clean; 59 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.4 MiB/s wr, 212 op/s 2023-12-11T17:47:53.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:47:53 smithi181 ceph-mon[103627]: pgmap v52: 129 pgs: 129 active+clean; 59 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.4 MiB/s wr, 212 op/s 2023-12-11T17:47:53.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:47:53 smithi059 ceph-mon[98847]: pgmap v52: 129 pgs: 129 active+clean; 59 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.4 MiB/s wr, 212 op/s 2023-12-11T17:47:55.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:47:55 smithi181 ceph-mon[103627]: pgmap v53: 129 pgs: 129 active+clean; 66 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.1 MiB/s wr, 235 op/s 2023-12-11T17:47:55.906 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:47:55 smithi059 ceph-mon[98847]: pgmap v53: 129 pgs: 129 active+clean; 66 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.1 MiB/s wr, 235 op/s 2023-12-11T17:47:56.032 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:47:55 smithi115 ceph-mon[102748]: pgmap v53: 129 pgs: 129 active+clean; 66 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.1 MiB/s wr, 235 op/s 2023-12-11T17:47:57.187 INFO:tasks.workunit.client.0.smithi059.stdout: 24 608546 673 455082 505 152382 1246 2023-12-11T17:47:57.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:47:57 smithi181 ceph-mon[103627]: pgmap v54: 129 pgs: 129 active+clean; 66 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 921 KiB/s wr, 62 op/s 2023-12-11T17:47:57.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:47:57 smithi059 ceph-mon[98847]: pgmap v54: 129 pgs: 129 active+clean; 66 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 921 KiB/s wr, 62 op/s 2023-12-11T17:47:58.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:47:57 smithi115 ceph-mon[102748]: pgmap v54: 129 pgs: 129 active+clean; 66 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 921 KiB/s wr, 62 op/s 2023-12-11T17:47:59.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:47:59 smithi181 ceph-mon[103627]: pgmap v55: 129 pgs: 129 active+clean; 66 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 921 KiB/s wr, 62 op/s 2023-12-11T17:47:59.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:47:59 smithi059 ceph-mon[98847]: pgmap v55: 129 pgs: 129 active+clean; 66 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 921 KiB/s wr, 62 op/s 2023-12-11T17:48:00.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:47:59 smithi115 ceph-mon[102748]: pgmap v55: 129 pgs: 129 active+clean; 66 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 921 KiB/s wr, 62 op/s 2023-12-11T17:48:01.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:48:01 smithi181 ceph-mon[103627]: pgmap v56: 129 pgs: 129 active+clean; 74 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.7 MiB/s wr, 85 op/s 2023-12-11T17:48:01.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:48:01 smithi059 ceph-mon[98847]: pgmap v56: 129 pgs: 129 active+clean; 74 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.7 MiB/s wr, 85 op/s 2023-12-11T17:48:02.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:48:01 smithi115 ceph-mon[102748]: pgmap v56: 129 pgs: 129 active+clean; 74 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.7 MiB/s wr, 85 op/s 2023-12-11T17:48:03.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:48:03 smithi181 ceph-mon[103627]: pgmap v57: 129 pgs: 129 active+clean; 74 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 46 op/s 2023-12-11T17:48:03.910 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:48:03 smithi059 ceph-mon[98847]: pgmap v57: 129 pgs: 129 active+clean; 74 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 46 op/s 2023-12-11T17:48:04.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:48:03 smithi115 ceph-mon[102748]: pgmap v57: 129 pgs: 129 active+clean; 74 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 46 op/s 2023-12-11T17:48:04.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:48:04 smithi181 ceph-mon[103627]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-12-11T17:48:04.920 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:48:04 smithi059 ceph-mon[98847]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-12-11T17:48:05.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:48:04 smithi115 ceph-mon[102748]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-12-11T17:48:05.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:48:05 smithi181 ceph-mon[103627]: pgmap v58: 129 pgs: 129 active+clean; 83 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.4 MiB/s wr, 141 op/s 2023-12-11T17:48:05.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:48:05 smithi059 ceph-mon[98847]: pgmap v58: 129 pgs: 129 active+clean; 83 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.4 MiB/s wr, 141 op/s 2023-12-11T17:48:06.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:48:05 smithi115 ceph-mon[102748]: pgmap v58: 129 pgs: 129 active+clean; 83 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.4 MiB/s wr, 141 op/s 2023-12-11T17:48:07.187 INFO:tasks.workunit.client.0.smithi059.stdout: 37 905871 690 654587 678 319536 1459 2023-12-11T17:48:07.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:48:07 smithi181 ceph-mon[103627]: pgmap v59: 129 pgs: 129 active+clean; 83 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.7 MiB/s wr, 118 op/s 2023-12-11T17:48:07.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:48:07 smithi059 ceph-mon[98847]: pgmap v59: 129 pgs: 129 active+clean; 83 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.7 MiB/s wr, 118 op/s 2023-12-11T17:48:08.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:48:07 smithi115 ceph-mon[102748]: pgmap v59: 129 pgs: 129 active+clean; 83 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.7 MiB/s wr, 118 op/s 2023-12-11T17:48:09.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:48:09 smithi181 ceph-mon[103627]: pgmap v60: 129 pgs: 129 active+clean; 83 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.7 MiB/s wr, 118 op/s 2023-12-11T17:48:09.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:48:09 smithi059 ceph-mon[98847]: pgmap v60: 129 pgs: 129 active+clean; 83 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.7 MiB/s wr, 118 op/s 2023-12-11T17:48:10.032 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:48:09 smithi115 ceph-mon[102748]: pgmap v60: 129 pgs: 129 active+clean; 83 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.7 MiB/s wr, 118 op/s 2023-12-11T17:48:11.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:48:11 smithi181 ceph-mon[103627]: pgmap v61: 129 pgs: 129 active+clean; 146 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 7.2 MiB/s wr, 429 op/s 2023-12-11T17:48:11.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:48:11 smithi059 ceph-mon[98847]: pgmap v61: 129 pgs: 129 active+clean; 146 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 7.2 MiB/s wr, 429 op/s 2023-12-11T17:48:12.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:48:11 smithi115 ceph-mon[102748]: pgmap v61: 129 pgs: 129 active+clean; 146 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 7.2 MiB/s wr, 429 op/s 2023-12-11T17:48:13.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:48:13 smithi181 ceph-mon[103627]: pgmap v62: 129 pgs: 129 active+clean; 146 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.4 MiB/s wr, 405 op/s 2023-12-11T17:48:13.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:48:13 smithi059 ceph-mon[98847]: pgmap v62: 129 pgs: 129 active+clean; 146 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.4 MiB/s wr, 405 op/s 2023-12-11T17:48:14.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:48:13 smithi115 ceph-mon[102748]: pgmap v62: 129 pgs: 129 active+clean; 146 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.4 MiB/s wr, 405 op/s 2023-12-11T17:48:15.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:48:15 smithi115 ceph-mon[102748]: pgmap v63: 129 pgs: 129 active+clean; 172 MiB data, 499 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 8.7 MiB/s wr, 495 op/s 2023-12-11T17:48:15.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:48:15 smithi181 ceph-mon[103627]: pgmap v63: 129 pgs: 129 active+clean; 172 MiB data, 499 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 8.7 MiB/s wr, 495 op/s 2023-12-11T17:48:15.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:48:15 smithi059 ceph-mon[98847]: pgmap v63: 129 pgs: 129 active+clean; 172 MiB data, 499 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 8.7 MiB/s wr, 495 op/s 2023-12-11T17:48:17.187 INFO:tasks.workunit.client.0.smithi059.stdout: 39 19323 72 14265 113 6745 245 2023-12-11T17:48:17.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:48:17 smithi115 ceph-mon[102748]: pgmap v64: 129 pgs: 129 active+clean; 197 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 9.9 MiB/s wr, 518 op/s 2023-12-11T17:48:17.897 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:48:17 smithi181 ceph-mon[103627]: pgmap v64: 129 pgs: 129 active+clean; 197 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 9.9 MiB/s wr, 518 op/s 2023-12-11T17:48:17.951 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:48:17 smithi059 ceph-mon[98847]: pgmap v64: 129 pgs: 129 active+clean; 197 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 9.9 MiB/s wr, 518 op/s 2023-12-11T17:48:19.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:48:19 smithi115 ceph-mon[102748]: pgmap v65: 129 pgs: 129 active+clean; 197 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 9.9 MiB/s wr, 518 op/s 2023-12-11T17:48:19.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:48:19 smithi181 ceph-mon[103627]: pgmap v65: 129 pgs: 129 active+clean; 197 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 9.9 MiB/s wr, 518 op/s 2023-12-11T17:48:19.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:48:19 smithi059 ceph-mon[98847]: pgmap v65: 129 pgs: 129 active+clean; 197 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 9.9 MiB/s wr, 518 op/s 2023-12-11T17:48:21.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:48:21 smithi115 ceph-mon[102748]: pgmap v66: 129 pgs: 129 active+clean; 201 MiB data, 601 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 11 MiB/s wr, 688 op/s 2023-12-11T17:48:21.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:48:21 smithi181 ceph-mon[103627]: pgmap v66: 129 pgs: 129 active+clean; 201 MiB data, 601 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 11 MiB/s wr, 688 op/s 2023-12-11T17:48:21.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:48:21 smithi059 ceph-mon[98847]: pgmap v66: 129 pgs: 129 active+clean; 201 MiB data, 601 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 11 MiB/s wr, 688 op/s 2023-12-11T17:48:23.782 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:48:23 smithi115 ceph-mon[102748]: pgmap v67: 129 pgs: 129 active+clean; 201 MiB data, 601 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.4 MiB/s wr, 378 op/s 2023-12-11T17:48:23.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:48:23 smithi181 ceph-mon[103627]: pgmap v67: 129 pgs: 129 active+clean; 201 MiB data, 601 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.4 MiB/s wr, 378 op/s 2023-12-11T17:48:23.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:48:23 smithi059 ceph-mon[98847]: pgmap v67: 129 pgs: 129 active+clean; 201 MiB data, 601 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.4 MiB/s wr, 378 op/s 2023-12-11T17:48:24.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:48:24 smithi181 ceph-mon[103627]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-12-11T17:48:24.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:48:24 smithi059 ceph-mon[98847]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-12-11T17:48:25.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:48:24 smithi115 ceph-mon[102748]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-12-11T17:48:25.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:48:25 smithi181 ceph-mon[103627]: pgmap v68: 129 pgs: 129 active+clean; 205 MiB data, 603 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.7 MiB/s wr, 390 op/s 2023-12-11T17:48:25.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:48:25 smithi181 ceph-mon[103627]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:48:25.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:48:25 smithi181 ceph-mon[103627]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:48:25.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:48:25 smithi181 ceph-mon[103627]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:48:25.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:48:25 smithi059 ceph-mon[98847]: pgmap v68: 129 pgs: 129 active+clean; 205 MiB data, 603 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.7 MiB/s wr, 390 op/s 2023-12-11T17:48:25.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:48:25 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:48:25.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:48:25 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:48:25.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:48:25 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:48:26.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:48:25 smithi115 ceph-mon[102748]: pgmap v68: 129 pgs: 129 active+clean; 205 MiB data, 603 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.7 MiB/s wr, 390 op/s 2023-12-11T17:48:26.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:48:25 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:48:26.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:48:25 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:48:26.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:48:25 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:48:26.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:48:26 smithi181 ceph-mon[103627]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:48:26.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:48:26 smithi181 ceph-mon[103627]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:48:26.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:48:26 smithi181 ceph-mon[103627]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:48:26.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:48:26 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:48:26.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:48:26 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:48:26.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:48:26 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:48:27.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:48:26 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:48:27.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:48:26 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:48:27.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:48:26 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:48:27.187 INFO:tasks.workunit.client.0.smithi059.stdout: 45 72786 339 56619 182 28272 377 2023-12-11T17:48:27.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:48:27 smithi181 ceph-mon[103627]: pgmap v69: 129 pgs: 129 active+clean; 205 MiB data, 604 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.5 MiB/s wr, 310 op/s 2023-12-11T17:48:27.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:48:27 smithi059 ceph-mon[98847]: pgmap v69: 129 pgs: 129 active+clean; 205 MiB data, 604 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.5 MiB/s wr, 310 op/s 2023-12-11T17:48:28.032 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:48:27 smithi115 ceph-mon[102748]: pgmap v69: 129 pgs: 129 active+clean; 205 MiB data, 604 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.5 MiB/s wr, 310 op/s 2023-12-11T17:48:29.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:48:29 smithi181 ceph-mon[103627]: pgmap v70: 129 pgs: 129 active+clean; 205 MiB data, 604 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 192 op/s 2023-12-11T17:48:29.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:48:29 smithi059 ceph-mon[98847]: pgmap v70: 129 pgs: 129 active+clean; 205 MiB data, 604 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 192 op/s 2023-12-11T17:48:30.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:48:29 smithi115 ceph-mon[102748]: pgmap v70: 129 pgs: 129 active+clean; 205 MiB data, 604 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 192 op/s 2023-12-11T17:48:31.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:48:31 smithi181 ceph-mon[103627]: pgmap v71: 129 pgs: 129 active+clean; 210 MiB data, 617 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.8 MiB/s wr, 238 op/s 2023-12-11T17:48:31.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:48:31 smithi059 ceph-mon[98847]: pgmap v71: 129 pgs: 129 active+clean; 210 MiB data, 617 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.8 MiB/s wr, 238 op/s 2023-12-11T17:48:32.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:48:31 smithi115 ceph-mon[102748]: pgmap v71: 129 pgs: 129 active+clean; 210 MiB data, 617 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.8 MiB/s wr, 238 op/s 2023-12-11T17:48:33.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:48:33 smithi181 ceph-mon[103627]: pgmap v72: 129 pgs: 129 active+clean; 210 MiB data, 617 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 846 KiB/s wr, 67 op/s 2023-12-11T17:48:33.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:48:33 smithi059 ceph-mon[98847]: pgmap v72: 129 pgs: 129 active+clean; 210 MiB data, 617 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 846 KiB/s wr, 67 op/s 2023-12-11T17:48:34.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:48:33 smithi115 ceph-mon[102748]: pgmap v72: 129 pgs: 129 active+clean; 210 MiB data, 617 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 846 KiB/s wr, 67 op/s 2023-12-11T17:48:35.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:48:35 smithi181 ceph-mon[103627]: pgmap v73: 129 pgs: 129 active+clean; 216 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.4 MiB/s wr, 94 op/s 2023-12-11T17:48:35.915 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:48:35 smithi059 ceph-mon[98847]: pgmap v73: 129 pgs: 129 active+clean; 216 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.4 MiB/s wr, 94 op/s 2023-12-11T17:48:36.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:48:35 smithi115 ceph-mon[102748]: pgmap v73: 129 pgs: 129 active+clean; 216 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.4 MiB/s wr, 94 op/s 2023-12-11T17:48:37.187 INFO:tasks.workunit.client.0.smithi059.stdout: 71 782542 1082 556185 1340 279116 805 2023-12-11T17:48:37.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:48:37 smithi181 ceph-mon[103627]: pgmap v74: 129 pgs: 129 active+clean; 220 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.4 MiB/s wr, 95 op/s 2023-12-11T17:48:37.911 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:48:37 smithi059 ceph-mon[98847]: pgmap v74: 129 pgs: 129 active+clean; 220 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.4 MiB/s wr, 95 op/s 2023-12-11T17:48:38.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:48:37 smithi115 ceph-mon[102748]: pgmap v74: 129 pgs: 129 active+clean; 220 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.4 MiB/s wr, 95 op/s 2023-12-11T17:48:39.896 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:48:39 smithi181 ceph-mon[103627]: pgmap v75: 129 pgs: 129 active+clean; 220 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.4 MiB/s wr, 86 op/s 2023-12-11T17:48:39.908 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:48:39 smithi059 ceph-mon[98847]: pgmap v75: 129 pgs: 129 active+clean; 220 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.4 MiB/s wr, 86 op/s 2023-12-11T17:48:40.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:48:39 smithi115 ceph-mon[102748]: pgmap v75: 129 pgs: 129 active+clean; 220 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.4 MiB/s wr, 86 op/s 2023-12-11T17:48:41.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:48:41 smithi181 ceph-mon[103627]: pgmap v76: 129 pgs: 129 active+clean; 225 MiB data, 624 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.0 MiB/s wr, 135 op/s 2023-12-11T17:48:41.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:48:41 smithi059 ceph-mon[98847]: pgmap v76: 129 pgs: 129 active+clean; 225 MiB data, 624 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.0 MiB/s wr, 135 op/s 2023-12-11T17:48:42.032 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:48:41 smithi115 ceph-mon[102748]: pgmap v76: 129 pgs: 129 active+clean; 225 MiB data, 624 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.0 MiB/s wr, 135 op/s 2023-12-11T17:48:43.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:48:43 smithi181 ceph-mon[103627]: pgmap v77: 129 pgs: 129 active+clean; 225 MiB data, 624 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 90 op/s 2023-12-11T17:48:43.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:48:43 smithi059 ceph-mon[98847]: pgmap v77: 129 pgs: 129 active+clean; 225 MiB data, 624 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 90 op/s 2023-12-11T17:48:44.032 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:48:43 smithi115 ceph-mon[102748]: pgmap v77: 129 pgs: 129 active+clean; 225 MiB data, 624 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 90 op/s 2023-12-11T17:48:45.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:48:45 smithi181 ceph-mon[103627]: pgmap v78: 129 pgs: 129 active+clean; 270 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.4 MiB/s wr, 284 op/s 2023-12-11T17:48:45.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:48:45 smithi059 ceph-mon[98847]: pgmap v78: 129 pgs: 129 active+clean; 270 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.4 MiB/s wr, 284 op/s 2023-12-11T17:48:46.032 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:48:45 smithi115 ceph-mon[102748]: pgmap v78: 129 pgs: 129 active+clean; 270 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.4 MiB/s wr, 284 op/s 2023-12-11T17:48:47.194 INFO:tasks.workunit.client.0.smithi059.stdout: 87 581397 686 416389 761 186949 193 2023-12-11T17:48:47.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:48:47 smithi181 ceph-mon[103627]: pgmap v79: 129 pgs: 129 active+clean; 329 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 10 MiB/s wr, 498 op/s 2023-12-11T17:48:47.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:48:47 smithi059 ceph-mon[98847]: pgmap v79: 129 pgs: 129 active+clean; 329 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 10 MiB/s wr, 498 op/s 2023-12-11T17:48:48.032 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:48:47 smithi115 ceph-mon[102748]: pgmap v79: 129 pgs: 129 active+clean; 329 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 10 MiB/s wr, 498 op/s 2023-12-11T17:48:49.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:48:49 smithi181 ceph-mon[103627]: pgmap v80: 129 pgs: 129 active+clean; 329 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 9.6 MiB/s wr, 484 op/s 2023-12-11T17:48:49.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:48:49 smithi059 ceph-mon[98847]: pgmap v80: 129 pgs: 129 active+clean; 329 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 9.6 MiB/s wr, 484 op/s 2023-12-11T17:48:50.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:48:49 smithi115 ceph-mon[102748]: pgmap v80: 129 pgs: 129 active+clean; 329 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 9.6 MiB/s wr, 484 op/s 2023-12-11T17:48:51.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:48:51 smithi059 ceph-mon[98847]: pgmap v81: 129 pgs: 129 active+clean; 336 MiB data, 674 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 10 MiB/s wr, 554 op/s 2023-12-11T17:48:52.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:48:51 smithi115 ceph-mon[102748]: pgmap v81: 129 pgs: 129 active+clean; 336 MiB data, 674 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 10 MiB/s wr, 554 op/s 2023-12-11T17:48:52.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:48:51 smithi181 ceph-mon[103627]: pgmap v81: 129 pgs: 129 active+clean; 336 MiB data, 674 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 10 MiB/s wr, 554 op/s 2023-12-11T17:48:53.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:48:53 smithi059 ceph-mon[98847]: pgmap v82: 129 pgs: 129 active+clean; 336 MiB data, 674 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 9.7 MiB/s wr, 505 op/s 2023-12-11T17:48:54.032 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:48:53 smithi115 ceph-mon[102748]: pgmap v82: 129 pgs: 129 active+clean; 336 MiB data, 674 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 9.7 MiB/s wr, 505 op/s 2023-12-11T17:48:54.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:48:53 smithi181 ceph-mon[103627]: pgmap v82: 129 pgs: 129 active+clean; 336 MiB data, 674 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 9.7 MiB/s wr, 505 op/s 2023-12-11T17:48:55.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:48:55 smithi181 ceph-mon[103627]: pgmap v83: 129 pgs: 129 active+clean; 342 MiB data, 680 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 10 MiB/s wr, 537 op/s 2023-12-11T17:48:55.927 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:48:55 smithi059 ceph-mon[98847]: pgmap v83: 129 pgs: 129 active+clean; 342 MiB data, 680 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 10 MiB/s wr, 537 op/s 2023-12-11T17:48:56.032 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:48:55 smithi115 ceph-mon[102748]: pgmap v83: 129 pgs: 129 active+clean; 342 MiB data, 680 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 10 MiB/s wr, 537 op/s 2023-12-11T17:48:57.195 INFO:tasks.workunit.client.0.smithi059.stdout: 109 1090289 1127 807082 1348 288434 177 2023-12-11T17:48:57.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:48:57 smithi181 ceph-mon[103627]: pgmap v84: 129 pgs: 129 active+clean; 345 MiB data, 673 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 6.7 MiB/s wr, 370 op/s 2023-12-11T17:48:57.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:48:57 smithi059 ceph-mon[98847]: pgmap v84: 129 pgs: 129 active+clean; 345 MiB data, 673 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 6.7 MiB/s wr, 370 op/s 2023-12-11T17:48:58.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:48:57 smithi115 ceph-mon[102748]: pgmap v84: 129 pgs: 129 active+clean; 345 MiB data, 673 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 6.7 MiB/s wr, 370 op/s 2023-12-11T17:48:59.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:48:59 smithi181 ceph-mon[103627]: pgmap v85: 129 pgs: 129 active+clean; 345 MiB data, 673 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 129 op/s 2023-12-11T17:48:59.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:48:59 smithi059 ceph-mon[98847]: pgmap v85: 129 pgs: 129 active+clean; 345 MiB data, 673 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 129 op/s 2023-12-11T17:49:00.032 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:48:59 smithi115 ceph-mon[102748]: pgmap v85: 129 pgs: 129 active+clean; 345 MiB data, 673 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 129 op/s 2023-12-11T17:49:01.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:49:01 smithi181 ceph-mon[103627]: pgmap v86: 129 pgs: 129 active+clean; 352 MiB data, 710 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.3 MiB/s wr, 251 op/s 2023-12-11T17:49:01.913 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:49:01 smithi059 ceph-mon[98847]: pgmap v86: 129 pgs: 129 active+clean; 352 MiB data, 710 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.3 MiB/s wr, 251 op/s 2023-12-11T17:49:02.032 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:49:01 smithi115 ceph-mon[102748]: pgmap v86: 129 pgs: 129 active+clean; 352 MiB data, 710 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.3 MiB/s wr, 251 op/s 2023-12-11T17:49:04.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:49:03 smithi115 ceph-mon[102748]: pgmap v87: 129 pgs: 129 active+clean; 352 MiB data, 710 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.6 MiB/s wr, 181 op/s 2023-12-11T17:49:04.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:49:03 smithi181 ceph-mon[103627]: pgmap v87: 129 pgs: 129 active+clean; 352 MiB data, 710 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.6 MiB/s wr, 181 op/s 2023-12-11T17:49:04.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:49:03 smithi059 ceph-mon[98847]: pgmap v87: 129 pgs: 129 active+clean; 352 MiB data, 710 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.6 MiB/s wr, 181 op/s 2023-12-11T17:49:06.032 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:49:05 smithi115 ceph-mon[102748]: pgmap v88: 129 pgs: 129 active+clean; 355 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.9 MiB/s wr, 244 op/s 2023-12-11T17:49:06.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:49:05 smithi181 ceph-mon[103627]: pgmap v88: 129 pgs: 129 active+clean; 355 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.9 MiB/s wr, 244 op/s 2023-12-11T17:49:06.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:49:05 smithi059 ceph-mon[98847]: pgmap v88: 129 pgs: 129 active+clean; 355 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.9 MiB/s wr, 244 op/s 2023-12-11T17:49:07.195 INFO:tasks.workunit.client.0.smithi059.stdout: 125 1314046 858 983029 936 312604 1122 2023-12-11T17:49:08.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:49:07 smithi115 ceph-mon[102748]: pgmap v89: 129 pgs: 129 active+clean; 358 MiB data, 711 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.8 MiB/s wr, 251 op/s 2023-12-11T17:49:08.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:49:07 smithi181 ceph-mon[103627]: pgmap v89: 129 pgs: 129 active+clean; 358 MiB data, 711 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.8 MiB/s wr, 251 op/s 2023-12-11T17:49:08.155 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:49:07 smithi059 ceph-mon[98847]: pgmap v89: 129 pgs: 129 active+clean; 358 MiB data, 711 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.8 MiB/s wr, 251 op/s 2023-12-11T17:49:10.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:49:09 smithi115 ceph-mon[102748]: pgmap v90: 129 pgs: 129 active+clean; 358 MiB data, 711 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.4 MiB/s wr, 224 op/s 2023-12-11T17:49:10.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:49:09 smithi181 ceph-mon[103627]: pgmap v90: 129 pgs: 129 active+clean; 358 MiB data, 711 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.4 MiB/s wr, 224 op/s 2023-12-11T17:49:10.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:49:09 smithi059 ceph-mon[98847]: pgmap v90: 129 pgs: 129 active+clean; 358 MiB data, 711 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.4 MiB/s wr, 224 op/s 2023-12-11T17:49:11.035 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:49:10 smithi115 ceph-mon[102748]: pgmap v91: 129 pgs: 129 active+clean; 370 MiB data, 730 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.5 MiB/s wr, 288 op/s 2023-12-11T17:49:11.146 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:49:10 smithi181 ceph-mon[103627]: pgmap v91: 129 pgs: 129 active+clean; 370 MiB data, 730 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.5 MiB/s wr, 288 op/s 2023-12-11T17:49:11.158 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:49:10 smithi059 ceph-mon[98847]: pgmap v91: 129 pgs: 129 active+clean; 370 MiB data, 730 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.5 MiB/s wr, 288 op/s 2023-12-11T17:49:13.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:49:12 smithi059 ceph-mon[98847]: pgmap v92: 129 pgs: 129 active+clean; 370 MiB data, 730 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.8 MiB/s wr, 165 op/s 2023-12-11T17:49:13.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:49:12 smithi115 ceph-mon[102748]: pgmap v92: 129 pgs: 129 active+clean; 370 MiB data, 730 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.8 MiB/s wr, 165 op/s 2023-12-11T17:49:13.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:49:12 smithi181 ceph-mon[103627]: pgmap v92: 129 pgs: 129 active+clean; 370 MiB data, 730 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.8 MiB/s wr, 165 op/s 2023-12-11T17:49:15.896 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:49:15 smithi181 ceph-mon[103627]: pgmap v93: 129 pgs: 129 active+clean; 418 MiB data, 805 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 5.9 MiB/s wr, 306 op/s 2023-12-11T17:49:15.907 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:49:15 smithi059 ceph-mon[98847]: pgmap v93: 129 pgs: 129 active+clean; 418 MiB data, 805 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 5.9 MiB/s wr, 306 op/s 2023-12-11T17:49:16.034 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:49:15 smithi115 ceph-mon[102748]: pgmap v93: 129 pgs: 129 active+clean; 418 MiB data, 805 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 5.9 MiB/s wr, 306 op/s 2023-12-11T17:49:17.196 INFO:tasks.workunit.client.0.smithi059.stdout: 131 313514 380 229858 201 92244 525 2023-12-11T17:49:17.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:49:17 smithi181 ceph-mon[103627]: pgmap v94: 129 pgs: 129 active+clean; 454 MiB data, 890 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 8.7 MiB/s wr, 377 op/s 2023-12-11T17:49:17.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:49:17 smithi059 ceph-mon[98847]: pgmap v94: 129 pgs: 129 active+clean; 454 MiB data, 890 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 8.7 MiB/s wr, 377 op/s 2023-12-11T17:49:18.032 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:49:17 smithi115 ceph-mon[102748]: pgmap v94: 129 pgs: 129 active+clean; 454 MiB data, 890 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 8.7 MiB/s wr, 377 op/s 2023-12-11T17:49:19.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:49:19 smithi181 ceph-mon[103627]: pgmap v95: 129 pgs: 129 active+clean; 454 MiB data, 890 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 8.3 MiB/s wr, 339 op/s 2023-12-11T17:49:19.910 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:49:19 smithi059 ceph-mon[98847]: pgmap v95: 129 pgs: 129 active+clean; 454 MiB data, 890 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 8.3 MiB/s wr, 339 op/s 2023-12-11T17:49:20.032 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:49:19 smithi115 ceph-mon[102748]: pgmap v95: 129 pgs: 129 active+clean; 454 MiB data, 890 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 8.3 MiB/s wr, 339 op/s 2023-12-11T17:49:21.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:49:21 smithi181 ceph-mon[103627]: pgmap v96: 129 pgs: 129 active+clean; 556 MiB data, 1001 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 17 MiB/s wr, 752 op/s 2023-12-11T17:49:21.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:49:21 smithi059 ceph-mon[98847]: pgmap v96: 129 pgs: 129 active+clean; 556 MiB data, 1001 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 17 MiB/s wr, 752 op/s 2023-12-11T17:49:22.032 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:49:21 smithi115 ceph-mon[102748]: pgmap v96: 129 pgs: 129 active+clean; 556 MiB data, 1001 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 17 MiB/s wr, 752 op/s 2023-12-11T17:49:23.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:49:23 smithi181 ceph-mon[103627]: pgmap v97: 129 pgs: 129 active+clean; 556 MiB data, 1001 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 16 MiB/s wr, 688 op/s 2023-12-11T17:49:23.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:49:23 smithi059 ceph-mon[98847]: pgmap v97: 129 pgs: 129 active+clean; 556 MiB data, 1001 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 16 MiB/s wr, 688 op/s 2023-12-11T17:49:24.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:49:23 smithi115 ceph-mon[102748]: pgmap v97: 129 pgs: 129 active+clean; 556 MiB data, 1001 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 16 MiB/s wr, 688 op/s 2023-12-11T17:49:25.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:49:25 smithi181 ceph-mon[103627]: pgmap v98: 129 pgs: 129 active+clean; 562 MiB data, 1011 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 17 MiB/s wr, 724 op/s 2023-12-11T17:49:25.911 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:49:25 smithi059 ceph-mon[98847]: pgmap v98: 129 pgs: 129 active+clean; 562 MiB data, 1011 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 17 MiB/s wr, 724 op/s 2023-12-11T17:49:26.032 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:49:25 smithi115 ceph-mon[102748]: pgmap v98: 129 pgs: 129 active+clean; 562 MiB data, 1011 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 17 MiB/s wr, 724 op/s 2023-12-11T17:49:27.032 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:49:26 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:49:27.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:49:26 smithi181 ceph-mon[103627]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:49:27.165 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:49:26 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:49:27.206 INFO:tasks.workunit.client.0.smithi059.stdout: 142 1137315 528 830744 619 337824 795 2023-12-11T17:49:28.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:49:27 smithi115 ceph-mon[102748]: pgmap v99: 129 pgs: 129 active+clean; 564 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 13 MiB/s wr, 599 op/s 2023-12-11T17:49:28.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:49:27 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:49:28.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:49:27 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:49:28.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:49:27 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:49:28.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:49:27 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:49:28.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:49:27 smithi181 ceph-mon[103627]: pgmap v99: 129 pgs: 129 active+clean; 564 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 13 MiB/s wr, 599 op/s 2023-12-11T17:49:28.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:49:27 smithi181 ceph-mon[103627]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:49:28.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:49:27 smithi181 ceph-mon[103627]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:49:28.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:49:27 smithi181 ceph-mon[103627]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:49:28.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:49:27 smithi181 ceph-mon[103627]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:49:28.159 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:49:27 smithi059 ceph-mon[98847]: pgmap v99: 129 pgs: 129 active+clean; 564 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 13 MiB/s wr, 599 op/s 2023-12-11T17:49:28.159 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:49:27 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:49:28.159 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:49:27 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:49:28.159 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:49:27 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:49:28.159 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:49:27 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:49:30.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:49:29 smithi115 ceph-mon[102748]: pgmap v100: 129 pgs: 129 active+clean; 564 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 9.7 MiB/s wr, 464 op/s 2023-12-11T17:49:30.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:49:29 smithi181 ceph-mon[103627]: pgmap v100: 129 pgs: 129 active+clean; 564 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 9.7 MiB/s wr, 464 op/s 2023-12-11T17:49:30.164 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:49:29 smithi059 ceph-mon[98847]: pgmap v100: 129 pgs: 129 active+clean; 564 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 9.7 MiB/s wr, 464 op/s 2023-12-11T17:49:31.032 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:49:30 smithi115 ceph-mon[102748]: pgmap v101: 129 pgs: 129 active+clean; 569 MiB data, 1021 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 10 MiB/s wr, 511 op/s 2023-12-11T17:49:31.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:49:30 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:49:31.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:49:30 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:49:31.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:49:30 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:49:31.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:49:30 smithi181 ceph-mon[103627]: pgmap v101: 129 pgs: 129 active+clean; 569 MiB data, 1021 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 10 MiB/s wr, 511 op/s 2023-12-11T17:49:31.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:49:30 smithi181 ceph-mon[103627]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:49:31.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:49:30 smithi181 ceph-mon[103627]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:49:31.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:49:30 smithi181 ceph-mon[103627]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:49:31.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:49:30 smithi059 ceph-mon[98847]: pgmap v101: 129 pgs: 129 active+clean; 569 MiB data, 1021 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 10 MiB/s wr, 511 op/s 2023-12-11T17:49:31.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:49:30 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:49:31.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:49:30 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:49:31.155 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:49:30 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:49:33.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:49:32 smithi115 ceph-mon[102748]: pgmap v102: 129 pgs: 129 active+clean; 569 MiB data, 1021 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 98 op/s 2023-12-11T17:49:33.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:49:32 smithi181 ceph-mon[103627]: pgmap v102: 129 pgs: 129 active+clean; 569 MiB data, 1021 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 98 op/s 2023-12-11T17:49:33.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:49:32 smithi059 ceph-mon[98847]: pgmap v102: 129 pgs: 129 active+clean; 569 MiB data, 1021 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 98 op/s 2023-12-11T17:49:35.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:49:35 smithi181 ceph-mon[103627]: pgmap v103: 129 pgs: 129 active+clean; 576 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.0 MiB/s wr, 139 op/s 2023-12-11T17:49:35.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:49:35 smithi059 ceph-mon[98847]: pgmap v103: 129 pgs: 129 active+clean; 576 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.0 MiB/s wr, 139 op/s 2023-12-11T17:49:36.032 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:49:35 smithi115 ceph-mon[102748]: pgmap v103: 129 pgs: 129 active+clean; 576 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.0 MiB/s wr, 139 op/s 2023-12-11T17:49:37.202 INFO:tasks.workunit.client.0.smithi059.stdout: 153 2193190 774 1594394 568 684357 947 2023-12-11T17:49:37.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:49:37 smithi181 ceph-mon[103627]: pgmap v104: 129 pgs: 129 active+clean; 577 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 150 op/s 2023-12-11T17:49:37.911 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:49:37 smithi059 ceph-mon[98847]: pgmap v104: 129 pgs: 129 active+clean; 577 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 150 op/s 2023-12-11T17:49:38.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:49:37 smithi115 ceph-mon[102748]: pgmap v104: 129 pgs: 129 active+clean; 577 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 150 op/s 2023-12-11T17:49:40.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:49:39 smithi115 ceph-mon[102748]: pgmap v105: 129 pgs: 129 active+clean; 577 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 134 op/s 2023-12-11T17:49:40.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:49:39 smithi181 ceph-mon[103627]: pgmap v105: 129 pgs: 129 active+clean; 577 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 134 op/s 2023-12-11T17:49:40.157 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:49:39 smithi059 ceph-mon[98847]: pgmap v105: 129 pgs: 129 active+clean; 577 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 134 op/s 2023-12-11T17:49:42.032 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:49:41 smithi115 ceph-mon[102748]: pgmap v106: 129 pgs: 129 active+clean; 583 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.9 MiB/s wr, 265 op/s 2023-12-11T17:49:42.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:49:41 smithi181 ceph-mon[103627]: pgmap v106: 129 pgs: 129 active+clean; 583 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.9 MiB/s wr, 265 op/s 2023-12-11T17:49:42.158 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:49:41 smithi059 ceph-mon[98847]: pgmap v106: 129 pgs: 129 active+clean; 583 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.9 MiB/s wr, 265 op/s 2023-12-11T17:49:44.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:49:43 smithi115 ceph-mon[102748]: pgmap v107: 129 pgs: 129 active+clean; 583 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.4 MiB/s wr, 218 op/s 2023-12-11T17:49:44.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:49:43 smithi181 ceph-mon[103627]: pgmap v107: 129 pgs: 129 active+clean; 583 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.4 MiB/s wr, 218 op/s 2023-12-11T17:49:44.158 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:49:43 smithi059 ceph-mon[98847]: pgmap v107: 129 pgs: 129 active+clean; 583 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.4 MiB/s wr, 218 op/s 2023-12-11T17:49:46.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:49:45 smithi115 ceph-mon[102748]: pgmap v108: 129 pgs: 129 active+clean; 589 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.9 MiB/s wr, 238 op/s 2023-12-11T17:49:46.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:49:45 smithi181 ceph-mon[103627]: pgmap v108: 129 pgs: 129 active+clean; 589 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.9 MiB/s wr, 238 op/s 2023-12-11T17:49:46.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:49:45 smithi059 ceph-mon[98847]: pgmap v108: 129 pgs: 129 active+clean; 589 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.9 MiB/s wr, 238 op/s 2023-12-11T17:49:47.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:49:46 smithi181 ceph-mon[103627]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-12-11T17:49:47.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:49:46 smithi181 ceph-mon[103627]: pgmap v109: 129 pgs: 129 active+clean; 590 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.4 MiB/s wr, 232 op/s 2023-12-11T17:49:47.164 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:49:46 smithi059 ceph-mon[98847]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-12-11T17:49:47.164 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:49:46 smithi059 ceph-mon[98847]: pgmap v109: 129 pgs: 129 active+clean; 590 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.4 MiB/s wr, 232 op/s 2023-12-11T17:49:47.211 INFO:tasks.workunit.client.0.smithi059.stdout: 164 2296666 700 1653382 561 724105 735 2023-12-11T17:49:47.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:49:46 smithi115 ceph-mon[102748]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-12-11T17:49:47.284 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:49:46 smithi115 ceph-mon[102748]: pgmap v109: 129 pgs: 129 active+clean; 590 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.4 MiB/s wr, 232 op/s 2023-12-11T17:49:49.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:49:48 smithi115 ceph-mon[102748]: pgmap v110: 129 pgs: 129 active+clean; 590 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.3 MiB/s wr, 185 op/s 2023-12-11T17:49:49.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:49:48 smithi181 ceph-mon[103627]: pgmap v110: 129 pgs: 129 active+clean; 590 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.3 MiB/s wr, 185 op/s 2023-12-11T17:49:49.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:49:48 smithi059 ceph-mon[98847]: pgmap v110: 129 pgs: 129 active+clean; 590 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.3 MiB/s wr, 185 op/s 2023-12-11T17:49:51.896 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:49:51 smithi181 ceph-mon[103627]: pgmap v111: 129 pgs: 129 active+clean; 708 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 11 MiB/s wr, 741 op/s 2023-12-11T17:49:51.911 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:49:51 smithi059 ceph-mon[98847]: pgmap v111: 129 pgs: 129 active+clean; 708 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 11 MiB/s wr, 741 op/s 2023-12-11T17:49:52.032 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:49:51 smithi115 ceph-mon[102748]: pgmap v111: 129 pgs: 129 active+clean; 708 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 11 MiB/s wr, 741 op/s 2023-12-11T17:49:53.895 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:49:53 smithi181 ceph-mon[103627]: pgmap v112: 129 pgs: 129 active+clean; 708 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 11 MiB/s wr, 611 op/s 2023-12-11T17:49:53.917 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:49:53 smithi059 ceph-mon[98847]: pgmap v112: 129 pgs: 129 active+clean; 708 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 11 MiB/s wr, 611 op/s 2023-12-11T17:49:54.032 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:49:53 smithi115 ceph-mon[102748]: pgmap v112: 129 pgs: 129 active+clean; 708 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 11 MiB/s wr, 611 op/s 2023-12-11T17:49:56.032 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:49:55 smithi115 ceph-mon[102748]: pgmap v113: 129 pgs: 129 active+clean; 720 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 12 MiB/s wr, 666 op/s 2023-12-11T17:49:56.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:49:55 smithi181 ceph-mon[103627]: pgmap v113: 129 pgs: 129 active+clean; 720 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 12 MiB/s wr, 666 op/s 2023-12-11T17:49:56.157 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:49:55 smithi059 ceph-mon[98847]: pgmap v113: 129 pgs: 129 active+clean; 720 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 12 MiB/s wr, 666 op/s 2023-12-11T17:49:57.210 INFO:tasks.workunit.client.0.smithi059.stdout: 171 1410202 523 1015286 416 442898 523 2023-12-11T17:49:58.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:49:57 smithi115 ceph-mon[102748]: pgmap v114: 129 pgs: 129 active+clean; 740 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 13 MiB/s wr, 750 op/s 2023-12-11T17:49:58.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:49:57 smithi181 ceph-mon[103627]: pgmap v114: 129 pgs: 129 active+clean; 740 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 13 MiB/s wr, 750 op/s 2023-12-11T17:49:58.156 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:49:57 smithi059 ceph-mon[98847]: pgmap v114: 129 pgs: 129 active+clean; 740 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 13 MiB/s wr, 750 op/s 2023-12-11T17:50:00.032 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:49:59 smithi115 ceph-mon[102748]: pgmap v115: 129 pgs: 129 active+clean; 740 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 13 MiB/s wr, 716 op/s 2023-12-11T17:50:00.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:49:59 smithi181 ceph-mon[103627]: pgmap v115: 129 pgs: 129 active+clean; 740 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 13 MiB/s wr, 716 op/s 2023-12-11T17:50:00.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:49:59 smithi059 ceph-mon[98847]: pgmap v115: 129 pgs: 129 active+clean; 740 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 13 MiB/s wr, 716 op/s 2023-12-11T17:50:01.032 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:50:00 smithi115 ceph-mon[102748]: overall HEALTH_OK 2023-12-11T17:50:01.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:50:00 smithi181 ceph-mon[103627]: overall HEALTH_OK 2023-12-11T17:50:01.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:50:00 smithi059 ceph-mon[98847]: overall HEALTH_OK 2023-12-11T17:50:02.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:50:01 smithi115 ceph-mon[102748]: pgmap v116: 129 pgs: 129 active+clean; 746 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 14 MiB/s wr, 800 op/s 2023-12-11T17:50:02.146 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:50:01 smithi181 ceph-mon[103627]: pgmap v116: 129 pgs: 129 active+clean; 746 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 14 MiB/s wr, 800 op/s 2023-12-11T17:50:02.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:50:01 smithi059 ceph-mon[98847]: pgmap v116: 129 pgs: 129 active+clean; 746 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 14 MiB/s wr, 800 op/s 2023-12-11T17:50:03.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:50:02 smithi115 ceph-mon[102748]: pgmap v117: 129 pgs: 129 active+clean; 746 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.6 MiB/s wr, 243 op/s 2023-12-11T17:50:03.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:50:02 smithi181 ceph-mon[103627]: pgmap v117: 129 pgs: 129 active+clean; 746 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.6 MiB/s wr, 243 op/s 2023-12-11T17:50:03.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:50:02 smithi059 ceph-mon[98847]: pgmap v117: 129 pgs: 129 active+clean; 746 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.6 MiB/s wr, 243 op/s 2023-12-11T17:50:06.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:50:05 smithi115 ceph-mon[102748]: pgmap v118: 129 pgs: 129 active+clean; 750 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.1 MiB/s wr, 267 op/s 2023-12-11T17:50:06.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:50:05 smithi181 ceph-mon[103627]: pgmap v118: 129 pgs: 129 active+clean; 750 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.1 MiB/s wr, 267 op/s 2023-12-11T17:50:06.163 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:50:05 smithi059 ceph-mon[98847]: pgmap v118: 129 pgs: 129 active+clean; 750 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.1 MiB/s wr, 267 op/s 2023-12-11T17:50:07.210 INFO:tasks.workunit.client.0.smithi059.stdout: 184 1596229 738 1145207 785 508362 634 2023-12-11T17:50:08.032 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:50:07 smithi115 ceph-mon[102748]: pgmap v119: 129 pgs: 129 active+clean; 752 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.0 MiB/s wr, 246 op/s 2023-12-11T17:50:08.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:50:07 smithi181 ceph-mon[103627]: pgmap v119: 129 pgs: 129 active+clean; 752 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.0 MiB/s wr, 246 op/s 2023-12-11T17:50:08.160 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:50:07 smithi059 ceph-mon[98847]: pgmap v119: 129 pgs: 129 active+clean; 752 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.0 MiB/s wr, 246 op/s 2023-12-11T17:50:10.032 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:50:09 smithi115 ceph-mon[102748]: pgmap v120: 129 pgs: 129 active+clean; 752 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.2 MiB/s wr, 141 op/s 2023-12-11T17:50:10.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:50:09 smithi181 ceph-mon[103627]: pgmap v120: 129 pgs: 129 active+clean; 752 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.2 MiB/s wr, 141 op/s 2023-12-11T17:50:10.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:50:09 smithi059 ceph-mon[98847]: pgmap v120: 129 pgs: 129 active+clean; 752 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.2 MiB/s wr, 141 op/s 2023-12-11T17:50:12.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:50:11 smithi115 ceph-mon[102748]: pgmap v121: 129 pgs: 129 active+clean; 760 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.0 MiB/s wr, 171 op/s 2023-12-11T17:50:12.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:50:11 smithi181 ceph-mon[103627]: pgmap v121: 129 pgs: 129 active+clean; 760 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.0 MiB/s wr, 171 op/s 2023-12-11T17:50:12.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:50:11 smithi059 ceph-mon[98847]: pgmap v121: 129 pgs: 129 active+clean; 760 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.0 MiB/s wr, 171 op/s 2023-12-11T17:50:14.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:50:13 smithi115 ceph-mon[102748]: pgmap v122: 129 pgs: 129 active+clean; 760 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 87 op/s 2023-12-11T17:50:14.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:50:13 smithi181 ceph-mon[103627]: pgmap v122: 129 pgs: 129 active+clean; 760 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 87 op/s 2023-12-11T17:50:14.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:50:13 smithi059 ceph-mon[98847]: pgmap v122: 129 pgs: 129 active+clean; 760 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 87 op/s 2023-12-11T17:50:16.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:50:15 smithi115 ceph-mon[102748]: pgmap v123: 129 pgs: 129 active+clean; 766 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 108 op/s 2023-12-11T17:50:16.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:50:15 smithi181 ceph-mon[103627]: pgmap v123: 129 pgs: 129 active+clean; 766 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 108 op/s 2023-12-11T17:50:16.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:50:15 smithi059 ceph-mon[98847]: pgmap v123: 129 pgs: 129 active+clean; 766 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 108 op/s 2023-12-11T17:50:17.214 INFO:tasks.workunit.client.0.smithi059.stdout: 218 1133712 1654 804956 1528 362034 1161 2023-12-11T17:50:18.032 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:50:17 smithi115 ceph-mon[102748]: pgmap v124: 129 pgs: 129 active+clean; 776 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.4 MiB/s wr, 106 op/s 2023-12-11T17:50:18.147 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:50:17 smithi181 ceph-mon[103627]: pgmap v124: 129 pgs: 129 active+clean; 776 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.4 MiB/s wr, 106 op/s 2023-12-11T17:50:18.155 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:50:17 smithi059 ceph-mon[98847]: pgmap v124: 129 pgs: 129 active+clean; 776 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.4 MiB/s wr, 106 op/s 2023-12-11T17:50:20.034 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:50:19 smithi115 ceph-mon[102748]: pgmap v125: 129 pgs: 129 active+clean; 776 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.2 MiB/s wr, 72 op/s 2023-12-11T17:50:20.146 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:50:19 smithi181 ceph-mon[103627]: pgmap v125: 129 pgs: 129 active+clean; 776 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.2 MiB/s wr, 72 op/s 2023-12-11T17:50:20.156 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:50:19 smithi059 ceph-mon[98847]: pgmap v125: 129 pgs: 129 active+clean; 776 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.2 MiB/s wr, 72 op/s 2023-12-11T17:50:22.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:50:21 smithi181 ceph-mon[103627]: pgmap v126: 129 pgs: 129 active+clean; 834 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 7.3 MiB/s wr, 327 op/s 2023-12-11T17:50:22.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:50:21 smithi059 ceph-mon[98847]: pgmap v126: 129 pgs: 129 active+clean; 834 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 7.3 MiB/s wr, 327 op/s 2023-12-11T17:50:22.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:50:21 smithi115 ceph-mon[102748]: pgmap v126: 129 pgs: 129 active+clean; 834 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 7.3 MiB/s wr, 327 op/s 2023-12-11T17:50:23.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:50:22 smithi181 ceph-mon[103627]: pgmap v127: 129 pgs: 129 active+clean; 838 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.9 MiB/s wr, 322 op/s 2023-12-11T17:50:23.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:50:22 smithi059 ceph-mon[98847]: pgmap v127: 129 pgs: 129 active+clean; 838 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.9 MiB/s wr, 322 op/s 2023-12-11T17:50:23.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:50:22 smithi115 ceph-mon[102748]: pgmap v127: 129 pgs: 129 active+clean; 838 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.9 MiB/s wr, 322 op/s 2023-12-11T17:50:26.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:50:25 smithi115 ceph-mon[102748]: pgmap v128: 129 pgs: 129 active+clean; 864 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 9.2 MiB/s wr, 391 op/s 2023-12-11T17:50:26.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:50:25 smithi181 ceph-mon[103627]: pgmap v128: 129 pgs: 129 active+clean; 864 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 9.2 MiB/s wr, 391 op/s 2023-12-11T17:50:26.167 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:50:25 smithi059 ceph-mon[98847]: pgmap v128: 129 pgs: 129 active+clean; 864 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 9.2 MiB/s wr, 391 op/s 2023-12-11T17:50:27.210 INFO:tasks.workunit.client.0.smithi059.stdout: 224 1437791 407 1004299 391 431901 199 2023-12-11T17:50:28.032 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:50:27 smithi115 ceph-mon[102748]: pgmap v129: 129 pgs: 129 active+clean; 972 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 18 MiB/s wr, 810 op/s 2023-12-11T17:50:28.032 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:50:27 smithi115 ceph-mon[102748]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-12-11T17:50:28.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:50:27 smithi181 ceph-mon[103627]: pgmap v129: 129 pgs: 129 active+clean; 972 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 18 MiB/s wr, 810 op/s 2023-12-11T17:50:28.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:50:27 smithi181 ceph-mon[103627]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-12-11T17:50:28.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:50:27 smithi059 ceph-mon[98847]: pgmap v129: 129 pgs: 129 active+clean; 972 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 18 MiB/s wr, 810 op/s 2023-12-11T17:50:28.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:50:27 smithi059 ceph-mon[98847]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-12-11T17:50:30.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:50:29 smithi115 ceph-mon[102748]: pgmap v130: 129 pgs: 129 active+clean; 972 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 17 MiB/s wr, 788 op/s 2023-12-11T17:50:30.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:50:29 smithi181 ceph-mon[103627]: pgmap v130: 129 pgs: 129 active+clean; 972 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 17 MiB/s wr, 788 op/s 2023-12-11T17:50:30.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:50:29 smithi059 ceph-mon[98847]: pgmap v130: 129 pgs: 129 active+clean; 972 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 17 MiB/s wr, 788 op/s 2023-12-11T17:50:31.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:50:30 smithi181 ceph-mon[103627]: pgmap v131: 129 pgs: 129 active+clean; 979 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 18 MiB/s wr, 859 op/s 2023-12-11T17:50:31.158 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:50:30 smithi059 ceph-mon[98847]: pgmap v131: 129 pgs: 129 active+clean; 979 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 18 MiB/s wr, 859 op/s 2023-12-11T17:50:31.285 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:50:30 smithi115 ceph-mon[102748]: pgmap v131: 129 pgs: 129 active+clean; 979 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 18 MiB/s wr, 859 op/s 2023-12-11T17:50:32.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:50:31 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:50:32.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:50:31 smithi181 ceph-mon[103627]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:50:32.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:50:31 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:50:33.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:50:32 smithi115 ceph-mon[102748]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-12-11T17:50:33.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:50:32 smithi115 ceph-mon[102748]: pgmap v132: 129 pgs: 129 active+clean; 979 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 13 MiB/s wr, 606 op/s 2023-12-11T17:50:33.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:50:32 smithi181 ceph-mon[103627]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-12-11T17:50:33.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:50:32 smithi181 ceph-mon[103627]: pgmap v132: 129 pgs: 129 active+clean; 979 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 13 MiB/s wr, 606 op/s 2023-12-11T17:50:33.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:50:32 smithi059 ceph-mon[98847]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-12-11T17:50:33.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:50:32 smithi059 ceph-mon[98847]: pgmap v132: 129 pgs: 129 active+clean; 979 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 13 MiB/s wr, 606 op/s 2023-12-11T17:50:36.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:50:35 smithi115 ceph-mon[102748]: pgmap v133: 129 pgs: 129 active+clean; 985 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 13 MiB/s wr, 598 op/s 2023-12-11T17:50:36.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:50:35 smithi181 ceph-mon[103627]: pgmap v133: 129 pgs: 129 active+clean; 985 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 13 MiB/s wr, 598 op/s 2023-12-11T17:50:36.159 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:50:35 smithi059 ceph-mon[98847]: pgmap v133: 129 pgs: 129 active+clean; 985 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 13 MiB/s wr, 598 op/s 2023-12-11T17:50:37.217 INFO:tasks.workunit.client.0.smithi059.stdout: 238 2032508 857 1397632 570 608963 1049 2023-12-11T17:50:38.032 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:50:37 smithi115 ceph-mon[102748]: pgmap v134: 129 pgs: 129 active+clean; 989 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 11 MiB/s wr, 563 op/s 2023-12-11T17:50:38.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:50:37 smithi181 ceph-mon[103627]: pgmap v134: 129 pgs: 129 active+clean; 989 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 11 MiB/s wr, 563 op/s 2023-12-11T17:50:38.157 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:50:37 smithi059 ceph-mon[98847]: pgmap v134: 129 pgs: 129 active+clean; 989 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 11 MiB/s wr, 563 op/s 2023-12-11T17:50:40.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:50:39 smithi115 ceph-mon[102748]: pgmap v135: 129 pgs: 129 active+clean; 989 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 123 op/s 2023-12-11T17:50:40.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:50:39 smithi181 ceph-mon[103627]: pgmap v135: 129 pgs: 129 active+clean; 989 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 123 op/s 2023-12-11T17:50:40.158 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:50:39 smithi059 ceph-mon[98847]: pgmap v135: 129 pgs: 129 active+clean; 989 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 123 op/s 2023-12-11T17:50:41.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:50:40 smithi181 ceph-mon[103627]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:50:41.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:50:40 smithi181 ceph-mon[103627]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:50:41.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:50:40 smithi181 ceph-mon[103627]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:50:41.155 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:50:40 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:50:41.155 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:50:40 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:50:41.155 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:50:40 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:50:41.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:50:40 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:50:41.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:50:40 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:50:41.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:50:40 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:50:42.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:50:41 smithi181 ceph-mon[103627]: pgmap v136: 129 pgs: 129 active+clean; 998 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.6 MiB/s wr, 209 op/s 2023-12-11T17:50:42.157 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:50:41 smithi059 ceph-mon[98847]: pgmap v136: 129 pgs: 129 active+clean; 998 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.6 MiB/s wr, 209 op/s 2023-12-11T17:50:42.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:50:41 smithi115 ceph-mon[102748]: pgmap v136: 129 pgs: 129 active+clean; 998 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.6 MiB/s wr, 209 op/s 2023-12-11T17:50:43.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:50:42 smithi181 ceph-mon[103627]: pgmap v137: 129 pgs: 129 active+clean; 1000 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 154 op/s 2023-12-11T17:50:43.158 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:50:42 smithi059 ceph-mon[98847]: pgmap v137: 129 pgs: 129 active+clean; 1000 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 154 op/s 2023-12-11T17:50:43.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:50:42 smithi115 ceph-mon[102748]: pgmap v137: 129 pgs: 129 active+clean; 1000 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 154 op/s 2023-12-11T17:50:46.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:50:45 smithi115 ceph-mon[102748]: pgmap v138: 129 pgs: 129 active+clean; 1004 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 164 op/s 2023-12-11T17:50:46.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:50:45 smithi181 ceph-mon[103627]: pgmap v138: 129 pgs: 129 active+clean; 1004 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 164 op/s 2023-12-11T17:50:46.158 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:50:45 smithi059 ceph-mon[98847]: pgmap v138: 129 pgs: 129 active+clean; 1004 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 164 op/s 2023-12-11T17:50:47.218 INFO:tasks.workunit.client.0.smithi059.stdout: 245 2322014 528 1597160 473 728975 1077 2023-12-11T17:50:48.032 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:50:47 smithi115 ceph-mon[102748]: pgmap v139: 129 pgs: 129 active+clean; 1008 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 219 op/s 2023-12-11T17:50:48.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:50:47 smithi181 ceph-mon[103627]: pgmap v139: 129 pgs: 129 active+clean; 1008 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 219 op/s 2023-12-11T17:50:48.155 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:50:47 smithi059 ceph-mon[98847]: pgmap v139: 129 pgs: 129 active+clean; 1008 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 219 op/s 2023-12-11T17:50:50.034 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:50:49 smithi115 ceph-mon[102748]: pgmap v140: 129 pgs: 129 active+clean; 1008 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 186 op/s 2023-12-11T17:50:50.149 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:50:49 smithi181 ceph-mon[103627]: pgmap v140: 129 pgs: 129 active+clean; 1008 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 186 op/s 2023-12-11T17:50:50.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:50:49 smithi059 ceph-mon[98847]: pgmap v140: 129 pgs: 129 active+clean; 1008 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 186 op/s 2023-12-11T17:50:52.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:50:51 smithi181 ceph-mon[103627]: pgmap v141: 129 pgs: 129 active+clean; 1011 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.2 MiB/s wr, 261 op/s 2023-12-11T17:50:52.162 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:50:51 smithi059 ceph-mon[98847]: pgmap v141: 129 pgs: 129 active+clean; 1011 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.2 MiB/s wr, 261 op/s 2023-12-11T17:50:52.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:50:51 smithi115 ceph-mon[102748]: pgmap v141: 129 pgs: 129 active+clean; 1011 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.2 MiB/s wr, 261 op/s 2023-12-11T17:50:54.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:50:53 smithi181 ceph-mon[103627]: pgmap v142: 129 pgs: 129 active+clean; 1022 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.3 MiB/s wr, 249 op/s 2023-12-11T17:50:54.157 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:50:53 smithi059 ceph-mon[98847]: pgmap v142: 129 pgs: 129 active+clean; 1022 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.3 MiB/s wr, 249 op/s 2023-12-11T17:50:54.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:50:53 smithi115 ceph-mon[102748]: pgmap v142: 129 pgs: 129 active+clean; 1022 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.3 MiB/s wr, 249 op/s 2023-12-11T17:50:56.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:50:55 smithi115 ceph-mon[102748]: pgmap v143: 129 pgs: 129 active+clean; 1.0 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.8 MiB/s wr, 318 op/s 2023-12-11T17:50:56.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:50:55 smithi181 ceph-mon[103627]: pgmap v143: 129 pgs: 129 active+clean; 1.0 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.8 MiB/s wr, 318 op/s 2023-12-11T17:50:56.161 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:50:55 smithi059 ceph-mon[98847]: pgmap v143: 129 pgs: 129 active+clean; 1.0 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.8 MiB/s wr, 318 op/s 2023-12-11T17:50:57.237 INFO:tasks.workunit.client.0.smithi059.stdout: 253 1829709 541 1251898 306 601484 756 2023-12-11T17:50:58.032 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:50:57 smithi115 ceph-mon[102748]: pgmap v144: 129 pgs: 129 active+clean; 1.1 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.4 MiB/s wr, 484 op/s 2023-12-11T17:50:58.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:50:57 smithi181 ceph-mon[103627]: pgmap v144: 129 pgs: 129 active+clean; 1.1 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.4 MiB/s wr, 484 op/s 2023-12-11T17:50:58.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:50:57 smithi059 ceph-mon[98847]: pgmap v144: 129 pgs: 129 active+clean; 1.1 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.4 MiB/s wr, 484 op/s 2023-12-11T17:51:00.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:50:59 smithi115 ceph-mon[102748]: pgmap v145: 129 pgs: 129 active+clean; 1.1 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 8.1 MiB/s wr, 535 op/s 2023-12-11T17:51:00.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:50:59 smithi181 ceph-mon[103627]: pgmap v145: 129 pgs: 129 active+clean; 1.1 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 8.1 MiB/s wr, 535 op/s 2023-12-11T17:51:00.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:50:59 smithi059 ceph-mon[98847]: pgmap v145: 129 pgs: 129 active+clean; 1.1 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 8.1 MiB/s wr, 535 op/s 2023-12-11T17:51:02.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:51:01 smithi115 ceph-mon[102748]: pgmap v146: 129 pgs: 129 active+clean; 1.1 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 10 MiB/s wr, 651 op/s 2023-12-11T17:51:02.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:51:01 smithi181 ceph-mon[103627]: pgmap v146: 129 pgs: 129 active+clean; 1.1 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 10 MiB/s wr, 651 op/s 2023-12-11T17:51:02.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:51:01 smithi059 ceph-mon[98847]: pgmap v146: 129 pgs: 129 active+clean; 1.1 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 10 MiB/s wr, 651 op/s 2023-12-11T17:51:04.032 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:51:03 smithi115 ceph-mon[102748]: pgmap v147: 129 pgs: 129 active+clean; 1.1 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 11 MiB/s wr, 628 op/s 2023-12-11T17:51:04.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:51:03 smithi181 ceph-mon[103627]: pgmap v147: 129 pgs: 129 active+clean; 1.1 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 11 MiB/s wr, 628 op/s 2023-12-11T17:51:04.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:51:03 smithi059 ceph-mon[98847]: pgmap v147: 129 pgs: 129 active+clean; 1.1 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 11 MiB/s wr, 628 op/s 2023-12-11T17:51:06.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:51:05 smithi181 ceph-mon[103627]: pgmap v148: 129 pgs: 129 active+clean; 1.1 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 10 MiB/s wr, 596 op/s 2023-12-11T17:51:06.158 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:51:05 smithi059 ceph-mon[98847]: pgmap v148: 129 pgs: 129 active+clean; 1.1 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 10 MiB/s wr, 596 op/s 2023-12-11T17:51:06.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:51:05 smithi115 ceph-mon[102748]: pgmap v148: 129 pgs: 129 active+clean; 1.1 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 10 MiB/s wr, 596 op/s 2023-12-11T17:51:07.242 INFO:tasks.workunit.client.0.smithi059.stdout: 266 1038025 949 702405 560 337070 755 2023-12-11T17:51:08.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:51:07 smithi181 ceph-mon[103627]: pgmap v149: 129 pgs: 129 active+clean; 1.1 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 8.9 MiB/s wr, 524 op/s 2023-12-11T17:51:08.159 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:51:07 smithi059 ceph-mon[98847]: pgmap v149: 129 pgs: 129 active+clean; 1.1 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 8.9 MiB/s wr, 524 op/s 2023-12-11T17:51:08.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:51:07 smithi115 ceph-mon[102748]: pgmap v149: 129 pgs: 129 active+clean; 1.1 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 8.9 MiB/s wr, 524 op/s 2023-12-11T17:51:10.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:51:09 smithi181 ceph-mon[103627]: pgmap v150: 129 pgs: 129 active+clean; 1.1 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.0 MiB/s wr, 373 op/s 2023-12-11T17:51:10.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:51:09 smithi059 ceph-mon[98847]: pgmap v150: 129 pgs: 129 active+clean; 1.1 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.0 MiB/s wr, 373 op/s 2023-12-11T17:51:10.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:51:09 smithi115 ceph-mon[102748]: pgmap v150: 129 pgs: 129 active+clean; 1.1 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.0 MiB/s wr, 373 op/s 2023-12-11T17:51:12.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:51:11 smithi181 ceph-mon[103627]: pgmap v151: 129 pgs: 129 active+clean; 1.1 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.6 MiB/s wr, 303 op/s 2023-12-11T17:51:12.155 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:51:11 smithi059 ceph-mon[98847]: pgmap v151: 129 pgs: 129 active+clean; 1.1 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.6 MiB/s wr, 303 op/s 2023-12-11T17:51:12.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:51:11 smithi115 ceph-mon[102748]: pgmap v151: 129 pgs: 129 active+clean; 1.1 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.6 MiB/s wr, 303 op/s 2023-12-11T17:51:13.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:51:12 smithi181 ceph-mon[103627]: pgmap v152: 129 pgs: 129 active+clean; 1.1 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.6 MiB/s wr, 195 op/s 2023-12-11T17:51:13.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:51:12 smithi059 ceph-mon[98847]: pgmap v152: 129 pgs: 129 active+clean; 1.1 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.6 MiB/s wr, 195 op/s 2023-12-11T17:51:13.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:51:12 smithi115 ceph-mon[102748]: pgmap v152: 129 pgs: 129 active+clean; 1.1 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.6 MiB/s wr, 195 op/s 2023-12-11T17:51:16.032 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:51:15 smithi115 ceph-mon[102748]: pgmap v153: 129 pgs: 129 active+clean; 1.1 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 165 op/s 2023-12-11T17:51:16.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:51:15 smithi181 ceph-mon[103627]: pgmap v153: 129 pgs: 129 active+clean; 1.1 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 165 op/s 2023-12-11T17:51:16.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:51:15 smithi059 ceph-mon[98847]: pgmap v153: 129 pgs: 129 active+clean; 1.1 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 165 op/s 2023-12-11T17:51:17.249 INFO:tasks.workunit.client.0.smithi059.stdout: 284 2005933 1053 1347422 821 666301 1161 2023-12-11T17:51:18.032 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:51:17 smithi115 ceph-mon[102748]: pgmap v154: 129 pgs: 129 active+clean; 1.1 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 127 op/s 2023-12-11T17:51:18.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:51:17 smithi181 ceph-mon[103627]: pgmap v154: 129 pgs: 129 active+clean; 1.1 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 127 op/s 2023-12-11T17:51:18.157 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:51:17 smithi059 ceph-mon[98847]: pgmap v154: 129 pgs: 129 active+clean; 1.1 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 127 op/s 2023-12-11T17:51:20.032 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:51:19 smithi115 ceph-mon[102748]: pgmap v155: 129 pgs: 129 active+clean; 1.1 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 129 op/s 2023-12-11T17:51:20.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:51:19 smithi181 ceph-mon[103627]: pgmap v155: 129 pgs: 129 active+clean; 1.1 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 129 op/s 2023-12-11T17:51:20.157 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:51:19 smithi059 ceph-mon[98847]: pgmap v155: 129 pgs: 129 active+clean; 1.1 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 129 op/s 2023-12-11T17:51:22.148 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:51:21 smithi181 ceph-mon[103627]: pgmap v156: 129 pgs: 129 active+clean; 1.1 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.5 MiB/s wr, 141 op/s 2023-12-11T17:51:22.159 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:51:21 smithi059 ceph-mon[98847]: pgmap v156: 129 pgs: 129 active+clean; 1.1 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.5 MiB/s wr, 141 op/s 2023-12-11T17:51:22.284 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:51:21 smithi115 ceph-mon[102748]: pgmap v156: 129 pgs: 129 active+clean; 1.1 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.5 MiB/s wr, 141 op/s 2023-12-11T17:51:24.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:51:23 smithi181 ceph-mon[103627]: pgmap v157: 129 pgs: 129 active+clean; 1.1 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 99 op/s 2023-12-11T17:51:24.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:51:23 smithi059 ceph-mon[98847]: pgmap v157: 129 pgs: 129 active+clean; 1.1 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 99 op/s 2023-12-11T17:51:24.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:51:23 smithi115 ceph-mon[102748]: pgmap v157: 129 pgs: 129 active+clean; 1.1 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 99 op/s 2023-12-11T17:51:26.165 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:51:25 smithi059 ceph-mon[98847]: pgmap v158: 129 pgs: 129 active+clean; 1.2 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 8.5 MiB/s wr, 431 op/s 2023-12-11T17:51:26.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:51:25 smithi115 ceph-mon[102748]: pgmap v158: 129 pgs: 129 active+clean; 1.2 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 8.5 MiB/s wr, 431 op/s 2023-12-11T17:51:26.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:51:25 smithi181 ceph-mon[103627]: pgmap v158: 129 pgs: 129 active+clean; 1.2 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 8.5 MiB/s wr, 431 op/s 2023-12-11T17:51:27.264 INFO:tasks.workunit.client.0.smithi059.stdout: 289 1629999 368 1080121 323 551443 593 2023-12-11T17:51:27.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:51:26 smithi115 ceph-mon[102748]: pgmap v159: 129 pgs: 129 active+clean; 1.3 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 11 MiB/s wr, 544 op/s 2023-12-11T17:51:27.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:51:26 smithi181 ceph-mon[103627]: pgmap v159: 129 pgs: 129 active+clean; 1.3 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 11 MiB/s wr, 544 op/s 2023-12-11T17:51:27.409 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:51:26 smithi059 ceph-mon[98847]: pgmap v159: 129 pgs: 129 active+clean; 1.3 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 11 MiB/s wr, 544 op/s 2023-12-11T17:51:29.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:51:29 smithi181 ceph-mon[103627]: pgmap v160: 129 pgs: 129 active+clean; 1.3 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 14 MiB/s wr, 698 op/s 2023-12-11T17:51:29.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:51:29 smithi059 ceph-mon[98847]: pgmap v160: 129 pgs: 129 active+clean; 1.3 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 14 MiB/s wr, 698 op/s 2023-12-11T17:51:29.532 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:51:29 smithi115 ceph-mon[102748]: pgmap v160: 129 pgs: 129 active+clean; 1.3 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 14 MiB/s wr, 698 op/s 2023-12-11T17:51:32.032 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:51:31 smithi115 ceph-mon[102748]: pgmap v161: 129 pgs: 129 active+clean; 1.3 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 14 MiB/s wr, 746 op/s 2023-12-11T17:51:32.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:51:31 smithi181 ceph-mon[103627]: pgmap v161: 129 pgs: 129 active+clean; 1.3 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 14 MiB/s wr, 746 op/s 2023-12-11T17:51:32.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:51:31 smithi059 ceph-mon[98847]: pgmap v161: 129 pgs: 129 active+clean; 1.3 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 14 MiB/s wr, 746 op/s 2023-12-11T17:51:34.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:51:33 smithi181 ceph-mon[103627]: pgmap v162: 129 pgs: 129 active+clean; 1.3 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 14 MiB/s wr, 707 op/s 2023-12-11T17:51:34.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:51:33 smithi059 ceph-mon[98847]: pgmap v162: 129 pgs: 129 active+clean; 1.3 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 14 MiB/s wr, 707 op/s 2023-12-11T17:51:34.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:51:33 smithi115 ceph-mon[102748]: pgmap v162: 129 pgs: 129 active+clean; 1.3 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 14 MiB/s wr, 707 op/s 2023-12-11T17:51:36.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:51:35 smithi181 ceph-mon[103627]: pgmap v163: 129 pgs: 129 active+clean; 1.3 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 14 MiB/s wr, 729 op/s 2023-12-11T17:51:36.163 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:51:35 smithi059 ceph-mon[98847]: pgmap v163: 129 pgs: 129 active+clean; 1.3 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 14 MiB/s wr, 729 op/s 2023-12-11T17:51:36.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:51:35 smithi115 ceph-mon[102748]: pgmap v163: 129 pgs: 129 active+clean; 1.3 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 14 MiB/s wr, 729 op/s 2023-12-11T17:51:37.264 INFO:tasks.workunit.client.0.smithi059.stdout: 297 2413480 566 1596898 440 830932 841 2023-12-11T17:51:38.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:51:37 smithi181 ceph-mon[103627]: pgmap v164: 129 pgs: 129 active+clean; 1.3 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 7.7 MiB/s wr, 394 op/s 2023-12-11T17:51:38.157 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:51:37 smithi059 ceph-mon[98847]: pgmap v164: 129 pgs: 129 active+clean; 1.3 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 7.7 MiB/s wr, 394 op/s 2023-12-11T17:51:38.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:51:37 smithi115 ceph-mon[102748]: pgmap v164: 129 pgs: 129 active+clean; 1.3 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 7.7 MiB/s wr, 394 op/s 2023-12-11T17:51:40.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:51:39 smithi115 ceph-mon[102748]: pgmap v165: 129 pgs: 129 active+clean; 1.3 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.0 MiB/s wr, 274 op/s 2023-12-11T17:51:40.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:51:39 smithi181 ceph-mon[103627]: pgmap v165: 129 pgs: 129 active+clean; 1.3 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.0 MiB/s wr, 274 op/s 2023-12-11T17:51:40.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:51:39 smithi059 ceph-mon[98847]: pgmap v165: 129 pgs: 129 active+clean; 1.3 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.0 MiB/s wr, 274 op/s 2023-12-11T17:51:41.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:51:40 smithi115 ceph-mon[102748]: pgmap v166: 129 pgs: 129 active+clean; 1.3 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.3 MiB/s wr, 143 op/s 2023-12-11T17:51:41.394 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:51:40 smithi181 ceph-mon[103627]: pgmap v166: 129 pgs: 129 active+clean; 1.3 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.3 MiB/s wr, 143 op/s 2023-12-11T17:51:41.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:51:40 smithi059 ceph-mon[98847]: pgmap v166: 129 pgs: 129 active+clean; 1.3 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.3 MiB/s wr, 143 op/s 2023-12-11T17:51:42.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:51:42 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:51:42.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:51:42 smithi181 ceph-mon[103627]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:51:42.411 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:51:42 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:51:43.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:51:43 smithi181 ceph-mon[103627]: pgmap v167: 129 pgs: 129 active+clean; 1.3 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 87 op/s 2023-12-11T17:51:43.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:51:43 smithi059 ceph-mon[98847]: pgmap v167: 129 pgs: 129 active+clean; 1.3 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 87 op/s 2023-12-11T17:51:43.532 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:51:43 smithi115 ceph-mon[102748]: pgmap v167: 129 pgs: 129 active+clean; 1.3 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 87 op/s 2023-12-11T17:51:46.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:51:45 smithi181 ceph-mon[103627]: pgmap v168: 129 pgs: 129 active+clean; 1.3 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.0 MiB/s wr, 163 op/s 2023-12-11T17:51:46.155 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:51:45 smithi059 ceph-mon[98847]: pgmap v168: 129 pgs: 129 active+clean; 1.3 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.0 MiB/s wr, 163 op/s 2023-12-11T17:51:46.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:51:45 smithi115 ceph-mon[102748]: pgmap v168: 129 pgs: 129 active+clean; 1.3 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.0 MiB/s wr, 163 op/s 2023-12-11T17:51:47.272 INFO:tasks.workunit.client.0.smithi059.stdout: 306 2312801 456 1526832 412 812948 1132 2023-12-11T17:51:48.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:51:47 smithi181 ceph-mon[103627]: pgmap v169: 129 pgs: 129 active+clean; 1.3 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.6 MiB/s wr, 153 op/s 2023-12-11T17:51:48.155 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:51:47 smithi059 ceph-mon[98847]: pgmap v169: 129 pgs: 129 active+clean; 1.3 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.6 MiB/s wr, 153 op/s 2023-12-11T17:51:48.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:51:47 smithi115 ceph-mon[102748]: pgmap v169: 129 pgs: 129 active+clean; 1.3 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.6 MiB/s wr, 153 op/s 2023-12-11T17:51:50.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:51:49 smithi115 ceph-mon[102748]: pgmap v170: 129 pgs: 129 active+clean; 1.3 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.6 MiB/s wr, 206 op/s 2023-12-11T17:51:50.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:51:49 smithi181 ceph-mon[103627]: pgmap v170: 129 pgs: 129 active+clean; 1.3 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.6 MiB/s wr, 206 op/s 2023-12-11T17:51:50.406 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:51:49 smithi059 ceph-mon[98847]: pgmap v170: 129 pgs: 129 active+clean; 1.3 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.6 MiB/s wr, 206 op/s 2023-12-11T17:51:52.285 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:51:51 smithi115 ceph-mon[102748]: pgmap v171: 129 pgs: 129 active+clean; 1.3 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.2 MiB/s wr, 303 op/s 2023-12-11T17:51:52.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:51:51 smithi181 ceph-mon[103627]: pgmap v171: 129 pgs: 129 active+clean; 1.3 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.2 MiB/s wr, 303 op/s 2023-12-11T17:51:52.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:51:51 smithi059 ceph-mon[98847]: pgmap v171: 129 pgs: 129 active+clean; 1.3 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.2 MiB/s wr, 303 op/s 2023-12-11T17:51:53.398 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:51:53 smithi181 ceph-mon[103627]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:51:53.398 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:51:53 smithi181 ceph-mon[103627]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:51:53.398 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:51:53 smithi181 ceph-mon[103627]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:51:53.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:51:53 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:51:53.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:51:53 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:51:53.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:51:53 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:51:53.535 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:51:53 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:51:53.535 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:51:53 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:51:53.535 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:51:53 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:51:54.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:51:54 smithi181 ceph-mon[103627]: pgmap v172: 129 pgs: 129 active+clean; 1.3 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.7 MiB/s wr, 288 op/s 2023-12-11T17:51:54.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:51:54 smithi059 ceph-mon[98847]: pgmap v172: 129 pgs: 129 active+clean; 1.3 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.7 MiB/s wr, 288 op/s 2023-12-11T17:51:54.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:51:54 smithi115 ceph-mon[102748]: pgmap v172: 129 pgs: 129 active+clean; 1.3 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.7 MiB/s wr, 288 op/s 2023-12-11T17:51:56.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:51:55 smithi181 ceph-mon[103627]: pgmap v173: 129 pgs: 129 active+clean; 1.4 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.3 MiB/s wr, 556 op/s 2023-12-11T17:51:56.168 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:51:55 smithi059 ceph-mon[98847]: pgmap v173: 129 pgs: 129 active+clean; 1.4 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.3 MiB/s wr, 556 op/s 2023-12-11T17:51:56.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:51:55 smithi115 ceph-mon[102748]: pgmap v173: 129 pgs: 129 active+clean; 1.4 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.3 MiB/s wr, 556 op/s 2023-12-11T17:51:57.286 INFO:tasks.workunit.client.0.smithi059.stdout: 310 1859366 310 1232116 351 672588 718 2023-12-11T17:51:58.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:51:57 smithi059 ceph-mon[98847]: pgmap v174: 129 pgs: 129 active+clean; 1.4 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 6.4 MiB/s wr, 609 op/s 2023-12-11T17:51:58.286 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:51:57 smithi115 ceph-mon[102748]: pgmap v174: 129 pgs: 129 active+clean; 1.4 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 6.4 MiB/s wr, 609 op/s 2023-12-11T17:51:58.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:51:57 smithi181 ceph-mon[103627]: pgmap v174: 129 pgs: 129 active+clean; 1.4 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 6.4 MiB/s wr, 609 op/s 2023-12-11T17:52:00.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:51:59 smithi115 ceph-mon[102748]: pgmap v175: 129 pgs: 129 active+clean; 1.4 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.6 MiB/s wr, 622 op/s 2023-12-11T17:52:00.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:51:59 smithi181 ceph-mon[103627]: pgmap v175: 129 pgs: 129 active+clean; 1.4 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.6 MiB/s wr, 622 op/s 2023-12-11T17:52:00.422 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:51:59 smithi059 ceph-mon[98847]: pgmap v175: 129 pgs: 129 active+clean; 1.4 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.6 MiB/s wr, 622 op/s 2023-12-11T17:52:02.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:52:01 smithi115 ceph-mon[102748]: pgmap v176: 129 pgs: 129 active+clean; 1.4 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 10 MiB/s wr, 849 op/s 2023-12-11T17:52:02.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:52:01 smithi181 ceph-mon[103627]: pgmap v176: 129 pgs: 129 active+clean; 1.4 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 10 MiB/s wr, 849 op/s 2023-12-11T17:52:02.406 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:52:01 smithi059 ceph-mon[98847]: pgmap v176: 129 pgs: 129 active+clean; 1.4 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 10 MiB/s wr, 849 op/s 2023-12-11T17:52:04.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:52:03 smithi115 ceph-mon[102748]: pgmap v177: 129 pgs: 129 active+clean; 1.4 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 9.5 MiB/s wr, 752 op/s 2023-12-11T17:52:04.394 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:52:03 smithi181 ceph-mon[103627]: pgmap v177: 129 pgs: 129 active+clean; 1.4 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 9.5 MiB/s wr, 752 op/s 2023-12-11T17:52:04.407 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:52:03 smithi059 ceph-mon[98847]: pgmap v177: 129 pgs: 129 active+clean; 1.4 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 9.5 MiB/s wr, 752 op/s 2023-12-11T17:52:06.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:52:05 smithi115 ceph-mon[102748]: pgmap v178: 129 pgs: 129 active+clean; 1.4 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 9.8 MiB/s wr, 798 op/s 2023-12-11T17:52:06.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:52:05 smithi181 ceph-mon[103627]: pgmap v178: 129 pgs: 129 active+clean; 1.4 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 9.8 MiB/s wr, 798 op/s 2023-12-11T17:52:06.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:52:05 smithi059 ceph-mon[98847]: pgmap v178: 129 pgs: 129 active+clean; 1.4 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 9.8 MiB/s wr, 798 op/s 2023-12-11T17:52:07.284 INFO:tasks.workunit.client.0.smithi059.stdout: 321 1655179 618 1106471 593 641618 1199 2023-12-11T17:52:08.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:52:07 smithi115 ceph-mon[102748]: pgmap v179: 129 pgs: 129 active+clean; 1.4 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 6.6 MiB/s wr, 576 op/s 2023-12-11T17:52:08.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:52:07 smithi181 ceph-mon[103627]: pgmap v179: 129 pgs: 129 active+clean; 1.4 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 6.6 MiB/s wr, 576 op/s 2023-12-11T17:52:08.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:52:07 smithi059 ceph-mon[98847]: pgmap v179: 129 pgs: 129 active+clean; 1.4 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 6.6 MiB/s wr, 576 op/s 2023-12-11T17:52:10.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:52:09 smithi115 ceph-mon[102748]: pgmap v180: 129 pgs: 129 active+clean; 1.4 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.0 MiB/s wr, 447 op/s 2023-12-11T17:52:10.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:52:09 smithi181 ceph-mon[103627]: pgmap v180: 129 pgs: 129 active+clean; 1.4 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.0 MiB/s wr, 447 op/s 2023-12-11T17:52:10.406 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:52:09 smithi059 ceph-mon[98847]: pgmap v180: 129 pgs: 129 active+clean; 1.4 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.0 MiB/s wr, 447 op/s 2023-12-11T17:52:11.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:52:11 smithi181 ceph-mon[103627]: pgmap v181: 129 pgs: 129 active+clean; 1.5 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.5 MiB/s wr, 511 op/s 2023-12-11T17:52:11.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:52:11 smithi059 ceph-mon[98847]: pgmap v181: 129 pgs: 129 active+clean; 1.5 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.5 MiB/s wr, 511 op/s 2023-12-11T17:52:11.532 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:52:11 smithi115 ceph-mon[102748]: pgmap v181: 129 pgs: 129 active+clean; 1.5 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.5 MiB/s wr, 511 op/s 2023-12-11T17:52:13.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:52:13 smithi115 ceph-mon[102748]: pgmap v182: 129 pgs: 129 active+clean; 1.5 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.9 MiB/s wr, 230 op/s 2023-12-11T17:52:13.645 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:52:13 smithi181 ceph-mon[103627]: pgmap v182: 129 pgs: 129 active+clean; 1.5 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.9 MiB/s wr, 230 op/s 2023-12-11T17:52:13.668 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:52:13 smithi059 ceph-mon[98847]: pgmap v182: 129 pgs: 129 active+clean; 1.5 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.9 MiB/s wr, 230 op/s 2023-12-11T17:52:16.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:52:15 smithi181 ceph-mon[103627]: pgmap v183: 129 pgs: 129 active+clean; 1.5 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.4 MiB/s wr, 270 op/s 2023-12-11T17:52:16.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:52:15 smithi059 ceph-mon[98847]: pgmap v183: 129 pgs: 129 active+clean; 1.5 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.4 MiB/s wr, 270 op/s 2023-12-11T17:52:16.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:52:15 smithi115 ceph-mon[102748]: pgmap v183: 129 pgs: 129 active+clean; 1.5 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.4 MiB/s wr, 270 op/s 2023-12-11T17:52:17.290 INFO:tasks.workunit.client.0.smithi059.stdout: 328 2288051 485 1525362 426 899844 1110 2023-12-11T17:52:18.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:52:17 smithi181 ceph-mon[103627]: pgmap v184: 129 pgs: 129 active+clean; 1.5 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.4 MiB/s wr, 233 op/s 2023-12-11T17:52:18.155 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:52:17 smithi059 ceph-mon[98847]: pgmap v184: 129 pgs: 129 active+clean; 1.5 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.4 MiB/s wr, 233 op/s 2023-12-11T17:52:18.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:52:17 smithi115 ceph-mon[102748]: pgmap v184: 129 pgs: 129 active+clean; 1.5 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.4 MiB/s wr, 233 op/s 2023-12-11T17:52:20.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:52:19 smithi115 ceph-mon[102748]: pgmap v185: 129 pgs: 129 active+clean; 1.5 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.8 MiB/s wr, 190 op/s 2023-12-11T17:52:20.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:52:19 smithi181 ceph-mon[103627]: pgmap v185: 129 pgs: 129 active+clean; 1.5 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.8 MiB/s wr, 190 op/s 2023-12-11T17:52:20.408 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:52:19 smithi059 ceph-mon[98847]: pgmap v185: 129 pgs: 129 active+clean; 1.5 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.8 MiB/s wr, 190 op/s 2023-12-11T17:52:22.285 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:52:21 smithi115 ceph-mon[102748]: pgmap v186: 129 pgs: 129 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.7 MiB/s wr, 400 op/s 2023-12-11T17:52:22.397 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:52:21 smithi181 ceph-mon[103627]: pgmap v186: 129 pgs: 129 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.7 MiB/s wr, 400 op/s 2023-12-11T17:52:22.411 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:52:21 smithi059 ceph-mon[98847]: pgmap v186: 129 pgs: 129 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.7 MiB/s wr, 400 op/s 2023-12-11T17:52:24.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:52:24 smithi181 ceph-mon[103627]: pgmap v187: 129 pgs: 129 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.8 MiB/s wr, 315 op/s 2023-12-11T17:52:24.421 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:52:24 smithi059 ceph-mon[98847]: pgmap v187: 129 pgs: 129 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.8 MiB/s wr, 315 op/s 2023-12-11T17:52:24.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:52:24 smithi115 ceph-mon[102748]: pgmap v187: 129 pgs: 129 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.8 MiB/s wr, 315 op/s 2023-12-11T17:52:26.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:52:25 smithi181 ceph-mon[103627]: pgmap v188: 129 pgs: 129 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.0 MiB/s wr, 442 op/s 2023-12-11T17:52:26.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:52:25 smithi059 ceph-mon[98847]: pgmap v188: 129 pgs: 129 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.0 MiB/s wr, 442 op/s 2023-12-11T17:52:26.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:52:25 smithi115 ceph-mon[102748]: pgmap v188: 129 pgs: 129 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.0 MiB/s wr, 442 op/s 2023-12-11T17:52:27.290 INFO:tasks.workunit.client.0.smithi059.stdout: 334 1942120 342 1293509 332 790654 675 2023-12-11T17:52:28.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:52:27 smithi115 ceph-mon[102748]: pgmap v189: 129 pgs: 129 active+clean; 1.5 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.8 MiB/s wr, 445 op/s 2023-12-11T17:52:28.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:52:27 smithi181 ceph-mon[103627]: pgmap v189: 129 pgs: 129 active+clean; 1.5 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.8 MiB/s wr, 445 op/s 2023-12-11T17:52:28.417 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:52:27 smithi059 ceph-mon[98847]: pgmap v189: 129 pgs: 129 active+clean; 1.5 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.8 MiB/s wr, 445 op/s 2023-12-11T17:52:30.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:52:30 smithi115 ceph-mon[102748]: pgmap v190: 129 pgs: 129 active+clean; 1.5 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.2 MiB/s wr, 472 op/s 2023-12-11T17:52:30.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:52:30 smithi181 ceph-mon[103627]: pgmap v190: 129 pgs: 129 active+clean; 1.5 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.2 MiB/s wr, 472 op/s 2023-12-11T17:52:30.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:52:30 smithi059 ceph-mon[98847]: pgmap v190: 129 pgs: 129 active+clean; 1.5 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.2 MiB/s wr, 472 op/s 2023-12-11T17:52:32.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:52:32 smithi181 ceph-mon[103627]: pgmap v191: 129 pgs: 129 active+clean; 1.6 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 9.3 MiB/s wr, 694 op/s 2023-12-11T17:52:32.416 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:52:32 smithi059 ceph-mon[98847]: pgmap v191: 129 pgs: 129 active+clean; 1.6 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 9.3 MiB/s wr, 694 op/s 2023-12-11T17:52:32.532 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:52:32 smithi115 ceph-mon[102748]: pgmap v191: 129 pgs: 129 active+clean; 1.6 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 9.3 MiB/s wr, 694 op/s 2023-12-11T17:52:33.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:52:33 smithi181 ceph-mon[103627]: pgmap v192: 129 pgs: 129 active+clean; 1.6 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 9.2 MiB/s wr, 534 op/s 2023-12-11T17:52:33.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:52:33 smithi059 ceph-mon[98847]: pgmap v192: 129 pgs: 129 active+clean; 1.6 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 9.2 MiB/s wr, 534 op/s 2023-12-11T17:52:33.532 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:52:33 smithi115 ceph-mon[102748]: pgmap v192: 129 pgs: 129 active+clean; 1.6 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 9.2 MiB/s wr, 534 op/s 2023-12-11T17:52:36.156 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:52:35 smithi059 ceph-mon[98847]: pgmap v193: 129 pgs: 129 active+clean; 1.6 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 11 MiB/s wr, 614 op/s 2023-12-11T17:52:36.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:52:35 smithi115 ceph-mon[102748]: pgmap v193: 129 pgs: 129 active+clean; 1.6 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 11 MiB/s wr, 614 op/s 2023-12-11T17:52:36.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:52:35 smithi181 ceph-mon[103627]: pgmap v193: 129 pgs: 129 active+clean; 1.6 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 11 MiB/s wr, 614 op/s 2023-12-11T17:52:37.291 INFO:tasks.workunit.client.0.smithi059.stdout: 338 2043448 215 1360233 184 824712 891 2023-12-11T17:52:38.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:52:37 smithi115 ceph-mon[102748]: pgmap v194: 129 pgs: 129 active+clean; 1.6 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 8.4 MiB/s wr, 507 op/s 2023-12-11T17:52:38.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:52:37 smithi181 ceph-mon[103627]: pgmap v194: 129 pgs: 129 active+clean; 1.6 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 8.4 MiB/s wr, 507 op/s 2023-12-11T17:52:38.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:52:37 smithi059 ceph-mon[98847]: pgmap v194: 129 pgs: 129 active+clean; 1.6 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 8.4 MiB/s wr, 507 op/s 2023-12-11T17:52:40.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:52:39 smithi115 ceph-mon[102748]: pgmap v195: 129 pgs: 129 active+clean; 1.6 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 7.3 MiB/s wr, 459 op/s 2023-12-11T17:52:40.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:52:39 smithi181 ceph-mon[103627]: pgmap v195: 129 pgs: 129 active+clean; 1.6 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 7.3 MiB/s wr, 459 op/s 2023-12-11T17:52:40.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:52:39 smithi059 ceph-mon[98847]: pgmap v195: 129 pgs: 129 active+clean; 1.6 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 7.3 MiB/s wr, 459 op/s 2023-12-11T17:52:42.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:52:41 smithi115 ceph-mon[102748]: pgmap v196: 129 pgs: 129 active+clean; 1.6 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 7.4 MiB/s wr, 463 op/s 2023-12-11T17:52:42.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:52:41 smithi181 ceph-mon[103627]: pgmap v196: 129 pgs: 129 active+clean; 1.6 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 7.4 MiB/s wr, 463 op/s 2023-12-11T17:52:42.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:52:41 smithi059 ceph-mon[98847]: pgmap v196: 129 pgs: 129 active+clean; 1.6 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 7.4 MiB/s wr, 463 op/s 2023-12-11T17:52:44.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:52:43 smithi115 ceph-mon[102748]: pgmap v197: 129 pgs: 129 active+clean; 1.6 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.2 MiB/s wr, 245 op/s 2023-12-11T17:52:44.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:52:43 smithi181 ceph-mon[103627]: pgmap v197: 129 pgs: 129 active+clean; 1.6 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.2 MiB/s wr, 245 op/s 2023-12-11T17:52:44.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:52:43 smithi059 ceph-mon[98847]: pgmap v197: 129 pgs: 129 active+clean; 1.6 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.2 MiB/s wr, 245 op/s 2023-12-11T17:52:46.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:52:45 smithi115 ceph-mon[102748]: pgmap v198: 129 pgs: 129 active+clean; 1.6 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.6 MiB/s wr, 212 op/s 2023-12-11T17:52:46.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:52:45 smithi181 ceph-mon[103627]: pgmap v198: 129 pgs: 129 active+clean; 1.6 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.6 MiB/s wr, 212 op/s 2023-12-11T17:52:46.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:52:45 smithi059 ceph-mon[98847]: pgmap v198: 129 pgs: 129 active+clean; 1.6 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.6 MiB/s wr, 212 op/s 2023-12-11T17:52:48.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:52:47 smithi115 ceph-mon[102748]: pgmap v199: 129 pgs: 129 active+clean; 1.6 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.3 MiB/s wr, 123 op/s 2023-12-11T17:52:48.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:52:47 smithi181 ceph-mon[103627]: pgmap v199: 129 pgs: 129 active+clean; 1.6 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.3 MiB/s wr, 123 op/s 2023-12-11T17:52:48.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:52:47 smithi059 ceph-mon[98847]: pgmap v199: 129 pgs: 129 active+clean; 1.6 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.3 MiB/s wr, 123 op/s 2023-12-11T17:52:50.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:52:49 smithi115 ceph-mon[102748]: pgmap v200: 129 pgs: 129 active+clean; 1.6 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.2 MiB/s wr, 102 op/s 2023-12-11T17:52:50.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:52:49 smithi181 ceph-mon[103627]: pgmap v200: 129 pgs: 129 active+clean; 1.6 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.2 MiB/s wr, 102 op/s 2023-12-11T17:52:50.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:52:49 smithi059 ceph-mon[98847]: pgmap v200: 129 pgs: 129 active+clean; 1.6 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.2 MiB/s wr, 102 op/s 2023-12-11T17:52:52.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:52:51 smithi115 ceph-mon[102748]: pgmap v201: 129 pgs: 129 active+clean; 1.6 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 998 KiB/s wr, 97 op/s 2023-12-11T17:52:52.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:52:51 smithi181 ceph-mon[103627]: pgmap v201: 129 pgs: 129 active+clean; 1.6 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 998 KiB/s wr, 97 op/s 2023-12-11T17:52:52.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:52:51 smithi059 ceph-mon[98847]: pgmap v201: 129 pgs: 129 active+clean; 1.6 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 998 KiB/s wr, 97 op/s 2023-12-11T17:52:54.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:52:53 smithi115 ceph-mon[102748]: pgmap v202: 129 pgs: 129 active+clean; 1.6 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 139 KiB/s wr, 44 op/s 2023-12-11T17:52:54.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:52:53 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:52:54.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:52:53 smithi181 ceph-mon[103627]: pgmap v202: 129 pgs: 129 active+clean; 1.6 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 139 KiB/s wr, 44 op/s 2023-12-11T17:52:54.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:52:53 smithi181 ceph-mon[103627]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:52:54.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:52:53 smithi059 ceph-mon[98847]: pgmap v202: 129 pgs: 129 active+clean; 1.6 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 139 KiB/s wr, 44 op/s 2023-12-11T17:52:54.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:52:53 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:52:55.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:52:54 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:52:55.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:52:54 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:52:55.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:52:54 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:52:55.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:52:54 smithi181 ceph-mon[103627]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:52:55.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:52:54 smithi181 ceph-mon[103627]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:52:55.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:52:54 smithi181 ceph-mon[103627]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:52:55.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:52:54 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:52:55.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:52:54 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:52:55.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:52:54 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:52:56.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:52:55 smithi115 ceph-mon[102748]: pgmap v203: 129 pgs: 129 active+clean; 1.6 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 1.5 MiB/s wr, 128 op/s 2023-12-11T17:52:56.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:52:55 smithi181 ceph-mon[103627]: pgmap v203: 129 pgs: 129 active+clean; 1.6 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 1.5 MiB/s wr, 128 op/s 2023-12-11T17:52:56.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:52:55 smithi059 ceph-mon[98847]: pgmap v203: 129 pgs: 129 active+clean; 1.6 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 1.5 MiB/s wr, 128 op/s 2023-12-11T17:52:58.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:52:57 smithi115 ceph-mon[102748]: pgmap v204: 129 pgs: 129 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 1.9 MiB/s wr, 148 op/s 2023-12-11T17:52:58.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:52:57 smithi181 ceph-mon[103627]: pgmap v204: 129 pgs: 129 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 1.9 MiB/s wr, 148 op/s 2023-12-11T17:52:58.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:52:57 smithi059 ceph-mon[98847]: pgmap v204: 129 pgs: 129 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 1.9 MiB/s wr, 148 op/s 2023-12-11T17:53:00.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:52:59 smithi115 ceph-mon[102748]: pgmap v205: 129 pgs: 129 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 2.3 MiB/s wr, 164 op/s 2023-12-11T17:53:00.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:52:59 smithi181 ceph-mon[103627]: pgmap v205: 129 pgs: 129 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 2.3 MiB/s wr, 164 op/s 2023-12-11T17:53:00.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:52:59 smithi059 ceph-mon[98847]: pgmap v205: 129 pgs: 129 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 2.3 MiB/s wr, 164 op/s 2023-12-11T17:53:02.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:53:01 smithi115 ceph-mon[102748]: pgmap v206: 129 pgs: 129 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 512 KiB/s rd, 3.0 MiB/s wr, 187 op/s 2023-12-11T17:53:02.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:53:01 smithi181 ceph-mon[103627]: pgmap v206: 129 pgs: 129 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 512 KiB/s rd, 3.0 MiB/s wr, 187 op/s 2023-12-11T17:53:02.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:53:01 smithi059 ceph-mon[98847]: pgmap v206: 129 pgs: 129 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 512 KiB/s rd, 3.0 MiB/s wr, 187 op/s 2023-12-11T17:53:04.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:53:03 smithi115 ceph-mon[102748]: pgmap v207: 129 pgs: 129 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 506 KiB/s rd, 2.9 MiB/s wr, 184 op/s 2023-12-11T17:53:04.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:53:03 smithi181 ceph-mon[103627]: pgmap v207: 129 pgs: 129 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 506 KiB/s rd, 2.9 MiB/s wr, 184 op/s 2023-12-11T17:53:04.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:53:03 smithi059 ceph-mon[98847]: pgmap v207: 129 pgs: 129 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 506 KiB/s rd, 2.9 MiB/s wr, 184 op/s 2023-12-11T17:53:05.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:53:04 smithi115 ceph-mon[102748]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-12-11T17:53:05.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:53:04 smithi181 ceph-mon[103627]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-12-11T17:53:05.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:53:04 smithi059 ceph-mon[98847]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-12-11T17:53:06.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:53:05 smithi115 ceph-mon[102748]: pgmap v208: 129 pgs: 129 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 516 KiB/s rd, 3.0 MiB/s wr, 198 op/s 2023-12-11T17:53:06.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:53:05 smithi181 ceph-mon[103627]: pgmap v208: 129 pgs: 129 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 516 KiB/s rd, 3.0 MiB/s wr, 198 op/s 2023-12-11T17:53:06.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:53:05 smithi059 ceph-mon[98847]: pgmap v208: 129 pgs: 129 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 516 KiB/s rd, 3.0 MiB/s wr, 198 op/s 2023-12-11T17:53:08.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:53:07 smithi115 ceph-mon[102748]: pgmap v209: 129 pgs: 129 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 512 KiB/s rd, 1.7 MiB/s wr, 103 op/s 2023-12-11T17:53:08.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:53:07 smithi181 ceph-mon[103627]: pgmap v209: 129 pgs: 129 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 512 KiB/s rd, 1.7 MiB/s wr, 103 op/s 2023-12-11T17:53:08.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:53:07 smithi059 ceph-mon[98847]: pgmap v209: 129 pgs: 129 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 512 KiB/s rd, 1.7 MiB/s wr, 103 op/s 2023-12-11T17:53:10.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:53:10 smithi115 ceph-mon[102748]: pgmap v210: 129 pgs: 129 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 512 KiB/s rd, 1.1 MiB/s wr, 65 op/s 2023-12-11T17:53:10.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:53:10 smithi115 ceph-mon[102748]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-12-11T17:53:10.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:53:10 smithi181 ceph-mon[103627]: pgmap v210: 129 pgs: 129 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 512 KiB/s rd, 1.1 MiB/s wr, 65 op/s 2023-12-11T17:53:10.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:53:10 smithi181 ceph-mon[103627]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-12-11T17:53:10.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:53:10 smithi059 ceph-mon[98847]: pgmap v210: 129 pgs: 129 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 512 KiB/s rd, 1.1 MiB/s wr, 65 op/s 2023-12-11T17:53:10.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:53:10 smithi059 ceph-mon[98847]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-12-11T17:53:12.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:53:12 smithi115 ceph-mon[102748]: pgmap v211: 129 pgs: 129 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 667 KiB/s wr, 48 op/s 2023-12-11T17:53:12.396 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:53:12 smithi181 ceph-mon[103627]: pgmap v211: 129 pgs: 129 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 667 KiB/s wr, 48 op/s 2023-12-11T17:53:12.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:53:12 smithi059 ceph-mon[98847]: pgmap v211: 129 pgs: 129 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 667 KiB/s wr, 48 op/s 2023-12-11T17:53:14.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:53:14 smithi115 ceph-mon[102748]: pgmap v212: 129 pgs: 129 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 137 KiB/s wr, 24 op/s 2023-12-11T17:53:14.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:53:14 smithi181 ceph-mon[103627]: pgmap v212: 129 pgs: 129 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 137 KiB/s wr, 24 op/s 2023-12-11T17:53:14.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:53:14 smithi059 ceph-mon[98847]: pgmap v212: 129 pgs: 129 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 137 KiB/s wr, 24 op/s 2023-12-11T17:53:16.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:53:16 smithi115 ceph-mon[102748]: pgmap v213: 129 pgs: 129 active+clean; 1.7 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 47 op/s 2023-12-11T17:53:16.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:53:16 smithi181 ceph-mon[103627]: pgmap v213: 129 pgs: 129 active+clean; 1.7 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 47 op/s 2023-12-11T17:53:16.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:53:16 smithi059 ceph-mon[98847]: pgmap v213: 129 pgs: 129 active+clean; 1.7 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 47 op/s 2023-12-11T17:53:18.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:53:18 smithi115 ceph-mon[102748]: pgmap v214: 129 pgs: 129 active+clean; 1.7 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 40 op/s 2023-12-11T17:53:18.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:53:18 smithi181 ceph-mon[103627]: pgmap v214: 129 pgs: 129 active+clean; 1.7 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 40 op/s 2023-12-11T17:53:18.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:53:18 smithi059 ceph-mon[98847]: pgmap v214: 129 pgs: 129 active+clean; 1.7 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 40 op/s 2023-12-11T17:53:20.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:53:20 smithi115 ceph-mon[102748]: pgmap v215: 129 pgs: 129 active+clean; 1.7 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 42 op/s 2023-12-11T17:53:20.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:53:20 smithi181 ceph-mon[103627]: pgmap v215: 129 pgs: 129 active+clean; 1.7 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 42 op/s 2023-12-11T17:53:20.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:53:20 smithi059 ceph-mon[98847]: pgmap v215: 129 pgs: 129 active+clean; 1.7 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 42 op/s 2023-12-11T17:53:22.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:53:22 smithi115 ceph-mon[102748]: pgmap v216: 129 pgs: 129 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.4 MiB/s wr, 55 op/s 2023-12-11T17:53:22.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:53:22 smithi115 ceph-mon[102748]: osdmap e92: 12 total, 12 up, 12 in 2023-12-11T17:53:22.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:53:22 smithi181 ceph-mon[103627]: pgmap v216: 129 pgs: 129 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.4 MiB/s wr, 55 op/s 2023-12-11T17:53:22.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:53:22 smithi181 ceph-mon[103627]: osdmap e92: 12 total, 12 up, 12 in 2023-12-11T17:53:22.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:53:22 smithi059 ceph-mon[98847]: pgmap v216: 129 pgs: 129 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.4 MiB/s wr, 55 op/s 2023-12-11T17:53:22.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:53:22 smithi059 ceph-mon[98847]: osdmap e92: 12 total, 12 up, 12 in 2023-12-11T17:53:24.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:53:24 smithi181 ceph-mon[103627]: pgmap v218: 129 pgs: 129 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.8 MiB/s wr, 61 op/s 2023-12-11T17:53:24.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:53:24 smithi059 ceph-mon[98847]: pgmap v218: 129 pgs: 129 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.8 MiB/s wr, 61 op/s 2023-12-11T17:53:24.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:53:24 smithi115 ceph-mon[102748]: pgmap v218: 129 pgs: 129 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.8 MiB/s wr, 61 op/s 2023-12-11T17:53:26.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:53:26 smithi181 ceph-mon[103627]: pgmap v219: 129 pgs: 129 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 37 op/s 2023-12-11T17:53:26.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:53:26 smithi059 ceph-mon[98847]: pgmap v219: 129 pgs: 129 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 37 op/s 2023-12-11T17:53:26.532 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:53:26 smithi115 ceph-mon[102748]: pgmap v219: 129 pgs: 129 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 37 op/s 2023-12-11T17:53:28.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:53:28 smithi181 ceph-mon[103627]: pgmap v220: 129 pgs: 129 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 171 KiB/s wr, 30 op/s 2023-12-11T17:53:28.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:53:28 smithi059 ceph-mon[98847]: pgmap v220: 129 pgs: 129 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 171 KiB/s wr, 30 op/s 2023-12-11T17:53:28.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:53:28 smithi115 ceph-mon[102748]: pgmap v220: 129 pgs: 129 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 171 KiB/s wr, 30 op/s 2023-12-11T17:53:30.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:53:30 smithi181 ceph-mon[103627]: pgmap v221: 129 pgs: 129 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 168 KiB/s wr, 26 op/s 2023-12-11T17:53:30.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:53:30 smithi181 ceph-mon[103627]: osdmap e93: 12 total, 12 up, 12 in 2023-12-11T17:53:30.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:53:30 smithi059 ceph-mon[98847]: pgmap v221: 129 pgs: 129 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 168 KiB/s wr, 26 op/s 2023-12-11T17:53:30.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:53:30 smithi059 ceph-mon[98847]: osdmap e93: 12 total, 12 up, 12 in 2023-12-11T17:53:30.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:53:30 smithi115 ceph-mon[102748]: pgmap v221: 129 pgs: 129 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 168 KiB/s wr, 26 op/s 2023-12-11T17:53:30.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:53:30 smithi115 ceph-mon[102748]: osdmap e93: 12 total, 12 up, 12 in 2023-12-11T17:53:32.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:53:32 smithi181 ceph-mon[103627]: pgmap v223: 129 pgs: 129 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 558 KiB/s rd, 587 KiB/s wr, 17 op/s 2023-12-11T17:53:32.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:53:32 smithi059 ceph-mon[98847]: pgmap v223: 129 pgs: 129 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 558 KiB/s rd, 587 KiB/s wr, 17 op/s 2023-12-11T17:53:32.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:53:32 smithi115 ceph-mon[102748]: pgmap v223: 129 pgs: 129 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 558 KiB/s rd, 587 KiB/s wr, 17 op/s 2023-12-11T17:53:34.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:53:34 smithi181 ceph-mon[103627]: pgmap v224: 129 pgs: 129 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 509 KiB/s rd, 536 KiB/s wr, 17 op/s 2023-12-11T17:53:34.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:53:34 smithi059 ceph-mon[98847]: pgmap v224: 129 pgs: 129 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 509 KiB/s rd, 536 KiB/s wr, 17 op/s 2023-12-11T17:53:34.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:53:34 smithi115 ceph-mon[102748]: pgmap v224: 129 pgs: 129 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 509 KiB/s rd, 536 KiB/s wr, 17 op/s 2023-12-11T17:53:36.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:53:36 smithi181 ceph-mon[103627]: pgmap v225: 129 pgs: 129 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 524 KiB/s rd, 552 KiB/s wr, 18 op/s 2023-12-11T17:53:36.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:53:36 smithi059 ceph-mon[98847]: pgmap v225: 129 pgs: 129 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 524 KiB/s rd, 552 KiB/s wr, 18 op/s 2023-12-11T17:53:36.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:53:36 smithi115 ceph-mon[102748]: pgmap v225: 129 pgs: 129 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 524 KiB/s rd, 552 KiB/s wr, 18 op/s 2023-12-11T17:53:38.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:53:38 smithi181 ceph-mon[103627]: pgmap v226: 129 pgs: 129 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 525 KiB/s rd, 553 KiB/s wr, 17 op/s 2023-12-11T17:53:38.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:53:38 smithi059 ceph-mon[98847]: pgmap v226: 129 pgs: 129 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 525 KiB/s rd, 553 KiB/s wr, 17 op/s 2023-12-11T17:53:38.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:53:38 smithi115 ceph-mon[102748]: pgmap v226: 129 pgs: 129 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 525 KiB/s rd, 553 KiB/s wr, 17 op/s 2023-12-11T17:53:40.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:53:40 smithi181 ceph-mon[103627]: pgmap v227: 129 pgs: 129 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 525 KiB/s rd, 553 KiB/s wr, 18 op/s 2023-12-11T17:53:40.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:53:40 smithi059 ceph-mon[98847]: pgmap v227: 129 pgs: 129 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 525 KiB/s rd, 553 KiB/s wr, 18 op/s 2023-12-11T17:53:40.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:53:40 smithi115 ceph-mon[102748]: pgmap v227: 129 pgs: 129 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 525 KiB/s rd, 553 KiB/s wr, 18 op/s 2023-12-11T17:53:42.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:53:42 smithi181 ceph-mon[103627]: pgmap v228: 129 pgs: 129 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 36 KiB/s wr, 14 op/s 2023-12-11T17:53:42.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:53:42 smithi059 ceph-mon[98847]: pgmap v228: 129 pgs: 129 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 36 KiB/s wr, 14 op/s 2023-12-11T17:53:42.532 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:53:42 smithi115 ceph-mon[102748]: pgmap v228: 129 pgs: 129 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 36 KiB/s wr, 14 op/s 2023-12-11T17:53:44.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:53:44 smithi181 ceph-mon[103627]: pgmap v229: 129 pgs: 129 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 32 KiB/s wr, 13 op/s 2023-12-11T17:53:44.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:53:44 smithi059 ceph-mon[98847]: pgmap v229: 129 pgs: 129 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 32 KiB/s wr, 13 op/s 2023-12-11T17:53:44.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:53:44 smithi115 ceph-mon[102748]: pgmap v229: 129 pgs: 129 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 32 KiB/s wr, 13 op/s 2023-12-11T17:53:46.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:53:46 smithi181 ceph-mon[103627]: pgmap v230: 129 pgs: 129 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 43 KiB/s wr, 15 op/s 2023-12-11T17:53:46.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:53:46 smithi059 ceph-mon[98847]: pgmap v230: 129 pgs: 129 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 43 KiB/s wr, 15 op/s 2023-12-11T17:53:46.532 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:53:46 smithi115 ceph-mon[102748]: pgmap v230: 129 pgs: 129 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 43 KiB/s wr, 15 op/s 2023-12-11T17:53:48.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:53:48 smithi181 ceph-mon[103627]: pgmap v231: 129 pgs: 129 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 20 KiB/s wr, 11 op/s 2023-12-11T17:53:48.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:53:48 smithi059 ceph-mon[98847]: pgmap v231: 129 pgs: 129 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 20 KiB/s wr, 11 op/s 2023-12-11T17:53:48.532 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:53:48 smithi115 ceph-mon[102748]: pgmap v231: 129 pgs: 129 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 20 KiB/s wr, 11 op/s 2023-12-11T17:53:50.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:53:50 smithi181 ceph-mon[103627]: pgmap v232: 129 pgs: 129 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 18 KiB/s wr, 10 op/s 2023-12-11T17:53:50.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:53:50 smithi059 ceph-mon[98847]: pgmap v232: 129 pgs: 129 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 18 KiB/s wr, 10 op/s 2023-12-11T17:53:50.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:53:50 smithi115 ceph-mon[102748]: pgmap v232: 129 pgs: 129 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 18 KiB/s wr, 10 op/s 2023-12-11T17:53:52.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:53:52 smithi181 ceph-mon[103627]: pgmap v233: 129 pgs: 129 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 28 KiB/s wr, 13 op/s 2023-12-11T17:53:52.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:53:52 smithi059 ceph-mon[98847]: pgmap v233: 129 pgs: 129 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 28 KiB/s wr, 13 op/s 2023-12-11T17:53:52.532 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:53:52 smithi115 ceph-mon[102748]: pgmap v233: 129 pgs: 129 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 28 KiB/s wr, 13 op/s 2023-12-11T17:53:54.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:53:54 smithi181 ceph-mon[103627]: pgmap v234: 129 pgs: 129 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 25 KiB/s wr, 11 op/s 2023-12-11T17:53:54.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:53:54 smithi059 ceph-mon[98847]: pgmap v234: 129 pgs: 129 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 25 KiB/s wr, 11 op/s 2023-12-11T17:53:54.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:53:54 smithi115 ceph-mon[102748]: pgmap v234: 129 pgs: 129 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 25 KiB/s wr, 11 op/s 2023-12-11T17:53:55.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:53:55 smithi181 ceph-mon[103627]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:53:55.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:53:55 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:53:55.532 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:53:55 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:53:56.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:53:56 smithi181 ceph-mon[103627]: pgmap v235: 129 pgs: 129 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 30 KiB/s wr, 12 op/s 2023-12-11T17:53:56.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:53:56 smithi181 ceph-mon[103627]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:53:56.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:53:56 smithi181 ceph-mon[103627]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:53:56.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:53:56 smithi181 ceph-mon[103627]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:53:56.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:53:56 smithi059 ceph-mon[98847]: pgmap v235: 129 pgs: 129 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 30 KiB/s wr, 12 op/s 2023-12-11T17:53:56.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:53:56 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:53:56.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:53:56 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:53:56.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:53:56 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:53:56.532 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:53:56 smithi115 ceph-mon[102748]: pgmap v235: 129 pgs: 129 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 30 KiB/s wr, 12 op/s 2023-12-11T17:53:56.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:53:56 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:53:56.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:53:56 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:53:56.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:53:56 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:53:58.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:53:58 smithi181 ceph-mon[103627]: pgmap v236: 129 pgs: 129 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 10 op/s 2023-12-11T17:53:58.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:53:58 smithi059 ceph-mon[98847]: pgmap v236: 129 pgs: 129 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 10 op/s 2023-12-11T17:53:58.532 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:53:58 smithi115 ceph-mon[102748]: pgmap v236: 129 pgs: 129 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 10 op/s 2023-12-11T17:54:00.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:54:00 smithi181 ceph-mon[103627]: pgmap v237: 129 pgs: 129 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 9 op/s 2023-12-11T17:54:00.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:54:00 smithi059 ceph-mon[98847]: pgmap v237: 129 pgs: 129 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 9 op/s 2023-12-11T17:54:00.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:54:00 smithi115 ceph-mon[102748]: pgmap v237: 129 pgs: 129 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 9 op/s 2023-12-11T17:54:02.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:54:02 smithi181 ceph-mon[103627]: pgmap v238: 129 pgs: 129 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 26 KiB/s wr, 12 op/s 2023-12-11T17:54:02.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:54:02 smithi059 ceph-mon[98847]: pgmap v238: 129 pgs: 129 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 26 KiB/s wr, 12 op/s 2023-12-11T17:54:02.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:54:02 smithi115 ceph-mon[102748]: pgmap v238: 129 pgs: 129 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 26 KiB/s wr, 12 op/s 2023-12-11T17:54:04.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:54:04 smithi181 ceph-mon[103627]: pgmap v239: 129 pgs: 129 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 11 op/s 2023-12-11T17:54:04.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:54:04 smithi059 ceph-mon[98847]: pgmap v239: 129 pgs: 129 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 11 op/s 2023-12-11T17:54:04.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:54:04 smithi115 ceph-mon[102748]: pgmap v239: 129 pgs: 129 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 11 op/s 2023-12-11T17:54:06.145 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:54:05 smithi181 ceph-mon[103627]: pgmap v240: 129 pgs: 129 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 45 KiB/s wr, 25 op/s 2023-12-11T17:54:06.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:54:05 smithi059 ceph-mon[98847]: pgmap v240: 129 pgs: 129 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 45 KiB/s wr, 25 op/s 2023-12-11T17:54:06.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:54:05 smithi115 ceph-mon[102748]: pgmap v240: 129 pgs: 129 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 45 KiB/s wr, 25 op/s 2023-12-11T17:54:08.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:54:07 smithi059 ceph-mon[98847]: pgmap v241: 129 pgs: 129 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 62 KiB/s wr, 26 op/s 2023-12-11T17:54:08.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:54:07 smithi059 ceph-mon[98847]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-12-11T17:54:08.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:54:07 smithi115 ceph-mon[102748]: pgmap v241: 129 pgs: 129 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 62 KiB/s wr, 26 op/s 2023-12-11T17:54:08.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:54:07 smithi115 ceph-mon[102748]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-12-11T17:54:08.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:54:07 smithi181 ceph-mon[103627]: pgmap v241: 129 pgs: 129 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 62 KiB/s wr, 26 op/s 2023-12-11T17:54:08.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:54:07 smithi181 ceph-mon[103627]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-12-11T17:54:10.154 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:54:09 smithi059 ceph-mon[98847]: pgmap v242: 129 pgs: 129 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 62 KiB/s wr, 24 op/s 2023-12-11T17:54:10.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:54:09 smithi115 ceph-mon[102748]: pgmap v242: 129 pgs: 129 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 62 KiB/s wr, 24 op/s 2023-12-11T17:54:10.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:54:09 smithi181 ceph-mon[103627]: pgmap v242: 129 pgs: 129 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 62 KiB/s wr, 24 op/s 2023-12-11T17:54:12.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:54:11 smithi115 ceph-mon[102748]: pgmap v243: 129 pgs: 129 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 70 KiB/s wr, 27 op/s 2023-12-11T17:54:12.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:54:11 smithi115 ceph-mon[102748]: osdmap e94: 12 total, 12 up, 12 in 2023-12-11T17:54:12.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:54:11 smithi181 ceph-mon[103627]: pgmap v243: 129 pgs: 129 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 70 KiB/s wr, 27 op/s 2023-12-11T17:54:12.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:54:11 smithi181 ceph-mon[103627]: osdmap e94: 12 total, 12 up, 12 in 2023-12-11T17:54:12.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:54:11 smithi059 ceph-mon[98847]: pgmap v243: 129 pgs: 129 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 70 KiB/s wr, 27 op/s 2023-12-11T17:54:12.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:54:11 smithi059 ceph-mon[98847]: osdmap e94: 12 total, 12 up, 12 in 2023-12-11T17:54:14.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:54:13 smithi115 ceph-mon[102748]: pgmap v245: 129 pgs: 3 active+clean+snaptrim, 118 active+clean, 8 active+clean+snaptrim_wait; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 74 KiB/s wr, 28 op/s 2023-12-11T17:54:14.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:54:13 smithi181 ceph-mon[103627]: pgmap v245: 129 pgs: 3 active+clean+snaptrim, 118 active+clean, 8 active+clean+snaptrim_wait; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 74 KiB/s wr, 28 op/s 2023-12-11T17:54:14.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:54:13 smithi059 ceph-mon[98847]: pgmap v245: 129 pgs: 3 active+clean+snaptrim, 118 active+clean, 8 active+clean+snaptrim_wait; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 74 KiB/s wr, 28 op/s 2023-12-11T17:54:16.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:54:15 smithi115 ceph-mon[102748]: pgmap v246: 129 pgs: 3 active+clean+snaptrim, 118 active+clean, 8 active+clean+snaptrim_wait; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 206 KiB/s wr, 14 op/s 2023-12-11T17:54:16.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:54:15 smithi181 ceph-mon[103627]: pgmap v246: 129 pgs: 3 active+clean+snaptrim, 118 active+clean, 8 active+clean+snaptrim_wait; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 206 KiB/s wr, 14 op/s 2023-12-11T17:54:16.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:54:15 smithi059 ceph-mon[98847]: pgmap v246: 129 pgs: 3 active+clean+snaptrim, 118 active+clean, 8 active+clean+snaptrim_wait; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 206 KiB/s wr, 14 op/s 2023-12-11T17:54:18.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:54:17 smithi115 ceph-mon[102748]: pgmap v247: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 180 KiB/s wr, 12 op/s 2023-12-11T17:54:18.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:54:17 smithi181 ceph-mon[103627]: pgmap v247: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 180 KiB/s wr, 12 op/s 2023-12-11T17:54:18.406 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:54:17 smithi059 ceph-mon[98847]: pgmap v247: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 180 KiB/s wr, 12 op/s 2023-12-11T17:54:20.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:54:19 smithi115 ceph-mon[102748]: pgmap v248: 129 pgs: 129 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 180 KiB/s wr, 12 op/s 2023-12-11T17:54:20.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:54:19 smithi115 ceph-mon[102748]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-12-11T17:54:20.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:54:19 smithi115 ceph-mon[102748]: osdmap e95: 12 total, 12 up, 12 in 2023-12-11T17:54:20.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:54:19 smithi181 ceph-mon[103627]: pgmap v248: 129 pgs: 129 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 180 KiB/s wr, 12 op/s 2023-12-11T17:54:20.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:54:19 smithi181 ceph-mon[103627]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-12-11T17:54:20.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:54:19 smithi181 ceph-mon[103627]: osdmap e95: 12 total, 12 up, 12 in 2023-12-11T17:54:20.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:54:19 smithi059 ceph-mon[98847]: pgmap v248: 129 pgs: 129 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 180 KiB/s wr, 12 op/s 2023-12-11T17:54:20.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:54:19 smithi059 ceph-mon[98847]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-12-11T17:54:20.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:54:19 smithi059 ceph-mon[98847]: osdmap e95: 12 total, 12 up, 12 in 2023-12-11T17:54:22.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:54:21 smithi115 ceph-mon[102748]: pgmap v250: 129 pgs: 129 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 277 KiB/s wr, 17 op/s 2023-12-11T17:54:22.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:54:21 smithi181 ceph-mon[103627]: pgmap v250: 129 pgs: 129 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 277 KiB/s wr, 17 op/s 2023-12-11T17:54:22.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:54:21 smithi059 ceph-mon[98847]: pgmap v250: 129 pgs: 129 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 277 KiB/s wr, 17 op/s 2023-12-11T17:54:24.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:54:23 smithi115 ceph-mon[102748]: pgmap v251: 129 pgs: 129 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 256 KiB/s wr, 15 op/s 2023-12-11T17:54:24.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:54:23 smithi181 ceph-mon[103627]: pgmap v251: 129 pgs: 129 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 256 KiB/s wr, 15 op/s 2023-12-11T17:54:24.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:54:23 smithi059 ceph-mon[98847]: pgmap v251: 129 pgs: 129 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 256 KiB/s wr, 15 op/s 2023-12-11T17:54:25.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:54:24 smithi115 ceph-mon[102748]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-12-11T17:54:25.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:54:24 smithi181 ceph-mon[103627]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-12-11T17:54:25.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:54:24 smithi059 ceph-mon[98847]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-12-11T17:54:26.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:54:25 smithi115 ceph-mon[102748]: pgmap v252: 129 pgs: 129 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 258 KiB/s wr, 30 op/s 2023-12-11T17:54:26.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:54:25 smithi181 ceph-mon[103627]: pgmap v252: 129 pgs: 129 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 258 KiB/s wr, 30 op/s 2023-12-11T17:54:26.406 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:54:25 smithi059 ceph-mon[98847]: pgmap v252: 129 pgs: 129 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 258 KiB/s wr, 30 op/s 2023-12-11T17:54:28.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:54:27 smithi115 ceph-mon[102748]: pgmap v253: 129 pgs: 129 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 344 KiB/s wr, 35 op/s 2023-12-11T17:54:28.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:54:27 smithi181 ceph-mon[103627]: pgmap v253: 129 pgs: 129 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 344 KiB/s wr, 35 op/s 2023-12-11T17:54:28.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:54:27 smithi059 ceph-mon[98847]: pgmap v253: 129 pgs: 129 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 344 KiB/s wr, 35 op/s 2023-12-11T17:54:30.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:54:29 smithi115 ceph-mon[102748]: pgmap v254: 129 pgs: 129 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 345 KiB/s wr, 37 op/s 2023-12-11T17:54:30.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:54:29 smithi181 ceph-mon[103627]: pgmap v254: 129 pgs: 129 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 345 KiB/s wr, 37 op/s 2023-12-11T17:54:30.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:54:29 smithi059 ceph-mon[98847]: pgmap v254: 129 pgs: 129 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 345 KiB/s wr, 37 op/s 2023-12-11T17:54:32.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:54:31 smithi115 ceph-mon[102748]: pgmap v255: 129 pgs: 129 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 346 KiB/s rd, 286 KiB/s wr, 35 op/s 2023-12-11T17:54:32.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:54:31 smithi181 ceph-mon[103627]: pgmap v255: 129 pgs: 129 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 346 KiB/s rd, 286 KiB/s wr, 35 op/s 2023-12-11T17:54:32.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:54:31 smithi059 ceph-mon[98847]: pgmap v255: 129 pgs: 129 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 346 KiB/s rd, 286 KiB/s wr, 35 op/s 2023-12-11T17:54:34.286 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:54:33 smithi115 ceph-mon[102748]: pgmap v256: 129 pgs: 129 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 247 KiB/s wr, 31 op/s 2023-12-11T17:54:34.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:54:33 smithi181 ceph-mon[103627]: pgmap v256: 129 pgs: 129 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 247 KiB/s wr, 31 op/s 2023-12-11T17:54:34.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:54:33 smithi059 ceph-mon[98847]: pgmap v256: 129 pgs: 129 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 247 KiB/s wr, 31 op/s 2023-12-11T17:54:36.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:54:35 smithi115 ceph-mon[102748]: pgmap v257: 129 pgs: 129 active+clean; 1.7 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 48 op/s 2023-12-11T17:54:36.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:54:35 smithi181 ceph-mon[103627]: pgmap v257: 129 pgs: 129 active+clean; 1.7 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 48 op/s 2023-12-11T17:54:36.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:54:35 smithi059 ceph-mon[98847]: pgmap v257: 129 pgs: 129 active+clean; 1.7 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 48 op/s 2023-12-11T17:54:38.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:54:37 smithi115 ceph-mon[102748]: pgmap v258: 129 pgs: 129 active+clean; 1.7 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 37 op/s 2023-12-11T17:54:38.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:54:37 smithi181 ceph-mon[103627]: pgmap v258: 129 pgs: 129 active+clean; 1.7 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 37 op/s 2023-12-11T17:54:38.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:54:37 smithi059 ceph-mon[98847]: pgmap v258: 129 pgs: 129 active+clean; 1.7 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 37 op/s 2023-12-11T17:54:40.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:54:39 smithi115 ceph-mon[102748]: pgmap v259: 129 pgs: 129 active+clean; 1.7 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.9 MiB/s wr, 37 op/s 2023-12-11T17:54:40.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:54:39 smithi181 ceph-mon[103627]: pgmap v259: 129 pgs: 129 active+clean; 1.7 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.9 MiB/s wr, 37 op/s 2023-12-11T17:54:40.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:54:39 smithi059 ceph-mon[98847]: pgmap v259: 129 pgs: 129 active+clean; 1.7 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.9 MiB/s wr, 37 op/s 2023-12-11T17:54:42.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:54:41 smithi115 ceph-mon[102748]: pgmap v260: 129 pgs: 129 active+clean; 1.7 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 8.3 MiB/s wr, 56 op/s 2023-12-11T17:54:42.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:54:41 smithi181 ceph-mon[103627]: pgmap v260: 129 pgs: 129 active+clean; 1.7 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 8.3 MiB/s wr, 56 op/s 2023-12-11T17:54:42.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:54:41 smithi059 ceph-mon[98847]: pgmap v260: 129 pgs: 129 active+clean; 1.7 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 8.3 MiB/s wr, 56 op/s 2023-12-11T17:54:44.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:54:43 smithi115 ceph-mon[102748]: pgmap v261: 129 pgs: 129 active+clean; 1.7 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 9.2 MiB/s wr, 64 op/s 2023-12-11T17:54:44.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:54:43 smithi181 ceph-mon[103627]: pgmap v261: 129 pgs: 129 active+clean; 1.7 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 9.2 MiB/s wr, 64 op/s 2023-12-11T17:54:44.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:54:43 smithi059 ceph-mon[98847]: pgmap v261: 129 pgs: 129 active+clean; 1.7 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 9.2 MiB/s wr, 64 op/s 2023-12-11T17:54:46.035 INFO:tasks.workunit.client.0.smithi059.stdout: 2023-12-11T17:54:46.035 INFO:tasks.workunit.client.0.smithi059.stdout:Final score for writes: 338 2023-12-11T17:54:46.036 INFO:tasks.workunit.client.0.smithi059.stdout:Final score for reads : 872855 2023-12-11T17:54:46.036 INFO:tasks.workunit.client.0.smithi059.stdout: 2023-12-11T17:54:46.036 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-12-11T17:54:46.036 DEBUG:teuthology.orchestra.run.smithi059:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-12-11T17:54:46.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:54:45 smithi115 ceph-mon[102748]: pgmap v262: 129 pgs: 129 active+clean; 1.7 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 10 MiB/s wr, 81 op/s 2023-12-11T17:54:46.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:54:45 smithi181 ceph-mon[103627]: pgmap v262: 129 pgs: 129 active+clean; 1.7 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 10 MiB/s wr, 81 op/s 2023-12-11T17:54:46.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:54:45 smithi059 ceph-mon[98847]: pgmap v262: 129 pgs: 129 active+clean; 1.7 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 10 MiB/s wr, 81 op/s 2023-12-11T17:54:48.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:54:47 smithi115 ceph-mon[102748]: pgmap v263: 129 pgs: 129 active+clean; 1.7 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 9.0 MiB/s wr, 68 op/s 2023-12-11T17:54:48.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:54:47 smithi115 ceph-mon[102748]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-12-11T17:54:48.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:54:47 smithi181 ceph-mon[103627]: pgmap v263: 129 pgs: 129 active+clean; 1.7 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 9.0 MiB/s wr, 68 op/s 2023-12-11T17:54:48.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:54:47 smithi181 ceph-mon[103627]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-12-11T17:54:48.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:54:47 smithi059 ceph-mon[98847]: pgmap v263: 129 pgs: 129 active+clean; 1.7 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 9.0 MiB/s wr, 68 op/s 2023-12-11T17:54:48.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:54:47 smithi059 ceph-mon[98847]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-12-11T17:54:50.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:54:50 smithi115 ceph-mon[102748]: pgmap v264: 129 pgs: 129 active+clean; 1.7 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 8.1 MiB/s wr, 65 op/s 2023-12-11T17:54:50.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:54:50 smithi181 ceph-mon[103627]: pgmap v264: 129 pgs: 129 active+clean; 1.7 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 8.1 MiB/s wr, 65 op/s 2023-12-11T17:54:50.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:54:50 smithi059 ceph-mon[98847]: pgmap v264: 129 pgs: 129 active+clean; 1.7 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 8.1 MiB/s wr, 65 op/s 2023-12-11T17:54:52.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:54:52 smithi115 ceph-mon[102748]: pgmap v265: 129 pgs: 129 active+clean; 1.7 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 8.1 MiB/s wr, 103 op/s 2023-12-11T17:54:52.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:54:52 smithi181 ceph-mon[103627]: pgmap v265: 129 pgs: 129 active+clean; 1.7 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 8.1 MiB/s wr, 103 op/s 2023-12-11T17:54:52.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:54:52 smithi059 ceph-mon[98847]: pgmap v265: 129 pgs: 129 active+clean; 1.7 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 8.1 MiB/s wr, 103 op/s 2023-12-11T17:54:54.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:54:54 smithi115 ceph-mon[102748]: pgmap v266: 129 pgs: 129 active+clean; 1.7 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.8 MiB/s wr, 106 op/s 2023-12-11T17:54:54.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:54:54 smithi181 ceph-mon[103627]: pgmap v266: 129 pgs: 129 active+clean; 1.7 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.8 MiB/s wr, 106 op/s 2023-12-11T17:54:54.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:54:54 smithi059 ceph-mon[98847]: pgmap v266: 129 pgs: 129 active+clean; 1.7 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.8 MiB/s wr, 106 op/s 2023-12-11T17:54:56.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:54:56 smithi115 ceph-mon[102748]: pgmap v267: 129 pgs: 129 active+clean; 1.7 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.5 MiB/s wr, 156 op/s 2023-12-11T17:54:56.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:54:56 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:54:56.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:54:56 smithi181 ceph-mon[103627]: pgmap v267: 129 pgs: 129 active+clean; 1.7 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.5 MiB/s wr, 156 op/s 2023-12-11T17:54:56.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:54:56 smithi181 ceph-mon[103627]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:54:56.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:54:56 smithi059 ceph-mon[98847]: pgmap v267: 129 pgs: 129 active+clean; 1.7 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.5 MiB/s wr, 156 op/s 2023-12-11T17:54:56.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:54:56 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:54:57.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:54:57 smithi181 ceph-mon[103627]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:54:57.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:54:57 smithi181 ceph-mon[103627]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:54:57.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:54:57 smithi181 ceph-mon[103627]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:54:57.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:54:57 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:54:57.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:54:57 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:54:57.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:54:57 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:54:57.532 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:54:57 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:54:57.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:54:57 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:54:57.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:54:57 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:54:58.396 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:54:58 smithi181 ceph-mon[103627]: pgmap v268: 129 pgs: 129 active+clean; 1.7 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 163 op/s 2023-12-11T17:54:58.396 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:54:58 smithi181 ceph-mon[103627]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-12-11T17:54:58.406 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:54:58 smithi059 ceph-mon[98847]: pgmap v268: 129 pgs: 129 active+clean; 1.7 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 163 op/s 2023-12-11T17:54:58.406 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:54:58 smithi059 ceph-mon[98847]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-12-11T17:54:58.532 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:54:58 smithi115 ceph-mon[102748]: pgmap v268: 129 pgs: 129 active+clean; 1.7 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 163 op/s 2023-12-11T17:54:58.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:54:58 smithi115 ceph-mon[102748]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-12-11T17:55:00.396 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:55:00 smithi181 ceph-mon[103627]: pgmap v269: 129 pgs: 129 active+clean; 1.7 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 173 op/s 2023-12-11T17:55:00.406 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:55:00 smithi059 ceph-mon[98847]: pgmap v269: 129 pgs: 129 active+clean; 1.7 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 173 op/s 2023-12-11T17:55:00.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:55:00 smithi115 ceph-mon[102748]: pgmap v269: 129 pgs: 129 active+clean; 1.7 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 173 op/s 2023-12-11T17:55:02.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:55:02 smithi181 ceph-mon[103627]: pgmap v270: 129 pgs: 129 active+clean; 1.8 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.9 MiB/s wr, 282 op/s 2023-12-11T17:55:02.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:55:02 smithi059 ceph-mon[98847]: pgmap v270: 129 pgs: 129 active+clean; 1.8 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.9 MiB/s wr, 282 op/s 2023-12-11T17:55:02.532 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:55:02 smithi115 ceph-mon[102748]: pgmap v270: 129 pgs: 129 active+clean; 1.8 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.9 MiB/s wr, 282 op/s 2023-12-11T17:55:04.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:55:04 smithi181 ceph-mon[103627]: pgmap v271: 129 pgs: 129 active+clean; 1.8 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 258 op/s 2023-12-11T17:55:04.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:55:04 smithi059 ceph-mon[98847]: pgmap v271: 129 pgs: 129 active+clean; 1.8 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 258 op/s 2023-12-11T17:55:04.534 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:55:04 smithi115 ceph-mon[102748]: pgmap v271: 129 pgs: 129 active+clean; 1.8 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 258 op/s 2023-12-11T17:55:06.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:55:06 smithi181 ceph-mon[103627]: pgmap v272: 129 pgs: 129 active+clean; 1.8 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.2 MiB/s wr, 318 op/s 2023-12-11T17:55:06.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:55:06 smithi059 ceph-mon[98847]: pgmap v272: 129 pgs: 129 active+clean; 1.8 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.2 MiB/s wr, 318 op/s 2023-12-11T17:55:06.534 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:55:06 smithi115 ceph-mon[102748]: pgmap v272: 129 pgs: 129 active+clean; 1.8 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.2 MiB/s wr, 318 op/s 2023-12-11T17:55:07.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:55:07 smithi181 ceph-mon[103627]: osdmap e96: 12 total, 12 up, 12 in 2023-12-11T17:55:07.406 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:55:07 smithi059 ceph-mon[98847]: osdmap e96: 12 total, 12 up, 12 in 2023-12-11T17:55:07.532 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:55:07 smithi115 ceph-mon[102748]: osdmap e96: 12 total, 12 up, 12 in 2023-12-11T17:55:08.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:55:08 smithi181 ceph-mon[103627]: pgmap v274: 129 pgs: 129 active+clean; 1.8 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.1 MiB/s wr, 311 op/s 2023-12-11T17:55:08.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:55:08 smithi059 ceph-mon[98847]: pgmap v274: 129 pgs: 129 active+clean; 1.8 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.1 MiB/s wr, 311 op/s 2023-12-11T17:55:08.532 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:55:08 smithi115 ceph-mon[102748]: pgmap v274: 129 pgs: 129 active+clean; 1.8 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.1 MiB/s wr, 311 op/s 2023-12-11T17:55:10.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:55:10 smithi181 ceph-mon[103627]: pgmap v275: 129 pgs: 4 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 110 active+clean; 1.8 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.1 MiB/s wr, 311 op/s 2023-12-11T17:55:10.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:55:10 smithi059 ceph-mon[98847]: pgmap v275: 129 pgs: 4 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 110 active+clean; 1.8 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.1 MiB/s wr, 311 op/s 2023-12-11T17:55:10.532 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:55:10 smithi115 ceph-mon[102748]: pgmap v275: 129 pgs: 4 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 110 active+clean; 1.8 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.1 MiB/s wr, 311 op/s 2023-12-11T17:55:12.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:55:12 smithi181 ceph-mon[103627]: pgmap v276: 129 pgs: 4 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 110 active+clean; 1.8 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 244 op/s 2023-12-11T17:55:12.406 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:55:12 smithi059 ceph-mon[98847]: pgmap v276: 129 pgs: 4 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 110 active+clean; 1.8 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 244 op/s 2023-12-11T17:55:12.532 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:55:12 smithi115 ceph-mon[102748]: pgmap v276: 129 pgs: 4 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 110 active+clean; 1.8 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 244 op/s 2023-12-11T17:55:14.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:55:14 smithi181 ceph-mon[103627]: pgmap v277: 129 pgs: 129 active+clean; 1.8 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 229 op/s 2023-12-11T17:55:14.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:55:14 smithi059 ceph-mon[98847]: pgmap v277: 129 pgs: 129 active+clean; 1.8 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 229 op/s 2023-12-11T17:55:14.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:55:14 smithi115 ceph-mon[102748]: pgmap v277: 129 pgs: 129 active+clean; 1.8 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 229 op/s 2023-12-11T17:55:16.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:55:15 smithi115 ceph-mon[102748]: pgmap v278: 129 pgs: 129 active+clean; 1.8 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.8 MiB/s wr, 192 op/s 2023-12-11T17:55:16.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:55:15 smithi115 ceph-mon[102748]: osdmap e97: 12 total, 12 up, 12 in 2023-12-11T17:55:16.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:55:15 smithi181 ceph-mon[103627]: pgmap v278: 129 pgs: 129 active+clean; 1.8 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.8 MiB/s wr, 192 op/s 2023-12-11T17:55:16.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:55:15 smithi181 ceph-mon[103627]: osdmap e97: 12 total, 12 up, 12 in 2023-12-11T17:55:16.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:55:15 smithi059 ceph-mon[98847]: pgmap v278: 129 pgs: 129 active+clean; 1.8 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.8 MiB/s wr, 192 op/s 2023-12-11T17:55:16.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:55:15 smithi059 ceph-mon[98847]: osdmap e97: 12 total, 12 up, 12 in 2023-12-11T17:55:18.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:55:17 smithi115 ceph-mon[102748]: pgmap v280: 129 pgs: 129 active+clean; 1.8 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.2 MiB/s wr, 179 op/s 2023-12-11T17:55:18.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:55:17 smithi181 ceph-mon[103627]: pgmap v280: 129 pgs: 129 active+clean; 1.8 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.2 MiB/s wr, 179 op/s 2023-12-11T17:55:18.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:55:17 smithi059 ceph-mon[98847]: pgmap v280: 129 pgs: 129 active+clean; 1.8 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.2 MiB/s wr, 179 op/s 2023-12-11T17:55:20.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:55:19 smithi115 ceph-mon[102748]: pgmap v281: 129 pgs: 129 active+clean; 1.8 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.2 MiB/s wr, 178 op/s 2023-12-11T17:55:20.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:55:19 smithi181 ceph-mon[103627]: pgmap v281: 129 pgs: 129 active+clean; 1.8 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.2 MiB/s wr, 178 op/s 2023-12-11T17:55:20.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:55:19 smithi059 ceph-mon[98847]: pgmap v281: 129 pgs: 129 active+clean; 1.8 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.2 MiB/s wr, 178 op/s 2023-12-11T17:55:22.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:55:21 smithi115 ceph-mon[102748]: pgmap v282: 129 pgs: 129 active+clean; 1.8 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.5 MiB/s wr, 190 op/s 2023-12-11T17:55:22.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:55:21 smithi181 ceph-mon[103627]: pgmap v282: 129 pgs: 129 active+clean; 1.8 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.5 MiB/s wr, 190 op/s 2023-12-11T17:55:22.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:55:21 smithi059 ceph-mon[98847]: pgmap v282: 129 pgs: 129 active+clean; 1.8 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.5 MiB/s wr, 190 op/s 2023-12-11T17:55:24.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:55:23 smithi115 ceph-mon[102748]: pgmap v283: 129 pgs: 129 active+clean; 1.8 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.5 MiB/s wr, 189 op/s 2023-12-11T17:55:24.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:55:23 smithi181 ceph-mon[103627]: pgmap v283: 129 pgs: 129 active+clean; 1.8 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.5 MiB/s wr, 189 op/s 2023-12-11T17:55:24.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:55:23 smithi059 ceph-mon[98847]: pgmap v283: 129 pgs: 129 active+clean; 1.8 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.5 MiB/s wr, 189 op/s 2023-12-11T17:55:26.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:55:25 smithi115 ceph-mon[102748]: pgmap v284: 129 pgs: 129 active+clean; 1.8 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 158 op/s 2023-12-11T17:55:26.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:55:25 smithi181 ceph-mon[103627]: pgmap v284: 129 pgs: 129 active+clean; 1.8 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 158 op/s 2023-12-11T17:55:26.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:55:25 smithi059 ceph-mon[98847]: pgmap v284: 129 pgs: 129 active+clean; 1.8 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 158 op/s 2023-12-11T17:55:27.285 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:55:26 smithi115 ceph-mon[102748]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-12-11T17:55:27.398 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:55:26 smithi181 ceph-mon[103627]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-12-11T17:55:27.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:55:26 smithi059 ceph-mon[98847]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-12-11T17:55:28.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:55:27 smithi115 ceph-mon[102748]: pgmap v285: 129 pgs: 129 active+clean; 1.8 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 147 op/s 2023-12-11T17:55:28.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:55:27 smithi181 ceph-mon[103627]: pgmap v285: 129 pgs: 129 active+clean; 1.8 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 147 op/s 2023-12-11T17:55:28.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:55:27 smithi059 ceph-mon[98847]: pgmap v285: 129 pgs: 129 active+clean; 1.8 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 147 op/s 2023-12-11T17:55:30.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:55:29 smithi115 ceph-mon[102748]: pgmap v286: 129 pgs: 129 active+clean; 1.8 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 138 op/s 2023-12-11T17:55:30.396 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:55:29 smithi181 ceph-mon[103627]: pgmap v286: 129 pgs: 129 active+clean; 1.8 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 138 op/s 2023-12-11T17:55:30.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:55:29 smithi059 ceph-mon[98847]: pgmap v286: 129 pgs: 129 active+clean; 1.8 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 138 op/s 2023-12-11T17:55:32.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:55:31 smithi115 ceph-mon[102748]: pgmap v287: 129 pgs: 129 active+clean; 1.8 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.5 MiB/s wr, 223 op/s 2023-12-11T17:55:32.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:55:31 smithi181 ceph-mon[103627]: pgmap v287: 129 pgs: 129 active+clean; 1.8 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.5 MiB/s wr, 223 op/s 2023-12-11T17:55:32.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:55:31 smithi059 ceph-mon[98847]: pgmap v287: 129 pgs: 129 active+clean; 1.8 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.5 MiB/s wr, 223 op/s 2023-12-11T17:55:34.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:55:33 smithi115 ceph-mon[102748]: pgmap v288: 129 pgs: 129 active+clean; 1.8 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 186 op/s 2023-12-11T17:55:34.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:55:33 smithi181 ceph-mon[103627]: pgmap v288: 129 pgs: 129 active+clean; 1.8 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 186 op/s 2023-12-11T17:55:34.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:55:33 smithi059 ceph-mon[98847]: pgmap v288: 129 pgs: 129 active+clean; 1.8 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 186 op/s 2023-12-11T17:55:36.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:55:35 smithi115 ceph-mon[102748]: pgmap v289: 129 pgs: 129 active+clean; 1.8 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 221 op/s 2023-12-11T17:55:36.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:55:35 smithi181 ceph-mon[103627]: pgmap v289: 129 pgs: 129 active+clean; 1.8 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 221 op/s 2023-12-11T17:55:36.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:55:35 smithi059 ceph-mon[98847]: pgmap v289: 129 pgs: 129 active+clean; 1.8 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 221 op/s 2023-12-11T17:55:38.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:55:37 smithi115 ceph-mon[102748]: pgmap v290: 129 pgs: 129 active+clean; 1.8 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 206 op/s 2023-12-11T17:55:38.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:55:37 smithi181 ceph-mon[103627]: pgmap v290: 129 pgs: 129 active+clean; 1.8 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 206 op/s 2023-12-11T17:55:38.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:55:37 smithi059 ceph-mon[98847]: pgmap v290: 129 pgs: 129 active+clean; 1.8 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 206 op/s 2023-12-11T17:55:40.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:55:39 smithi115 ceph-mon[102748]: pgmap v291: 129 pgs: 129 active+clean; 1.8 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 194 op/s 2023-12-11T17:55:40.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:55:39 smithi181 ceph-mon[103627]: pgmap v291: 129 pgs: 129 active+clean; 1.8 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 194 op/s 2023-12-11T17:55:40.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:55:39 smithi059 ceph-mon[98847]: pgmap v291: 129 pgs: 129 active+clean; 1.8 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 194 op/s 2023-12-11T17:55:42.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:55:41 smithi115 ceph-mon[102748]: pgmap v292: 129 pgs: 129 active+clean; 1.8 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.9 MiB/s wr, 235 op/s 2023-12-11T17:55:42.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:55:41 smithi115 ceph-mon[102748]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-12-11T17:55:42.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:55:41 smithi181 ceph-mon[103627]: pgmap v292: 129 pgs: 129 active+clean; 1.8 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.9 MiB/s wr, 235 op/s 2023-12-11T17:55:42.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:55:41 smithi181 ceph-mon[103627]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-12-11T17:55:42.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:55:41 smithi059 ceph-mon[98847]: pgmap v292: 129 pgs: 129 active+clean; 1.8 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.9 MiB/s wr, 235 op/s 2023-12-11T17:55:42.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:55:41 smithi059 ceph-mon[98847]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-12-11T17:55:44.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:55:43 smithi115 ceph-mon[102748]: pgmap v293: 129 pgs: 129 active+clean; 1.8 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 156 op/s 2023-12-11T17:55:44.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:55:43 smithi181 ceph-mon[103627]: pgmap v293: 129 pgs: 129 active+clean; 1.8 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 156 op/s 2023-12-11T17:55:44.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:55:43 smithi059 ceph-mon[98847]: pgmap v293: 129 pgs: 129 active+clean; 1.8 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 156 op/s 2023-12-11T17:55:46.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:55:45 smithi115 ceph-mon[102748]: pgmap v294: 129 pgs: 129 active+clean; 1.9 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.2 MiB/s wr, 223 op/s 2023-12-11T17:55:46.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:55:45 smithi181 ceph-mon[103627]: pgmap v294: 129 pgs: 129 active+clean; 1.9 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.2 MiB/s wr, 223 op/s 2023-12-11T17:55:46.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:55:45 smithi059 ceph-mon[98847]: pgmap v294: 129 pgs: 129 active+clean; 1.9 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.2 MiB/s wr, 223 op/s 2023-12-11T17:55:48.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:55:47 smithi115 ceph-mon[102748]: pgmap v295: 129 pgs: 129 active+clean; 1.9 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.6 MiB/s wr, 221 op/s 2023-12-11T17:55:48.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:55:47 smithi181 ceph-mon[103627]: pgmap v295: 129 pgs: 129 active+clean; 1.9 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.6 MiB/s wr, 221 op/s 2023-12-11T17:55:48.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:55:47 smithi059 ceph-mon[98847]: pgmap v295: 129 pgs: 129 active+clean; 1.9 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.6 MiB/s wr, 221 op/s 2023-12-11T17:55:50.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:55:49 smithi115 ceph-mon[102748]: pgmap v296: 129 pgs: 129 active+clean; 1.9 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.3 MiB/s wr, 231 op/s 2023-12-11T17:55:50.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:55:50 smithi181 ceph-mon[103627]: pgmap v296: 129 pgs: 129 active+clean; 1.9 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.3 MiB/s wr, 231 op/s 2023-12-11T17:55:50.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:55:49 smithi059 ceph-mon[98847]: pgmap v296: 129 pgs: 129 active+clean; 1.9 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.3 MiB/s wr, 231 op/s 2023-12-11T17:55:52.284 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:55:52 smithi115 ceph-mon[102748]: pgmap v297: 129 pgs: 129 active+clean; 1.9 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.6 MiB/s wr, 281 op/s 2023-12-11T17:55:52.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:55:52 smithi181 ceph-mon[103627]: pgmap v297: 129 pgs: 129 active+clean; 1.9 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.6 MiB/s wr, 281 op/s 2023-12-11T17:55:52.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:55:52 smithi059 ceph-mon[98847]: pgmap v297: 129 pgs: 129 active+clean; 1.9 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.6 MiB/s wr, 281 op/s 2023-12-11T17:55:54.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:55:54 smithi115 ceph-mon[102748]: pgmap v298: 129 pgs: 129 active+clean; 1.9 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 233 op/s 2023-12-11T17:55:54.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:55:54 smithi181 ceph-mon[103627]: pgmap v298: 129 pgs: 129 active+clean; 1.9 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 233 op/s 2023-12-11T17:55:54.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:55:54 smithi059 ceph-mon[98847]: pgmap v298: 129 pgs: 129 active+clean; 1.9 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 233 op/s 2023-12-11T17:55:56.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:55:56 smithi115 ceph-mon[102748]: pgmap v299: 129 pgs: 129 active+clean; 1.9 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.4 MiB/s wr, 267 op/s 2023-12-11T17:55:56.395 INFO:journalctl@ceph.mon.c.smithi181.stdout:Dec 11 17:55:56 smithi181 ceph-mon[103627]: pgmap v299: 129 pgs: 129 active+clean; 1.9 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.4 MiB/s wr, 267 op/s 2023-12-11T17:55:56.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:55:56 smithi059 ceph-mon[98847]: pgmap v299: 129 pgs: 129 active+clean; 1.9 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.4 MiB/s wr, 267 op/s 2023-12-11T17:55:57.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:55:57 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:55:57.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:55:57 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:56:12.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:56:12 smithi059 ceph-mon[98847]: pgmap v300: 129 pgs: 129 active+clean; 1.9 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 199 op/s 2023-12-11T17:56:12.532 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:56:12 smithi115 ceph-mon[102748]: pgmap v300: 129 pgs: 129 active+clean; 1.9 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 199 op/s 2023-12-11T17:56:13.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:56:13 smithi059 ceph-mon[98847]: pgmap v301: 129 pgs: 129 active+clean; 1.9 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.3 MiB/s wr, 176 op/s 2023-12-11T17:56:13.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:56:13 smithi059 ceph-mon[98847]: pgmap v302: 129 pgs: 129 active+clean; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 183 op/s 2023-12-11T17:56:13.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:56:13 smithi059 ceph-mon[98847]: pgmap v303: 129 pgs: 129 active+clean; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 132 op/s 2023-12-11T17:56:13.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:56:13 smithi059 ceph-mon[98847]: pgmap v304: 129 pgs: 129 active+clean; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 130 op/s 2023-12-11T17:56:13.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:56:13 smithi059 ceph-mon[98847]: pgmap v305: 129 pgs: 129 active+clean; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 76 op/s 2023-12-11T17:56:13.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:56:13 smithi059 ceph-mon[98847]: mon.b calling monitor election 2023-12-11T17:56:13.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:56:13 smithi059 ceph-mon[98847]: mon.a calling monitor election 2023-12-11T17:56:13.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:56:13 smithi059 ceph-mon[98847]: pgmap v306: 129 pgs: 129 active+clean; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 558 KiB/s rd, 487 KiB/s wr, 48 op/s 2023-12-11T17:56:13.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:56:13 smithi059 ceph-mon[98847]: pgmap v307: 129 pgs: 129 active+clean; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 411 KiB/s rd, 346 KiB/s wr, 33 op/s 2023-12-11T17:56:13.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:56:13 smithi059 ceph-mon[98847]: mon.a is new leader, mons a,b in quorum (ranks 0,2) 2023-12-11T17:56:13.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:56:13 smithi059 ceph-mon[98847]: monmap e3: 3 mons at {a=[v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0],b=[v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0],c=[v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0]} removed_ranks: {} disallowed_leaders: {} 2023-12-11T17:56:13.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:56:13 smithi059 ceph-mon[98847]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-12-11T17:56:13.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:56:13 smithi059 ceph-mon[98847]: osdmap e97: 12 total, 12 up, 12 in 2023-12-11T17:56:13.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:56:13 smithi059 ceph-mon[98847]: mgrmap e25: x(active, since 9m), standbys: z, y 2023-12-11T17:56:13.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:56:13 smithi059 ceph-mon[98847]: Health check failed: 1/3 mons down, quorum a,b (MON_DOWN) 2023-12-11T17:56:13.406 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:56:13 smithi059 ceph-mon[98847]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,b 2023-12-11T17:56:13.406 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:56:13 smithi059 ceph-mon[98847]: [WRN] MON_DOWN: 1/3 mons down, quorum a,b 2023-12-11T17:56:13.406 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:56:13 smithi059 ceph-mon[98847]: mon.c (rank 1) addr [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] is down (out of quorum) 2023-12-11T17:56:13.532 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:56:13 smithi115 ceph-mon[102748]: pgmap v301: 129 pgs: 129 active+clean; 1.9 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.3 MiB/s wr, 176 op/s 2023-12-11T17:56:13.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:56:13 smithi115 ceph-mon[102748]: pgmap v302: 129 pgs: 129 active+clean; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 183 op/s 2023-12-11T17:56:13.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:56:13 smithi115 ceph-mon[102748]: pgmap v303: 129 pgs: 129 active+clean; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 132 op/s 2023-12-11T17:56:13.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:56:13 smithi115 ceph-mon[102748]: pgmap v304: 129 pgs: 129 active+clean; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 130 op/s 2023-12-11T17:56:13.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:56:13 smithi115 ceph-mon[102748]: pgmap v305: 129 pgs: 129 active+clean; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 76 op/s 2023-12-11T17:56:13.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:56:13 smithi115 ceph-mon[102748]: mon.b calling monitor election 2023-12-11T17:56:13.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:56:13 smithi115 ceph-mon[102748]: mon.a calling monitor election 2023-12-11T17:56:13.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:56:13 smithi115 ceph-mon[102748]: pgmap v306: 129 pgs: 129 active+clean; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 558 KiB/s rd, 487 KiB/s wr, 48 op/s 2023-12-11T17:56:13.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:56:13 smithi115 ceph-mon[102748]: pgmap v307: 129 pgs: 129 active+clean; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 411 KiB/s rd, 346 KiB/s wr, 33 op/s 2023-12-11T17:56:13.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:56:13 smithi115 ceph-mon[102748]: mon.a is new leader, mons a,b in quorum (ranks 0,2) 2023-12-11T17:56:13.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:56:13 smithi115 ceph-mon[102748]: monmap e3: 3 mons at {a=[v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0],b=[v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0],c=[v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0]} removed_ranks: {} disallowed_leaders: {} 2023-12-11T17:56:13.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:56:13 smithi115 ceph-mon[102748]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-12-11T17:56:13.534 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:56:13 smithi115 ceph-mon[102748]: osdmap e97: 12 total, 12 up, 12 in 2023-12-11T17:56:13.534 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:56:13 smithi115 ceph-mon[102748]: mgrmap e25: x(active, since 9m), standbys: z, y 2023-12-11T17:56:13.534 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:56:13 smithi115 ceph-mon[102748]: Health check failed: 1/3 mons down, quorum a,b (MON_DOWN) 2023-12-11T17:56:13.534 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:56:13 smithi115 ceph-mon[102748]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,b 2023-12-11T17:56:13.534 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:56:13 smithi115 ceph-mon[102748]: [WRN] MON_DOWN: 1/3 mons down, quorum a,b 2023-12-11T17:56:13.534 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:56:13 smithi115 ceph-mon[102748]: mon.c (rank 1) addr [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] is down (out of quorum) 2023-12-11T17:56:14.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:56:14 smithi059 ceph-mon[98847]: pgmap v308: 129 pgs: 129 active+clean; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 122 KiB/s wr, 4 op/s 2023-12-11T17:56:14.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:56:14 smithi115 ceph-mon[102748]: pgmap v308: 129 pgs: 129 active+clean; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 122 KiB/s wr, 4 op/s 2023-12-11T17:56:16.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:56:16 smithi059 ceph-mon[98847]: pgmap v309: 129 pgs: 1 active+clean+laggy, 128 active+clean; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1.6 KiB/s wr, 2 op/s 2023-12-11T17:56:16.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:56:16 smithi115 ceph-mon[102748]: pgmap v309: 129 pgs: 1 active+clean+laggy, 128 active+clean; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1.6 KiB/s wr, 2 op/s 2023-12-11T17:56:17.904 INFO:journalctl@ceph.osd.6.smithi059.stdout:Dec 11 17:56:17 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-6[133999]: 2023-12-11T17:56:17.640+0000 7f6dafb58700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.181:6820 osd.8 since back 2023-12-11T17:55:57.255576+0000 front 2023-12-11T17:55:54.955213+0000 (oldest deadline 2023-12-11T17:56:17.255010+0000) 2023-12-11T17:56:17.904 INFO:journalctl@ceph.osd.6.smithi059.stdout:Dec 11 17:56:17 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-6[133999]: 2023-12-11T17:56:17.640+0000 7f6dafb58700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.181:6828 osd.11 since back 2023-12-11T17:55:54.955300+0000 front 2023-12-11T17:55:54.955265+0000 (oldest deadline 2023-12-11T17:56:17.255010+0000) 2023-12-11T17:56:18.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:56:18 smithi115 ceph-mon[102748]: pgmap v310: 129 pgs: 1 active+clean+laggy, 128 active+clean; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1.5 KiB/s wr, 1 op/s 2023-12-11T17:56:18.283 INFO:journalctl@ceph.osd.1.smithi115.stdout:Dec 11 17:56:17 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-1[107812]: 2023-12-11T17:56:17.878+0000 7f3bf3990700 -1 osd.1 97 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-12-11T17:55:52.633801+0000 front 2023-12-11T17:55:52.633706+0000 (oldest deadline 2023-12-11T17:56:17.333559+0000) 2023-12-11T17:56:18.283 INFO:journalctl@ceph.osd.1.smithi115.stdout:Dec 11 17:56:17 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-1[107812]: 2023-12-11T17:56:17.878+0000 7f3bf3990700 -1 osd.1 97 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2023-12-11T17:55:52.633718+0000 front 2023-12-11T17:55:52.633748+0000 (oldest deadline 2023-12-11T17:56:17.333559+0000) 2023-12-11T17:56:18.283 INFO:journalctl@ceph.osd.1.smithi115.stdout:Dec 11 17:56:17 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-1[107812]: 2023-12-11T17:56:17.878+0000 7f3bf3990700 -1 osd.1 97 heartbeat_check: no reply from 172.21.15.181:6820 osd.8 since back 2023-12-11T17:55:52.633760+0000 front 2023-12-11T17:55:52.633735+0000 (oldest deadline 2023-12-11T17:56:17.333559+0000) 2023-12-11T17:56:18.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:56:18 smithi059 ceph-mon[98847]: pgmap v310: 129 pgs: 1 active+clean+laggy, 128 active+clean; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1.5 KiB/s wr, 1 op/s 2023-12-11T17:56:18.904 INFO:journalctl@ceph.osd.6.smithi059.stdout:Dec 11 17:56:18 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-6[133999]: 2023-12-11T17:56:18.687+0000 7f6dafb58700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.181:6820 osd.8 since back 2023-12-11T17:55:57.255576+0000 front 2023-12-11T17:55:54.955213+0000 (oldest deadline 2023-12-11T17:56:17.255010+0000) 2023-12-11T17:56:18.904 INFO:journalctl@ceph.osd.6.smithi059.stdout:Dec 11 17:56:18 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-6[133999]: 2023-12-11T17:56:18.687+0000 7f6dafb58700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.181:6828 osd.11 since back 2023-12-11T17:55:54.955300+0000 front 2023-12-11T17:55:54.955265+0000 (oldest deadline 2023-12-11T17:56:17.255010+0000) 2023-12-11T17:56:18.904 INFO:journalctl@ceph.osd.0.smithi059.stdout:Dec 11 17:56:18 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-0[117294]: 2023-12-11T17:56:18.456+0000 7efcf51d8700 -1 osd.0 97 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-12-11T17:55:55.882654+0000 front 2023-12-11T17:55:55.882549+0000 (oldest deadline 2023-12-11T17:56:18.182726+0000) 2023-12-11T17:56:18.905 INFO:journalctl@ceph.osd.0.smithi059.stdout:Dec 11 17:56:18 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-0[117294]: 2023-12-11T17:56:18.456+0000 7efcf51d8700 -1 osd.0 97 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2023-12-11T17:55:55.882636+0000 front 2023-12-11T17:55:55.882730+0000 (oldest deadline 2023-12-11T17:56:18.182726+0000) 2023-12-11T17:56:18.905 INFO:journalctl@ceph.osd.0.smithi059.stdout:Dec 11 17:56:18 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-0[117294]: 2023-12-11T17:56:18.456+0000 7efcf51d8700 -1 osd.0 97 heartbeat_check: no reply from 172.21.15.181:6820 osd.8 since back 2023-12-11T17:55:55.882754+0000 front 2023-12-11T17:55:55.882871+0000 (oldest deadline 2023-12-11T17:56:18.182726+0000) 2023-12-11T17:56:18.905 INFO:journalctl@ceph.osd.0.smithi059.stdout:Dec 11 17:56:18 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-0[117294]: 2023-12-11T17:56:18.456+0000 7efcf51d8700 -1 osd.0 97 heartbeat_check: no reply from 172.21.15.181:6828 osd.11 since back 2023-12-11T17:55:55.882720+0000 front 2023-12-11T17:55:55.882909+0000 (oldest deadline 2023-12-11T17:56:18.182726+0000) 2023-12-11T17:56:19.033 INFO:journalctl@ceph.osd.1.smithi115.stdout:Dec 11 17:56:18 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-1[107812]: 2023-12-11T17:56:18.866+0000 7f3bf3990700 -1 osd.1 97 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-12-11T17:55:52.633801+0000 front 2023-12-11T17:55:52.633706+0000 (oldest deadline 2023-12-11T17:56:17.333559+0000) 2023-12-11T17:56:19.033 INFO:journalctl@ceph.osd.1.smithi115.stdout:Dec 11 17:56:18 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-1[107812]: 2023-12-11T17:56:18.866+0000 7f3bf3990700 -1 osd.1 97 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2023-12-11T17:55:52.633718+0000 front 2023-12-11T17:55:52.633748+0000 (oldest deadline 2023-12-11T17:56:17.333559+0000) 2023-12-11T17:56:19.033 INFO:journalctl@ceph.osd.1.smithi115.stdout:Dec 11 17:56:18 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-1[107812]: 2023-12-11T17:56:18.866+0000 7f3bf3990700 -1 osd.1 97 heartbeat_check: no reply from 172.21.15.181:6820 osd.8 since back 2023-12-11T17:55:52.633760+0000 front 2023-12-11T17:55:52.633735+0000 (oldest deadline 2023-12-11T17:56:17.333559+0000) 2023-12-11T17:56:19.033 INFO:journalctl@ceph.osd.4.smithi115.stdout:Dec 11 17:56:18 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-4[116520]: 2023-12-11T17:56:18.634+0000 7ff7103a4700 -1 osd.4 97 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-12-11T17:55:56.290542+0000 front 2023-12-11T17:55:56.290684+0000 (oldest deadline 2023-12-11T17:56:17.990646+0000) 2023-12-11T17:56:19.034 INFO:journalctl@ceph.osd.4.smithi115.stdout:Dec 11 17:56:18 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-4[116520]: 2023-12-11T17:56:18.634+0000 7ff7103a4700 -1 osd.4 97 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2023-12-11T17:55:56.290469+0000 front 2023-12-11T17:55:56.290565+0000 (oldest deadline 2023-12-11T17:56:17.990646+0000) 2023-12-11T17:56:19.034 INFO:journalctl@ceph.osd.4.smithi115.stdout:Dec 11 17:56:18 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-4[116520]: 2023-12-11T17:56:18.634+0000 7ff7103a4700 -1 osd.4 97 heartbeat_check: no reply from 172.21.15.181:6820 osd.8 since back 2023-12-11T17:55:56.290811+0000 front 2023-12-11T17:55:56.290721+0000 (oldest deadline 2023-12-11T17:56:17.990646+0000) 2023-12-11T17:56:19.034 INFO:journalctl@ceph.osd.4.smithi115.stdout:Dec 11 17:56:18 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-4[116520]: 2023-12-11T17:56:18.634+0000 7ff7103a4700 -1 osd.4 97 heartbeat_check: no reply from 172.21.15.181:6828 osd.11 since back 2023-12-11T17:55:56.290758+0000 front 2023-12-11T17:55:56.290747+0000 (oldest deadline 2023-12-11T17:56:17.990646+0000) 2023-12-11T17:56:19.034 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 17:56:18 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T17:56:18.743+0000 7f1ebb27b700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-12-11T17:55:56.935374+0000 front 2023-12-11T17:55:56.935482+0000 (oldest deadline 2023-12-11T17:56:18.635441+0000) 2023-12-11T17:56:19.034 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 17:56:18 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T17:56:18.743+0000 7f1ebb27b700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2023-12-11T17:55:56.935446+0000 front 2023-12-11T17:55:56.935516+0000 (oldest deadline 2023-12-11T17:56:18.635441+0000) 2023-12-11T17:56:19.034 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 17:56:18 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T17:56:18.743+0000 7f1ebb27b700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.181:6820 osd.8 since back 2023-12-11T17:55:56.935637+0000 front 2023-12-11T17:55:56.935655+0000 (oldest deadline 2023-12-11T17:56:18.635441+0000) 2023-12-11T17:56:19.034 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 17:56:18 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T17:56:18.743+0000 7f1ebb27b700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.181:6828 osd.11 since back 2023-12-11T17:55:56.936093+0000 front 2023-12-11T17:55:56.935592+0000 (oldest deadline 2023-12-11T17:56:18.635441+0000) 2023-12-11T17:56:19.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:56:19 smithi059 ceph-mon[98847]: osd.2 reported failed by osd.1 2023-12-11T17:56:19.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:56:19 smithi059 ceph-mon[98847]: osd.5 reported failed by osd.1 2023-12-11T17:56:19.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:56:19 smithi059 ceph-mon[98847]: osd.8 reported failed by osd.1 2023-12-11T17:56:19.532 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:56:19 smithi115 ceph-mon[102748]: osd.2 reported failed by osd.1 2023-12-11T17:56:19.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:56:19 smithi115 ceph-mon[102748]: osd.5 reported failed by osd.1 2023-12-11T17:56:19.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:56:19 smithi115 ceph-mon[102748]: osd.8 reported failed by osd.1 2023-12-11T17:56:19.704 INFO:journalctl@ceph.osd.0.smithi059.stdout:Dec 11 17:56:19 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-0[117294]: 2023-12-11T17:56:19.408+0000 7efcf51d8700 -1 osd.0 97 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-12-11T17:55:55.882654+0000 front 2023-12-11T17:55:55.882549+0000 (oldest deadline 2023-12-11T17:56:18.182726+0000) 2023-12-11T17:56:19.704 INFO:journalctl@ceph.osd.0.smithi059.stdout:Dec 11 17:56:19 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-0[117294]: 2023-12-11T17:56:19.408+0000 7efcf51d8700 -1 osd.0 97 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2023-12-11T17:55:55.882636+0000 front 2023-12-11T17:55:55.882730+0000 (oldest deadline 2023-12-11T17:56:18.182726+0000) 2023-12-11T17:56:19.704 INFO:journalctl@ceph.osd.0.smithi059.stdout:Dec 11 17:56:19 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-0[117294]: 2023-12-11T17:56:19.408+0000 7efcf51d8700 -1 osd.0 97 heartbeat_check: no reply from 172.21.15.181:6820 osd.8 since back 2023-12-11T17:55:55.882754+0000 front 2023-12-11T17:55:55.882871+0000 (oldest deadline 2023-12-11T17:56:18.182726+0000) 2023-12-11T17:56:19.704 INFO:journalctl@ceph.osd.0.smithi059.stdout:Dec 11 17:56:19 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-0[117294]: 2023-12-11T17:56:19.408+0000 7efcf51d8700 -1 osd.0 97 heartbeat_check: no reply from 172.21.15.181:6828 osd.11 since back 2023-12-11T17:55:55.882720+0000 front 2023-12-11T17:55:55.882909+0000 (oldest deadline 2023-12-11T17:56:18.182726+0000) 2023-12-11T17:56:20.033 INFO:journalctl@ceph.osd.1.smithi115.stdout:Dec 11 17:56:19 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-1[107812]: 2023-12-11T17:56:19.855+0000 7f3bf3990700 -1 osd.1 97 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-12-11T17:55:52.633801+0000 front 2023-12-11T17:55:52.633706+0000 (oldest deadline 2023-12-11T17:56:17.333559+0000) 2023-12-11T17:56:20.033 INFO:journalctl@ceph.osd.1.smithi115.stdout:Dec 11 17:56:19 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-1[107812]: 2023-12-11T17:56:19.855+0000 7f3bf3990700 -1 osd.1 97 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2023-12-11T17:55:52.633718+0000 front 2023-12-11T17:55:52.633748+0000 (oldest deadline 2023-12-11T17:56:17.333559+0000) 2023-12-11T17:56:20.033 INFO:journalctl@ceph.osd.1.smithi115.stdout:Dec 11 17:56:19 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-1[107812]: 2023-12-11T17:56:19.855+0000 7f3bf3990700 -1 osd.1 97 heartbeat_check: no reply from 172.21.15.181:6820 osd.8 since back 2023-12-11T17:55:52.633760+0000 front 2023-12-11T17:55:52.633735+0000 (oldest deadline 2023-12-11T17:56:17.333559+0000) 2023-12-11T17:56:20.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 17:56:19 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T17:56:19.779+0000 7f1ebb27b700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-12-11T17:55:56.935374+0000 front 2023-12-11T17:55:56.935482+0000 (oldest deadline 2023-12-11T17:56:18.635441+0000) 2023-12-11T17:56:20.034 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 17:56:19 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T17:56:19.779+0000 7f1ebb27b700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2023-12-11T17:55:56.935446+0000 front 2023-12-11T17:55:56.935516+0000 (oldest deadline 2023-12-11T17:56:18.635441+0000) 2023-12-11T17:56:20.034 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 17:56:19 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T17:56:19.779+0000 7f1ebb27b700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.181:6820 osd.8 since back 2023-12-11T17:55:56.935637+0000 front 2023-12-11T17:55:56.935655+0000 (oldest deadline 2023-12-11T17:56:18.635441+0000) 2023-12-11T17:56:20.034 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 17:56:19 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T17:56:19.779+0000 7f1ebb27b700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.181:6828 osd.11 since back 2023-12-11T17:55:56.936093+0000 front 2023-12-11T17:55:56.935592+0000 (oldest deadline 2023-12-11T17:56:18.635441+0000) 2023-12-11T17:56:20.034 INFO:journalctl@ceph.osd.4.smithi115.stdout:Dec 11 17:56:19 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-4[116520]: 2023-12-11T17:56:19.663+0000 7ff7103a4700 -1 osd.4 97 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-12-11T17:55:56.290542+0000 front 2023-12-11T17:55:56.290684+0000 (oldest deadline 2023-12-11T17:56:17.990646+0000) 2023-12-11T17:56:20.034 INFO:journalctl@ceph.osd.4.smithi115.stdout:Dec 11 17:56:19 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-4[116520]: 2023-12-11T17:56:19.663+0000 7ff7103a4700 -1 osd.4 97 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2023-12-11T17:55:56.290469+0000 front 2023-12-11T17:55:56.290565+0000 (oldest deadline 2023-12-11T17:56:17.990646+0000) 2023-12-11T17:56:20.034 INFO:journalctl@ceph.osd.4.smithi115.stdout:Dec 11 17:56:19 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-4[116520]: 2023-12-11T17:56:19.663+0000 7ff7103a4700 -1 osd.4 97 heartbeat_check: no reply from 172.21.15.181:6820 osd.8 since back 2023-12-11T17:55:56.290811+0000 front 2023-12-11T17:55:56.290721+0000 (oldest deadline 2023-12-11T17:56:17.990646+0000) 2023-12-11T17:56:20.034 INFO:journalctl@ceph.osd.4.smithi115.stdout:Dec 11 17:56:19 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-4[116520]: 2023-12-11T17:56:19.663+0000 7ff7103a4700 -1 osd.4 97 heartbeat_check: no reply from 172.21.15.181:6828 osd.11 since back 2023-12-11T17:55:56.290758+0000 front 2023-12-11T17:55:56.290747+0000 (oldest deadline 2023-12-11T17:56:17.990646+0000) 2023-12-11T17:56:20.121 INFO:journalctl@ceph.osd.6.smithi059.stdout:Dec 11 17:56:19 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-6[133999]: 2023-12-11T17:56:19.701+0000 7f6dafb58700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-12-11T17:55:57.255391+0000 front 2023-12-11T17:55:57.255405+0000 (oldest deadline 2023-12-11T17:56:19.555520+0000) 2023-12-11T17:56:20.122 INFO:journalctl@ceph.osd.6.smithi059.stdout:Dec 11 17:56:19 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-6[133999]: 2023-12-11T17:56:19.701+0000 7f6dafb58700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2023-12-11T17:55:57.255520+0000 front 2023-12-11T17:55:57.255550+0000 (oldest deadline 2023-12-11T17:56:19.555520+0000) 2023-12-11T17:56:20.122 INFO:journalctl@ceph.osd.6.smithi059.stdout:Dec 11 17:56:19 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-6[133999]: 2023-12-11T17:56:19.701+0000 7f6dafb58700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.181:6820 osd.8 since back 2023-12-11T17:55:57.255576+0000 front 2023-12-11T17:55:54.955213+0000 (oldest deadline 2023-12-11T17:56:17.255010+0000) 2023-12-11T17:56:20.122 INFO:journalctl@ceph.osd.6.smithi059.stdout:Dec 11 17:56:19 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-6[133999]: 2023-12-11T17:56:19.701+0000 7f6dafb58700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.181:6828 osd.11 since back 2023-12-11T17:55:54.955300+0000 front 2023-12-11T17:55:54.955265+0000 (oldest deadline 2023-12-11T17:56:17.255010+0000) 2023-12-11T17:56:20.381 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:56:20 smithi115 ceph-mon[102748]: pgmap v311: 129 pgs: 1 active+clean+laggy, 128 active+clean; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 597 B/s rd, 1 op/s 2023-12-11T17:56:20.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:56:20 smithi059 ceph-mon[98847]: pgmap v311: 129 pgs: 1 active+clean+laggy, 128 active+clean; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 597 B/s rd, 1 op/s 2023-12-11T17:56:20.405 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 17:56:20 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T17:56:20.156+0000 7f86220d5700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-12-11T17:55:56.631215+0000 front 2023-12-11T17:55:56.631149+0000 (oldest deadline 2023-12-11T17:56:19.531227+0000) 2023-12-11T17:56:20.405 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 17:56:20 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T17:56:20.156+0000 7f86220d5700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2023-12-11T17:55:56.631403+0000 front 2023-12-11T17:55:56.631306+0000 (oldest deadline 2023-12-11T17:56:19.531227+0000) 2023-12-11T17:56:20.405 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 17:56:20 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T17:56:20.156+0000 7f86220d5700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.181:6820 osd.8 since back 2023-12-11T17:55:56.631231+0000 front 2023-12-11T17:55:56.631424+0000 (oldest deadline 2023-12-11T17:56:19.531227+0000) 2023-12-11T17:56:20.405 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 17:56:20 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T17:56:20.156+0000 7f86220d5700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.181:6828 osd.11 since back 2023-12-11T17:55:56.631444+0000 front 2023-12-11T17:55:56.631344+0000 (oldest deadline 2023-12-11T17:56:19.531227+0000) 2023-12-11T17:56:20.655 INFO:journalctl@ceph.osd.7.smithi115.stdout:Dec 11 17:56:20 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-7[124665]: 2023-12-11T17:56:20.378+0000 7f9a71cae700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-12-11T17:55:55.092841+0000 front 2023-12-11T17:55:55.092881+0000 (oldest deadline 2023-12-11T17:56:19.793035+0000) 2023-12-11T17:56:20.656 INFO:journalctl@ceph.osd.7.smithi115.stdout:Dec 11 17:56:20 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-7[124665]: 2023-12-11T17:56:20.378+0000 7f9a71cae700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2023-12-11T17:55:55.092987+0000 front 2023-12-11T17:55:55.093028+0000 (oldest deadline 2023-12-11T17:56:19.793035+0000) 2023-12-11T17:56:20.656 INFO:journalctl@ceph.osd.7.smithi115.stdout:Dec 11 17:56:20 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-7[124665]: 2023-12-11T17:56:20.378+0000 7f9a71cae700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.181:6820 osd.8 since back 2023-12-11T17:55:55.093014+0000 front 2023-12-11T17:55:55.092965+0000 (oldest deadline 2023-12-11T17:56:19.793035+0000) 2023-12-11T17:56:20.656 INFO:journalctl@ceph.osd.7.smithi115.stdout:Dec 11 17:56:20 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-7[124665]: 2023-12-11T17:56:20.378+0000 7f9a71cae700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.181:6828 osd.11 since back 2023-12-11T17:55:55.093040+0000 front 2023-12-11T17:55:55.093127+0000 (oldest deadline 2023-12-11T17:56:19.793035+0000) 2023-12-11T17:56:20.746 INFO:journalctl@ceph.osd.0.smithi059.stdout:Dec 11 17:56:20 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-0[117294]: 2023-12-11T17:56:20.405+0000 7efcf51d8700 -1 osd.0 97 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-12-11T17:55:55.882654+0000 front 2023-12-11T17:55:55.882549+0000 (oldest deadline 2023-12-11T17:56:18.182726+0000) 2023-12-11T17:56:20.746 INFO:journalctl@ceph.osd.0.smithi059.stdout:Dec 11 17:56:20 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-0[117294]: 2023-12-11T17:56:20.405+0000 7efcf51d8700 -1 osd.0 97 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2023-12-11T17:55:55.882636+0000 front 2023-12-11T17:55:55.882730+0000 (oldest deadline 2023-12-11T17:56:18.182726+0000) 2023-12-11T17:56:20.746 INFO:journalctl@ceph.osd.0.smithi059.stdout:Dec 11 17:56:20 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-0[117294]: 2023-12-11T17:56:20.405+0000 7efcf51d8700 -1 osd.0 97 heartbeat_check: no reply from 172.21.15.181:6820 osd.8 since back 2023-12-11T17:55:55.882754+0000 front 2023-12-11T17:55:55.882871+0000 (oldest deadline 2023-12-11T17:56:18.182726+0000) 2023-12-11T17:56:20.746 INFO:journalctl@ceph.osd.0.smithi059.stdout:Dec 11 17:56:20 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-0[117294]: 2023-12-11T17:56:20.405+0000 7efcf51d8700 -1 osd.0 97 heartbeat_check: no reply from 172.21.15.181:6828 osd.11 since back 2023-12-11T17:55:55.882720+0000 front 2023-12-11T17:55:55.882909+0000 (oldest deadline 2023-12-11T17:56:18.182726+0000) 2023-12-11T17:56:21.033 INFO:journalctl@ceph.osd.1.smithi115.stdout:Dec 11 17:56:20 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-1[107812]: 2023-12-11T17:56:20.817+0000 7f3bf3990700 -1 osd.1 97 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-12-11T17:55:52.633801+0000 front 2023-12-11T17:55:52.633706+0000 (oldest deadline 2023-12-11T17:56:17.333559+0000) 2023-12-11T17:56:21.033 INFO:journalctl@ceph.osd.1.smithi115.stdout:Dec 11 17:56:20 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-1[107812]: 2023-12-11T17:56:20.817+0000 7f3bf3990700 -1 osd.1 97 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2023-12-11T17:55:52.633718+0000 front 2023-12-11T17:55:52.633748+0000 (oldest deadline 2023-12-11T17:56:17.333559+0000) 2023-12-11T17:56:21.033 INFO:journalctl@ceph.osd.1.smithi115.stdout:Dec 11 17:56:20 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-1[107812]: 2023-12-11T17:56:20.817+0000 7f3bf3990700 -1 osd.1 97 heartbeat_check: no reply from 172.21.15.181:6820 osd.8 since back 2023-12-11T17:55:52.633760+0000 front 2023-12-11T17:55:52.633735+0000 (oldest deadline 2023-12-11T17:56:17.333559+0000) 2023-12-11T17:56:21.033 INFO:journalctl@ceph.osd.4.smithi115.stdout:Dec 11 17:56:20 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-4[116520]: 2023-12-11T17:56:20.653+0000 7ff7103a4700 -1 osd.4 97 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-12-11T17:55:56.290542+0000 front 2023-12-11T17:55:56.290684+0000 (oldest deadline 2023-12-11T17:56:17.990646+0000) 2023-12-11T17:56:21.033 INFO:journalctl@ceph.osd.4.smithi115.stdout:Dec 11 17:56:20 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-4[116520]: 2023-12-11T17:56:20.653+0000 7ff7103a4700 -1 osd.4 97 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2023-12-11T17:55:56.290469+0000 front 2023-12-11T17:55:56.290565+0000 (oldest deadline 2023-12-11T17:56:17.990646+0000) 2023-12-11T17:56:21.033 INFO:journalctl@ceph.osd.4.smithi115.stdout:Dec 11 17:56:20 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-4[116520]: 2023-12-11T17:56:20.653+0000 7ff7103a4700 -1 osd.4 97 heartbeat_check: no reply from 172.21.15.181:6820 osd.8 since back 2023-12-11T17:55:56.290811+0000 front 2023-12-11T17:55:56.290721+0000 (oldest deadline 2023-12-11T17:56:17.990646+0000) 2023-12-11T17:56:21.033 INFO:journalctl@ceph.osd.4.smithi115.stdout:Dec 11 17:56:20 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-4[116520]: 2023-12-11T17:56:20.653+0000 7ff7103a4700 -1 osd.4 97 heartbeat_check: no reply from 172.21.15.181:6828 osd.11 since back 2023-12-11T17:55:56.290758+0000 front 2023-12-11T17:55:56.290747+0000 (oldest deadline 2023-12-11T17:56:17.990646+0000) 2023-12-11T17:56:21.034 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 17:56:20 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T17:56:20.734+0000 7f1ebb27b700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-12-11T17:55:56.935374+0000 front 2023-12-11T17:55:56.935482+0000 (oldest deadline 2023-12-11T17:56:18.635441+0000) 2023-12-11T17:56:21.034 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 17:56:20 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T17:56:20.734+0000 7f1ebb27b700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2023-12-11T17:55:56.935446+0000 front 2023-12-11T17:55:56.935516+0000 (oldest deadline 2023-12-11T17:56:18.635441+0000) 2023-12-11T17:56:21.034 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 17:56:20 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T17:56:20.734+0000 7f1ebb27b700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.181:6820 osd.8 since back 2023-12-11T17:55:56.935637+0000 front 2023-12-11T17:55:56.935655+0000 (oldest deadline 2023-12-11T17:56:18.635441+0000) 2023-12-11T17:56:21.034 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 17:56:20 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T17:56:20.734+0000 7f1ebb27b700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.181:6828 osd.11 since back 2023-12-11T17:55:56.936093+0000 front 2023-12-11T17:55:56.935592+0000 (oldest deadline 2023-12-11T17:56:18.635441+0000) 2023-12-11T17:56:21.109 INFO:journalctl@ceph.osd.6.smithi059.stdout:Dec 11 17:56:20 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-6[133999]: 2023-12-11T17:56:20.741+0000 7f6dafb58700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-12-11T17:55:57.255391+0000 front 2023-12-11T17:55:57.255405+0000 (oldest deadline 2023-12-11T17:56:19.555520+0000) 2023-12-11T17:56:21.109 INFO:journalctl@ceph.osd.6.smithi059.stdout:Dec 11 17:56:20 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-6[133999]: 2023-12-11T17:56:20.741+0000 7f6dafb58700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2023-12-11T17:55:57.255520+0000 front 2023-12-11T17:55:57.255550+0000 (oldest deadline 2023-12-11T17:56:19.555520+0000) 2023-12-11T17:56:21.110 INFO:journalctl@ceph.osd.6.smithi059.stdout:Dec 11 17:56:20 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-6[133999]: 2023-12-11T17:56:20.741+0000 7f6dafb58700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.181:6820 osd.8 since back 2023-12-11T17:55:57.255576+0000 front 2023-12-11T17:55:54.955213+0000 (oldest deadline 2023-12-11T17:56:17.255010+0000) 2023-12-11T17:56:21.110 INFO:journalctl@ceph.osd.6.smithi059.stdout:Dec 11 17:56:20 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-6[133999]: 2023-12-11T17:56:20.741+0000 7f6dafb58700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.181:6828 osd.11 since back 2023-12-11T17:55:54.955300+0000 front 2023-12-11T17:55:54.955265+0000 (oldest deadline 2023-12-11T17:56:17.255010+0000) 2023-12-11T17:56:21.392 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 17:56:21 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T17:56:21.107+0000 7f86220d5700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-12-11T17:55:56.631215+0000 front 2023-12-11T17:55:56.631149+0000 (oldest deadline 2023-12-11T17:56:19.531227+0000) 2023-12-11T17:56:21.392 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 17:56:21 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T17:56:21.107+0000 7f86220d5700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2023-12-11T17:55:56.631403+0000 front 2023-12-11T17:55:56.631306+0000 (oldest deadline 2023-12-11T17:56:19.531227+0000) 2023-12-11T17:56:21.393 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 17:56:21 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T17:56:21.107+0000 7f86220d5700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.181:6820 osd.8 since back 2023-12-11T17:55:56.631231+0000 front 2023-12-11T17:55:56.631424+0000 (oldest deadline 2023-12-11T17:56:19.531227+0000) 2023-12-11T17:56:21.393 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 17:56:21 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T17:56:21.107+0000 7f86220d5700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.181:6828 osd.11 since back 2023-12-11T17:55:56.631444+0000 front 2023-12-11T17:55:56.631344+0000 (oldest deadline 2023-12-11T17:56:19.531227+0000) 2023-12-11T17:56:21.633 INFO:journalctl@ceph.osd.7.smithi115.stdout:Dec 11 17:56:21 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-7[124665]: 2023-12-11T17:56:21.345+0000 7f9a71cae700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-12-11T17:55:55.092841+0000 front 2023-12-11T17:55:55.092881+0000 (oldest deadline 2023-12-11T17:56:19.793035+0000) 2023-12-11T17:56:21.633 INFO:journalctl@ceph.osd.7.smithi115.stdout:Dec 11 17:56:21 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-7[124665]: 2023-12-11T17:56:21.345+0000 7f9a71cae700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2023-12-11T17:55:55.092987+0000 front 2023-12-11T17:55:55.093028+0000 (oldest deadline 2023-12-11T17:56:19.793035+0000) 2023-12-11T17:56:21.633 INFO:journalctl@ceph.osd.7.smithi115.stdout:Dec 11 17:56:21 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-7[124665]: 2023-12-11T17:56:21.345+0000 7f9a71cae700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.181:6820 osd.8 since back 2023-12-11T17:55:55.093014+0000 front 2023-12-11T17:55:55.092965+0000 (oldest deadline 2023-12-11T17:56:19.793035+0000) 2023-12-11T17:56:21.633 INFO:journalctl@ceph.osd.7.smithi115.stdout:Dec 11 17:56:21 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-7[124665]: 2023-12-11T17:56:21.345+0000 7f9a71cae700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.181:6828 osd.11 since back 2023-12-11T17:55:55.093040+0000 front 2023-12-11T17:55:55.093127+0000 (oldest deadline 2023-12-11T17:56:19.793035+0000) 2023-12-11T17:56:21.653 INFO:journalctl@ceph.osd.0.smithi059.stdout:Dec 11 17:56:21 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-0[117294]: 2023-12-11T17:56:21.390+0000 7efcf51d8700 -1 osd.0 97 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-12-11T17:55:55.882654+0000 front 2023-12-11T17:55:55.882549+0000 (oldest deadline 2023-12-11T17:56:18.182726+0000) 2023-12-11T17:56:21.653 INFO:journalctl@ceph.osd.0.smithi059.stdout:Dec 11 17:56:21 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-0[117294]: 2023-12-11T17:56:21.390+0000 7efcf51d8700 -1 osd.0 97 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2023-12-11T17:55:55.882636+0000 front 2023-12-11T17:55:55.882730+0000 (oldest deadline 2023-12-11T17:56:18.182726+0000) 2023-12-11T17:56:21.653 INFO:journalctl@ceph.osd.0.smithi059.stdout:Dec 11 17:56:21 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-0[117294]: 2023-12-11T17:56:21.390+0000 7efcf51d8700 -1 osd.0 97 heartbeat_check: no reply from 172.21.15.181:6820 osd.8 since back 2023-12-11T17:55:55.882754+0000 front 2023-12-11T17:55:55.882871+0000 (oldest deadline 2023-12-11T17:56:18.182726+0000) 2023-12-11T17:56:21.653 INFO:journalctl@ceph.osd.0.smithi059.stdout:Dec 11 17:56:21 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-0[117294]: 2023-12-11T17:56:21.390+0000 7efcf51d8700 -1 osd.0 97 heartbeat_check: no reply from 172.21.15.181:6828 osd.11 since back 2023-12-11T17:55:55.882720+0000 front 2023-12-11T17:55:55.882909+0000 (oldest deadline 2023-12-11T17:56:18.182726+0000) 2023-12-11T17:56:21.904 INFO:journalctl@ceph.osd.6.smithi059.stdout:Dec 11 17:56:21 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-6[133999]: 2023-12-11T17:56:21.708+0000 7f6dafb58700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-12-11T17:55:57.255391+0000 front 2023-12-11T17:55:57.255405+0000 (oldest deadline 2023-12-11T17:56:19.555520+0000) 2023-12-11T17:56:21.904 INFO:journalctl@ceph.osd.6.smithi059.stdout:Dec 11 17:56:21 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-6[133999]: 2023-12-11T17:56:21.708+0000 7f6dafb58700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2023-12-11T17:55:57.255520+0000 front 2023-12-11T17:55:57.255550+0000 (oldest deadline 2023-12-11T17:56:19.555520+0000) 2023-12-11T17:56:21.904 INFO:journalctl@ceph.osd.6.smithi059.stdout:Dec 11 17:56:21 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-6[133999]: 2023-12-11T17:56:21.708+0000 7f6dafb58700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.181:6820 osd.8 since back 2023-12-11T17:55:57.255576+0000 front 2023-12-11T17:55:54.955213+0000 (oldest deadline 2023-12-11T17:56:17.255010+0000) 2023-12-11T17:56:21.904 INFO:journalctl@ceph.osd.6.smithi059.stdout:Dec 11 17:56:21 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-6[133999]: 2023-12-11T17:56:21.708+0000 7f6dafb58700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.181:6828 osd.11 since back 2023-12-11T17:55:54.955300+0000 front 2023-12-11T17:55:54.955265+0000 (oldest deadline 2023-12-11T17:56:17.255010+0000) 2023-12-11T17:56:21.905 INFO:journalctl@ceph.osd.3.smithi059.stdout:Dec 11 17:56:21 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-3[126028]: 2023-12-11T17:56:21.650+0000 7fe38d5cc700 -1 osd.3 97 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-12-11T17:55:56.667369+0000 front 2023-12-11T17:55:56.667351+0000 (oldest deadline 2023-12-11T17:56:21.367365+0000) 2023-12-11T17:56:21.905 INFO:journalctl@ceph.osd.3.smithi059.stdout:Dec 11 17:56:21 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-3[126028]: 2023-12-11T17:56:21.650+0000 7fe38d5cc700 -1 osd.3 97 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2023-12-11T17:55:56.667482+0000 front 2023-12-11T17:55:56.667391+0000 (oldest deadline 2023-12-11T17:56:21.367365+0000) 2023-12-11T17:56:21.905 INFO:journalctl@ceph.osd.3.smithi059.stdout:Dec 11 17:56:21 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-3[126028]: 2023-12-11T17:56:21.650+0000 7fe38d5cc700 -1 osd.3 97 heartbeat_check: no reply from 172.21.15.181:6828 osd.11 since back 2023-12-11T17:55:56.667446+0000 front 2023-12-11T17:55:56.667624+0000 (oldest deadline 2023-12-11T17:56:21.367365+0000) 2023-12-11T17:56:22.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 17:56:21 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T17:56:21.740+0000 7f1ebb27b700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-12-11T17:55:56.935374+0000 front 2023-12-11T17:55:56.935482+0000 (oldest deadline 2023-12-11T17:56:18.635441+0000) 2023-12-11T17:56:22.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 17:56:21 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T17:56:21.740+0000 7f1ebb27b700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2023-12-11T17:55:56.935446+0000 front 2023-12-11T17:55:56.935516+0000 (oldest deadline 2023-12-11T17:56:18.635441+0000) 2023-12-11T17:56:22.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 17:56:21 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T17:56:21.740+0000 7f1ebb27b700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.181:6820 osd.8 since back 2023-12-11T17:55:56.935637+0000 front 2023-12-11T17:55:56.935655+0000 (oldest deadline 2023-12-11T17:56:18.635441+0000) 2023-12-11T17:56:22.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 17:56:21 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T17:56:21.740+0000 7f1ebb27b700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.181:6828 osd.11 since back 2023-12-11T17:55:56.936093+0000 front 2023-12-11T17:55:56.935592+0000 (oldest deadline 2023-12-11T17:56:18.635441+0000) 2023-12-11T17:56:22.033 INFO:journalctl@ceph.osd.1.smithi115.stdout:Dec 11 17:56:21 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-1[107812]: 2023-12-11T17:56:21.798+0000 7f3bf3990700 -1 osd.1 97 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-12-11T17:55:52.633801+0000 front 2023-12-11T17:55:52.633706+0000 (oldest deadline 2023-12-11T17:56:17.333559+0000) 2023-12-11T17:56:22.033 INFO:journalctl@ceph.osd.1.smithi115.stdout:Dec 11 17:56:21 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-1[107812]: 2023-12-11T17:56:21.798+0000 7f3bf3990700 -1 osd.1 97 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2023-12-11T17:55:52.633718+0000 front 2023-12-11T17:55:52.633748+0000 (oldest deadline 2023-12-11T17:56:17.333559+0000) 2023-12-11T17:56:22.034 INFO:journalctl@ceph.osd.1.smithi115.stdout:Dec 11 17:56:21 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-1[107812]: 2023-12-11T17:56:21.798+0000 7f3bf3990700 -1 osd.1 97 heartbeat_check: no reply from 172.21.15.181:6820 osd.8 since back 2023-12-11T17:55:52.633760+0000 front 2023-12-11T17:55:52.633735+0000 (oldest deadline 2023-12-11T17:56:17.333559+0000) 2023-12-11T17:56:22.034 INFO:journalctl@ceph.osd.4.smithi115.stdout:Dec 11 17:56:21 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-4[116520]: 2023-12-11T17:56:21.630+0000 7ff7103a4700 -1 osd.4 97 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-12-11T17:55:56.290542+0000 front 2023-12-11T17:55:56.290684+0000 (oldest deadline 2023-12-11T17:56:17.990646+0000) 2023-12-11T17:56:22.034 INFO:journalctl@ceph.osd.4.smithi115.stdout:Dec 11 17:56:21 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-4[116520]: 2023-12-11T17:56:21.630+0000 7ff7103a4700 -1 osd.4 97 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2023-12-11T17:55:56.290469+0000 front 2023-12-11T17:55:56.290565+0000 (oldest deadline 2023-12-11T17:56:17.990646+0000) 2023-12-11T17:56:22.034 INFO:journalctl@ceph.osd.4.smithi115.stdout:Dec 11 17:56:21 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-4[116520]: 2023-12-11T17:56:21.630+0000 7ff7103a4700 -1 osd.4 97 heartbeat_check: no reply from 172.21.15.181:6820 osd.8 since back 2023-12-11T17:55:56.290811+0000 front 2023-12-11T17:55:56.290721+0000 (oldest deadline 2023-12-11T17:56:17.990646+0000) 2023-12-11T17:56:22.034 INFO:journalctl@ceph.osd.4.smithi115.stdout:Dec 11 17:56:21 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-4[116520]: 2023-12-11T17:56:21.630+0000 7ff7103a4700 -1 osd.4 97 heartbeat_check: no reply from 172.21.15.181:6828 osd.11 since back 2023-12-11T17:55:56.290758+0000 front 2023-12-11T17:55:56.290747+0000 (oldest deadline 2023-12-11T17:56:17.990646+0000) 2023-12-11T17:56:22.404 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 17:56:22 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T17:56:22.080+0000 7f86220d5700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.181:6804 osd.2 since back 2023-12-11T17:55:56.631215+0000 front 2023-12-11T17:55:56.631149+0000 (oldest deadline 2023-12-11T17:56:19.531227+0000) 2023-12-11T17:56:22.405 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 17:56:22 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T17:56:22.080+0000 7f86220d5700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.181:6812 osd.5 since back 2023-12-11T17:55:56.631403+0000 front 2023-12-11T17:55:56.631306+0000 (oldest deadline 2023-12-11T17:56:19.531227+0000) 2023-12-11T17:56:22.405 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 17:56:22 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T17:56:22.080+0000 7f86220d5700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.181:6820 osd.8 since back 2023-12-11T17:55:56.631231+0000 front 2023-12-11T17:55:56.631424+0000 (oldest deadline 2023-12-11T17:56:19.531227+0000) 2023-12-11T17:56:22.405 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 17:56:22 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T17:56:22.080+0000 7f86220d5700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.181:6828 osd.11 since back 2023-12-11T17:55:56.631444+0000 front 2023-12-11T17:55:56.631344+0000 (oldest deadline 2023-12-11T17:56:19.531227+0000) 2023-12-11T17:56:22.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:56:22 smithi059 ceph-mon[98847]: pgmap v312: 129 pgs: 2 active+clean+laggy, 127 active+clean; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 597 B/s rd, 1 op/s 2023-12-11T17:56:22.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:56:22 smithi059 ceph-mon[98847]: osd.2 reported failed by osd.4 2023-12-11T17:56:22.406 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:56:22 smithi059 ceph-mon[98847]: osd.2 failed (root=default,host=smithi181) (2 reporters from different osd after 25.000377 >= grace 20.000000) 2023-12-11T17:56:22.406 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:56:22 smithi059 ceph-mon[98847]: osd.5 reported failed by osd.4 2023-12-11T17:56:22.406 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:56:22 smithi059 ceph-mon[98847]: osd.5 failed (root=default,host=smithi181) (2 reporters from different osd after 25.000557 >= grace 20.000000) 2023-12-11T17:56:22.406 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:56:22 smithi059 ceph-mon[98847]: osd.8 reported failed by osd.4 2023-12-11T17:56:22.406 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:56:22 smithi059 ceph-mon[98847]: osd.8 failed (root=default,host=smithi181) (2 reporters from different osd after 25.000695 >= grace 20.000000) 2023-12-11T17:56:22.406 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:56:22 smithi059 ceph-mon[98847]: osd.11 reported failed by osd.4 2023-12-11T17:56:22.406 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:56:22 smithi059 ceph-mon[98847]: osd.2 reported failed by osd.6 2023-12-11T17:56:22.406 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:56:22 smithi059 ceph-mon[98847]: osd.5 reported failed by osd.6 2023-12-11T17:56:22.406 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:56:22 smithi059 ceph-mon[98847]: osd.8 reported failed by osd.6 2023-12-11T17:56:22.407 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:56:22 smithi059 ceph-mon[98847]: osd.11 reported failed by osd.6 2023-12-11T17:56:22.407 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:56:22 smithi059 ceph-mon[98847]: osd.11 failed (root=default,host=smithi181) (2 reporters from different osd after 25.078269 >= grace 20.000000) 2023-12-11T17:56:22.407 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:56:22 smithi059 ceph-mon[98847]: osd.2 reported failed by osd.10 2023-12-11T17:56:22.407 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:56:22 smithi059 ceph-mon[98847]: osd.5 reported failed by osd.10 2023-12-11T17:56:22.407 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:56:22 smithi059 ceph-mon[98847]: osd.8 reported failed by osd.10 2023-12-11T17:56:22.407 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:56:22 smithi059 ceph-mon[98847]: osd.11 reported failed by osd.10 2023-12-11T17:56:22.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:56:22 smithi115 ceph-mon[102748]: pgmap v312: 129 pgs: 2 active+clean+laggy, 127 active+clean; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 597 B/s rd, 1 op/s 2023-12-11T17:56:22.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:56:22 smithi115 ceph-mon[102748]: osd.2 reported failed by osd.4 2023-12-11T17:56:22.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:56:22 smithi115 ceph-mon[102748]: osd.2 failed (root=default,host=smithi181) (2 reporters from different osd after 25.000377 >= grace 20.000000) 2023-12-11T17:56:22.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:56:22 smithi115 ceph-mon[102748]: osd.5 reported failed by osd.4 2023-12-11T17:56:22.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:56:22 smithi115 ceph-mon[102748]: osd.5 failed (root=default,host=smithi181) (2 reporters from different osd after 25.000557 >= grace 20.000000) 2023-12-11T17:56:22.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:56:22 smithi115 ceph-mon[102748]: osd.8 reported failed by osd.4 2023-12-11T17:56:22.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:56:22 smithi115 ceph-mon[102748]: osd.8 failed (root=default,host=smithi181) (2 reporters from different osd after 25.000695 >= grace 20.000000) 2023-12-11T17:56:22.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:56:22 smithi115 ceph-mon[102748]: osd.11 reported failed by osd.4 2023-12-11T17:56:22.534 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:56:22 smithi115 ceph-mon[102748]: osd.2 reported failed by osd.6 2023-12-11T17:56:22.534 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:56:22 smithi115 ceph-mon[102748]: osd.5 reported failed by osd.6 2023-12-11T17:56:22.534 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:56:22 smithi115 ceph-mon[102748]: osd.8 reported failed by osd.6 2023-12-11T17:56:22.534 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:56:22 smithi115 ceph-mon[102748]: osd.11 reported failed by osd.6 2023-12-11T17:56:22.534 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:56:22 smithi115 ceph-mon[102748]: osd.11 failed (root=default,host=smithi181) (2 reporters from different osd after 25.078269 >= grace 20.000000) 2023-12-11T17:56:22.534 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:56:22 smithi115 ceph-mon[102748]: osd.2 reported failed by osd.10 2023-12-11T17:56:22.534 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:56:22 smithi115 ceph-mon[102748]: osd.5 reported failed by osd.10 2023-12-11T17:56:22.534 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:56:22 smithi115 ceph-mon[102748]: osd.8 reported failed by osd.10 2023-12-11T17:56:22.534 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:56:22 smithi115 ceph-mon[102748]: osd.11 reported failed by osd.10 2023-12-11T17:56:23.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:56:23 smithi115 ceph-mon[102748]: Health check failed: 4 osds down (OSD_DOWN) 2023-12-11T17:56:23.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:56:23 smithi115 ceph-mon[102748]: Health check failed: 1 host (4 osds) down (OSD_HOST_DOWN) 2023-12-11T17:56:23.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:56:23 smithi115 ceph-mon[102748]: osdmap e98: 12 total, 8 up, 12 in 2023-12-11T17:56:23.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:56:23 smithi115 ceph-mon[102748]: pgmap v314: 129 pgs: 15 peering, 38 stale+active+clean, 3 active+clean+laggy, 73 active+clean; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s wr, 5 op/s 2023-12-11T17:56:23.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:56:23 smithi115 ceph-mon[102748]: osdmap e99: 12 total, 8 up, 12 in 2023-12-11T17:56:23.654 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:56:23 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T17:56:23.204+0000 7f45eaf9f700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-11T17:56:23.654 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:56:23 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T17:56:23.204+0000 7f45eaf9f700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-11T17:56:23.654 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:56:23 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T17:56:23.204+0000 7f45eaf9f700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-11T17:56:23.654 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:56:23 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T17:56:23.204+0000 7f45eaf9f700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-11T17:56:23.655 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:56:23 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T17:56:23.204+0000 7f45eaf9f700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-11T17:56:23.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:56:23 smithi059 ceph-mon[98847]: Health check failed: 4 osds down (OSD_DOWN) 2023-12-11T17:56:23.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:56:23 smithi059 ceph-mon[98847]: Health check failed: 1 host (4 osds) down (OSD_HOST_DOWN) 2023-12-11T17:56:23.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:56:23 smithi059 ceph-mon[98847]: osdmap e98: 12 total, 8 up, 12 in 2023-12-11T17:56:23.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:56:23 smithi059 ceph-mon[98847]: pgmap v314: 129 pgs: 15 peering, 38 stale+active+clean, 3 active+clean+laggy, 73 active+clean; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s wr, 5 op/s 2023-12-11T17:56:23.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:56:23 smithi059 ceph-mon[98847]: osdmap e99: 12 total, 8 up, 12 in 2023-12-11T17:56:25.154 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:56:24 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T17:56:24.841+0000 7f4637bd5700 -1 log_channel(cephadm) log [ERR] : host smithi181 `cephadm gather-facts` failed: Cannot decode JSON: 2023-12-11T17:56:25.154 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:56:24 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: Traceback (most recent call last): 2023-12-11T17:56:25.154 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:56:24 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: File "/usr/share/ceph/mgr/cephadm/serve.py", line 1512, in _run_cephadm_json 2023-12-11T17:56:25.155 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:56:24 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: return json.loads(''.join(out)) 2023-12-11T17:56:25.155 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:56:24 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: File "/lib64/python3.6/json/__init__.py", line 354, in loads 2023-12-11T17:56:25.155 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:56:24 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: return _default_decoder.decode(s) 2023-12-11T17:56:25.155 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:56:24 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: File "/lib64/python3.6/json/decoder.py", line 339, in decode 2023-12-11T17:56:25.155 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:56:24 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: obj, end = self.raw_decode(s, idx=_w(s, 0).end()) 2023-12-11T17:56:25.155 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:56:24 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: File "/lib64/python3.6/json/decoder.py", line 357, in raw_decode 2023-12-11T17:56:25.155 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:56:24 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: raise JSONDecodeError("Expecting value", s, err.value) from None 2023-12-11T17:56:25.155 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:56:24 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0) 2023-12-11T17:56:25.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:56:25 smithi115 ceph-mon[102748]: mgrmap e26: x(active, since 10m), standbys: z, y 2023-12-11T17:56:25.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:56:25 smithi115 ceph-mon[102748]: pgmap v316: 129 pgs: 40 active+clean, 1 active+clean+laggy, 10 stale+active+clean, 2 down, 15 peering, 39 active+undersized+degraded, 22 undersized+degraded+peered; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 537 KiB/s wr, 28 op/s; 52425/300060 objects degraded (17.472%) 2023-12-11T17:56:25.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:56:25 smithi115 ceph-mon[102748]: host smithi181 `cephadm gather-facts` failed: Cannot decode JSON: 2023-12-11T17:56:25.533 INFO:journalctl@ceph.mon.b.smithi115.stdout: Traceback (most recent call last): 2023-12-11T17:56:25.533 INFO:journalctl@ceph.mon.b.smithi115.stdout: File "/usr/share/ceph/mgr/cephadm/serve.py", line 1512, in _run_cephadm_json 2023-12-11T17:56:25.533 INFO:journalctl@ceph.mon.b.smithi115.stdout: return json.loads(''.join(out)) 2023-12-11T17:56:25.533 INFO:journalctl@ceph.mon.b.smithi115.stdout: File "/lib64/python3.6/json/__init__.py", line 354, in loads 2023-12-11T17:56:25.533 INFO:journalctl@ceph.mon.b.smithi115.stdout: return _default_decoder.decode(s) 2023-12-11T17:56:25.534 INFO:journalctl@ceph.mon.b.smithi115.stdout: File "/lib64/python3.6/json/decoder.py", line 339, in decode 2023-12-11T17:56:25.534 INFO:journalctl@ceph.mon.b.smithi115.stdout: obj, end = self.raw_decode(s, idx=_w(s, 0).end()) 2023-12-11T17:56:25.534 INFO:journalctl@ceph.mon.b.smithi115.stdout: File "/lib64/python3.6/json/decoder.py", line 357, in raw_decode 2023-12-11T17:56:25.534 INFO:journalctl@ceph.mon.b.smithi115.stdout: raise JSONDecodeError("Expecting value", s, err.value) from None 2023-12-11T17:56:25.534 INFO:journalctl@ceph.mon.b.smithi115.stdout: json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0) 2023-12-11T17:56:25.534 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:56:25 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:56:25.534 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:56:25 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:56:25.534 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:56:25 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:56:25.534 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:56:25 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:56:25.534 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:56:25 smithi115 ceph-mon[102748]: Adjusting osd_memory_target on smithi181 to 196.2M 2023-12-11T17:56:25.534 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:56:25 smithi115 ceph-mon[102748]: Unable to set osd_memory_target on smithi181 to 205735936: error parsing value: Value '205735936' is below minimum 939524096 2023-12-11T17:56:25.534 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:56:25 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:56:25.534 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:56:25 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:56:25.534 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:56:25 smithi115 ceph-mon[102748]: pgmap v317: 129 pgs: 40 active+clean, 1 active+clean+laggy, 10 stale+active+clean, 2 down, 15 peering, 39 active+undersized+degraded, 22 undersized+degraded+peered; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 646 KiB/s wr, 34 op/s; 52425/300060 objects degraded (17.472%) 2023-12-11T17:56:25.534 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:56:25 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:56:25.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:56:25 smithi059 ceph-mon[98847]: mgrmap e26: x(active, since 10m), standbys: z, y 2023-12-11T17:56:25.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:56:25 smithi059 ceph-mon[98847]: pgmap v316: 129 pgs: 40 active+clean, 1 active+clean+laggy, 10 stale+active+clean, 2 down, 15 peering, 39 active+undersized+degraded, 22 undersized+degraded+peered; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 537 KiB/s wr, 28 op/s; 52425/300060 objects degraded (17.472%) 2023-12-11T17:56:25.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:56:25 smithi059 ceph-mon[98847]: host smithi181 `cephadm gather-facts` failed: Cannot decode JSON: 2023-12-11T17:56:25.655 INFO:journalctl@ceph.mon.a.smithi059.stdout: Traceback (most recent call last): 2023-12-11T17:56:25.655 INFO:journalctl@ceph.mon.a.smithi059.stdout: File "/usr/share/ceph/mgr/cephadm/serve.py", line 1512, in _run_cephadm_json 2023-12-11T17:56:25.655 INFO:journalctl@ceph.mon.a.smithi059.stdout: return json.loads(''.join(out)) 2023-12-11T17:56:25.655 INFO:journalctl@ceph.mon.a.smithi059.stdout: File "/lib64/python3.6/json/__init__.py", line 354, in loads 2023-12-11T17:56:25.655 INFO:journalctl@ceph.mon.a.smithi059.stdout: return _default_decoder.decode(s) 2023-12-11T17:56:25.655 INFO:journalctl@ceph.mon.a.smithi059.stdout: File "/lib64/python3.6/json/decoder.py", line 339, in decode 2023-12-11T17:56:25.655 INFO:journalctl@ceph.mon.a.smithi059.stdout: obj, end = self.raw_decode(s, idx=_w(s, 0).end()) 2023-12-11T17:56:25.655 INFO:journalctl@ceph.mon.a.smithi059.stdout: File "/lib64/python3.6/json/decoder.py", line 357, in raw_decode 2023-12-11T17:56:25.655 INFO:journalctl@ceph.mon.a.smithi059.stdout: raise JSONDecodeError("Expecting value", s, err.value) from None 2023-12-11T17:56:25.655 INFO:journalctl@ceph.mon.a.smithi059.stdout: json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0) 2023-12-11T17:56:25.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:56:25 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:56:25.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:56:25 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:56:25.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:56:25 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:56:25.656 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:56:25 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:56:25.656 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:56:25 smithi059 ceph-mon[98847]: Adjusting osd_memory_target on smithi181 to 196.2M 2023-12-11T17:56:25.656 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:56:25 smithi059 ceph-mon[98847]: Unable to set osd_memory_target on smithi181 to 205735936: error parsing value: Value '205735936' is below minimum 939524096 2023-12-11T17:56:25.656 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:56:25 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:56:25.656 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:56:25 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:56:25.656 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:56:25 smithi059 ceph-mon[98847]: pgmap v317: 129 pgs: 40 active+clean, 1 active+clean+laggy, 10 stale+active+clean, 2 down, 15 peering, 39 active+undersized+degraded, 22 undersized+degraded+peered; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 646 KiB/s wr, 34 op/s; 52425/300060 objects degraded (17.472%) 2023-12-11T17:56:25.656 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:56:25 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:56:26.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:56:26 smithi115 ceph-mon[102748]: Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED) 2023-12-11T17:56:26.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:56:26 smithi115 ceph-mon[102748]: Health check failed: Reduced data availability: 2 pgs down (PG_AVAILABILITY) 2023-12-11T17:56:26.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:56:26 smithi115 ceph-mon[102748]: Health check failed: Degraded data redundancy: 52425/300060 objects degraded (17.472%), 61 pgs degraded (PG_DEGRADED) 2023-12-11T17:56:26.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:56:26 smithi059 ceph-mon[98847]: Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED) 2023-12-11T17:56:26.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:56:26 smithi059 ceph-mon[98847]: Health check failed: Reduced data availability: 2 pgs down (PG_AVAILABILITY) 2023-12-11T17:56:26.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:56:26 smithi059 ceph-mon[98847]: Health check failed: Degraded data redundancy: 52425/300060 objects degraded (17.472%), 61 pgs degraded (PG_DEGRADED) 2023-12-11T17:56:27.532 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:56:27 smithi115 ceph-mon[102748]: pgmap v318: 129 pgs: 27 active+clean, 2 stale+active+clean, 2 down, 15 peering, 56 active+undersized+degraded, 27 undersized+degraded+peered; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 647 KiB/s wr, 45 op/s; 73763/300315 objects degraded (24.562%) 2023-12-11T17:56:27.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:56:27 smithi059 ceph-mon[98847]: pgmap v318: 129 pgs: 27 active+clean, 2 stale+active+clean, 2 down, 15 peering, 56 active+undersized+degraded, 27 undersized+degraded+peered; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 647 KiB/s wr, 45 op/s; 73763/300315 objects degraded (24.562%) 2023-12-11T17:56:30.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:56:29 smithi115 ceph-mon[102748]: pgmap v319: 129 pgs: 27 active+clean, 2 stale+active+clean, 2 down, 67 active+undersized+degraded, 31 undersized+degraded+peered; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 611 KiB/s wr, 44 op/s; 83348/300323 objects degraded (27.753%) 2023-12-11T17:56:30.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:56:29 smithi059 ceph-mon[98847]: pgmap v319: 129 pgs: 27 active+clean, 2 stale+active+clean, 2 down, 67 active+undersized+degraded, 31 undersized+degraded+peered; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 611 KiB/s wr, 44 op/s; 83348/300323 objects degraded (27.753%) 2023-12-11T17:56:32.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:56:31 smithi115 ceph-mon[102748]: pgmap v320: 129 pgs: 27 active+clean, 2 stale+active+clean, 2 down, 67 active+undersized+degraded, 31 undersized+degraded+peered; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s wr, 29 op/s; 83348/300323 objects degraded (27.753%) 2023-12-11T17:56:32.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:56:31 smithi059 ceph-mon[98847]: pgmap v320: 129 pgs: 27 active+clean, 2 stale+active+clean, 2 down, 67 active+undersized+degraded, 31 undersized+degraded+peered; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s wr, 29 op/s; 83348/300323 objects degraded (27.753%) 2023-12-11T17:56:34.285 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:56:33 smithi115 ceph-mon[102748]: pgmap v321: 129 pgs: 27 active+clean, 2 stale+active+clean, 2 down, 67 active+undersized+degraded, 31 undersized+degraded+peered; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s wr, 26 op/s; 83348/300323 objects degraded (27.753%) 2023-12-11T17:56:34.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:56:33 smithi059 ceph-mon[98847]: pgmap v321: 129 pgs: 27 active+clean, 2 stale+active+clean, 2 down, 67 active+undersized+degraded, 31 undersized+degraded+peered; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s wr, 26 op/s; 83348/300323 objects degraded (27.753%) 2023-12-11T17:56:35.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:56:34 smithi115 ceph-mon[102748]: Health check update: Degraded data redundancy: 83348/300323 objects degraded (27.753%), 98 pgs degraded (PG_DEGRADED) 2023-12-11T17:56:35.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:56:34 smithi059 ceph-mon[98847]: Health check update: Degraded data redundancy: 83348/300323 objects degraded (27.753%), 98 pgs degraded (PG_DEGRADED) 2023-12-11T17:56:36.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:56:35 smithi115 ceph-mon[102748]: pgmap v322: 129 pgs: 27 active+clean, 2 stale+active+clean, 2 down, 67 active+undersized+degraded, 31 undersized+degraded+peered; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 961 B/s wr, 7 op/s; 83348/300323 objects degraded (27.753%) 2023-12-11T17:56:36.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:56:35 smithi059 ceph-mon[98847]: pgmap v322: 129 pgs: 27 active+clean, 2 stale+active+clean, 2 down, 67 active+undersized+degraded, 31 undersized+degraded+peered; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 961 B/s wr, 7 op/s; 83348/300323 objects degraded (27.753%) 2023-12-11T17:56:38.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:56:37 smithi115 ceph-mon[102748]: pgmap v323: 129 pgs: 27 active+clean, 2 stale+active+clean, 2 down, 67 active+undersized+degraded, 31 undersized+degraded+peered; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s wr, 6 op/s; 83348/300323 objects degraded (27.753%) 2023-12-11T17:56:38.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:56:37 smithi059 ceph-mon[98847]: pgmap v323: 129 pgs: 27 active+clean, 2 stale+active+clean, 2 down, 67 active+undersized+degraded, 31 undersized+degraded+peered; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s wr, 6 op/s; 83348/300323 objects degraded (27.753%) 2023-12-11T17:56:40.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:56:39 smithi115 ceph-mon[102748]: pgmap v324: 129 pgs: 27 active+clean, 2 stale+active+clean, 2 down, 67 active+undersized+degraded, 31 undersized+degraded+peered; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 2 op/s; 83348/300323 objects degraded (27.753%) 2023-12-11T17:56:40.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:56:39 smithi059 ceph-mon[98847]: pgmap v324: 129 pgs: 27 active+clean, 2 stale+active+clean, 2 down, 67 active+undersized+degraded, 31 undersized+degraded+peered; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 2 op/s; 83348/300323 objects degraded (27.753%) 2023-12-11T17:56:42.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:56:41 smithi115 ceph-mon[102748]: pgmap v325: 129 pgs: 27 active+clean, 2 stale+active+clean, 2 down, 67 active+undersized+degraded, 31 undersized+degraded+peered; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 2 op/s; 83348/300323 objects degraded (27.753%) 2023-12-11T17:56:42.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:56:41 smithi059 ceph-mon[98847]: pgmap v325: 129 pgs: 27 active+clean, 2 stale+active+clean, 2 down, 67 active+undersized+degraded, 31 undersized+degraded+peered; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 2 op/s; 83348/300323 objects degraded (27.753%) 2023-12-11T17:56:44.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:56:43 smithi115 ceph-mon[102748]: pgmap v326: 129 pgs: 27 active+clean, 2 stale+active+clean, 2 down, 67 active+undersized+degraded, 31 undersized+degraded+peered; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s; 83348/300323 objects degraded (27.753%) 2023-12-11T17:56:44.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:56:43 smithi059 ceph-mon[98847]: pgmap v326: 129 pgs: 27 active+clean, 2 stale+active+clean, 2 down, 67 active+undersized+degraded, 31 undersized+degraded+peered; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s; 83348/300323 objects degraded (27.753%) 2023-12-11T17:56:46.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:56:45 smithi115 ceph-mon[102748]: pgmap v327: 129 pgs: 27 active+clean, 2 stale+active+clean, 2 down, 67 active+undersized+degraded, 31 undersized+degraded+peered; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 0 B/s wr, 2 op/s; 83348/300323 objects degraded (27.753%) 2023-12-11T17:56:46.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:56:45 smithi115 ceph-mon[102748]: mgrmap e27: x(active, since 10m), standbys: y 2023-12-11T17:56:46.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:56:45 smithi059 ceph-mon[98847]: pgmap v327: 129 pgs: 27 active+clean, 2 stale+active+clean, 2 down, 67 active+undersized+degraded, 31 undersized+degraded+peered; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 0 B/s wr, 2 op/s; 83348/300323 objects degraded (27.753%) 2023-12-11T17:56:46.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:56:45 smithi059 ceph-mon[98847]: mgrmap e27: x(active, since 10m), standbys: y 2023-12-11T17:56:48.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:56:47 smithi115 ceph-mon[102748]: pgmap v328: 129 pgs: 27 active+clean, 2 stale+active+clean, 2 down, 67 active+undersized+degraded, 31 undersized+degraded+peered; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s; 83348/300323 objects degraded (27.753%) 2023-12-11T17:56:48.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:56:47 smithi059 ceph-mon[98847]: pgmap v328: 129 pgs: 27 active+clean, 2 stale+active+clean, 2 down, 67 active+undersized+degraded, 31 undersized+degraded+peered; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s; 83348/300323 objects degraded (27.753%) 2023-12-11T17:56:50.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:56:49 smithi115 ceph-mon[102748]: pgmap v329: 129 pgs: 27 active+clean, 2 stale+active+clean, 2 down, 67 active+undersized+degraded, 31 undersized+degraded+peered; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 2 op/s; 83348/300323 objects degraded (27.753%) 2023-12-11T17:56:50.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:56:49 smithi059 ceph-mon[98847]: pgmap v329: 129 pgs: 27 active+clean, 2 stale+active+clean, 2 down, 67 active+undersized+degraded, 31 undersized+degraded+peered; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 2 op/s; 83348/300323 objects degraded (27.753%) 2023-12-11T17:56:52.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:56:51 smithi115 ceph-mon[102748]: pgmap v330: 129 pgs: 27 active+clean, 2 stale+active+clean, 2 down, 67 active+undersized+degraded, 31 undersized+degraded+peered; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 2 op/s; 83348/300323 objects degraded (27.753%) 2023-12-11T17:56:52.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:56:51 smithi059 ceph-mon[98847]: pgmap v330: 129 pgs: 27 active+clean, 2 stale+active+clean, 2 down, 67 active+undersized+degraded, 31 undersized+degraded+peered; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 2 op/s; 83348/300323 objects degraded (27.753%) 2023-12-11T17:56:54.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:56:53 smithi115 ceph-mon[102748]: pgmap v331: 129 pgs: 27 active+clean, 2 stale+active+clean, 2 down, 67 active+undersized+degraded, 31 undersized+degraded+peered; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s; 83348/300323 objects degraded (27.753%) 2023-12-11T17:56:54.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:56:53 smithi059 ceph-mon[98847]: pgmap v331: 129 pgs: 27 active+clean, 2 stale+active+clean, 2 down, 67 active+undersized+degraded, 31 undersized+degraded+peered; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s; 83348/300323 objects degraded (27.753%) 2023-12-11T17:56:56.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:56:55 smithi115 ceph-mon[102748]: pgmap v332: 129 pgs: 27 active+clean, 2 stale+active+clean, 2 down, 67 active+undersized+degraded, 31 undersized+degraded+peered; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 2 op/s; 83348/300323 objects degraded (27.753%) 2023-12-11T17:56:56.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:56:55 smithi059 ceph-mon[98847]: pgmap v332: 129 pgs: 27 active+clean, 2 stale+active+clean, 2 down, 67 active+undersized+degraded, 31 undersized+degraded+peered; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 2 op/s; 83348/300323 objects degraded (27.753%) 2023-12-11T17:56:58.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:56:57 smithi115 ceph-mon[102748]: pgmap v333: 129 pgs: 27 active+clean, 2 stale+active+clean, 2 down, 67 active+undersized+degraded, 31 undersized+degraded+peered; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s; 83348/300323 objects degraded (27.753%) 2023-12-11T17:56:58.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:56:57 smithi059 ceph-mon[98847]: pgmap v333: 129 pgs: 27 active+clean, 2 stale+active+clean, 2 down, 67 active+undersized+degraded, 31 undersized+degraded+peered; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s; 83348/300323 objects degraded (27.753%) 2023-12-11T17:57:00.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:56:59 smithi115 ceph-mon[102748]: pgmap v334: 129 pgs: 27 active+clean, 2 stale+active+clean, 2 down, 67 active+undersized+degraded, 31 undersized+degraded+peered; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 2 op/s; 83348/300323 objects degraded (27.753%) 2023-12-11T17:57:00.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:56:59 smithi059 ceph-mon[98847]: pgmap v334: 129 pgs: 27 active+clean, 2 stale+active+clean, 2 down, 67 active+undersized+degraded, 31 undersized+degraded+peered; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 2 op/s; 83348/300323 objects degraded (27.753%) 2023-12-11T17:57:02.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:57:01 smithi115 ceph-mon[102748]: pgmap v335: 129 pgs: 27 active+clean, 2 stale+active+clean, 2 down, 67 active+undersized+degraded, 31 undersized+degraded+peered; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s; 83348/300323 objects degraded (27.753%) 2023-12-11T17:57:02.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:57:01 smithi059 ceph-mon[98847]: pgmap v335: 129 pgs: 27 active+clean, 2 stale+active+clean, 2 down, 67 active+undersized+degraded, 31 undersized+degraded+peered; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s; 83348/300323 objects degraded (27.753%) 2023-12-11T17:57:04.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:57:03 smithi115 ceph-mon[102748]: pgmap v336: 129 pgs: 27 active+clean, 2 stale+active+clean, 2 down, 67 active+undersized+degraded, 31 undersized+degraded+peered; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s; 83348/300323 objects degraded (27.753%) 2023-12-11T17:57:04.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:57:03 smithi059 ceph-mon[98847]: pgmap v336: 129 pgs: 27 active+clean, 2 stale+active+clean, 2 down, 67 active+undersized+degraded, 31 undersized+degraded+peered; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s; 83348/300323 objects degraded (27.753%) 2023-12-11T17:57:06.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:57:05 smithi115 ceph-mon[102748]: pgmap v337: 129 pgs: 27 active+clean, 2 stale+active+clean, 2 down, 67 active+undersized+degraded, 31 undersized+degraded+peered; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 2 op/s; 83348/300323 objects degraded (27.753%) 2023-12-11T17:57:06.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:57:05 smithi059 ceph-mon[98847]: pgmap v337: 129 pgs: 27 active+clean, 2 stale+active+clean, 2 down, 67 active+undersized+degraded, 31 undersized+degraded+peered; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 2 op/s; 83348/300323 objects degraded (27.753%) 2023-12-11T17:57:08.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:57:07 smithi115 ceph-mon[102748]: pgmap v338: 129 pgs: 27 active+clean, 2 stale+active+clean, 2 down, 67 active+undersized+degraded, 31 undersized+degraded+peered; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s; 83348/300323 objects degraded (27.753%) 2023-12-11T17:57:08.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:57:07 smithi059 ceph-mon[98847]: pgmap v338: 129 pgs: 27 active+clean, 2 stale+active+clean, 2 down, 67 active+undersized+degraded, 31 undersized+degraded+peered; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s; 83348/300323 objects degraded (27.753%) 2023-12-11T17:57:10.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:57:09 smithi115 ceph-mon[102748]: pgmap v339: 129 pgs: 27 active+clean, 2 stale+active+clean, 2 down, 67 active+undersized+degraded, 31 undersized+degraded+peered; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 2 op/s; 83348/300323 objects degraded (27.753%) 2023-12-11T17:57:10.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:57:09 smithi059 ceph-mon[98847]: pgmap v339: 129 pgs: 27 active+clean, 2 stale+active+clean, 2 down, 67 active+undersized+degraded, 31 undersized+degraded+peered; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 2 op/s; 83348/300323 objects degraded (27.753%) 2023-12-11T17:57:12.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:57:11 smithi115 ceph-mon[102748]: pgmap v340: 129 pgs: 27 active+clean, 2 stale+active+clean, 2 down, 67 active+undersized+degraded, 31 undersized+degraded+peered; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s; 83348/300323 objects degraded (27.753%) 2023-12-11T17:57:12.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:57:11 smithi059 ceph-mon[98847]: pgmap v340: 129 pgs: 27 active+clean, 2 stale+active+clean, 2 down, 67 active+undersized+degraded, 31 undersized+degraded+peered; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s; 83348/300323 objects degraded (27.753%) 2023-12-11T17:57:14.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:57:13 smithi115 ceph-mon[102748]: pgmap v341: 129 pgs: 27 active+clean, 2 stale+active+clean, 2 down, 67 active+undersized+degraded, 31 undersized+degraded+peered; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s; 83348/300323 objects degraded (27.753%) 2023-12-11T17:57:14.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:57:13 smithi059 ceph-mon[98847]: pgmap v341: 129 pgs: 27 active+clean, 2 stale+active+clean, 2 down, 67 active+undersized+degraded, 31 undersized+degraded+peered; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s; 83348/300323 objects degraded (27.753%) 2023-12-11T17:57:15.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:57:14 smithi115 ceph-mon[102748]: MDS daemon mds.l is removed because it is dead or otherwise unavailable. 2023-12-11T17:57:15.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:57:14 smithi115 ceph-mon[102748]: MDS daemon mds.f is removed because it is dead or otherwise unavailable. 2023-12-11T17:57:15.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:57:14 smithi115 ceph-mon[102748]: MDS daemon mds.c is removed because it is dead or otherwise unavailable. 2023-12-11T17:57:15.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:57:14 smithi115 ceph-mon[102748]: Replacing daemon mds.i as rank 1 with standby daemon mds.k 2023-12-11T17:57:15.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:57:14 smithi115 ceph-mon[102748]: MDS daemon mds.i is removed because it is dead or otherwise unavailable. 2023-12-11T17:57:15.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:57:14 smithi115 ceph-mon[102748]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-12-11T17:57:15.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:57:14 smithi115 ceph-mon[102748]: osdmap e100: 12 total, 8 up, 12 in 2023-12-11T17:57:15.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:57:14 smithi115 ceph-mon[102748]: fsmap cephfs:3/3 {0=b=up:active,1=k=up:replay,2=h=up:active} 1 up:standby-replay 4 up:standby 2023-12-11T17:57:15.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:57:14 smithi059 ceph-mon[98847]: MDS daemon mds.l is removed because it is dead or otherwise unavailable. 2023-12-11T17:57:15.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:57:14 smithi059 ceph-mon[98847]: MDS daemon mds.f is removed because it is dead or otherwise unavailable. 2023-12-11T17:57:15.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:57:14 smithi059 ceph-mon[98847]: MDS daemon mds.c is removed because it is dead or otherwise unavailable. 2023-12-11T17:57:15.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:57:14 smithi059 ceph-mon[98847]: Replacing daemon mds.i as rank 1 with standby daemon mds.k 2023-12-11T17:57:15.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:57:14 smithi059 ceph-mon[98847]: MDS daemon mds.i is removed because it is dead or otherwise unavailable. 2023-12-11T17:57:15.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:57:14 smithi059 ceph-mon[98847]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-12-11T17:57:15.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:57:14 smithi059 ceph-mon[98847]: osdmap e100: 12 total, 8 up, 12 in 2023-12-11T17:57:15.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:57:14 smithi059 ceph-mon[98847]: fsmap cephfs:3/3 {0=b=up:active,1=k=up:replay,2=h=up:active} 1 up:standby-replay 4 up:standby 2023-12-11T17:57:16.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:57:15 smithi115 ceph-mon[102748]: pgmap v342: 129 pgs: 27 active+clean, 2 stale+active+clean, 2 down, 67 active+undersized+degraded, 31 undersized+degraded+peered; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 2 op/s; 83348/300323 objects degraded (27.753%) 2023-12-11T17:57:16.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:57:15 smithi059 ceph-mon[98847]: pgmap v342: 129 pgs: 27 active+clean, 2 stale+active+clean, 2 down, 67 active+undersized+degraded, 31 undersized+degraded+peered; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 2 op/s; 83348/300323 objects degraded (27.753%) 2023-12-11T17:57:18.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:57:17 smithi115 ceph-mon[102748]: pgmap v344: 129 pgs: 27 active+clean, 2 stale+active+clean, 2 down, 67 active+undersized+degraded, 31 undersized+degraded+peered; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s; 83348/300323 objects degraded (27.753%) 2023-12-11T17:57:18.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:57:17 smithi059 ceph-mon[98847]: pgmap v344: 129 pgs: 27 active+clean, 2 stale+active+clean, 2 down, 67 active+undersized+degraded, 31 undersized+degraded+peered; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s; 83348/300323 objects degraded (27.753%) 2023-12-11T17:57:20.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:57:19 smithi115 ceph-mon[102748]: pgmap v345: 129 pgs: 27 active+clean, 2 stale+active+clean, 2 down, 67 active+undersized+degraded, 31 undersized+degraded+peered; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s; 83348/300323 objects degraded (27.753%) 2023-12-11T17:57:20.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:57:19 smithi059 ceph-mon[98847]: pgmap v345: 129 pgs: 27 active+clean, 2 stale+active+clean, 2 down, 67 active+undersized+degraded, 31 undersized+degraded+peered; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s; 83348/300323 objects degraded (27.753%) 2023-12-11T17:57:22.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:57:21 smithi115 ceph-mon[102748]: pgmap v346: 129 pgs: 27 active+clean, 2 stale+active+clean, 2 down, 67 active+undersized+degraded, 31 undersized+degraded+peered; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 2 op/s; 83348/300323 objects degraded (27.753%) 2023-12-11T17:57:22.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:57:21 smithi059 ceph-mon[98847]: pgmap v346: 129 pgs: 27 active+clean, 2 stale+active+clean, 2 down, 67 active+undersized+degraded, 31 undersized+degraded+peered; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 2 op/s; 83348/300323 objects degraded (27.753%) 2023-12-11T17:57:23.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:57:23 smithi059 ceph-mon[98847]: Health check update: Reduced data availability: 33 pgs inactive, 2 pgs down, 2 pgs stale (PG_AVAILABILITY) 2023-12-11T17:57:23.534 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:57:23 smithi115 ceph-mon[102748]: Health check update: Reduced data availability: 33 pgs inactive, 2 pgs down, 2 pgs stale (PG_AVAILABILITY) 2023-12-11T17:57:24.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:57:24 smithi059 ceph-mon[98847]: pgmap v347: 129 pgs: 27 active+clean, 2 stale+active+clean, 2 down, 67 active+undersized+degraded, 31 undersized+degraded+peered; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 2 op/s; 83348/300323 objects degraded (27.753%) 2023-12-11T17:57:24.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:57:24 smithi115 ceph-mon[102748]: pgmap v347: 129 pgs: 27 active+clean, 2 stale+active+clean, 2 down, 67 active+undersized+degraded, 31 undersized+degraded+peered; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 2 op/s; 83348/300323 objects degraded (27.753%) 2023-12-11T17:57:26.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:57:25 smithi115 ceph-mon[102748]: pgmap v348: 129 pgs: 27 active+clean, 2 stale+active+clean, 2 down, 67 active+undersized+degraded, 31 undersized+degraded+peered; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 2 op/s; 83348/300323 objects degraded (27.753%) 2023-12-11T17:57:26.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:57:25 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:57:26.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:57:25 smithi115 ceph-mon[102748]: Health check update: Degraded data redundancy: 83348/300323 objects degraded (27.753%), 98 pgs degraded, 98 pgs undersized (PG_DEGRADED) 2023-12-11T17:57:26.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:57:25 smithi059 ceph-mon[98847]: pgmap v348: 129 pgs: 27 active+clean, 2 stale+active+clean, 2 down, 67 active+undersized+degraded, 31 undersized+degraded+peered; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 2 op/s; 83348/300323 objects degraded (27.753%) 2023-12-11T17:57:26.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:57:25 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:57:26.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:57:25 smithi059 ceph-mon[98847]: Health check update: Degraded data redundancy: 83348/300323 objects degraded (27.753%), 98 pgs degraded, 98 pgs undersized (PG_DEGRADED) 2023-12-11T17:57:28.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:57:27 smithi115 ceph-mon[102748]: pgmap v349: 129 pgs: 27 active+clean, 2 stale+active+clean, 2 down, 67 active+undersized+degraded, 31 undersized+degraded+peered; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 2 op/s; 83348/300323 objects degraded (27.753%) 2023-12-11T17:57:28.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:57:27 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:57:28.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:57:27 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:57:28.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:57:27 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:57:28.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:57:27 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:57:28.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:57:27 smithi059 ceph-mon[98847]: pgmap v349: 129 pgs: 27 active+clean, 2 stale+active+clean, 2 down, 67 active+undersized+degraded, 31 undersized+degraded+peered; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 2 op/s; 83348/300323 objects degraded (27.753%) 2023-12-11T17:57:28.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:57:27 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:57:28.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:57:27 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:57:28.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:57:27 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:57:28.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:57:27 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:57:29.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:57:28 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:57:29.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:57:28 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:57:29.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:57:28 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:57:29.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:57:28 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:57:29.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:57:28 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:57:29.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:57:28 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:57:29.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:57:28 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:57:29.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:57:28 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:57:29.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:57:28 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:57:29.284 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:57:28 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:57:29.284 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:57:28 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:57:29.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:57:28 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:57:29.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:57:28 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:57:29.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:57:28 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:57:29.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:57:28 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:57:29.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:57:28 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:57:29.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:57:28 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:57:29.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:57:28 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:57:29.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:57:28 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-12-11T17:57:29.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:57:28 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:57:29.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:57:28 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:57:29.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:57:28 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:57:30.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:57:29 smithi115 ceph-mon[102748]: Adjusting osd_memory_target on smithi059 to 197.2M 2023-12-11T17:57:30.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:57:29 smithi115 ceph-mon[102748]: Unable to set osd_memory_target on smithi059 to 206875648: error parsing value: Value '206875648' is below minimum 939524096 2023-12-11T17:57:30.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:57:29 smithi115 ceph-mon[102748]: Adjusting osd_memory_target on smithi115 to 196.2M 2023-12-11T17:57:30.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:57:29 smithi115 ceph-mon[102748]: Unable to set osd_memory_target on smithi115 to 205751705: error parsing value: Value '205751705' is below minimum 939524096 2023-12-11T17:57:30.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:57:29 smithi115 ceph-mon[102748]: pgmap v350: 129 pgs: 27 active+clean, 2 stale+active+clean, 2 down, 67 active+undersized+degraded, 31 undersized+degraded+peered; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 2 op/s; 83348/300323 objects degraded (27.753%) 2023-12-11T17:57:30.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:57:29 smithi115 ceph-mon[102748]: Health check cleared: CEPHADM_REFRESH_FAILED (was: failed to probe daemons or devices) 2023-12-11T17:57:30.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:57:29 smithi059 ceph-mon[98847]: Adjusting osd_memory_target on smithi059 to 197.2M 2023-12-11T17:57:30.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:57:29 smithi059 ceph-mon[98847]: Unable to set osd_memory_target on smithi059 to 206875648: error parsing value: Value '206875648' is below minimum 939524096 2023-12-11T17:57:30.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:57:29 smithi059 ceph-mon[98847]: Adjusting osd_memory_target on smithi115 to 196.2M 2023-12-11T17:57:30.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:57:29 smithi059 ceph-mon[98847]: Unable to set osd_memory_target on smithi115 to 205751705: error parsing value: Value '205751705' is below minimum 939524096 2023-12-11T17:57:30.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:57:29 smithi059 ceph-mon[98847]: pgmap v350: 129 pgs: 27 active+clean, 2 stale+active+clean, 2 down, 67 active+undersized+degraded, 31 undersized+degraded+peered; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 2 op/s; 83348/300323 objects degraded (27.753%) 2023-12-11T17:57:30.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:57:29 smithi059 ceph-mon[98847]: Health check cleared: CEPHADM_REFRESH_FAILED (was: failed to probe daemons or devices) 2023-12-11T17:57:32.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:57:31 smithi115 ceph-mon[102748]: pgmap v351: 129 pgs: 27 active+clean, 2 stale+active+clean, 2 down, 67 active+undersized+degraded, 31 undersized+degraded+peered; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 2 op/s; 83348/300323 objects degraded (27.753%) 2023-12-11T17:57:32.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:57:31 smithi059 ceph-mon[98847]: pgmap v351: 129 pgs: 27 active+clean, 2 stale+active+clean, 2 down, 67 active+undersized+degraded, 31 undersized+degraded+peered; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 2 op/s; 83348/300323 objects degraded (27.753%) 2023-12-11T17:57:34.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:57:33 smithi115 ceph-mon[102748]: pgmap v352: 129 pgs: 27 active+clean, 2 stale+active+clean, 2 down, 67 active+undersized+degraded, 31 undersized+degraded+peered; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s; 83348/300323 objects degraded (27.753%) 2023-12-11T17:57:34.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:57:33 smithi059 ceph-mon[98847]: pgmap v352: 129 pgs: 27 active+clean, 2 stale+active+clean, 2 down, 67 active+undersized+degraded, 31 undersized+degraded+peered; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s; 83348/300323 objects degraded (27.753%) 2023-12-11T17:57:36.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:57:35 smithi115 ceph-mon[102748]: pgmap v353: 129 pgs: 27 active+clean, 2 stale+active+clean, 2 down, 67 active+undersized+degraded, 31 undersized+degraded+peered; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 895 B/s rd, 2 op/s; 83348/300323 objects degraded (27.753%) 2023-12-11T17:57:36.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:57:35 smithi059 ceph-mon[98847]: pgmap v353: 129 pgs: 27 active+clean, 2 stale+active+clean, 2 down, 67 active+undersized+degraded, 31 undersized+degraded+peered; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 895 B/s rd, 2 op/s; 83348/300323 objects degraded (27.753%) 2023-12-11T17:57:38.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:57:37 smithi115 ceph-mon[102748]: pgmap v354: 129 pgs: 27 active+clean, 2 stale+active+clean, 2 down, 67 active+undersized+degraded, 31 undersized+degraded+peered; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s; 83348/300323 objects degraded (27.753%) 2023-12-11T17:57:38.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:57:37 smithi059 ceph-mon[98847]: pgmap v354: 129 pgs: 27 active+clean, 2 stale+active+clean, 2 down, 67 active+undersized+degraded, 31 undersized+degraded+peered; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s; 83348/300323 objects degraded (27.753%) 2023-12-11T17:57:40.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:57:39 smithi115 ceph-mon[102748]: pgmap v355: 129 pgs: 27 active+clean, 2 stale+active+clean, 2 down, 67 active+undersized+degraded, 31 undersized+degraded+peered; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 895 B/s rd, 1 op/s; 83348/300323 objects degraded (27.753%) 2023-12-11T17:57:40.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:57:39 smithi059 ceph-mon[98847]: pgmap v355: 129 pgs: 27 active+clean, 2 stale+active+clean, 2 down, 67 active+undersized+degraded, 31 undersized+degraded+peered; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 895 B/s rd, 1 op/s; 83348/300323 objects degraded (27.753%) 2023-12-11T17:57:42.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:57:41 smithi115 ceph-mon[102748]: pgmap v356: 129 pgs: 27 active+clean, 2 stale+active+clean, 2 down, 67 active+undersized+degraded, 31 undersized+degraded+peered; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 2 op/s; 83348/300323 objects degraded (27.753%) 2023-12-11T17:57:42.406 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:57:41 smithi059 ceph-mon[98847]: pgmap v356: 129 pgs: 27 active+clean, 2 stale+active+clean, 2 down, 67 active+undersized+degraded, 31 undersized+degraded+peered; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 2 op/s; 83348/300323 objects degraded (27.753%) 2023-12-11T17:57:44.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:57:43 smithi115 ceph-mon[102748]: pgmap v357: 129 pgs: 27 active+clean, 2 stale+active+clean, 2 down, 67 active+undersized+degraded, 31 undersized+degraded+peered; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s; 83348/300323 objects degraded (27.753%) 2023-12-11T17:57:44.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:57:43 smithi059 ceph-mon[98847]: pgmap v357: 129 pgs: 27 active+clean, 2 stale+active+clean, 2 down, 67 active+undersized+degraded, 31 undersized+degraded+peered; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s; 83348/300323 objects degraded (27.753%) 2023-12-11T17:57:46.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:57:45 smithi115 ceph-mon[102748]: pgmap v358: 129 pgs: 27 active+clean, 2 stale+active+clean, 2 down, 67 active+undersized+degraded, 31 undersized+degraded+peered; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 2 op/s; 83348/300323 objects degraded (27.753%) 2023-12-11T17:57:46.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:57:45 smithi059 ceph-mon[98847]: pgmap v358: 129 pgs: 27 active+clean, 2 stale+active+clean, 2 down, 67 active+undersized+degraded, 31 undersized+degraded+peered; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 2 op/s; 83348/300323 objects degraded (27.753%) 2023-12-11T17:57:48.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:57:47 smithi115 ceph-mon[102748]: pgmap v359: 129 pgs: 27 active+clean, 2 stale+active+clean, 2 down, 67 active+undersized+degraded, 31 undersized+degraded+peered; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s; 83348/300323 objects degraded (27.753%) 2023-12-11T17:57:48.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:57:47 smithi059 ceph-mon[98847]: pgmap v359: 129 pgs: 27 active+clean, 2 stale+active+clean, 2 down, 67 active+undersized+degraded, 31 undersized+degraded+peered; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s; 83348/300323 objects degraded (27.753%) 2023-12-11T17:57:50.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:57:49 smithi115 ceph-mon[102748]: pgmap v360: 129 pgs: 27 active+clean, 2 stale+active+clean, 2 down, 67 active+undersized+degraded, 31 undersized+degraded+peered; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s; 83348/300323 objects degraded (27.753%) 2023-12-11T17:57:50.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:57:49 smithi059 ceph-mon[98847]: pgmap v360: 129 pgs: 27 active+clean, 2 stale+active+clean, 2 down, 67 active+undersized+degraded, 31 undersized+degraded+peered; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s; 83348/300323 objects degraded (27.753%) 2023-12-11T17:57:52.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:57:51 smithi115 ceph-mon[102748]: pgmap v361: 129 pgs: 27 active+clean, 2 stale+active+clean, 2 down, 67 active+undersized+degraded, 31 undersized+degraded+peered; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 2 op/s; 83348/300323 objects degraded (27.753%) 2023-12-11T17:57:52.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:57:51 smithi059 ceph-mon[98847]: pgmap v361: 129 pgs: 27 active+clean, 2 stale+active+clean, 2 down, 67 active+undersized+degraded, 31 undersized+degraded+peered; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 2 op/s; 83348/300323 objects degraded (27.753%) 2023-12-11T17:57:54.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:57:53 smithi115 ceph-mon[102748]: pgmap v362: 129 pgs: 27 active+clean, 2 stale+active+clean, 2 down, 67 active+undersized+degraded, 31 undersized+degraded+peered; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s; 83348/300323 objects degraded (27.753%) 2023-12-11T17:57:54.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:57:53 smithi059 ceph-mon[98847]: pgmap v362: 129 pgs: 27 active+clean, 2 stale+active+clean, 2 down, 67 active+undersized+degraded, 31 undersized+degraded+peered; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s; 83348/300323 objects degraded (27.753%) 2023-12-11T17:57:56.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:57:55 smithi115 ceph-mon[102748]: pgmap v363: 129 pgs: 27 active+clean, 2 stale+active+clean, 2 down, 67 active+undersized+degraded, 31 undersized+degraded+peered; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 2 op/s; 83348/300323 objects degraded (27.753%) 2023-12-11T17:57:56.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:57:55 smithi059 ceph-mon[98847]: pgmap v363: 129 pgs: 27 active+clean, 2 stale+active+clean, 2 down, 67 active+undersized+degraded, 31 undersized+degraded+peered; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 2 op/s; 83348/300323 objects degraded (27.753%) 2023-12-11T17:57:58.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:57:58 smithi115 ceph-mon[102748]: pgmap v364: 129 pgs: 27 active+clean, 2 stale+active+clean, 2 down, 67 active+undersized+degraded, 31 undersized+degraded+peered; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s; 83348/300323 objects degraded (27.753%) 2023-12-11T17:57:58.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:57:58 smithi059 ceph-mon[98847]: pgmap v364: 129 pgs: 27 active+clean, 2 stale+active+clean, 2 down, 67 active+undersized+degraded, 31 undersized+degraded+peered; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s; 83348/300323 objects degraded (27.753%) 2023-12-11T17:58:00.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:00 smithi115 ceph-mon[102748]: pgmap v365: 129 pgs: 27 active+clean, 2 stale+active+clean, 2 down, 67 active+undersized+degraded, 31 undersized+degraded+peered; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s; 83348/300323 objects degraded (27.753%) 2023-12-11T17:58:00.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:00 smithi059 ceph-mon[98847]: pgmap v365: 129 pgs: 27 active+clean, 2 stale+active+clean, 2 down, 67 active+undersized+degraded, 31 undersized+degraded+peered; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s; 83348/300323 objects degraded (27.753%) 2023-12-11T17:58:01.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:01 smithi115 ceph-mon[102748]: from='client.? 172.21.15.181:0/1353916201' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2023-12-11T17:58:01.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:01 smithi059 ceph-mon[98847]: from='client.? 172.21.15.181:0/1353916201' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2023-12-11T17:58:02.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:02 smithi115 ceph-mon[102748]: pgmap v366: 129 pgs: 27 active+clean, 2 stale+active+clean, 2 down, 67 active+undersized+degraded, 31 undersized+degraded+peered; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 2 op/s; 83348/300323 objects degraded (27.753%) 2023-12-11T17:58:02.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:02 smithi059 ceph-mon[98847]: pgmap v366: 129 pgs: 27 active+clean, 2 stale+active+clean, 2 down, 67 active+undersized+degraded, 31 undersized+degraded+peered; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 2 op/s; 83348/300323 objects degraded (27.753%) 2023-12-11T17:58:04.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:04 smithi115 ceph-mon[102748]: pgmap v367: 129 pgs: 27 active+clean, 2 stale+active+clean, 2 down, 67 active+undersized+degraded, 31 undersized+degraded+peered; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s; 83348/300323 objects degraded (27.753%) 2023-12-11T17:58:04.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:04 smithi059 ceph-mon[98847]: pgmap v367: 129 pgs: 27 active+clean, 2 stale+active+clean, 2 down, 67 active+undersized+degraded, 31 undersized+degraded+peered; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s; 83348/300323 objects degraded (27.753%) 2023-12-11T17:58:06.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:06 smithi115 ceph-mon[102748]: pgmap v368: 129 pgs: 27 active+clean, 2 stale+active+clean, 2 down, 67 active+undersized+degraded, 31 undersized+degraded+peered; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 2 op/s; 83348/300323 objects degraded (27.753%) 2023-12-11T17:58:06.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:06 smithi059 ceph-mon[98847]: pgmap v368: 129 pgs: 27 active+clean, 2 stale+active+clean, 2 down, 67 active+undersized+degraded, 31 undersized+degraded+peered; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 2 op/s; 83348/300323 objects degraded (27.753%) 2023-12-11T17:58:08.032 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:08 smithi059 ceph-mon[98847]: mon.c calling monitor election 2023-12-11T17:58:08.032 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:08 smithi059 ceph-mon[98847]: mon.b calling monitor election 2023-12-11T17:58:08.032 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:08 smithi059 ceph-mon[98847]: mon.a calling monitor election 2023-12-11T17:58:08.033 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:08 smithi059 ceph-mon[98847]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-12-11T17:58:08.033 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:08 smithi059 ceph-mon[98847]: monmap e3: 3 mons at {a=[v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0],b=[v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0],c=[v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0]} removed_ranks: {} disallowed_leaders: {} 2023-12-11T17:58:08.033 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:08 smithi059 ceph-mon[98847]: fsmap cephfs:3/3 {0=b=up:active,1=k=up:replay,2=h=up:active} 1 up:standby-replay 8 up:standby 2023-12-11T17:58:08.033 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:08 smithi059 ceph-mon[98847]: osdmap e100: 12 total, 8 up, 12 in 2023-12-11T17:58:08.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:08 smithi115 ceph-mon[102748]: mon.c calling monitor election 2023-12-11T17:58:08.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:08 smithi115 ceph-mon[102748]: mon.b calling monitor election 2023-12-11T17:58:08.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:08 smithi115 ceph-mon[102748]: mon.a calling monitor election 2023-12-11T17:58:08.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:08 smithi115 ceph-mon[102748]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-12-11T17:58:08.284 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:08 smithi115 ceph-mon[102748]: monmap e3: 3 mons at {a=[v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0],b=[v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0],c=[v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0]} removed_ranks: {} disallowed_leaders: {} 2023-12-11T17:58:08.284 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:08 smithi115 ceph-mon[102748]: fsmap cephfs:3/3 {0=b=up:active,1=k=up:replay,2=h=up:active} 1 up:standby-replay 8 up:standby 2023-12-11T17:58:08.284 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:08 smithi115 ceph-mon[102748]: osdmap e100: 12 total, 8 up, 12 in 2023-12-11T17:58:08.284 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:08 smithi115 ceph-mon[102748]: mgrmap e27: x(active, since 11m), standbys: y 2023-12-11T17:58:08.284 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:08 smithi115 ceph-mon[102748]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,b) 2023-12-11T17:58:08.284 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:08 smithi115 ceph-mon[102748]: Health detail: HEALTH_WARN 1 filesystem is degraded; 4 osds down; 1 host (4 osds) down; Reduced data availability: 33 pgs inactive, 2 pgs down, 2 pgs stale; Degraded data redundancy: 83348/300323 objects degraded (27.753%), 98 pgs degraded, 98 pgs undersized 2023-12-11T17:58:08.284 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:08 smithi115 ceph-mon[102748]: [WRN] FS_DEGRADED: 1 filesystem is degraded 2023-12-11T17:58:08.284 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:08 smithi115 ceph-mon[102748]: fs cephfs is degraded 2023-12-11T17:58:08.284 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:08 smithi115 ceph-mon[102748]: [WRN] OSD_DOWN: 4 osds down 2023-12-11T17:58:08.284 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:08 smithi115 ceph-mon[102748]: osd.2 (root=default,host=smithi181) is down 2023-12-11T17:58:08.284 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:08 smithi115 ceph-mon[102748]: osd.5 (root=default,host=smithi181) is down 2023-12-11T17:58:08.284 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:08 smithi115 ceph-mon[102748]: osd.8 (root=default,host=smithi181) is down 2023-12-11T17:58:08.284 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:08 smithi115 ceph-mon[102748]: osd.11 (root=default,host=smithi181) is down 2023-12-11T17:58:08.284 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:08 smithi115 ceph-mon[102748]: [WRN] OSD_HOST_DOWN: 1 host (4 osds) down 2023-12-11T17:58:08.285 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:08 smithi115 ceph-mon[102748]: host smithi181 (root=default) (4 osds) is down 2023-12-11T17:58:08.285 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:08 smithi115 ceph-mon[102748]: [WRN] PG_AVAILABILITY: Reduced data availability: 33 pgs inactive, 2 pgs down, 2 pgs stale 2023-12-11T17:58:08.285 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:08 smithi115 ceph-mon[102748]: pg 2.0 is stuck inactive for 104s, current state undersized+degraded+peered, last acting [7] 2023-12-11T17:58:08.285 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:08 smithi115 ceph-mon[102748]: pg 2.a is stuck inactive for 104s, current state undersized+degraded+peered, last acting [3] 2023-12-11T17:58:08.285 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:08 smithi115 ceph-mon[102748]: pg 2.c is stuck inactive for 104s, current state undersized+degraded+peered, last acting [0] 2023-12-11T17:58:08.285 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:08 smithi115 ceph-mon[102748]: pg 2.f is stuck inactive for 104s, current state undersized+degraded+peered, last acting [4] 2023-12-11T17:58:08.285 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:08 smithi115 ceph-mon[102748]: pg 2.10 is stuck inactive for 104s, current state undersized+degraded+peered, last acting [0] 2023-12-11T17:58:08.285 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:08 smithi115 ceph-mon[102748]: pg 2.1c is stuck inactive for 104s, current state undersized+degraded+peered, last acting [1] 2023-12-11T17:58:08.285 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:08 smithi115 ceph-mon[102748]: pg 2.1d is stuck inactive for 104s, current state undersized+degraded+peered, last acting [7] 2023-12-11T17:58:08.285 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:08 smithi115 ceph-mon[102748]: pg 2.1f is stuck inactive for 104s, current state undersized+degraded+peered, last acting [9] 2023-12-11T17:58:08.285 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:08 smithi115 ceph-mon[102748]: pg 3.0 is stuck inactive for 104s, current state undersized+degraded+peered, last acting [4] 2023-12-11T17:58:08.285 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:08 smithi115 ceph-mon[102748]: pg 3.8 is stuck inactive for 104s, current state undersized+degraded+peered, last acting [0] 2023-12-11T17:58:08.285 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:08 smithi115 ceph-mon[102748]: pg 3.9 is stuck inactive for 104s, current state undersized+degraded+peered, last acting [10] 2023-12-11T17:58:08.285 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:08 smithi115 ceph-mon[102748]: pg 3.e is stuck inactive for 104s, current state undersized+degraded+peered, last acting [4] 2023-12-11T17:58:08.285 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:08 smithi115 ceph-mon[102748]: pg 3.f is stuck inactive for 104s, current state undersized+degraded+peered, last acting [7] 2023-12-11T17:58:08.286 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:08 smithi115 ceph-mon[102748]: pg 3.15 is stuck inactive for 104s, current state undersized+degraded+peered, last acting [7] 2023-12-11T17:58:08.286 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:08 smithi115 ceph-mon[102748]: pg 3.1a is stuck inactive for 104s, current state undersized+degraded+peered, last acting [9] 2023-12-11T17:58:08.286 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:08 smithi115 ceph-mon[102748]: pg 3.1e is stuck inactive for 104s, current state undersized+degraded+peered, last acting [3] 2023-12-11T17:58:08.286 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:08 smithi115 ceph-mon[102748]: pg 3.20 is stuck stale for 104s, current state stale+active+clean, last acting [11,2,5] 2023-12-11T17:58:08.286 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:08 smithi115 ceph-mon[102748]: pg 3.21 is stuck inactive for 104s, current state undersized+degraded+peered, last acting [4] 2023-12-11T17:58:08.286 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:08 smithi115 ceph-mon[102748]: pg 3.25 is stuck stale for 104s, current state stale+active+clean, last acting [11,8,5] 2023-12-11T17:58:08.286 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:08 smithi115 ceph-mon[102748]: pg 3.26 is stuck inactive for 104s, current state undersized+degraded+peered, last acting [1] 2023-12-11T17:58:08.286 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:08 smithi115 ceph-mon[102748]: pg 3.29 is stuck inactive for 104s, current state undersized+degraded+peered, last acting [3] 2023-12-11T17:58:08.286 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:08 smithi115 ceph-mon[102748]: pg 3.2c is stuck inactive for 104s, current state undersized+degraded+peered, last acting [4] 2023-12-11T17:58:08.286 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:08 smithi115 ceph-mon[102748]: pg 3.37 is stuck inactive for 104s, current state undersized+degraded+peered, last acting [9] 2023-12-11T17:58:08.286 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:08 smithi115 ceph-mon[102748]: pg 3.3c is stuck inactive for 104s, current state undersized+degraded+peered, last acting [4] 2023-12-11T17:58:08.286 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:08 smithi115 ceph-mon[102748]: pg 4.2 is stuck inactive for 104s, current state undersized+degraded+peered, last acting [NONE,10,NONE,9] 2023-12-11T17:58:08.286 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:08 smithi115 ceph-mon[102748]: pg 4.3 is stuck inactive for 104s, current state undersized+degraded+peered, last acting [NONE,NONE,7,6] 2023-12-11T17:58:08.286 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:08 smithi115 ceph-mon[102748]: pg 4.5 is stuck inactive for 104s, current state undersized+degraded+peered, last acting [7,NONE,NONE,9] 2023-12-11T17:58:08.287 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:08 smithi115 ceph-mon[102748]: pg 4.6 is stuck inactive for 104s, current state undersized+degraded+peered, last acting [NONE,NONE,6,10] 2023-12-11T17:58:08.287 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:08 smithi115 ceph-mon[102748]: pg 4.8 is stuck inactive for 104s, current state undersized+degraded+peered, last acting [NONE,NONE,7,1] 2023-12-11T17:58:08.287 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:08 smithi115 ceph-mon[102748]: pg 4.f is stuck inactive for 104s, current state undersized+degraded+peered, last acting [9,3,NONE,NONE] 2023-12-11T17:58:08.287 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:08 smithi115 ceph-mon[102748]: pg 4.12 is down, acting [9,NONE,NONE,NONE] 2023-12-11T17:58:08.287 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:08 smithi115 ceph-mon[102748]: pg 4.13 is stuck inactive for 104s, current state undersized+degraded+peered, last acting [4,1,NONE,NONE] 2023-12-11T17:58:08.287 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:08 smithi115 ceph-mon[102748]: pg 4.15 is stuck inactive for 104s, current state undersized+degraded+peered, last acting [NONE,NONE,3,6] 2023-12-11T17:58:08.287 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:08 smithi115 ceph-mon[102748]: pg 4.1c is stuck inactive for 104s, current state undersized+degraded+peered, last acting [NONE,6,9,NONE] 2023-12-11T17:58:08.287 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:08 smithi115 ceph-mon[102748]: pg 4.1d is down, acting [NONE,NONE,7,NONE] 2023-12-11T17:58:08.287 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:08 smithi115 ceph-mon[102748]: [WRN] PG_DEGRADED: Degraded data redundancy: 83348/300323 objects degraded (27.753%), 98 pgs degraded, 98 pgs undersized 2023-12-11T17:58:08.287 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:08 smithi115 ceph-mon[102748]: pg 2.10 is undersized+degraded+peered, acting [0] 2023-12-11T17:58:08.287 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:08 smithi115 ceph-mon[102748]: pg 2.12 is stuck undersized for 103s, current state active+undersized+degraded, last acting [3,7] 2023-12-11T17:58:08.287 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:08 smithi115 ceph-mon[102748]: pg 2.13 is stuck undersized for 103s, current state active+undersized+degraded, last acting [10,3] 2023-12-11T17:58:08.287 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:08 smithi115 ceph-mon[102748]: pg 2.14 is stuck undersized for 103s, current state active+undersized+degraded, last acting [10,1] 2023-12-11T17:58:08.287 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:08 smithi115 ceph-mon[102748]: pg 2.15 is stuck undersized for 103s, current state active+undersized+degraded, last acting [0,10] 2023-12-11T17:58:08.287 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:08 smithi115 ceph-mon[102748]: pg 2.16 is stuck undersized for 103s, current state active+undersized+degraded, last acting [7,1] 2023-12-11T17:58:08.288 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:08 smithi115 ceph-mon[102748]: pg 2.17 is stuck undersized for 103s, current state active+undersized+degraded, last acting [10,9] 2023-12-11T17:58:08.288 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:08 smithi115 ceph-mon[102748]: pg 2.18 is stuck undersized for 103s, current state active+undersized+degraded, last acting [1,7] 2023-12-11T17:58:08.288 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:08 smithi115 ceph-mon[102748]: pg 2.1a is stuck undersized for 103s, current state active+undersized+degraded, last acting [10,6] 2023-12-11T17:58:08.288 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:08 smithi115 ceph-mon[102748]: pg 2.1b is stuck undersized for 103s, current state active+undersized+degraded, last acting [1,7] 2023-12-11T17:58:08.288 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:08 smithi115 ceph-mon[102748]: pg 2.1c is stuck undersized for 103s, current state undersized+degraded+peered, last acting [1] 2023-12-11T17:58:08.288 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:08 smithi115 ceph-mon[102748]: pg 2.1d is stuck undersized for 103s, current state undersized+degraded+peered, last acting [7] 2023-12-11T17:58:08.288 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:08 smithi115 ceph-mon[102748]: pg 2.1e is stuck undersized for 103s, current state active+undersized+degraded, last acting [3,0] 2023-12-11T17:58:08.288 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:08 smithi115 ceph-mon[102748]: pg 2.1f is stuck undersized for 103s, current state undersized+degraded+peered, last acting [9] 2023-12-11T17:58:08.288 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:08 smithi115 ceph-mon[102748]: pg 3.10 is stuck undersized for 103s, current state active+undersized+degraded, last acting [7,10] 2023-12-11T17:58:08.288 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:08 smithi115 ceph-mon[102748]: pg 3.14 is stuck undersized for 103s, current state active+undersized+degraded, last acting [1,4] 2023-12-11T17:58:08.288 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:08 smithi115 ceph-mon[102748]: pg 3.15 is stuck undersized for 103s, current state undersized+degraded+peered, last acting [7] 2023-12-11T17:58:08.288 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:08 smithi115 ceph-mon[102748]: pg 3.17 is stuck undersized for 103s, current state active+undersized+degraded, last acting [0,7] 2023-12-11T17:58:08.288 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:08 smithi115 ceph-mon[102748]: pg 3.18 is stuck undersized for 103s, current state active+undersized+degraded, last acting [3,6] 2023-12-11T17:58:08.288 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:08 smithi115 ceph-mon[102748]: pg 3.1a is stuck undersized for 103s, current state undersized+degraded+peered, last acting [9] 2023-12-11T17:58:08.288 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:08 smithi115 ceph-mon[102748]: pg 3.1b is stuck undersized for 103s, current state active+undersized+degraded, last acting [4,3] 2023-12-11T17:58:08.289 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:08 smithi115 ceph-mon[102748]: pg 3.1d is stuck undersized for 103s, current state active+undersized+degraded, last acting [4,1] 2023-12-11T17:58:08.289 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:08 smithi115 ceph-mon[102748]: pg 3.1e is stuck undersized for 103s, current state undersized+degraded+peered, last acting [3] 2023-12-11T17:58:08.289 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:08 smithi115 ceph-mon[102748]: pg 3.2d is stuck undersized for 103s, current state active+undersized+degraded, last acting [3,6] 2023-12-11T17:58:08.289 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:08 smithi115 ceph-mon[102748]: pg 3.2f is stuck undersized for 103s, current state active+undersized+degraded, last acting [1,0] 2023-12-11T17:58:08.289 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:08 smithi115 ceph-mon[102748]: pg 3.31 is stuck undersized for 103s, current state active+undersized+degraded, last acting [0,4] 2023-12-11T17:58:08.289 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:08 smithi115 ceph-mon[102748]: pg 3.32 is stuck undersized for 103s, current state active+undersized+degraded, last acting [0,10] 2023-12-11T17:58:08.289 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:08 smithi115 ceph-mon[102748]: pg 3.33 is stuck undersized for 103s, current state active+undersized+degraded, last acting [6,10] 2023-12-11T17:58:08.289 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:08 smithi115 ceph-mon[102748]: pg 3.34 is stuck undersized for 103s, current state active+undersized+degraded, last acting [4,10] 2023-12-11T17:58:08.289 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:08 smithi115 ceph-mon[102748]: pg 3.35 is stuck undersized for 103s, current state active+undersized+degraded, last acting [7,1] 2023-12-11T17:58:08.289 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:08 smithi115 ceph-mon[102748]: pg 3.36 is stuck undersized for 103s, current state active+undersized+degraded, last acting [10,9] 2023-12-11T17:58:08.289 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:08 smithi115 ceph-mon[102748]: pg 3.37 is stuck undersized for 103s, current state undersized+degraded+peered, last acting [9] 2023-12-11T17:58:08.289 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:08 smithi115 ceph-mon[102748]: pg 3.38 is stuck undersized for 103s, current state active+undersized+degraded, last acting [3,10] 2023-12-11T17:58:08.289 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:08 smithi115 ceph-mon[102748]: pg 3.39 is stuck undersized for 103s, current state active+undersized+degraded, last acting [7,9] 2023-12-11T17:58:08.289 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:08 smithi115 ceph-mon[102748]: pg 3.3a is stuck undersized for 103s, current state active+undersized+degraded, last acting [10,9] 2023-12-11T17:58:08.289 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:08 smithi115 ceph-mon[102748]: pg 3.3b is stuck undersized for 103s, current state active+undersized+degraded, last acting [3,6] 2023-12-11T17:58:08.290 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:08 smithi115 ceph-mon[102748]: pg 3.3c is stuck undersized for 103s, current state undersized+degraded+peered, last acting [4] 2023-12-11T17:58:08.290 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:08 smithi115 ceph-mon[102748]: pg 3.3d is stuck undersized for 103s, current state active+undersized+degraded, last acting [0,1] 2023-12-11T17:58:08.290 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:08 smithi115 ceph-mon[102748]: pg 3.3e is stuck undersized for 103s, current state active+undersized+degraded, last acting [0,10] 2023-12-11T17:58:08.290 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:08 smithi115 ceph-mon[102748]: pg 3.3f is stuck undersized for 103s, current state active+undersized+degraded, last acting [1,0] 2023-12-11T17:58:08.290 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:08 smithi115 ceph-mon[102748]: pg 4.10 is stuck undersized for 103s, current state active+undersized+degraded, last acting [9,0,NONE,4] 2023-12-11T17:58:08.290 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:08 smithi115 ceph-mon[102748]: pg 4.11 is stuck undersized for 103s, current state active+undersized+degraded, last acting [3,7,1,NONE] 2023-12-11T17:58:08.290 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:08 smithi115 ceph-mon[102748]: pg 4.13 is stuck undersized for 103s, current state undersized+degraded+peered, last acting [4,1,NONE,NONE] 2023-12-11T17:58:08.290 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:08 smithi115 ceph-mon[102748]: pg 4.14 is stuck undersized for 103s, current state active+undersized+degraded, last acting [NONE,9,7,6] 2023-12-11T17:58:08.290 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:08 smithi115 ceph-mon[102748]: pg 4.15 is stuck undersized for 103s, current state undersized+degraded+peered, last acting [NONE,NONE,3,6] 2023-12-11T17:58:08.290 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:08 smithi115 ceph-mon[102748]: pg 4.16 is stuck undersized for 103s, current state active+undersized+degraded, last acting [6,3,9,NONE] 2023-12-11T17:58:08.290 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:08 smithi115 ceph-mon[102748]: pg 4.19 is stuck undersized for 103s, current state active+undersized+degraded, last acting [NONE,3,9,0] 2023-12-11T17:58:08.290 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:08 smithi115 ceph-mon[102748]: pg 4.1a is stuck undersized for 103s, current state active+undersized+degraded, last acting [4,6,NONE,3] 2023-12-11T17:58:08.290 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:08 smithi115 ceph-mon[102748]: pg 4.1b is stuck undersized for 103s, current state active+undersized+degraded, last acting [4,3,9,NONE] 2023-12-11T17:58:08.290 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:08 smithi115 ceph-mon[102748]: pg 4.1c is stuck undersized for 103s, current state undersized+degraded+peered, last acting [NONE,6,9,NONE] 2023-12-11T17:58:08.290 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:08 smithi115 ceph-mon[102748]: pg 4.1f is stuck undersized for 103s, current state active+undersized+degraded, last acting [NONE,4,9,0] 2023-12-11T17:58:08.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:08 smithi059 ceph-mon[98847]: mgrmap e27: x(active, since 11m), standbys: y 2023-12-11T17:58:08.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:08 smithi059 ceph-mon[98847]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,b) 2023-12-11T17:58:08.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:08 smithi059 ceph-mon[98847]: Health detail: HEALTH_WARN 1 filesystem is degraded; 4 osds down; 1 host (4 osds) down; Reduced data availability: 33 pgs inactive, 2 pgs down, 2 pgs stale; Degraded data redundancy: 83348/300323 objects degraded (27.753%), 98 pgs degraded, 98 pgs undersized 2023-12-11T17:58:08.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:08 smithi059 ceph-mon[98847]: [WRN] FS_DEGRADED: 1 filesystem is degraded 2023-12-11T17:58:08.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:08 smithi059 ceph-mon[98847]: fs cephfs is degraded 2023-12-11T17:58:08.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:08 smithi059 ceph-mon[98847]: [WRN] OSD_DOWN: 4 osds down 2023-12-11T17:58:08.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:08 smithi059 ceph-mon[98847]: osd.2 (root=default,host=smithi181) is down 2023-12-11T17:58:08.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:08 smithi059 ceph-mon[98847]: osd.5 (root=default,host=smithi181) is down 2023-12-11T17:58:08.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:08 smithi059 ceph-mon[98847]: osd.8 (root=default,host=smithi181) is down 2023-12-11T17:58:08.406 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:08 smithi059 ceph-mon[98847]: osd.11 (root=default,host=smithi181) is down 2023-12-11T17:58:08.406 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:08 smithi059 ceph-mon[98847]: [WRN] OSD_HOST_DOWN: 1 host (4 osds) down 2023-12-11T17:58:08.406 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:08 smithi059 ceph-mon[98847]: host smithi181 (root=default) (4 osds) is down 2023-12-11T17:58:08.406 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:08 smithi059 ceph-mon[98847]: [WRN] PG_AVAILABILITY: Reduced data availability: 33 pgs inactive, 2 pgs down, 2 pgs stale 2023-12-11T17:58:08.406 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:08 smithi059 ceph-mon[98847]: pg 2.0 is stuck inactive for 104s, current state undersized+degraded+peered, last acting [7] 2023-12-11T17:58:08.406 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:08 smithi059 ceph-mon[98847]: pg 2.a is stuck inactive for 104s, current state undersized+degraded+peered, last acting [3] 2023-12-11T17:58:08.406 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:08 smithi059 ceph-mon[98847]: pg 2.c is stuck inactive for 104s, current state undersized+degraded+peered, last acting [0] 2023-12-11T17:58:08.406 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:08 smithi059 ceph-mon[98847]: pg 2.f is stuck inactive for 104s, current state undersized+degraded+peered, last acting [4] 2023-12-11T17:58:08.406 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:08 smithi059 ceph-mon[98847]: pg 2.10 is stuck inactive for 104s, current state undersized+degraded+peered, last acting [0] 2023-12-11T17:58:08.406 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:08 smithi059 ceph-mon[98847]: pg 2.1c is stuck inactive for 104s, current state undersized+degraded+peered, last acting [1] 2023-12-11T17:58:08.406 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:08 smithi059 ceph-mon[98847]: pg 2.1d is stuck inactive for 104s, current state undersized+degraded+peered, last acting [7] 2023-12-11T17:58:08.406 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:08 smithi059 ceph-mon[98847]: pg 2.1f is stuck inactive for 104s, current state undersized+degraded+peered, last acting [9] 2023-12-11T17:58:08.406 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:08 smithi059 ceph-mon[98847]: pg 3.0 is stuck inactive for 104s, current state undersized+degraded+peered, last acting [4] 2023-12-11T17:58:08.407 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:08 smithi059 ceph-mon[98847]: pg 3.8 is stuck inactive for 104s, current state undersized+degraded+peered, last acting [0] 2023-12-11T17:58:08.407 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:08 smithi059 ceph-mon[98847]: pg 3.9 is stuck inactive for 104s, current state undersized+degraded+peered, last acting [10] 2023-12-11T17:58:08.407 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:08 smithi059 ceph-mon[98847]: pg 3.e is stuck inactive for 104s, current state undersized+degraded+peered, last acting [4] 2023-12-11T17:58:08.407 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:08 smithi059 ceph-mon[98847]: pg 3.f is stuck inactive for 104s, current state undersized+degraded+peered, last acting [7] 2023-12-11T17:58:08.407 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:08 smithi059 ceph-mon[98847]: pg 3.15 is stuck inactive for 104s, current state undersized+degraded+peered, last acting [7] 2023-12-11T17:58:08.407 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:08 smithi059 ceph-mon[98847]: pg 3.1a is stuck inactive for 104s, current state undersized+degraded+peered, last acting [9] 2023-12-11T17:58:08.407 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:08 smithi059 ceph-mon[98847]: pg 3.1e is stuck inactive for 104s, current state undersized+degraded+peered, last acting [3] 2023-12-11T17:58:08.407 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:08 smithi059 ceph-mon[98847]: pg 3.20 is stuck stale for 104s, current state stale+active+clean, last acting [11,2,5] 2023-12-11T17:58:08.407 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:08 smithi059 ceph-mon[98847]: pg 3.21 is stuck inactive for 104s, current state undersized+degraded+peered, last acting [4] 2023-12-11T17:58:08.407 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:08 smithi059 ceph-mon[98847]: pg 3.25 is stuck stale for 104s, current state stale+active+clean, last acting [11,8,5] 2023-12-11T17:58:08.407 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:08 smithi059 ceph-mon[98847]: pg 3.26 is stuck inactive for 104s, current state undersized+degraded+peered, last acting [1] 2023-12-11T17:58:08.407 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:08 smithi059 ceph-mon[98847]: pg 3.29 is stuck inactive for 104s, current state undersized+degraded+peered, last acting [3] 2023-12-11T17:58:08.407 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:08 smithi059 ceph-mon[98847]: pg 3.2c is stuck inactive for 104s, current state undersized+degraded+peered, last acting [4] 2023-12-11T17:58:08.407 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:08 smithi059 ceph-mon[98847]: pg 3.37 is stuck inactive for 104s, current state undersized+degraded+peered, last acting [9] 2023-12-11T17:58:08.407 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:08 smithi059 ceph-mon[98847]: pg 3.3c is stuck inactive for 104s, current state undersized+degraded+peered, last acting [4] 2023-12-11T17:58:08.408 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:08 smithi059 ceph-mon[98847]: pg 4.2 is stuck inactive for 104s, current state undersized+degraded+peered, last acting [NONE,10,NONE,9] 2023-12-11T17:58:08.408 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:08 smithi059 ceph-mon[98847]: pg 4.3 is stuck inactive for 104s, current state undersized+degraded+peered, last acting [NONE,NONE,7,6] 2023-12-11T17:58:08.408 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:08 smithi059 ceph-mon[98847]: pg 4.5 is stuck inactive for 104s, current state undersized+degraded+peered, last acting [7,NONE,NONE,9] 2023-12-11T17:58:08.408 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:08 smithi059 ceph-mon[98847]: pg 4.6 is stuck inactive for 104s, current state undersized+degraded+peered, last acting [NONE,NONE,6,10] 2023-12-11T17:58:08.408 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:08 smithi059 ceph-mon[98847]: pg 4.8 is stuck inactive for 104s, current state undersized+degraded+peered, last acting [NONE,NONE,7,1] 2023-12-11T17:58:08.408 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:08 smithi059 ceph-mon[98847]: pg 4.f is stuck inactive for 104s, current state undersized+degraded+peered, last acting [9,3,NONE,NONE] 2023-12-11T17:58:08.408 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:08 smithi059 ceph-mon[98847]: pg 4.12 is down, acting [9,NONE,NONE,NONE] 2023-12-11T17:58:08.408 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:08 smithi059 ceph-mon[98847]: pg 4.13 is stuck inactive for 104s, current state undersized+degraded+peered, last acting [4,1,NONE,NONE] 2023-12-11T17:58:08.408 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:08 smithi059 ceph-mon[98847]: pg 4.15 is stuck inactive for 104s, current state undersized+degraded+peered, last acting [NONE,NONE,3,6] 2023-12-11T17:58:08.408 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:08 smithi059 ceph-mon[98847]: pg 4.1c is stuck inactive for 104s, current state undersized+degraded+peered, last acting [NONE,6,9,NONE] 2023-12-11T17:58:08.408 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:08 smithi059 ceph-mon[98847]: pg 4.1d is down, acting [NONE,NONE,7,NONE] 2023-12-11T17:58:08.408 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:08 smithi059 ceph-mon[98847]: [WRN] PG_DEGRADED: Degraded data redundancy: 83348/300323 objects degraded (27.753%), 98 pgs degraded, 98 pgs undersized 2023-12-11T17:58:08.408 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:08 smithi059 ceph-mon[98847]: pg 2.10 is undersized+degraded+peered, acting [0] 2023-12-11T17:58:08.408 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:08 smithi059 ceph-mon[98847]: pg 2.12 is stuck undersized for 103s, current state active+undersized+degraded, last acting [3,7] 2023-12-11T17:58:08.408 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:08 smithi059 ceph-mon[98847]: pg 2.13 is stuck undersized for 103s, current state active+undersized+degraded, last acting [10,3] 2023-12-11T17:58:08.409 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:08 smithi059 ceph-mon[98847]: pg 2.14 is stuck undersized for 103s, current state active+undersized+degraded, last acting [10,1] 2023-12-11T17:58:08.409 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:08 smithi059 ceph-mon[98847]: pg 2.15 is stuck undersized for 103s, current state active+undersized+degraded, last acting [0,10] 2023-12-11T17:58:08.409 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:08 smithi059 ceph-mon[98847]: pg 2.16 is stuck undersized for 103s, current state active+undersized+degraded, last acting [7,1] 2023-12-11T17:58:08.409 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:08 smithi059 ceph-mon[98847]: pg 2.17 is stuck undersized for 103s, current state active+undersized+degraded, last acting [10,9] 2023-12-11T17:58:08.409 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:08 smithi059 ceph-mon[98847]: pg 2.18 is stuck undersized for 103s, current state active+undersized+degraded, last acting [1,7] 2023-12-11T17:58:08.409 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:08 smithi059 ceph-mon[98847]: pg 2.1a is stuck undersized for 103s, current state active+undersized+degraded, last acting [10,6] 2023-12-11T17:58:08.409 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:08 smithi059 ceph-mon[98847]: pg 2.1b is stuck undersized for 103s, current state active+undersized+degraded, last acting [1,7] 2023-12-11T17:58:08.409 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:08 smithi059 ceph-mon[98847]: pg 2.1c is stuck undersized for 103s, current state undersized+degraded+peered, last acting [1] 2023-12-11T17:58:08.409 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:08 smithi059 ceph-mon[98847]: pg 2.1d is stuck undersized for 103s, current state undersized+degraded+peered, last acting [7] 2023-12-11T17:58:08.410 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:08 smithi059 ceph-mon[98847]: pg 2.1e is stuck undersized for 103s, current state active+undersized+degraded, last acting [3,0] 2023-12-11T17:58:08.410 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:08 smithi059 ceph-mon[98847]: pg 2.1f is stuck undersized for 103s, current state undersized+degraded+peered, last acting [9] 2023-12-11T17:58:08.410 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:08 smithi059 ceph-mon[98847]: pg 3.10 is stuck undersized for 103s, current state active+undersized+degraded, last acting [7,10] 2023-12-11T17:58:08.411 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:08 smithi059 ceph-mon[98847]: pg 3.14 is stuck undersized for 103s, current state active+undersized+degraded, last acting [1,4] 2023-12-11T17:58:08.411 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:08 smithi059 ceph-mon[98847]: pg 3.15 is stuck undersized for 103s, current state undersized+degraded+peered, last acting [7] 2023-12-11T17:58:08.411 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:08 smithi059 ceph-mon[98847]: pg 3.17 is stuck undersized for 103s, current state active+undersized+degraded, last acting [0,7] 2023-12-11T17:58:08.411 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:08 smithi059 ceph-mon[98847]: pg 3.18 is stuck undersized for 103s, current state active+undersized+degraded, last acting [3,6] 2023-12-11T17:58:08.411 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:08 smithi059 ceph-mon[98847]: pg 3.1a is stuck undersized for 103s, current state undersized+degraded+peered, last acting [9] 2023-12-11T17:58:08.411 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:08 smithi059 ceph-mon[98847]: pg 3.1b is stuck undersized for 103s, current state active+undersized+degraded, last acting [4,3] 2023-12-11T17:58:08.411 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:08 smithi059 ceph-mon[98847]: pg 3.1d is stuck undersized for 103s, current state active+undersized+degraded, last acting [4,1] 2023-12-11T17:58:08.411 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:08 smithi059 ceph-mon[98847]: pg 3.1e is stuck undersized for 103s, current state undersized+degraded+peered, last acting [3] 2023-12-11T17:58:08.411 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:08 smithi059 ceph-mon[98847]: pg 3.2d is stuck undersized for 103s, current state active+undersized+degraded, last acting [3,6] 2023-12-11T17:58:08.411 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:08 smithi059 ceph-mon[98847]: pg 3.2f is stuck undersized for 103s, current state active+undersized+degraded, last acting [1,0] 2023-12-11T17:58:08.411 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:08 smithi059 ceph-mon[98847]: pg 3.31 is stuck undersized for 103s, current state active+undersized+degraded, last acting [0,4] 2023-12-11T17:58:08.411 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:08 smithi059 ceph-mon[98847]: pg 3.32 is stuck undersized for 103s, current state active+undersized+degraded, last acting [0,10] 2023-12-11T17:58:08.411 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:08 smithi059 ceph-mon[98847]: pg 3.33 is stuck undersized for 103s, current state active+undersized+degraded, last acting [6,10] 2023-12-11T17:58:08.412 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:08 smithi059 ceph-mon[98847]: pg 3.34 is stuck undersized for 103s, current state active+undersized+degraded, last acting [4,10] 2023-12-11T17:58:08.412 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:08 smithi059 ceph-mon[98847]: pg 3.35 is stuck undersized for 103s, current state active+undersized+degraded, last acting [7,1] 2023-12-11T17:58:08.412 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:08 smithi059 ceph-mon[98847]: pg 3.36 is stuck undersized for 103s, current state active+undersized+degraded, last acting [10,9] 2023-12-11T17:58:08.412 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:08 smithi059 ceph-mon[98847]: pg 3.37 is stuck undersized for 103s, current state undersized+degraded+peered, last acting [9] 2023-12-11T17:58:08.412 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:08 smithi059 ceph-mon[98847]: pg 3.38 is stuck undersized for 103s, current state active+undersized+degraded, last acting [3,10] 2023-12-11T17:58:08.412 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:08 smithi059 ceph-mon[98847]: pg 3.39 is stuck undersized for 103s, current state active+undersized+degraded, last acting [7,9] 2023-12-11T17:58:08.412 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:08 smithi059 ceph-mon[98847]: pg 3.3a is stuck undersized for 103s, current state active+undersized+degraded, last acting [10,9] 2023-12-11T17:58:08.412 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:08 smithi059 ceph-mon[98847]: pg 3.3b is stuck undersized for 103s, current state active+undersized+degraded, last acting [3,6] 2023-12-11T17:58:08.412 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:08 smithi059 ceph-mon[98847]: pg 3.3c is stuck undersized for 103s, current state undersized+degraded+peered, last acting [4] 2023-12-11T17:58:08.412 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:08 smithi059 ceph-mon[98847]: pg 3.3d is stuck undersized for 103s, current state active+undersized+degraded, last acting [0,1] 2023-12-11T17:58:08.412 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:08 smithi059 ceph-mon[98847]: pg 3.3e is stuck undersized for 103s, current state active+undersized+degraded, last acting [0,10] 2023-12-11T17:58:08.412 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:08 smithi059 ceph-mon[98847]: pg 3.3f is stuck undersized for 103s, current state active+undersized+degraded, last acting [1,0] 2023-12-11T17:58:08.412 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:08 smithi059 ceph-mon[98847]: pg 4.10 is stuck undersized for 103s, current state active+undersized+degraded, last acting [9,0,NONE,4] 2023-12-11T17:58:08.412 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:08 smithi059 ceph-mon[98847]: pg 4.11 is stuck undersized for 103s, current state active+undersized+degraded, last acting [3,7,1,NONE] 2023-12-11T17:58:08.413 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:08 smithi059 ceph-mon[98847]: pg 4.13 is stuck undersized for 103s, current state undersized+degraded+peered, last acting [4,1,NONE,NONE] 2023-12-11T17:58:08.413 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:08 smithi059 ceph-mon[98847]: pg 4.14 is stuck undersized for 103s, current state active+undersized+degraded, last acting [NONE,9,7,6] 2023-12-11T17:58:08.413 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:08 smithi059 ceph-mon[98847]: pg 4.15 is stuck undersized for 103s, current state undersized+degraded+peered, last acting [NONE,NONE,3,6] 2023-12-11T17:58:08.413 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:08 smithi059 ceph-mon[98847]: pg 4.16 is stuck undersized for 103s, current state active+undersized+degraded, last acting [6,3,9,NONE] 2023-12-11T17:58:08.413 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:08 smithi059 ceph-mon[98847]: pg 4.19 is stuck undersized for 103s, current state active+undersized+degraded, last acting [NONE,3,9,0] 2023-12-11T17:58:08.413 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:08 smithi059 ceph-mon[98847]: pg 4.1a is stuck undersized for 103s, current state active+undersized+degraded, last acting [4,6,NONE,3] 2023-12-11T17:58:08.413 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:08 smithi059 ceph-mon[98847]: pg 4.1b is stuck undersized for 103s, current state active+undersized+degraded, last acting [4,3,9,NONE] 2023-12-11T17:58:08.413 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:08 smithi059 ceph-mon[98847]: pg 4.1c is stuck undersized for 103s, current state undersized+degraded+peered, last acting [NONE,6,9,NONE] 2023-12-11T17:58:08.413 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:08 smithi059 ceph-mon[98847]: pg 4.1f is stuck undersized for 103s, current state active+undersized+degraded, last acting [NONE,4,9,0] 2023-12-11T17:58:10.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:10 smithi059 ceph-mon[98847]: pgmap v370: 129 pgs: 27 active+clean, 2 stale+active+clean, 2 down, 67 active+undersized+degraded, 31 undersized+degraded+peered; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s; 83348/300323 objects degraded (27.753%) 2023-12-11T17:58:10.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:10 smithi115 ceph-mon[102748]: pgmap v370: 129 pgs: 27 active+clean, 2 stale+active+clean, 2 down, 67 active+undersized+degraded, 31 undersized+degraded+peered; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s; 83348/300323 objects degraded (27.753%) 2023-12-11T17:58:12.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:12 smithi059 ceph-mon[98847]: pgmap v371: 129 pgs: 27 active+clean, 2 stale+active+clean, 2 down, 67 active+undersized+degraded, 31 undersized+degraded+peered; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 2 op/s; 83348/300323 objects degraded (27.753%) 2023-12-11T17:58:12.532 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:12 smithi115 ceph-mon[102748]: pgmap v371: 129 pgs: 27 active+clean, 2 stale+active+clean, 2 down, 67 active+undersized+degraded, 31 undersized+degraded+peered; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 2 op/s; 83348/300323 objects degraded (27.753%) 2023-12-11T17:58:14.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:14 smithi059 ceph-mon[98847]: pgmap v372: 129 pgs: 27 active+clean, 2 stale+active+clean, 2 down, 67 active+undersized+degraded, 31 undersized+degraded+peered; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s; 83348/300323 objects degraded (27.753%) 2023-12-11T17:58:14.532 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:14 smithi115 ceph-mon[102748]: pgmap v372: 129 pgs: 27 active+clean, 2 stale+active+clean, 2 down, 67 active+undersized+degraded, 31 undersized+degraded+peered; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s; 83348/300323 objects degraded (27.753%) 2023-12-11T17:58:16.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:16 smithi059 ceph-mon[98847]: pgmap v373: 129 pgs: 27 active+clean, 2 stale+active+clean, 2 down, 67 active+undersized+degraded, 31 undersized+degraded+peered; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 2 op/s; 83348/300323 objects degraded (27.753%) 2023-12-11T17:58:16.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:16 smithi115 ceph-mon[102748]: pgmap v373: 129 pgs: 27 active+clean, 2 stale+active+clean, 2 down, 67 active+undersized+degraded, 31 undersized+degraded+peered; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 2 op/s; 83348/300323 objects degraded (27.753%) 2023-12-11T17:58:17.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:17 smithi059 ceph-mon[98847]: fsmap cephfs:3/3 {0=b=up:active,1=k=up:replay,2=h=up:active} 1 up:standby-replay 8 up:standby 2023-12-11T17:58:17.532 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:17 smithi115 ceph-mon[102748]: fsmap cephfs:3/3 {0=b=up:active,1=k=up:replay,2=h=up:active} 1 up:standby-replay 8 up:standby 2023-12-11T17:58:18.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:18 smithi059 ceph-mon[98847]: pgmap v374: 129 pgs: 27 active+clean, 2 stale+active+clean, 2 down, 67 active+undersized+degraded, 31 undersized+degraded+peered; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s; 83348/300323 objects degraded (27.753%) 2023-12-11T17:58:18.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:18 smithi115 ceph-mon[102748]: pgmap v374: 129 pgs: 27 active+clean, 2 stale+active+clean, 2 down, 67 active+undersized+degraded, 31 undersized+degraded+peered; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s; 83348/300323 objects degraded (27.753%) 2023-12-11T17:58:20.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:20 smithi059 ceph-mon[98847]: pgmap v375: 129 pgs: 27 active+clean, 2 stale+active+clean, 2 down, 67 active+undersized+degraded, 31 undersized+degraded+peered; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s; 83348/300323 objects degraded (27.753%) 2023-12-11T17:58:20.532 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:20 smithi115 ceph-mon[102748]: pgmap v375: 129 pgs: 27 active+clean, 2 stale+active+clean, 2 down, 67 active+undersized+degraded, 31 undersized+degraded+peered; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s; 83348/300323 objects degraded (27.753%) 2023-12-11T17:58:22.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:22 smithi059 ceph-mon[98847]: pgmap v376: 129 pgs: 27 active+clean, 2 stale+active+clean, 2 down, 67 active+undersized+degraded, 31 undersized+degraded+peered; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 2 op/s; 83348/300323 objects degraded (27.753%) 2023-12-11T17:58:22.532 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:22 smithi115 ceph-mon[102748]: pgmap v376: 129 pgs: 27 active+clean, 2 stale+active+clean, 2 down, 67 active+undersized+degraded, 31 undersized+degraded+peered; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 2 op/s; 83348/300323 objects degraded (27.753%) 2023-12-11T17:58:24.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:24 smithi059 ceph-mon[98847]: pgmap v377: 129 pgs: 27 active+clean, 2 stale+active+clean, 2 down, 67 active+undersized+degraded, 31 undersized+degraded+peered; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s; 83348/300323 objects degraded (27.753%) 2023-12-11T17:58:24.532 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:24 smithi115 ceph-mon[102748]: pgmap v377: 129 pgs: 27 active+clean, 2 stale+active+clean, 2 down, 67 active+undersized+degraded, 31 undersized+degraded+peered; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s; 83348/300323 objects degraded (27.753%) 2023-12-11T17:58:26.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:25 smithi115 ceph-mon[102748]: pgmap v378: 129 pgs: 27 active+clean, 2 stale+active+clean, 2 down, 67 active+undersized+degraded, 31 undersized+degraded+peered; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 2 op/s; 83348/300323 objects degraded (27.753%) 2023-12-11T17:58:26.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:25 smithi115 ceph-mon[102748]: from='osd.11 [v2:172.21.15.181:6824/540621004,v1:172.21.15.181:6825/540621004]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-12-11T17:58:26.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:25 smithi115 ceph-mon[102748]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-12-11T17:58:26.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:25 smithi115 ceph-mon[102748]: Standby manager daemon z started 2023-12-11T17:58:26.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:25 smithi115 ceph-mon[102748]: from='mgr.? 172.21.15.181:0/3404366026' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-12-11T17:58:26.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:25 smithi115 ceph-mon[102748]: from='mgr.? 172.21.15.181:0/3404366026' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-12-11T17:58:26.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:25 smithi115 ceph-mon[102748]: from='mgr.? 172.21.15.181:0/3404366026' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-12-11T17:58:26.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:25 smithi115 ceph-mon[102748]: from='mgr.? 172.21.15.181:0/3404366026' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-12-11T17:58:26.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:25 smithi115 ceph-mon[102748]: from='osd.2 [v2:172.21.15.181:6808/1449149834,v1:172.21.15.181:6810/1449149834]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-12-11T17:58:26.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:25 smithi115 ceph-mon[102748]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-12-11T17:58:26.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:25 smithi059 ceph-mon[98847]: pgmap v378: 129 pgs: 27 active+clean, 2 stale+active+clean, 2 down, 67 active+undersized+degraded, 31 undersized+degraded+peered; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 2 op/s; 83348/300323 objects degraded (27.753%) 2023-12-11T17:58:26.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:25 smithi059 ceph-mon[98847]: from='osd.11 [v2:172.21.15.181:6824/540621004,v1:172.21.15.181:6825/540621004]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-12-11T17:58:26.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:25 smithi059 ceph-mon[98847]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-12-11T17:58:26.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:25 smithi059 ceph-mon[98847]: Standby manager daemon z started 2023-12-11T17:58:26.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:25 smithi059 ceph-mon[98847]: from='mgr.? 172.21.15.181:0/3404366026' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-12-11T17:58:26.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:25 smithi059 ceph-mon[98847]: from='mgr.? 172.21.15.181:0/3404366026' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-12-11T17:58:26.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:25 smithi059 ceph-mon[98847]: from='mgr.? 172.21.15.181:0/3404366026' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-12-11T17:58:26.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:25 smithi059 ceph-mon[98847]: from='mgr.? 172.21.15.181:0/3404366026' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-12-11T17:58:26.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:25 smithi059 ceph-mon[98847]: from='osd.2 [v2:172.21.15.181:6808/1449149834,v1:172.21.15.181:6810/1449149834]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-12-11T17:58:26.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:25 smithi059 ceph-mon[98847]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-12-11T17:58:27.286 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:26 smithi115 ceph-mon[102748]: from='osd.11 [v2:172.21.15.181:6824/540621004,v1:172.21.15.181:6825/540621004]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2023-12-11T17:58:27.286 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:26 smithi115 ceph-mon[102748]: from='osd.2 [v2:172.21.15.181:6808/1449149834,v1:172.21.15.181:6810/1449149834]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2023-12-11T17:58:27.286 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:26 smithi115 ceph-mon[102748]: from='osd.8 [v2:172.21.15.181:6832/3914011605,v1:172.21.15.181:6833/3914011605]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-12-11T17:58:27.286 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:26 smithi115 ceph-mon[102748]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-12-11T17:58:27.286 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:26 smithi115 ceph-mon[102748]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-12-11T17:58:27.287 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:26 smithi115 ceph-mon[102748]: osdmap e101: 12 total, 8 up, 12 in 2023-12-11T17:58:27.287 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:26 smithi115 ceph-mon[102748]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2023-12-11T17:58:27.287 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:26 smithi115 ceph-mon[102748]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2023-12-11T17:58:27.287 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:26 smithi115 ceph-mon[102748]: mgrmap e28: x(active, since 12m), standbys: z, y 2023-12-11T17:58:27.287 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:26 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-12-11T17:58:27.287 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:26 smithi115 ceph-mon[102748]: fsmap cephfs:3/3 {0=b=up:active,1=k=up:replay,2=h=up:active} 1 up:standby-replay 8 up:standby 2023-12-11T17:58:27.287 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:26 smithi115 ceph-mon[102748]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-12-11T17:58:27.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:26 smithi059 ceph-mon[98847]: from='osd.11 [v2:172.21.15.181:6824/540621004,v1:172.21.15.181:6825/540621004]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2023-12-11T17:58:27.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:26 smithi059 ceph-mon[98847]: from='osd.2 [v2:172.21.15.181:6808/1449149834,v1:172.21.15.181:6810/1449149834]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2023-12-11T17:58:27.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:26 smithi059 ceph-mon[98847]: from='osd.8 [v2:172.21.15.181:6832/3914011605,v1:172.21.15.181:6833/3914011605]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-12-11T17:58:27.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:26 smithi059 ceph-mon[98847]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-12-11T17:58:27.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:26 smithi059 ceph-mon[98847]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-12-11T17:58:27.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:26 smithi059 ceph-mon[98847]: osdmap e101: 12 total, 8 up, 12 in 2023-12-11T17:58:27.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:26 smithi059 ceph-mon[98847]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2023-12-11T17:58:27.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:26 smithi059 ceph-mon[98847]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2023-12-11T17:58:27.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:26 smithi059 ceph-mon[98847]: mgrmap e28: x(active, since 12m), standbys: z, y 2023-12-11T17:58:27.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:26 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-12-11T17:58:27.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:26 smithi059 ceph-mon[98847]: fsmap cephfs:3/3 {0=b=up:active,1=k=up:replay,2=h=up:active} 1 up:standby-replay 8 up:standby 2023-12-11T17:58:27.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:26 smithi059 ceph-mon[98847]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-12-11T17:58:28.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:27 smithi115 ceph-mon[102748]: pgmap v380: 129 pgs: 2 unknown, 27 active+clean, 2 down, 67 active+undersized+degraded, 31 undersized+degraded+peered; 1.9 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s; 83348/300323 objects degraded (27.753%) 2023-12-11T17:58:28.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:27 smithi115 ceph-mon[102748]: from='osd.8 [v2:172.21.15.181:6832/3914011605,v1:172.21.15.181:6833/3914011605]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2023-12-11T17:58:28.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:27 smithi115 ceph-mon[102748]: from='osd.5 [v2:172.21.15.181:6809/3565629391,v1:172.21.15.181:6811/3565629391]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-12-11T17:58:28.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:27 smithi115 ceph-mon[102748]: Health check update: 2 osds down (OSD_DOWN) 2023-12-11T17:58:28.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:27 smithi115 ceph-mon[102748]: Health check cleared: OSD_HOST_DOWN (was: 1 host (4 osds) down) 2023-12-11T17:58:28.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:27 smithi115 ceph-mon[102748]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-12-11T17:58:28.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:27 smithi115 ceph-mon[102748]: osd.2 [v2:172.21.15.181:6808/1449149834,v1:172.21.15.181:6810/1449149834] boot 2023-12-11T17:58:28.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:27 smithi115 ceph-mon[102748]: osd.11 [v2:172.21.15.181:6824/540621004,v1:172.21.15.181:6825/540621004] boot 2023-12-11T17:58:28.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:27 smithi115 ceph-mon[102748]: osdmap e102: 12 total, 10 up, 12 in 2023-12-11T17:58:28.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:27 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-11T17:58:28.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:27 smithi115 ceph-mon[102748]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2023-12-11T17:58:28.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:27 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-11T17:58:28.284 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:27 smithi115 ceph-mon[102748]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-12-11T17:58:28.284 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:27 smithi115 ceph-mon[102748]: Health check update: Reduced data availability: 33 pgs inactive, 2 pgs down (PG_AVAILABILITY) 2023-12-11T17:58:28.284 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:27 smithi115 ceph-mon[102748]: from='osd.5 [v2:172.21.15.181:6809/3565629391,v1:172.21.15.181:6811/3565629391]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2023-12-11T17:58:28.284 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:27 smithi115 ceph-mon[102748]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-12-11T17:58:28.284 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:27 smithi115 ceph-mon[102748]: osd.8 [v2:172.21.15.181:6832/3914011605,v1:172.21.15.181:6833/3914011605] boot 2023-12-11T17:58:28.284 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:27 smithi115 ceph-mon[102748]: osdmap e103: 12 total, 11 up, 12 in 2023-12-11T17:58:28.284 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:27 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-11T17:58:28.284 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:27 smithi115 ceph-mon[102748]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2023-12-11T17:58:28.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:27 smithi059 ceph-mon[98847]: pgmap v380: 129 pgs: 2 unknown, 27 active+clean, 2 down, 67 active+undersized+degraded, 31 undersized+degraded+peered; 1.9 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s; 83348/300323 objects degraded (27.753%) 2023-12-11T17:58:28.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:27 smithi059 ceph-mon[98847]: from='osd.8 [v2:172.21.15.181:6832/3914011605,v1:172.21.15.181:6833/3914011605]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2023-12-11T17:58:28.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:27 smithi059 ceph-mon[98847]: from='osd.5 [v2:172.21.15.181:6809/3565629391,v1:172.21.15.181:6811/3565629391]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-12-11T17:58:28.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:27 smithi059 ceph-mon[98847]: Health check update: 2 osds down (OSD_DOWN) 2023-12-11T17:58:28.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:27 smithi059 ceph-mon[98847]: Health check cleared: OSD_HOST_DOWN (was: 1 host (4 osds) down) 2023-12-11T17:58:28.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:27 smithi059 ceph-mon[98847]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-12-11T17:58:28.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:27 smithi059 ceph-mon[98847]: osd.2 [v2:172.21.15.181:6808/1449149834,v1:172.21.15.181:6810/1449149834] boot 2023-12-11T17:58:28.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:27 smithi059 ceph-mon[98847]: osd.11 [v2:172.21.15.181:6824/540621004,v1:172.21.15.181:6825/540621004] boot 2023-12-11T17:58:28.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:27 smithi059 ceph-mon[98847]: osdmap e102: 12 total, 10 up, 12 in 2023-12-11T17:58:28.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:27 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-11T17:58:28.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:27 smithi059 ceph-mon[98847]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2023-12-11T17:58:28.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:27 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-11T17:58:28.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:27 smithi059 ceph-mon[98847]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-12-11T17:58:28.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:27 smithi059 ceph-mon[98847]: Health check update: Reduced data availability: 33 pgs inactive, 2 pgs down (PG_AVAILABILITY) 2023-12-11T17:58:28.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:27 smithi059 ceph-mon[98847]: from='osd.5 [v2:172.21.15.181:6809/3565629391,v1:172.21.15.181:6811/3565629391]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2023-12-11T17:58:28.406 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:27 smithi059 ceph-mon[98847]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-12-11T17:58:28.406 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:27 smithi059 ceph-mon[98847]: osd.8 [v2:172.21.15.181:6832/3914011605,v1:172.21.15.181:6833/3914011605] boot 2023-12-11T17:58:28.406 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:27 smithi059 ceph-mon[98847]: osdmap e103: 12 total, 11 up, 12 in 2023-12-11T17:58:28.406 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:27 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-11T17:58:28.406 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:27 smithi059 ceph-mon[98847]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2023-12-11T17:58:29.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:29 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:58:29.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:29 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:58:29.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:29 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:58:29.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:29 smithi115 ceph-mon[102748]: Health check cleared: OSD_DOWN (was: 1 osds down) 2023-12-11T17:58:29.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:29 smithi115 ceph-mon[102748]: osd.5 [v2:172.21.15.181:6809/3565629391,v1:172.21.15.181:6811/3565629391] boot 2023-12-11T17:58:29.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:29 smithi115 ceph-mon[102748]: osdmap e104: 12 total, 12 up, 12 in 2023-12-11T17:58:29.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:29 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-11T17:58:29.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:29 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:58:29.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:29 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:58:29.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:29 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:58:29.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:29 smithi059 ceph-mon[98847]: Health check cleared: OSD_DOWN (was: 1 osds down) 2023-12-11T17:58:29.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:29 smithi059 ceph-mon[98847]: osd.5 [v2:172.21.15.181:6809/3565629391,v1:172.21.15.181:6811/3565629391] boot 2023-12-11T17:58:29.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:29 smithi059 ceph-mon[98847]: osdmap e104: 12 total, 12 up, 12 in 2023-12-11T17:58:29.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:29 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-11T17:58:30.287 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:30 smithi115 ceph-mon[102748]: pgmap v383: 129 pgs: 2 unknown, 27 active+clean, 5 peering, 2 down, 64 active+undersized+degraded, 29 undersized+degraded+peered; 1.9 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s; 83186/300323 objects degraded (27.699%) 2023-12-11T17:58:30.288 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:30 smithi115 ceph-mon[102748]: Health check update: Degraded data redundancy: 83186/300323 objects degraded (27.699%), 93 pgs degraded, 93 pgs undersized (PG_DEGRADED) 2023-12-11T17:58:30.288 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:30 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:58:30.288 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:30 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:58:30.288 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:30 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:58:30.288 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:30 smithi115 ceph-mon[102748]: osdmap e105: 12 total, 12 up, 12 in 2023-12-11T17:58:30.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:30 smithi059 ceph-mon[98847]: pgmap v383: 129 pgs: 2 unknown, 27 active+clean, 5 peering, 2 down, 64 active+undersized+degraded, 29 undersized+degraded+peered; 1.9 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s; 83186/300323 objects degraded (27.699%) 2023-12-11T17:58:30.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:30 smithi059 ceph-mon[98847]: Health check update: Degraded data redundancy: 83186/300323 objects degraded (27.699%), 93 pgs degraded, 93 pgs undersized (PG_DEGRADED) 2023-12-11T17:58:30.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:30 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:58:30.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:30 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:58:30.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:30 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:58:30.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:30 smithi059 ceph-mon[98847]: osdmap e105: 12 total, 12 up, 12 in 2023-12-11T17:58:32.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:32 smithi115 ceph-mon[102748]: pgmap v386: 129 pgs: 2 unknown, 46 active+clean, 20 peering, 1 down, 39 active+undersized+degraded, 21 undersized+degraded+peered; 1.9 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s rd, 0 B/s wr, 3 op/s; 52661/300371 objects degraded (17.532%); 3.7 KiB/s, 3 keys/s, 1 objects/s recovering 2023-12-11T17:58:32.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:32 smithi059 ceph-mon[98847]: pgmap v386: 129 pgs: 2 unknown, 46 active+clean, 20 peering, 1 down, 39 active+undersized+degraded, 21 undersized+degraded+peered; 1.9 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s rd, 0 B/s wr, 3 op/s; 52661/300371 objects degraded (17.532%); 3.7 KiB/s, 3 keys/s, 1 objects/s recovering 2023-12-11T17:58:33.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:33 smithi059 ceph-mon[98847]: Health check update: Reduced data availability: 6 pgs inactive, 1 pg peering (PG_AVAILABILITY) 2023-12-11T17:58:33.532 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:33 smithi115 ceph-mon[102748]: Health check update: Reduced data availability: 6 pgs inactive, 1 pg peering (PG_AVAILABILITY) 2023-12-11T17:58:34.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:34 smithi059 ceph-mon[98847]: pgmap v387: 129 pgs: 100 active+clean, 15 peering, 10 active+undersized+degraded, 4 undersized+degraded+peered; 1.9 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 0 B/s wr, 35 op/s; 16675/300759 objects degraded (5.544%); 62 KiB/s, 2 keys/s, 5 objects/s recovering 2023-12-11T17:58:34.532 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:34 smithi115 ceph-mon[102748]: pgmap v387: 129 pgs: 100 active+clean, 15 peering, 10 active+undersized+degraded, 4 undersized+degraded+peered; 1.9 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 0 B/s wr, 35 op/s; 16675/300759 objects degraded (5.544%); 62 KiB/s, 2 keys/s, 5 objects/s recovering 2023-12-11T17:58:35.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:35 smithi059 ceph-mon[98847]: Health check update: Degraded data redundancy: 16675/300759 objects degraded (5.544%), 14 pgs degraded, 14 pgs undersized (PG_DEGRADED) 2023-12-11T17:58:35.532 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:35 smithi115 ceph-mon[102748]: Health check update: Degraded data redundancy: 16675/300759 objects degraded (5.544%), 14 pgs degraded, 14 pgs undersized (PG_DEGRADED) 2023-12-11T17:58:36.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:36 smithi059 ceph-mon[98847]: pgmap v388: 129 pgs: 129 active+clean; 1.9 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 320 B/s wr, 49 op/s; 62 KiB/s, 2 keys/s, 6 objects/s recovering 2023-12-11T17:58:36.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:36 smithi059 ceph-mon[98847]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 6 pgs inactive, 1 pg peering) 2023-12-11T17:58:36.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:36 smithi059 ceph-mon[98847]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 16675/300759 objects degraded (5.544%), 14 pgs degraded, 14 pgs undersized) 2023-12-11T17:58:36.532 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:36 smithi115 ceph-mon[102748]: pgmap v388: 129 pgs: 129 active+clean; 1.9 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 320 B/s wr, 49 op/s; 62 KiB/s, 2 keys/s, 6 objects/s recovering 2023-12-11T17:58:36.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:36 smithi115 ceph-mon[102748]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 6 pgs inactive, 1 pg peering) 2023-12-11T17:58:36.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:36 smithi115 ceph-mon[102748]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 16675/300759 objects degraded (5.544%), 14 pgs degraded, 14 pgs undersized) 2023-12-11T17:58:38.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:38 smithi059 ceph-mon[98847]: pgmap v389: 129 pgs: 129 active+clean; 1.9 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 255 B/s wr, 41 op/s; 50 KiB/s, 1 keys/s, 5 objects/s recovering 2023-12-11T17:58:38.532 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:38 smithi115 ceph-mon[102748]: pgmap v389: 129 pgs: 129 active+clean; 1.9 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 255 B/s wr, 41 op/s; 50 KiB/s, 1 keys/s, 5 objects/s recovering 2023-12-11T17:58:40.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:40 smithi059 ceph-mon[98847]: pgmap v390: 129 pgs: 129 active+clean; 1.9 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 21 MiB/s rd, 218 B/s wr, 36 op/s; 42 KiB/s, 1 keys/s, 4 objects/s recovering 2023-12-11T17:58:40.532 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:40 smithi115 ceph-mon[102748]: pgmap v390: 129 pgs: 129 active+clean; 1.9 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 21 MiB/s rd, 218 B/s wr, 36 op/s; 42 KiB/s, 1 keys/s, 4 objects/s recovering 2023-12-11T17:58:41.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:41 smithi059 ceph-mon[98847]: mds.? [v2:172.21.15.115:6832/3214606191,v1:172.21.15.115:6833/3214606191] up:resolve 2023-12-11T17:58:41.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:41 smithi059 ceph-mon[98847]: fsmap cephfs:3/3 {0=b=up:active,1=k=up:resolve,2=h=up:active} 1 up:standby-replay 8 up:standby 2023-12-11T17:58:41.540 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:41 smithi115 ceph-mon[102748]: mds.? [v2:172.21.15.115:6832/3214606191,v1:172.21.15.115:6833/3214606191] up:resolve 2023-12-11T17:58:41.541 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:41 smithi115 ceph-mon[102748]: fsmap cephfs:3/3 {0=b=up:active,1=k=up:resolve,2=h=up:active} 1 up:standby-replay 8 up:standby 2023-12-11T17:58:42.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:42 smithi059 ceph-mon[98847]: pgmap v391: 129 pgs: 129 active+clean; 1.9 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 26 MiB/s rd, 197 B/s wr, 33 op/s; 37 KiB/s, 0 keys/s, 3 objects/s recovering 2023-12-11T17:58:42.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:42 smithi059 ceph-mon[98847]: mds.? [v2:172.21.15.115:6832/3214606191,v1:172.21.15.115:6833/3214606191] up:reconnect 2023-12-11T17:58:42.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:42 smithi059 ceph-mon[98847]: fsmap cephfs:3/3 {0=b=up:active,1=k=up:reconnect,2=h=up:active} 1 up:standby-replay 8 up:standby 2023-12-11T17:58:42.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:42 smithi059 ceph-mon[98847]: reconnect by client.25132 172.21.15.59:0/1653421755 after 0.121998 2023-12-11T17:58:42.532 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:42 smithi115 ceph-mon[102748]: pgmap v391: 129 pgs: 129 active+clean; 1.9 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 26 MiB/s rd, 197 B/s wr, 33 op/s; 37 KiB/s, 0 keys/s, 3 objects/s recovering 2023-12-11T17:58:42.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:42 smithi115 ceph-mon[102748]: mds.? [v2:172.21.15.115:6832/3214606191,v1:172.21.15.115:6833/3214606191] up:reconnect 2023-12-11T17:58:42.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:42 smithi115 ceph-mon[102748]: fsmap cephfs:3/3 {0=b=up:active,1=k=up:reconnect,2=h=up:active} 1 up:standby-replay 8 up:standby 2023-12-11T17:58:42.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:42 smithi115 ceph-mon[102748]: reconnect by client.25132 172.21.15.59:0/1653421755 after 0.121998 2023-12-11T17:58:43.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:43 smithi059 ceph-mon[98847]: mds.? [v2:172.21.15.115:6832/3214606191,v1:172.21.15.115:6833/3214606191] up:rejoin 2023-12-11T17:58:43.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:43 smithi059 ceph-mon[98847]: fsmap cephfs:3/3 {0=b=up:active,1=k=up:rejoin,2=h=up:active} 1 up:standby-replay 8 up:standby 2023-12-11T17:58:43.535 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:43 smithi115 ceph-mon[102748]: mds.? [v2:172.21.15.115:6832/3214606191,v1:172.21.15.115:6833/3214606191] up:rejoin 2023-12-11T17:58:43.535 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:43 smithi115 ceph-mon[102748]: fsmap cephfs:3/3 {0=b=up:active,1=k=up:rejoin,2=h=up:active} 1 up:standby-replay 8 up:standby 2023-12-11T17:58:44.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:44 smithi059 ceph-mon[98847]: pgmap v392: 129 pgs: 129 active+clean; 1.9 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 25 MiB/s rd, 255 B/s wr, 30 op/s; 32 KiB/s, 0 keys/s, 2 objects/s recovering 2023-12-11T17:58:44.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:44 smithi059 ceph-mon[98847]: mds.? [v2:172.21.15.115:6832/3214606191,v1:172.21.15.115:6833/3214606191] up:clientreplay 2023-12-11T17:58:44.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:44 smithi059 ceph-mon[98847]: fsmap cephfs:3/3 {0=b=up:active,1=k=up:clientreplay,2=h=up:active} 1 up:standby-replay 8 up:standby 2023-12-11T17:58:44.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:44 smithi059 ceph-mon[98847]: daemon mds.k is now active in filesystem cephfs as rank 1 2023-12-11T17:58:44.532 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:44 smithi115 ceph-mon[102748]: pgmap v392: 129 pgs: 129 active+clean; 1.9 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 25 MiB/s rd, 255 B/s wr, 30 op/s; 32 KiB/s, 0 keys/s, 2 objects/s recovering 2023-12-11T17:58:44.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:44 smithi115 ceph-mon[102748]: mds.? [v2:172.21.15.115:6832/3214606191,v1:172.21.15.115:6833/3214606191] up:clientreplay 2023-12-11T17:58:44.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:44 smithi115 ceph-mon[102748]: fsmap cephfs:3/3 {0=b=up:active,1=k=up:clientreplay,2=h=up:active} 1 up:standby-replay 8 up:standby 2023-12-11T17:58:44.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:44 smithi115 ceph-mon[102748]: daemon mds.k is now active in filesystem cephfs as rank 1 2023-12-11T17:58:45.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:45 smithi059 ceph-mon[98847]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-12-11T17:58:45.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:45 smithi059 ceph-mon[98847]: Cluster is now healthy 2023-12-11T17:58:45.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:45 smithi059 ceph-mon[98847]: mds.? [v2:172.21.15.115:6832/3214606191,v1:172.21.15.115:6833/3214606191] up:active 2023-12-11T17:58:45.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:45 smithi059 ceph-mon[98847]: fsmap cephfs:3 {0=b=up:active,1=k=up:active,2=h=up:active} 1 up:standby-replay 8 up:standby 2023-12-11T17:58:45.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:45 smithi059 ceph-mon[98847]: fsmap cephfs:3 {0=b=up:active,1=k=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-12-11T17:58:45.532 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:45 smithi115 ceph-mon[102748]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-12-11T17:58:45.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:45 smithi115 ceph-mon[102748]: Cluster is now healthy 2023-12-11T17:58:45.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:45 smithi115 ceph-mon[102748]: mds.? [v2:172.21.15.115:6832/3214606191,v1:172.21.15.115:6833/3214606191] up:active 2023-12-11T17:58:45.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:45 smithi115 ceph-mon[102748]: fsmap cephfs:3 {0=b=up:active,1=k=up:active,2=h=up:active} 1 up:standby-replay 8 up:standby 2023-12-11T17:58:45.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:45 smithi115 ceph-mon[102748]: fsmap cephfs:3 {0=b=up:active,1=k=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-12-11T17:58:46.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:46 smithi115 ceph-mon[102748]: pgmap v393: 129 pgs: 129 active+clean; 1.9 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 22 MiB/s rd, 426 B/s wr, 25 op/s; 5.4 KiB/s, 0 objects/s recovering 2023-12-11T17:58:46.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:46 smithi059 ceph-mon[98847]: pgmap v393: 129 pgs: 129 active+clean; 1.9 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 22 MiB/s rd, 426 B/s wr, 25 op/s; 5.4 KiB/s, 0 objects/s recovering 2023-12-11T17:58:47.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:47 smithi115 ceph-mon[102748]: fsmap cephfs:3 {0=b=up:active,1=k=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-12-11T17:58:47.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:47 smithi115 ceph-mon[102748]: osdmap e106: 12 total, 12 up, 12 in 2023-12-11T17:58:47.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:47 smithi059 ceph-mon[98847]: fsmap cephfs:3 {0=b=up:active,1=k=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-12-11T17:58:47.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:47 smithi059 ceph-mon[98847]: osdmap e106: 12 total, 12 up, 12 in 2023-12-11T17:58:48.535 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:48 smithi115 ceph-mon[102748]: pgmap v395: 129 pgs: 129 active+clean; 1.9 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 19 KiB/s wr, 34 op/s 2023-12-11T17:58:48.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:48 smithi059 ceph-mon[98847]: pgmap v395: 129 pgs: 129 active+clean; 1.9 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 19 KiB/s wr, 34 op/s 2023-12-11T17:58:49.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:49 smithi115 ceph-mon[102748]: pgmap v396: 129 pgs: 7 active+clean+snaptrim, 21 active+clean+snaptrim_wait, 101 active+clean; 1.9 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 697 KiB/s wr, 59 op/s 2023-12-11T17:58:49.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:49 smithi059 ceph-mon[98847]: pgmap v396: 129 pgs: 7 active+clean+snaptrim, 21 active+clean+snaptrim_wait, 101 active+clean; 1.9 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 697 KiB/s wr, 59 op/s 2023-12-11T17:58:52.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:51 smithi115 ceph-mon[102748]: pgmap v397: 129 pgs: 13 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 82 active+clean; 1.9 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 23 MiB/s rd, 800 KiB/s wr, 149 op/s 2023-12-11T17:58:52.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:51 smithi059 ceph-mon[98847]: pgmap v397: 129 pgs: 13 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 82 active+clean; 1.9 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 23 MiB/s rd, 800 KiB/s wr, 149 op/s 2023-12-11T17:58:54.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:54 smithi115 ceph-mon[102748]: pgmap v398: 129 pgs: 15 active+clean+snaptrim, 37 active+clean+snaptrim_wait, 77 active+clean; 1.9 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 29 MiB/s rd, 1.3 MiB/s wr, 259 op/s 2023-12-11T17:58:54.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:54 smithi059 ceph-mon[98847]: pgmap v398: 129 pgs: 15 active+clean+snaptrim, 37 active+clean+snaptrim_wait, 77 active+clean; 1.9 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 29 MiB/s rd, 1.3 MiB/s wr, 259 op/s 2023-12-11T17:58:56.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:56 smithi115 ceph-mon[102748]: pgmap v399: 129 pgs: 14 active+clean+snaptrim, 36 active+clean+snaptrim_wait, 79 active+clean; 1.9 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 37 MiB/s rd, 1.9 MiB/s wr, 307 op/s 2023-12-11T17:58:56.406 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:56 smithi059 ceph-mon[98847]: pgmap v399: 129 pgs: 14 active+clean+snaptrim, 36 active+clean+snaptrim_wait, 79 active+clean; 1.9 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 37 MiB/s rd, 1.9 MiB/s wr, 307 op/s 2023-12-11T17:58:58.285 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:58:58 smithi115 ceph-mon[102748]: pgmap v400: 129 pgs: 9 active+clean+snaptrim, 18 active+clean+snaptrim_wait, 102 active+clean; 1.9 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 37 MiB/s rd, 1.9 MiB/s wr, 315 op/s 2023-12-11T17:58:58.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:58:58 smithi059 ceph-mon[98847]: pgmap v400: 129 pgs: 9 active+clean+snaptrim, 18 active+clean+snaptrim_wait, 102 active+clean; 1.9 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 37 MiB/s rd, 1.9 MiB/s wr, 315 op/s 2023-12-11T17:59:00.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:59:00 smithi115 ceph-mon[102748]: pgmap v401: 129 pgs: 6 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 111 active+clean; 1.9 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 41 MiB/s rd, 1.6 MiB/s wr, 299 op/s 2023-12-11T17:59:00.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:59:00 smithi059 ceph-mon[98847]: pgmap v401: 129 pgs: 6 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 111 active+clean; 1.9 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 41 MiB/s rd, 1.6 MiB/s wr, 299 op/s 2023-12-11T17:59:02.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:59:02 smithi115 ceph-mon[102748]: pgmap v402: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 1.9 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 41 MiB/s rd, 1.6 MiB/s wr, 324 op/s 2023-12-11T17:59:02.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:59:02 smithi059 ceph-mon[98847]: pgmap v402: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 1.9 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 41 MiB/s rd, 1.6 MiB/s wr, 324 op/s 2023-12-11T17:59:04.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:59:04 smithi115 ceph-mon[102748]: pgmap v403: 129 pgs: 129 active+clean; 1.9 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 35 MiB/s rd, 1.6 MiB/s wr, 306 op/s 2023-12-11T17:59:04.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:59:04 smithi059 ceph-mon[98847]: pgmap v403: 129 pgs: 129 active+clean; 1.9 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 35 MiB/s rd, 1.6 MiB/s wr, 306 op/s 2023-12-11T17:59:06.286 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:59:05 smithi115 ceph-mon[102748]: pgmap v404: 129 pgs: 129 active+clean; 1.9 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 30 MiB/s rd, 2.1 MiB/s wr, 302 op/s 2023-12-11T17:59:06.286 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:59:05 smithi115 ceph-mon[102748]: osdmap e107: 12 total, 12 up, 12 in 2023-12-11T17:59:06.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:59:05 smithi059 ceph-mon[98847]: pgmap v404: 129 pgs: 129 active+clean; 1.9 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 30 MiB/s rd, 2.1 MiB/s wr, 302 op/s 2023-12-11T17:59:06.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:59:05 smithi059 ceph-mon[98847]: osdmap e107: 12 total, 12 up, 12 in 2023-12-11T17:59:08.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:59:07 smithi115 ceph-mon[102748]: pgmap v406: 129 pgs: 129 active+clean; 1.9 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 24 MiB/s rd, 2.3 MiB/s wr, 331 op/s 2023-12-11T17:59:08.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:59:07 smithi059 ceph-mon[98847]: pgmap v406: 129 pgs: 129 active+clean; 1.9 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 24 MiB/s rd, 2.3 MiB/s wr, 331 op/s 2023-12-11T17:59:10.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:59:09 smithi115 ceph-mon[102748]: pgmap v407: 129 pgs: 129 active+clean; 1.9 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 2.3 MiB/s wr, 351 op/s 2023-12-11T17:59:10.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:59:09 smithi059 ceph-mon[98847]: pgmap v407: 129 pgs: 129 active+clean; 1.9 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 2.3 MiB/s wr, 351 op/s 2023-12-11T17:59:12.284 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:59:11 smithi115 ceph-mon[102748]: pgmap v408: 129 pgs: 129 active+clean; 1.9 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 2.0 MiB/s wr, 359 op/s 2023-12-11T17:59:12.284 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:59:11 smithi115 ceph-mon[102748]: osdmap e108: 12 total, 12 up, 12 in 2023-12-11T17:59:12.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:59:11 smithi059 ceph-mon[98847]: pgmap v408: 129 pgs: 129 active+clean; 1.9 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 2.0 MiB/s wr, 359 op/s 2023-12-11T17:59:12.406 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:59:11 smithi059 ceph-mon[98847]: osdmap e108: 12 total, 12 up, 12 in 2023-12-11T17:59:14.284 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:59:14 smithi115 ceph-mon[102748]: pgmap v410: 129 pgs: 7 active+clean+snaptrim, 23 active+clean+snaptrim_wait, 99 active+clean; 1.9 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.8 MiB/s wr, 322 op/s 2023-12-11T17:59:14.284 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:59:14 smithi115 ceph-mon[102748]: fsmap cephfs:3 {0=b=up:active,1=k=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-12-11T17:59:14.406 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:59:14 smithi059 ceph-mon[98847]: pgmap v410: 129 pgs: 7 active+clean+snaptrim, 23 active+clean+snaptrim_wait, 99 active+clean; 1.9 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.8 MiB/s wr, 322 op/s 2023-12-11T17:59:14.406 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:59:14 smithi059 ceph-mon[98847]: fsmap cephfs:3 {0=b=up:active,1=k=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-12-11T17:59:16.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:59:16 smithi115 ceph-mon[102748]: pgmap v411: 129 pgs: 15 active+clean+snaptrim, 42 active+clean+snaptrim_wait, 72 active+clean; 1.9 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 355 op/s 2023-12-11T17:59:16.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:59:16 smithi059 ceph-mon[98847]: pgmap v411: 129 pgs: 15 active+clean+snaptrim, 42 active+clean+snaptrim_wait, 72 active+clean; 1.9 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 355 op/s 2023-12-11T17:59:18.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:59:18 smithi059 ceph-mon[98847]: pgmap v412: 129 pgs: 19 active+clean+snaptrim, 56 active+clean+snaptrim_wait, 54 active+clean; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 307 op/s 2023-12-11T17:59:18.532 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:59:18 smithi115 ceph-mon[102748]: pgmap v412: 129 pgs: 19 active+clean+snaptrim, 56 active+clean+snaptrim_wait, 54 active+clean; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 307 op/s 2023-12-11T17:59:18.906 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 17:59:18 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T17:59:18.648+0000 7f4642beb700 -1 calc_pg_upmaps abort due to max <= 0 2023-12-11T17:59:19.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:59:19 smithi059 ceph-mon[98847]: fsmap cephfs:3 {0=b=up:active,1=k=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-12-11T17:59:19.536 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:59:19 smithi115 ceph-mon[102748]: fsmap cephfs:3 {0=b=up:active,1=k=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-12-11T17:59:20.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:59:20 smithi059 ceph-mon[98847]: pgmap v413: 129 pgs: 19 active+clean+snaptrim, 56 active+clean+snaptrim_wait, 54 active+clean; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 282 op/s 2023-12-11T17:59:20.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:59:20 smithi115 ceph-mon[102748]: pgmap v413: 129 pgs: 19 active+clean+snaptrim, 56 active+clean+snaptrim_wait, 54 active+clean; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 282 op/s 2023-12-11T17:59:22.409 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:59:22 smithi059 ceph-mon[98847]: pgmap v414: 129 pgs: 18 active+clean+snaptrim, 56 active+clean+snaptrim_wait, 55 active+clean; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 236 op/s 2023-12-11T17:59:22.536 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:59:22 smithi115 ceph-mon[102748]: pgmap v414: 129 pgs: 18 active+clean+snaptrim, 56 active+clean+snaptrim_wait, 55 active+clean; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 236 op/s 2023-12-11T17:59:24.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:59:24 smithi059 ceph-mon[98847]: pgmap v415: 129 pgs: 14 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 74 active+clean; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 208 op/s 2023-12-11T17:59:24.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:59:24 smithi115 ceph-mon[102748]: pgmap v415: 129 pgs: 14 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 74 active+clean; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 208 op/s 2023-12-11T17:59:26.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:59:26 smithi059 ceph-mon[98847]: pgmap v416: 129 pgs: 13 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 75 active+clean; 1.9 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 185 op/s 2023-12-11T17:59:26.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:59:26 smithi115 ceph-mon[102748]: pgmap v416: 129 pgs: 13 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 75 active+clean; 1.9 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 185 op/s 2023-12-11T17:59:28.406 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:59:28 smithi059 ceph-mon[98847]: pgmap v417: 129 pgs: 8 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 102 active+clean; 1.9 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.0 MiB/s wr, 135 op/s 2023-12-11T17:59:28.406 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:59:28 smithi059 ceph-mon[98847]: fsmap cephfs:3 {0=b=up:active,1=k=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-12-11T17:59:28.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:59:28 smithi115 ceph-mon[102748]: pgmap v417: 129 pgs: 8 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 102 active+clean; 1.9 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.0 MiB/s wr, 135 op/s 2023-12-11T17:59:28.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:59:28 smithi115 ceph-mon[102748]: fsmap cephfs:3 {0=b=up:active,1=k=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-12-11T17:59:30.407 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:59:30 smithi059 ceph-mon[98847]: pgmap v418: 129 pgs: 5 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 111 active+clean; 1.9 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.8 MiB/s wr, 124 op/s 2023-12-11T17:59:30.407 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:59:30 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:59:30.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:59:30 smithi115 ceph-mon[102748]: pgmap v418: 129 pgs: 5 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 111 active+clean; 1.9 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.8 MiB/s wr, 124 op/s 2023-12-11T17:59:30.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:59:30 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T17:59:31.782 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:59:31 smithi115 ceph-mon[102748]: pgmap v419: 129 pgs: 3 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 119 active+clean; 1.9 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.7 MiB/s wr, 126 op/s 2023-12-11T17:59:31.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:59:31 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:59:31.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:59:31 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:59:31.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:59:31 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:59:31.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:59:31 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:59:31.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:59:31 smithi059 ceph-mon[98847]: pgmap v419: 129 pgs: 3 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 119 active+clean; 1.9 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.7 MiB/s wr, 126 op/s 2023-12-11T17:59:31.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:59:31 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:59:31.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:59:31 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:59:31.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:59:31 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:59:31.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:59:31 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:59:33.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:59:33 smithi059 ceph-mon[98847]: fsmap cephfs:3 {0=b=up:active,1=k=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-12-11T17:59:33.537 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:59:33 smithi115 ceph-mon[102748]: fsmap cephfs:3 {0=b=up:active,1=k=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-12-11T17:59:34.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:59:34 smithi059 ceph-mon[98847]: pgmap v420: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 1.9 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 153 op/s 2023-12-11T17:59:34.537 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:59:34 smithi115 ceph-mon[102748]: pgmap v420: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 1.9 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 153 op/s 2023-12-11T17:59:35.535 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:59:35 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:59:35.535 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:59:35 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:59:35.535 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:59:35 smithi115 ceph-mon[102748]: pgmap v421: 129 pgs: 129 active+clean; 1.9 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.8 MiB/s wr, 185 op/s 2023-12-11T17:59:35.535 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:59:35 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:59:35.535 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:59:35 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:59:35.535 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:59:35 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:59:35.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:59:35 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:59:35.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:59:35 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:59:35.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:59:35 smithi059 ceph-mon[98847]: pgmap v421: 129 pgs: 129 active+clean; 1.9 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.8 MiB/s wr, 185 op/s 2023-12-11T17:59:35.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:59:35 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T17:59:35.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:59:35 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T17:59:35.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:59:35 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T17:59:37.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:59:37 smithi115 ceph-mon[102748]: pgmap v422: 129 pgs: 129 active+clean; 1.9 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 195 op/s 2023-12-11T17:59:37.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:59:37 smithi059 ceph-mon[98847]: pgmap v422: 129 pgs: 129 active+clean; 1.9 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 195 op/s 2023-12-11T17:59:39.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:59:39 smithi115 ceph-mon[102748]: pgmap v423: 129 pgs: 129 active+clean; 1.9 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 213 op/s 2023-12-11T17:59:39.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:59:39 smithi059 ceph-mon[98847]: pgmap v423: 129 pgs: 129 active+clean; 1.9 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 213 op/s 2023-12-11T17:59:41.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:59:41 smithi115 ceph-mon[102748]: osdmap e109: 12 total, 12 up, 12 in 2023-12-11T17:59:41.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:59:41 smithi059 ceph-mon[98847]: osdmap e109: 12 total, 12 up, 12 in 2023-12-11T17:59:42.282 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:59:42 smithi115 ceph-mon[102748]: pgmap v425: 129 pgs: 129 active+clean; 1.9 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 265 op/s 2023-12-11T17:59:42.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:59:42 smithi059 ceph-mon[98847]: pgmap v425: 129 pgs: 129 active+clean; 1.9 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 265 op/s 2023-12-11T17:59:44.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:59:44 smithi059 ceph-mon[98847]: pgmap v426: 129 pgs: 129 active+clean; 1.9 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 288 op/s 2023-12-11T17:59:44.532 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:59:44 smithi115 ceph-mon[102748]: pgmap v426: 129 pgs: 129 active+clean; 1.9 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 288 op/s 2023-12-11T17:59:46.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:59:46 smithi059 ceph-mon[98847]: pgmap v427: 129 pgs: 129 active+clean; 1.9 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 292 op/s 2023-12-11T17:59:46.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:59:46 smithi115 ceph-mon[102748]: pgmap v427: 129 pgs: 129 active+clean; 1.9 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 292 op/s 2023-12-11T17:59:48.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:59:48 smithi059 ceph-mon[98847]: pgmap v428: 129 pgs: 129 active+clean; 1.9 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 294 op/s 2023-12-11T17:59:48.532 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:59:48 smithi115 ceph-mon[102748]: pgmap v428: 129 pgs: 129 active+clean; 1.9 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 294 op/s 2023-12-11T17:59:50.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:59:50 smithi059 ceph-mon[98847]: pgmap v429: 129 pgs: 129 active+clean; 1.9 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.4 MiB/s wr, 296 op/s 2023-12-11T17:59:50.532 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:59:50 smithi115 ceph-mon[102748]: pgmap v429: 129 pgs: 129 active+clean; 1.9 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.4 MiB/s wr, 296 op/s 2023-12-11T17:59:52.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:59:52 smithi059 ceph-mon[98847]: pgmap v430: 129 pgs: 129 active+clean; 1.9 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 279 op/s 2023-12-11T17:59:52.532 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:59:52 smithi115 ceph-mon[102748]: pgmap v430: 129 pgs: 129 active+clean; 1.9 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 279 op/s 2023-12-11T17:59:54.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:59:54 smithi059 ceph-mon[98847]: pgmap v431: 129 pgs: 129 active+clean; 1.9 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 298 op/s 2023-12-11T17:59:54.532 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:59:54 smithi115 ceph-mon[102748]: pgmap v431: 129 pgs: 129 active+clean; 1.9 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 298 op/s 2023-12-11T17:59:56.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:59:56 smithi059 ceph-mon[98847]: pgmap v432: 129 pgs: 129 active+clean; 1.9 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 295 op/s 2023-12-11T17:59:56.535 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:59:56 smithi115 ceph-mon[102748]: pgmap v432: 129 pgs: 129 active+clean; 1.9 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 295 op/s 2023-12-11T17:59:58.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 17:59:58 smithi059 ceph-mon[98847]: pgmap v433: 129 pgs: 129 active+clean; 1.9 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.7 MiB/s wr, 267 op/s 2023-12-11T17:59:58.532 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 17:59:58 smithi115 ceph-mon[102748]: pgmap v433: 129 pgs: 129 active+clean; 1.9 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.7 MiB/s wr, 267 op/s 2023-12-11T18:00:00.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:00:00 smithi059 ceph-mon[98847]: pgmap v434: 129 pgs: 129 active+clean; 1.9 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 263 op/s 2023-12-11T18:00:00.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:00:00 smithi059 ceph-mon[98847]: overall HEALTH_OK 2023-12-11T18:00:00.532 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:00:00 smithi115 ceph-mon[102748]: pgmap v434: 129 pgs: 129 active+clean; 1.9 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 263 op/s 2023-12-11T18:00:00.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:00:00 smithi115 ceph-mon[102748]: overall HEALTH_OK 2023-12-11T18:00:02.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:00:02 smithi059 ceph-mon[98847]: pgmap v435: 129 pgs: 129 active+clean; 1.9 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 274 op/s 2023-12-11T18:00:02.532 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:00:02 smithi115 ceph-mon[102748]: pgmap v435: 129 pgs: 129 active+clean; 1.9 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 274 op/s 2023-12-11T18:00:04.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:00:04 smithi059 ceph-mon[98847]: pgmap v436: 129 pgs: 129 active+clean; 1.9 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 282 op/s 2023-12-11T18:00:04.534 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:00:04 smithi115 ceph-mon[102748]: pgmap v436: 129 pgs: 129 active+clean; 1.9 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 282 op/s 2023-12-11T18:00:06.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:00:06 smithi059 ceph-mon[98847]: pgmap v437: 129 pgs: 129 active+clean; 1.9 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.0 MiB/s wr, 285 op/s 2023-12-11T18:00:06.532 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:00:06 smithi115 ceph-mon[102748]: pgmap v437: 129 pgs: 129 active+clean; 1.9 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.0 MiB/s wr, 285 op/s 2023-12-11T18:00:08.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:00:08 smithi059 ceph-mon[98847]: pgmap v438: 129 pgs: 129 active+clean; 1.9 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 249 op/s 2023-12-11T18:00:08.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:00:08 smithi115 ceph-mon[102748]: pgmap v438: 129 pgs: 129 active+clean; 1.9 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 249 op/s 2023-12-11T18:00:10.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:00:10 smithi059 ceph-mon[98847]: pgmap v439: 129 pgs: 129 active+clean; 1.9 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 243 op/s 2023-12-11T18:00:10.532 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:00:10 smithi115 ceph-mon[102748]: pgmap v439: 129 pgs: 129 active+clean; 1.9 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 243 op/s 2023-12-11T18:00:12.407 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:00:12 smithi059 ceph-mon[98847]: pgmap v440: 129 pgs: 129 active+clean; 1.9 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 279 op/s 2023-12-11T18:00:12.407 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:00:12 smithi059 ceph-mon[98847]: osdmap e110: 12 total, 12 up, 12 in 2023-12-11T18:00:12.532 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:00:12 smithi115 ceph-mon[102748]: pgmap v440: 129 pgs: 129 active+clean; 1.9 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 279 op/s 2023-12-11T18:00:12.532 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:00:12 smithi115 ceph-mon[102748]: osdmap e110: 12 total, 12 up, 12 in 2023-12-11T18:00:14.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:00:14 smithi115 ceph-mon[102748]: pgmap v442: 129 pgs: 26 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 95 active+clean; 1.9 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.2 MiB/s wr, 293 op/s 2023-12-11T18:00:14.657 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:00:14 smithi059 ceph-mon[98847]: pgmap v442: 129 pgs: 26 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 95 active+clean; 1.9 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.2 MiB/s wr, 293 op/s 2023-12-11T18:00:16.536 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:00:16 smithi115 ceph-mon[102748]: pgmap v443: 129 pgs: 45 active+clean+snaptrim_wait, 16 active+clean+snaptrim, 68 active+clean; 1.9 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.8 MiB/s wr, 284 op/s 2023-12-11T18:00:16.658 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:00:16 smithi059 ceph-mon[98847]: pgmap v443: 129 pgs: 45 active+clean+snaptrim_wait, 16 active+clean+snaptrim, 68 active+clean; 1.9 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.8 MiB/s wr, 284 op/s 2023-12-11T18:00:17.536 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:00:17 smithi115 ceph-mon[102748]: pgmap v444: 129 pgs: 59 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 50 active+clean; 1.9 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.1 MiB/s wr, 280 op/s 2023-12-11T18:00:17.656 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:00:17 smithi059 ceph-mon[98847]: pgmap v444: 129 pgs: 59 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 50 active+clean; 1.9 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.1 MiB/s wr, 280 op/s 2023-12-11T18:00:19.155 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 18:00:18 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T18:00:18.651+0000 7f4642beb700 -1 calc_pg_upmaps abort due to max <= 0 2023-12-11T18:00:19.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:00:19 smithi115 ceph-mon[102748]: pgmap v445: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.9 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.2 MiB/s wr, 267 op/s 2023-12-11T18:00:19.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:00:19 smithi059 ceph-mon[98847]: pgmap v445: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.9 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.2 MiB/s wr, 267 op/s 2023-12-11T18:00:22.408 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:00:22 smithi059 ceph-mon[98847]: pgmap v446: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.9 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.0 MiB/s wr, 211 op/s 2023-12-11T18:00:22.534 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:00:22 smithi115 ceph-mon[102748]: pgmap v446: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.9 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.0 MiB/s wr, 211 op/s 2023-12-11T18:00:24.408 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:00:24 smithi059 ceph-mon[98847]: pgmap v447: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.9 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 177 op/s 2023-12-11T18:00:24.532 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:00:24 smithi115 ceph-mon[102748]: pgmap v447: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.9 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 177 op/s 2023-12-11T18:00:26.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:00:26 smithi059 ceph-mon[98847]: pgmap v448: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.9 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 149 op/s 2023-12-11T18:00:26.534 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:00:26 smithi115 ceph-mon[102748]: pgmap v448: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.9 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 149 op/s 2023-12-11T18:00:28.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:00:28 smithi059 ceph-mon[98847]: pgmap v449: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.9 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 101 op/s 2023-12-11T18:00:28.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:00:28 smithi115 ceph-mon[102748]: pgmap v449: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.9 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 101 op/s 2023-12-11T18:00:30.536 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:00:30 smithi115 ceph-mon[102748]: pgmap v450: 129 pgs: 56 active+clean+snaptrim_wait, 18 active+clean+snaptrim, 55 active+clean; 1.9 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.1 MiB/s wr, 98 op/s 2023-12-11T18:00:30.657 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:00:30 smithi059 ceph-mon[98847]: pgmap v450: 129 pgs: 56 active+clean+snaptrim_wait, 18 active+clean+snaptrim, 55 active+clean; 1.9 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.1 MiB/s wr, 98 op/s 2023-12-11T18:00:32.532 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:00:32 smithi115 ceph-mon[102748]: pgmap v451: 129 pgs: 56 active+clean+snaptrim_wait, 18 active+clean+snaptrim, 55 active+clean; 1.9 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.3 MiB/s wr, 99 op/s 2023-12-11T18:00:32.656 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:00:32 smithi059 ceph-mon[98847]: pgmap v451: 129 pgs: 56 active+clean+snaptrim_wait, 18 active+clean+snaptrim, 55 active+clean; 1.9 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.3 MiB/s wr, 99 op/s 2023-12-11T18:00:34.534 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:00:34 smithi115 ceph-mon[102748]: pgmap v452: 129 pgs: 48 active+clean+snaptrim_wait, 17 active+clean+snaptrim, 64 active+clean; 1.9 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 711 KiB/s wr, 96 op/s 2023-12-11T18:00:34.656 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:00:34 smithi059 ceph-mon[98847]: pgmap v452: 129 pgs: 48 active+clean+snaptrim_wait, 17 active+clean+snaptrim, 64 active+clean; 1.9 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 711 KiB/s wr, 96 op/s 2023-12-11T18:00:36.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:00:36 smithi059 ceph-mon[98847]: pgmap v453: 129 pgs: 49 active+clean+snaptrim_wait, 16 active+clean+snaptrim, 64 active+clean; 1.9 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 960 KiB/s rd, 975 KiB/s wr, 102 op/s 2023-12-11T18:00:36.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:00:36 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T18:00:36.536 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:00:36 smithi115 ceph-mon[102748]: pgmap v453: 129 pgs: 49 active+clean+snaptrim_wait, 16 active+clean+snaptrim, 64 active+clean; 1.9 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 960 KiB/s rd, 975 KiB/s wr, 102 op/s 2023-12-11T18:00:36.536 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:00:36 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T18:00:37.406 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:00:37 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T18:00:37.406 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:00:37 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T18:00:37.406 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:00:37 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T18:00:37.532 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:00:37 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T18:00:37.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:00:37 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T18:00:37.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:00:37 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T18:00:38.532 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:00:38 smithi115 ceph-mon[102748]: pgmap v454: 129 pgs: 49 active+clean+snaptrim_wait, 16 active+clean+snaptrim, 64 active+clean; 1.9 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 669 KiB/s rd, 621 KiB/s wr, 91 op/s 2023-12-11T18:00:38.656 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:00:38 smithi059 ceph-mon[98847]: pgmap v454: 129 pgs: 49 active+clean+snaptrim_wait, 16 active+clean+snaptrim, 64 active+clean; 1.9 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 669 KiB/s rd, 621 KiB/s wr, 91 op/s 2023-12-11T18:00:39.532 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:00:39 smithi115 ceph-mon[102748]: pgmap v455: 129 pgs: 33 active+clean+snaptrim_wait, 14 active+clean+snaptrim, 82 active+clean; 1.9 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 854 KiB/s rd, 702 KiB/s wr, 105 op/s 2023-12-11T18:00:39.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:00:39 smithi059 ceph-mon[98847]: pgmap v455: 129 pgs: 33 active+clean+snaptrim_wait, 14 active+clean+snaptrim, 82 active+clean; 1.9 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 854 KiB/s rd, 702 KiB/s wr, 105 op/s 2023-12-11T18:00:42.410 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:00:42 smithi059 ceph-mon[98847]: pgmap v456: 129 pgs: 33 active+clean+snaptrim_wait, 12 active+clean+snaptrim, 84 active+clean; 1.9 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.0 MiB/s wr, 113 op/s 2023-12-11T18:00:42.532 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:00:42 smithi115 ceph-mon[102748]: pgmap v456: 129 pgs: 33 active+clean+snaptrim_wait, 12 active+clean+snaptrim, 84 active+clean; 1.9 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.0 MiB/s wr, 113 op/s 2023-12-11T18:00:44.407 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:00:44 smithi059 ceph-mon[98847]: pgmap v457: 129 pgs: 25 active+clean+snaptrim_wait, 10 active+clean+snaptrim, 94 active+clean; 1.9 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 785 KiB/s rd, 746 KiB/s wr, 122 op/s 2023-12-11T18:00:44.532 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:00:44 smithi115 ceph-mon[102748]: pgmap v457: 129 pgs: 25 active+clean+snaptrim_wait, 10 active+clean+snaptrim, 94 active+clean; 1.9 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 785 KiB/s rd, 746 KiB/s wr, 122 op/s 2023-12-11T18:00:46.411 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:00:46 smithi059 ceph-mon[98847]: pgmap v458: 129 pgs: 25 active+clean+snaptrim_wait, 10 active+clean+snaptrim, 94 active+clean; 1.9 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 131 op/s 2023-12-11T18:00:46.532 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:00:46 smithi115 ceph-mon[102748]: pgmap v458: 129 pgs: 25 active+clean+snaptrim_wait, 10 active+clean+snaptrim, 94 active+clean; 1.9 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 131 op/s 2023-12-11T18:00:48.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:00:48 smithi059 ceph-mon[98847]: pgmap v459: 129 pgs: 25 active+clean+snaptrim_wait, 10 active+clean+snaptrim, 94 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 945 KiB/s rd, 829 KiB/s wr, 117 op/s 2023-12-11T18:00:48.532 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:00:48 smithi115 ceph-mon[102748]: pgmap v459: 129 pgs: 25 active+clean+snaptrim_wait, 10 active+clean+snaptrim, 94 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 945 KiB/s rd, 829 KiB/s wr, 117 op/s 2023-12-11T18:00:50.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:00:50 smithi059 ceph-mon[98847]: pgmap v460: 129 pgs: 25 active+clean+snaptrim_wait, 10 active+clean+snaptrim, 94 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1006 KiB/s rd, 889 KiB/s wr, 114 op/s 2023-12-11T18:00:50.532 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:00:50 smithi115 ceph-mon[102748]: pgmap v460: 129 pgs: 25 active+clean+snaptrim_wait, 10 active+clean+snaptrim, 94 active+clean; 1.9 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1006 KiB/s rd, 889 KiB/s wr, 114 op/s 2023-12-11T18:00:52.412 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:00:52 smithi059 ceph-mon[98847]: pgmap v461: 129 pgs: 19 active+clean+snaptrim_wait, 9 active+clean+snaptrim, 101 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 824 KiB/s rd, 809 KiB/s wr, 102 op/s 2023-12-11T18:00:52.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:00:52 smithi115 ceph-mon[102748]: pgmap v461: 129 pgs: 19 active+clean+snaptrim_wait, 9 active+clean+snaptrim, 101 active+clean; 1.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 824 KiB/s rd, 809 KiB/s wr, 102 op/s 2023-12-11T18:00:54.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:00:54 smithi059 ceph-mon[98847]: pgmap v462: 129 pgs: 19 active+clean+snaptrim_wait, 9 active+clean+snaptrim, 101 active+clean; 1.9 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 713 KiB/s rd, 703 KiB/s wr, 85 op/s 2023-12-11T18:00:54.532 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:00:54 smithi115 ceph-mon[102748]: pgmap v462: 129 pgs: 19 active+clean+snaptrim_wait, 9 active+clean+snaptrim, 101 active+clean; 1.9 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 713 KiB/s rd, 703 KiB/s wr, 85 op/s 2023-12-11T18:00:56.536 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:00:56 smithi115 ceph-mon[102748]: pgmap v463: 129 pgs: 13 active+clean+snaptrim_wait, 7 active+clean+snaptrim, 109 active+clean; 1.9 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 694 KiB/s rd, 677 KiB/s wr, 80 op/s 2023-12-11T18:00:56.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:00:56 smithi059 ceph-mon[98847]: pgmap v463: 129 pgs: 13 active+clean+snaptrim_wait, 7 active+clean+snaptrim, 109 active+clean; 1.9 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 694 KiB/s rd, 677 KiB/s wr, 80 op/s 2023-12-11T18:00:58.534 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:00:58 smithi115 ceph-mon[102748]: pgmap v464: 129 pgs: 7 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 119 active+clean; 1.9 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 687 KiB/s rd, 666 KiB/s wr, 75 op/s 2023-12-11T18:00:58.659 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:00:58 smithi059 ceph-mon[98847]: pgmap v464: 129 pgs: 7 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 119 active+clean; 1.9 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 687 KiB/s rd, 666 KiB/s wr, 75 op/s 2023-12-11T18:01:00.532 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:01:00 smithi115 ceph-mon[102748]: pgmap v465: 129 pgs: 7 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 120 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 107 op/s 2023-12-11T18:01:00.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:01:00 smithi059 ceph-mon[98847]: pgmap v465: 129 pgs: 7 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 120 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 107 op/s 2023-12-11T18:01:02.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:01:02 smithi115 ceph-mon[102748]: pgmap v466: 129 pgs: 7 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 120 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 145 op/s 2023-12-11T18:01:02.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:01:02 smithi059 ceph-mon[98847]: pgmap v466: 129 pgs: 7 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 120 active+clean; 2.0 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 145 op/s 2023-12-11T18:01:04.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:01:04 smithi115 ceph-mon[102748]: pgmap v467: 129 pgs: 7 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 120 active+clean; 1.9 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 183 op/s 2023-12-11T18:01:04.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:01:04 smithi059 ceph-mon[98847]: pgmap v467: 129 pgs: 7 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 120 active+clean; 1.9 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 183 op/s 2023-12-11T18:01:06.537 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:01:06 smithi115 ceph-mon[102748]: pgmap v468: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 1.9 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 219 op/s 2023-12-11T18:01:06.657 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:01:06 smithi059 ceph-mon[98847]: pgmap v468: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 1.9 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 219 op/s 2023-12-11T18:01:07.536 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:01:07 smithi115 ceph-mon[102748]: osdmap e111: 12 total, 12 up, 12 in 2023-12-11T18:01:07.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:01:07 smithi059 ceph-mon[98847]: osdmap e111: 12 total, 12 up, 12 in 2023-12-11T18:01:08.535 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:01:08 smithi115 ceph-mon[102748]: pgmap v470: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 1.9 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 260 op/s 2023-12-11T18:01:08.657 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:01:08 smithi059 ceph-mon[98847]: pgmap v470: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 1.9 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 260 op/s 2023-12-11T18:01:09.656 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:01:09 smithi059 ceph-mon[98847]: pgmap v471: 129 pgs: 29 active+clean+snaptrim_wait, 10 active+clean+snaptrim, 90 active+clean; 1.9 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.4 MiB/s wr, 247 op/s 2023-12-11T18:01:09.785 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:01:09 smithi115 ceph-mon[102748]: pgmap v471: 129 pgs: 29 active+clean+snaptrim_wait, 10 active+clean+snaptrim, 90 active+clean; 1.9 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.4 MiB/s wr, 247 op/s 2023-12-11T18:01:11.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:01:11 smithi059 ceph-mon[98847]: osdmap e112: 12 total, 12 up, 12 in 2023-12-11T18:01:11.536 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:01:11 smithi115 ceph-mon[102748]: osdmap e112: 12 total, 12 up, 12 in 2023-12-11T18:01:12.410 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:01:12 smithi059 ceph-mon[98847]: pgmap v473: 129 pgs: 42 active+clean+snaptrim_wait, 15 active+clean+snaptrim, 72 active+clean; 1.9 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.9 MiB/s wr, 220 op/s 2023-12-11T18:01:12.532 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:01:12 smithi115 ceph-mon[102748]: pgmap v473: 129 pgs: 42 active+clean+snaptrim_wait, 15 active+clean+snaptrim, 72 active+clean; 1.9 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.9 MiB/s wr, 220 op/s 2023-12-11T18:01:14.406 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:01:14 smithi059 ceph-mon[98847]: pgmap v474: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.9 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.3 MiB/s wr, 188 op/s 2023-12-11T18:01:14.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:01:14 smithi115 ceph-mon[102748]: pgmap v474: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.9 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.3 MiB/s wr, 188 op/s 2023-12-11T18:01:16.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:01:16 smithi059 ceph-mon[98847]: pgmap v475: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.9 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.6 MiB/s wr, 143 op/s 2023-12-11T18:01:16.406 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:01:16 smithi059 ceph-mon[98847]: osdmap e113: 12 total, 12 up, 12 in 2023-12-11T18:01:16.532 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:01:16 smithi115 ceph-mon[102748]: pgmap v475: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.9 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.6 MiB/s wr, 143 op/s 2023-12-11T18:01:16.532 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:01:16 smithi115 ceph-mon[102748]: osdmap e113: 12 total, 12 up, 12 in 2023-12-11T18:01:18.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:01:18 smithi059 ceph-mon[98847]: pgmap v477: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.9 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 959 KiB/s rd, 963 KiB/s wr, 121 op/s 2023-12-11T18:01:18.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:01:18 smithi115 ceph-mon[102748]: pgmap v477: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.9 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 959 KiB/s rd, 963 KiB/s wr, 121 op/s 2023-12-11T18:01:19.155 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 18:01:18 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T18:01:18.655+0000 7f4642beb700 -1 calc_pg_upmaps abort due to max <= 0 2023-12-11T18:01:20.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:01:20 smithi059 ceph-mon[98847]: pgmap v478: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.9 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 425 KiB/s rd, 364 KiB/s wr, 100 op/s 2023-12-11T18:01:20.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:01:20 smithi115 ceph-mon[102748]: pgmap v478: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.9 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 425 KiB/s rd, 364 KiB/s wr, 100 op/s 2023-12-11T18:01:22.536 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:01:22 smithi115 ceph-mon[102748]: pgmap v479: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.9 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 509 KiB/s rd, 469 KiB/s wr, 104 op/s 2023-12-11T18:01:22.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:01:22 smithi059 ceph-mon[98847]: pgmap v479: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.9 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 509 KiB/s rd, 469 KiB/s wr, 104 op/s 2023-12-11T18:01:24.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:01:24 smithi115 ceph-mon[102748]: pgmap v480: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.9 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 759 KiB/s rd, 747 KiB/s wr, 101 op/s 2023-12-11T18:01:24.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:01:24 smithi059 ceph-mon[98847]: pgmap v480: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.9 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 759 KiB/s rd, 747 KiB/s wr, 101 op/s 2023-12-11T18:01:26.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:01:26 smithi115 ceph-mon[102748]: pgmap v481: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.9 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 432 KiB/s rd, 475 KiB/s wr, 103 op/s 2023-12-11T18:01:26.656 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:01:26 smithi059 ceph-mon[98847]: pgmap v481: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.9 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 432 KiB/s rd, 475 KiB/s wr, 103 op/s 2023-12-11T18:01:28.532 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:01:28 smithi115 ceph-mon[102748]: pgmap v482: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.9 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 620 KiB/s rd, 691 KiB/s wr, 101 op/s 2023-12-11T18:01:28.656 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:01:28 smithi059 ceph-mon[98847]: pgmap v482: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.9 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 620 KiB/s rd, 691 KiB/s wr, 101 op/s 2023-12-11T18:01:30.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:01:30 smithi115 ceph-mon[102748]: pgmap v483: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.9 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 592 KiB/s rd, 669 KiB/s wr, 102 op/s 2023-12-11T18:01:30.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:01:30 smithi059 ceph-mon[98847]: pgmap v483: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.9 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 592 KiB/s rd, 669 KiB/s wr, 102 op/s 2023-12-11T18:01:32.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:01:32 smithi115 ceph-mon[102748]: pgmap v484: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.9 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 666 KiB/s rd, 748 KiB/s wr, 101 op/s 2023-12-11T18:01:32.658 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:01:32 smithi059 ceph-mon[98847]: pgmap v484: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.9 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 666 KiB/s rd, 748 KiB/s wr, 101 op/s 2023-12-11T18:01:33.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:01:33 smithi059 ceph-mon[98847]: pgmap v485: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.9 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 727 KiB/s rd, 769 KiB/s wr, 99 op/s 2023-12-11T18:01:33.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:01:33 smithi115 ceph-mon[102748]: pgmap v485: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.9 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 727 KiB/s rd, 769 KiB/s wr, 99 op/s 2023-12-11T18:01:36.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:01:36 smithi059 ceph-mon[98847]: pgmap v486: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.9 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 520 KiB/s rd, 530 KiB/s wr, 97 op/s 2023-12-11T18:01:36.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:01:36 smithi115 ceph-mon[102748]: pgmap v486: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.9 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 520 KiB/s rd, 530 KiB/s wr, 97 op/s 2023-12-11T18:01:37.535 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:01:37 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T18:01:37.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:01:37 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T18:01:38.535 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:01:38 smithi115 ceph-mon[102748]: pgmap v487: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.9 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 719 KiB/s rd, 777 KiB/s wr, 96 op/s 2023-12-11T18:01:38.535 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:01:38 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T18:01:38.536 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:01:38 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T18:01:38.536 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:01:38 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T18:01:38.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:01:38 smithi059 ceph-mon[98847]: pgmap v487: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.9 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 719 KiB/s rd, 777 KiB/s wr, 96 op/s 2023-12-11T18:01:38.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:01:38 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T18:01:38.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:01:38 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T18:01:38.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:01:38 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T18:01:40.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:01:40 smithi115 ceph-mon[102748]: pgmap v488: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.9 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 519 KiB/s wr, 95 op/s 2023-12-11T18:01:40.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:01:40 smithi059 ceph-mon[98847]: pgmap v488: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.9 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 519 KiB/s wr, 95 op/s 2023-12-11T18:01:42.534 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:01:42 smithi115 ceph-mon[102748]: pgmap v489: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.9 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 723 KiB/s rd, 668 KiB/s wr, 97 op/s 2023-12-11T18:01:42.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:01:42 smithi059 ceph-mon[98847]: pgmap v489: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.9 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 723 KiB/s rd, 668 KiB/s wr, 97 op/s 2023-12-11T18:01:44.532 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:01:44 smithi115 ceph-mon[102748]: pgmap v490: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.9 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 589 KiB/s wr, 91 op/s 2023-12-11T18:01:44.658 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:01:44 smithi059 ceph-mon[98847]: pgmap v490: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.9 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 589 KiB/s wr, 91 op/s 2023-12-11T18:01:46.534 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:01:46 smithi115 ceph-mon[102748]: pgmap v491: 129 pgs: 48 active+clean+snaptrim_wait, 17 active+clean+snaptrim, 64 active+clean; 1.9 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 692 KiB/s rd, 768 KiB/s wr, 100 op/s 2023-12-11T18:01:46.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:01:46 smithi059 ceph-mon[98847]: pgmap v491: 129 pgs: 48 active+clean+snaptrim_wait, 17 active+clean+snaptrim, 64 active+clean; 1.9 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 692 KiB/s rd, 768 KiB/s wr, 100 op/s 2023-12-11T18:01:48.657 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:01:48 smithi059 ceph-mon[98847]: pgmap v492: 129 pgs: 48 active+clean+snaptrim_wait, 16 active+clean+snaptrim, 65 active+clean; 1.9 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 689 KiB/s rd, 772 KiB/s wr, 102 op/s 2023-12-11T18:01:48.782 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:01:48 smithi115 ceph-mon[102748]: pgmap v492: 129 pgs: 48 active+clean+snaptrim_wait, 16 active+clean+snaptrim, 65 active+clean; 1.9 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 689 KiB/s rd, 772 KiB/s wr, 102 op/s 2023-12-11T18:01:49.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:01:49 smithi059 ceph-mon[98847]: pgmap v493: 129 pgs: 41 active+clean+snaptrim_wait, 15 active+clean+snaptrim, 73 active+clean; 1.9 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 625 KiB/s rd, 788 KiB/s wr, 102 op/s 2023-12-11T18:01:49.782 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:01:49 smithi115 ceph-mon[102748]: pgmap v493: 129 pgs: 41 active+clean+snaptrim_wait, 15 active+clean+snaptrim, 73 active+clean; 1.9 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 625 KiB/s rd, 788 KiB/s wr, 102 op/s 2023-12-11T18:01:52.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:01:52 smithi059 ceph-mon[98847]: pgmap v494: 129 pgs: 41 active+clean+snaptrim_wait, 14 active+clean+snaptrim, 74 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 720 KiB/s rd, 785 KiB/s wr, 104 op/s 2023-12-11T18:01:52.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:01:52 smithi115 ceph-mon[102748]: pgmap v494: 129 pgs: 41 active+clean+snaptrim_wait, 14 active+clean+snaptrim, 74 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 720 KiB/s rd, 785 KiB/s wr, 104 op/s 2023-12-11T18:01:54.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:01:54 smithi059 ceph-mon[98847]: pgmap v495: 129 pgs: 26 active+clean+snaptrim_wait, 11 active+clean+snaptrim, 92 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 710 KiB/s rd, 717 KiB/s wr, 109 op/s 2023-12-11T18:01:54.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:01:54 smithi115 ceph-mon[102748]: pgmap v495: 129 pgs: 26 active+clean+snaptrim_wait, 11 active+clean+snaptrim, 92 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 710 KiB/s rd, 717 KiB/s wr, 109 op/s 2023-12-11T18:01:56.532 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:01:56 smithi115 ceph-mon[102748]: pgmap v496: 129 pgs: 26 active+clean+snaptrim_wait, 10 active+clean+snaptrim, 93 active+clean; 1.9 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1011 KiB/s rd, 1.1 MiB/s wr, 129 op/s 2023-12-11T18:01:56.657 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:01:56 smithi059 ceph-mon[98847]: pgmap v496: 129 pgs: 26 active+clean+snaptrim_wait, 10 active+clean+snaptrim, 93 active+clean; 1.9 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1011 KiB/s rd, 1.1 MiB/s wr, 129 op/s 2023-12-11T18:01:58.532 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:01:58 smithi115 ceph-mon[102748]: pgmap v497: 129 pgs: 19 active+clean+snaptrim_wait, 9 active+clean+snaptrim, 101 active+clean; 1.9 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 781 KiB/s rd, 801 KiB/s wr, 129 op/s 2023-12-11T18:01:58.656 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:01:58 smithi059 ceph-mon[98847]: pgmap v497: 129 pgs: 19 active+clean+snaptrim_wait, 9 active+clean+snaptrim, 101 active+clean; 1.9 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 781 KiB/s rd, 801 KiB/s wr, 129 op/s 2023-12-11T18:02:00.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:02:00 smithi115 ceph-mon[102748]: pgmap v498: 129 pgs: 13 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 108 active+clean; 1.9 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 879 KiB/s rd, 818 KiB/s wr, 130 op/s 2023-12-11T18:02:00.659 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:02:00 smithi059 ceph-mon[98847]: pgmap v498: 129 pgs: 13 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 108 active+clean; 1.9 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 879 KiB/s rd, 818 KiB/s wr, 130 op/s 2023-12-11T18:02:02.534 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:02:02 smithi115 ceph-mon[102748]: pgmap v499: 129 pgs: 13 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 108 active+clean; 1.9 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 838 KiB/s wr, 130 op/s 2023-12-11T18:02:02.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:02:02 smithi059 ceph-mon[98847]: pgmap v499: 129 pgs: 13 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 108 active+clean; 1.9 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 838 KiB/s wr, 130 op/s 2023-12-11T18:02:04.540 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:02:04 smithi115 ceph-mon[102748]: pgmap v500: 129 pgs: 13 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 108 active+clean; 1.9 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 137 op/s 2023-12-11T18:02:04.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:02:04 smithi059 ceph-mon[98847]: pgmap v500: 129 pgs: 13 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 108 active+clean; 1.9 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 137 op/s 2023-12-11T18:02:06.532 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:02:06 smithi115 ceph-mon[102748]: pgmap v501: 129 pgs: 13 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 108 active+clean; 1.9 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 875 KiB/s rd, 971 KiB/s wr, 138 op/s 2023-12-11T18:02:06.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:02:06 smithi059 ceph-mon[98847]: pgmap v501: 129 pgs: 13 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 108 active+clean; 1.9 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 875 KiB/s rd, 971 KiB/s wr, 138 op/s 2023-12-11T18:02:08.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:02:08 smithi115 ceph-mon[102748]: pgmap v502: 129 pgs: 13 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 108 active+clean; 1.9 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 759 KiB/s rd, 697 KiB/s wr, 128 op/s 2023-12-11T18:02:08.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:02:08 smithi059 ceph-mon[98847]: pgmap v502: 129 pgs: 13 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 108 active+clean; 1.9 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 759 KiB/s rd, 697 KiB/s wr, 128 op/s 2023-12-11T18:02:10.532 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:02:10 smithi115 ceph-mon[102748]: pgmap v503: 129 pgs: 13 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 108 active+clean; 1.9 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 814 KiB/s rd, 971 KiB/s wr, 116 op/s 2023-12-11T18:02:10.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:02:10 smithi059 ceph-mon[98847]: pgmap v503: 129 pgs: 13 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 108 active+clean; 1.9 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 814 KiB/s rd, 971 KiB/s wr, 116 op/s 2023-12-11T18:02:12.532 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:02:12 smithi115 ceph-mon[102748]: pgmap v504: 129 pgs: 13 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 108 active+clean; 1.9 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 712 KiB/s rd, 953 KiB/s wr, 115 op/s 2023-12-11T18:02:12.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:02:12 smithi115 ceph-mon[102748]: osdmap e114: 12 total, 12 up, 12 in 2023-12-11T18:02:12.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:02:12 smithi059 ceph-mon[98847]: pgmap v504: 129 pgs: 13 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 108 active+clean; 1.9 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 712 KiB/s rd, 953 KiB/s wr, 115 op/s 2023-12-11T18:02:12.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:02:12 smithi059 ceph-mon[98847]: osdmap e114: 12 total, 12 up, 12 in 2023-12-11T18:02:14.536 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:02:14 smithi115 ceph-mon[102748]: pgmap v506: 129 pgs: 24 active+clean+snaptrim_wait, 10 active+clean+snaptrim, 95 active+clean; 1.9 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 663 KiB/s rd, 645 KiB/s wr, 123 op/s 2023-12-11T18:02:14.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:02:14 smithi059 ceph-mon[98847]: pgmap v506: 129 pgs: 24 active+clean+snaptrim_wait, 10 active+clean+snaptrim, 95 active+clean; 1.9 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 663 KiB/s rd, 645 KiB/s wr, 123 op/s 2023-12-11T18:02:16.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:02:16 smithi115 ceph-mon[102748]: pgmap v507: 129 pgs: 45 active+clean+snaptrim_wait, 16 active+clean+snaptrim, 68 active+clean; 1.9 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 113 op/s 2023-12-11T18:02:16.657 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:02:16 smithi059 ceph-mon[98847]: pgmap v507: 129 pgs: 45 active+clean+snaptrim_wait, 16 active+clean+snaptrim, 68 active+clean; 1.9 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 113 op/s 2023-12-11T18:02:18.535 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:02:18 smithi115 ceph-mon[102748]: pgmap v508: 129 pgs: 59 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 50 active+clean; 1.9 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 848 KiB/s rd, 885 KiB/s wr, 110 op/s 2023-12-11T18:02:18.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:02:18 smithi059 ceph-mon[98847]: pgmap v508: 129 pgs: 59 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 50 active+clean; 1.9 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 848 KiB/s rd, 885 KiB/s wr, 110 op/s 2023-12-11T18:02:19.157 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 18:02:18 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T18:02:18.660+0000 7f4642beb700 -1 calc_pg_upmaps abort due to max <= 0 2023-12-11T18:02:20.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:02:20 smithi059 ceph-mon[98847]: pgmap v509: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.9 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 952 KiB/s rd, 724 KiB/s wr, 110 op/s 2023-12-11T18:02:20.788 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:02:20 smithi115 ceph-mon[102748]: pgmap v509: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.9 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 952 KiB/s rd, 724 KiB/s wr, 110 op/s 2023-12-11T18:02:22.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:02:22 smithi059 ceph-mon[98847]: pgmap v510: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.9 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 959 KiB/s rd, 727 KiB/s wr, 102 op/s 2023-12-11T18:02:22.784 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:02:22 smithi115 ceph-mon[102748]: pgmap v510: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.9 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 959 KiB/s rd, 727 KiB/s wr, 102 op/s 2023-12-11T18:02:24.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:02:24 smithi059 ceph-mon[98847]: pgmap v511: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.9 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 966 KiB/s wr, 95 op/s 2023-12-11T18:02:24.784 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:02:24 smithi115 ceph-mon[102748]: pgmap v511: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.9 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 966 KiB/s wr, 95 op/s 2023-12-11T18:02:26.537 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:02:26 smithi115 ceph-mon[102748]: pgmap v512: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.9 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 716 KiB/s rd, 811 KiB/s wr, 93 op/s 2023-12-11T18:02:26.537 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:02:26 smithi115 ceph-mon[102748]: osdmap e115: 12 total, 12 up, 12 in 2023-12-11T18:02:26.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:02:26 smithi059 ceph-mon[98847]: pgmap v512: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.9 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 716 KiB/s rd, 811 KiB/s wr, 93 op/s 2023-12-11T18:02:26.656 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:02:26 smithi059 ceph-mon[98847]: osdmap e115: 12 total, 12 up, 12 in 2023-12-11T18:02:28.534 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:02:28 smithi115 ceph-mon[102748]: pgmap v514: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.9 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 859 KiB/s rd, 849 KiB/s wr, 101 op/s 2023-12-11T18:02:28.658 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:02:28 smithi059 ceph-mon[98847]: pgmap v514: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.9 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 859 KiB/s rd, 849 KiB/s wr, 101 op/s 2023-12-11T18:02:30.532 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:02:30 smithi115 ceph-mon[102748]: pgmap v515: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.9 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 702 KiB/s rd, 679 KiB/s wr, 106 op/s 2023-12-11T18:02:30.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:02:30 smithi059 ceph-mon[98847]: pgmap v515: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.9 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 702 KiB/s rd, 679 KiB/s wr, 106 op/s 2023-12-11T18:02:32.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:02:32 smithi115 ceph-mon[102748]: pgmap v516: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.9 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 949 KiB/s rd, 924 KiB/s wr, 117 op/s 2023-12-11T18:02:32.656 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:02:32 smithi059 ceph-mon[98847]: pgmap v516: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.9 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 949 KiB/s rd, 924 KiB/s wr, 117 op/s 2023-12-11T18:02:34.537 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:02:34 smithi115 ceph-mon[102748]: pgmap v517: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.9 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 697 KiB/s rd, 552 KiB/s wr, 119 op/s 2023-12-11T18:02:34.658 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:02:34 smithi059 ceph-mon[98847]: pgmap v517: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.9 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 697 KiB/s rd, 552 KiB/s wr, 119 op/s 2023-12-11T18:02:36.663 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:02:36 smithi059 ceph-mon[98847]: pgmap v518: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.9 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 729 KiB/s rd, 621 KiB/s wr, 122 op/s 2023-12-11T18:02:36.788 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:02:36 smithi115 ceph-mon[102748]: pgmap v518: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.9 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 729 KiB/s rd, 621 KiB/s wr, 122 op/s 2023-12-11T18:02:38.662 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:02:38 smithi059 ceph-mon[98847]: pgmap v519: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.9 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 862 KiB/s rd, 769 KiB/s wr, 114 op/s 2023-12-11T18:02:38.662 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:02:38 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T18:02:38.786 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:02:38 smithi115 ceph-mon[102748]: pgmap v519: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.9 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 862 KiB/s rd, 769 KiB/s wr, 114 op/s 2023-12-11T18:02:38.786 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:02:38 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T18:02:40.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:02:40 smithi059 ceph-mon[98847]: pgmap v520: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.9 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 478 KiB/s rd, 492 KiB/s wr, 103 op/s 2023-12-11T18:02:40.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:02:40 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T18:02:40.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:02:40 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T18:02:40.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:02:40 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T18:02:40.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:02:40 smithi115 ceph-mon[102748]: pgmap v520: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.9 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 478 KiB/s rd, 492 KiB/s wr, 103 op/s 2023-12-11T18:02:40.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:02:40 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T18:02:40.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:02:40 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T18:02:40.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:02:40 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T18:02:41.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:02:41 smithi115 ceph-mon[102748]: pgmap v521: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.9 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 819 KiB/s rd, 814 KiB/s wr, 100 op/s 2023-12-11T18:02:41.906 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:02:41 smithi059 ceph-mon[98847]: pgmap v521: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.9 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 819 KiB/s rd, 814 KiB/s wr, 100 op/s 2023-12-11T18:02:43.907 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:02:43 smithi059 ceph-mon[98847]: pgmap v522: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.9 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 606 KiB/s rd, 604 KiB/s wr, 91 op/s 2023-12-11T18:02:44.035 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:02:43 smithi115 ceph-mon[102748]: pgmap v522: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.9 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 606 KiB/s rd, 604 KiB/s wr, 91 op/s 2023-12-11T18:02:46.536 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:02:46 smithi115 ceph-mon[102748]: pgmap v523: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.9 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 611 KiB/s rd, 608 KiB/s wr, 83 op/s 2023-12-11T18:02:46.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:02:46 smithi059 ceph-mon[98847]: pgmap v523: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.9 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 611 KiB/s rd, 608 KiB/s wr, 83 op/s 2023-12-11T18:02:48.535 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:02:48 smithi115 ceph-mon[102748]: pgmap v524: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.9 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 892 KiB/s rd, 857 KiB/s wr, 94 op/s 2023-12-11T18:02:48.659 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:02:48 smithi059 ceph-mon[98847]: pgmap v524: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.9 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 892 KiB/s rd, 857 KiB/s wr, 94 op/s 2023-12-11T18:02:50.535 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:02:50 smithi115 ceph-mon[102748]: pgmap v525: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.9 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 973 KiB/s rd, 934 KiB/s wr, 90 op/s 2023-12-11T18:02:50.656 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:02:50 smithi059 ceph-mon[98847]: pgmap v525: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.9 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 973 KiB/s rd, 934 KiB/s wr, 90 op/s 2023-12-11T18:02:52.537 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:02:52 smithi115 ceph-mon[102748]: pgmap v526: 129 pgs: 48 active+clean+snaptrim_wait, 18 active+clean+snaptrim, 63 active+clean; 1.9 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 971 KiB/s rd, 930 KiB/s wr, 92 op/s 2023-12-11T18:02:52.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:02:52 smithi059 ceph-mon[98847]: pgmap v526: 129 pgs: 48 active+clean+snaptrim_wait, 18 active+clean+snaptrim, 63 active+clean; 1.9 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 971 KiB/s rd, 930 KiB/s wr, 92 op/s 2023-12-11T18:02:54.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:02:54 smithi059 ceph-mon[98847]: pgmap v527: 129 pgs: 48 active+clean+snaptrim_wait, 18 active+clean+snaptrim, 63 active+clean; 1.9 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 618 KiB/s rd, 603 KiB/s wr, 91 op/s 2023-12-11T18:02:54.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:02:54 smithi115 ceph-mon[102748]: pgmap v527: 129 pgs: 48 active+clean+snaptrim_wait, 18 active+clean+snaptrim, 63 active+clean; 1.9 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 618 KiB/s rd, 603 KiB/s wr, 91 op/s 2023-12-11T18:02:56.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:02:56 smithi059 ceph-mon[98847]: pgmap v528: 129 pgs: 48 active+clean+snaptrim_wait, 17 active+clean+snaptrim, 64 active+clean; 1.9 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 915 KiB/s rd, 951 KiB/s wr, 96 op/s 2023-12-11T18:02:56.785 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:02:56 smithi115 ceph-mon[102748]: pgmap v528: 129 pgs: 48 active+clean+snaptrim_wait, 17 active+clean+snaptrim, 64 active+clean; 1.9 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 915 KiB/s rd, 951 KiB/s wr, 96 op/s 2023-12-11T18:02:58.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:02:58 smithi059 ceph-mon[98847]: pgmap v529: 129 pgs: 48 active+clean+snaptrim_wait, 16 active+clean+snaptrim, 65 active+clean; 1.9 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 918 KiB/s rd, 948 KiB/s wr, 111 op/s 2023-12-11T18:02:58.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:02:58 smithi115 ceph-mon[102748]: pgmap v529: 129 pgs: 48 active+clean+snaptrim_wait, 16 active+clean+snaptrim, 65 active+clean; 1.9 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 918 KiB/s rd, 948 KiB/s wr, 111 op/s 2023-12-11T18:03:00.660 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:03:00 smithi059 ceph-mon[98847]: pgmap v530: 129 pgs: 48 active+clean+snaptrim_wait, 16 active+clean+snaptrim, 65 active+clean; 1.9 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 737 KiB/s rd, 868 KiB/s wr, 106 op/s 2023-12-11T18:03:00.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:03:00 smithi115 ceph-mon[102748]: pgmap v530: 129 pgs: 48 active+clean+snaptrim_wait, 16 active+clean+snaptrim, 65 active+clean; 1.9 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 737 KiB/s rd, 868 KiB/s wr, 106 op/s 2023-12-11T18:03:02.658 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:03:02 smithi059 ceph-mon[98847]: pgmap v531: 129 pgs: 41 active+clean+snaptrim_wait, 15 active+clean+snaptrim, 73 active+clean; 1.9 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 631 KiB/s wr, 116 op/s 2023-12-11T18:03:02.782 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:03:02 smithi115 ceph-mon[102748]: pgmap v531: 129 pgs: 41 active+clean+snaptrim_wait, 15 active+clean+snaptrim, 73 active+clean; 1.9 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 631 KiB/s wr, 116 op/s 2023-12-11T18:03:03.782 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:03:03 smithi115 ceph-mon[102748]: pgmap v532: 129 pgs: 36 active+clean+snaptrim_wait, 15 active+clean+snaptrim, 78 active+clean; 1.9 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 794 KiB/s rd, 763 KiB/s wr, 136 op/s 2023-12-11T18:03:03.909 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:03:03 smithi059 ceph-mon[98847]: pgmap v532: 129 pgs: 36 active+clean+snaptrim_wait, 15 active+clean+snaptrim, 78 active+clean; 1.9 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 794 KiB/s rd, 763 KiB/s wr, 136 op/s 2023-12-11T18:03:06.532 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:03:06 smithi115 ceph-mon[102748]: pgmap v533: 129 pgs: 36 active+clean+snaptrim_wait, 14 active+clean+snaptrim, 79 active+clean; 1.9 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 150 op/s 2023-12-11T18:03:06.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:03:06 smithi115 ceph-mon[102748]: osdmap e116: 12 total, 12 up, 12 in 2023-12-11T18:03:06.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:03:06 smithi059 ceph-mon[98847]: pgmap v533: 129 pgs: 36 active+clean+snaptrim_wait, 14 active+clean+snaptrim, 79 active+clean; 1.9 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 150 op/s 2023-12-11T18:03:06.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:03:06 smithi059 ceph-mon[98847]: osdmap e116: 12 total, 12 up, 12 in 2023-12-11T18:03:07.532 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:03:07 smithi115 ceph-mon[102748]: osdmap e117: 12 total, 12 up, 12 in 2023-12-11T18:03:07.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:03:07 smithi059 ceph-mon[98847]: osdmap e117: 12 total, 12 up, 12 in 2023-12-11T18:03:08.534 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:03:08 smithi115 ceph-mon[102748]: pgmap v536: 129 pgs: 33 active+clean+snaptrim_wait, 12 active+clean+snaptrim, 84 active+clean; 1.9 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1014 KiB/s wr, 150 op/s 2023-12-11T18:03:08.661 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:03:08 smithi059 ceph-mon[98847]: pgmap v536: 129 pgs: 33 active+clean+snaptrim_wait, 12 active+clean+snaptrim, 84 active+clean; 1.9 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1014 KiB/s wr, 150 op/s 2023-12-11T18:03:10.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:03:10 smithi115 ceph-mon[102748]: pgmap v537: 129 pgs: 36 active+clean+snaptrim_wait, 13 active+clean+snaptrim, 80 active+clean; 1.9 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 862 KiB/s rd, 960 KiB/s wr, 142 op/s 2023-12-11T18:03:10.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:03:10 smithi059 ceph-mon[98847]: pgmap v537: 129 pgs: 36 active+clean+snaptrim_wait, 13 active+clean+snaptrim, 80 active+clean; 1.9 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 862 KiB/s rd, 960 KiB/s wr, 142 op/s 2023-12-11T18:03:12.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:03:12 smithi059 ceph-mon[98847]: pgmap v538: 129 pgs: 51 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 59 active+clean; 1.9 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 128 op/s 2023-12-11T18:03:12.787 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:03:12 smithi115 ceph-mon[102748]: pgmap v538: 129 pgs: 51 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 59 active+clean; 1.9 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 128 op/s 2023-12-11T18:03:14.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:03:14 smithi059 ceph-mon[98847]: pgmap v539: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.9 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 116 op/s 2023-12-11T18:03:14.782 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:03:14 smithi115 ceph-mon[102748]: pgmap v539: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.9 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 116 op/s 2023-12-11T18:03:16.658 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:03:16 smithi059 ceph-mon[98847]: pgmap v540: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.9 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 965 KiB/s rd, 774 KiB/s wr, 112 op/s 2023-12-11T18:03:16.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:03:16 smithi115 ceph-mon[102748]: pgmap v540: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.9 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 965 KiB/s rd, 774 KiB/s wr, 112 op/s 2023-12-11T18:03:18.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:03:18 smithi059 ceph-mon[98847]: pgmap v541: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.9 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 125 op/s 2023-12-11T18:03:18.786 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:03:18 smithi115 ceph-mon[102748]: pgmap v541: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.9 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 125 op/s 2023-12-11T18:03:19.160 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 18:03:18 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T18:03:18.664+0000 7f4642beb700 -1 calc_pg_upmaps abort due to max <= 0 2023-12-11T18:03:20.658 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:03:20 smithi059 ceph-mon[98847]: pgmap v542: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.9 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 126 op/s 2023-12-11T18:03:20.786 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:03:20 smithi115 ceph-mon[102748]: pgmap v542: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.9 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 126 op/s 2023-12-11T18:03:21.788 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:03:21 smithi115 ceph-mon[102748]: pgmap v543: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.9 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 125 op/s 2023-12-11T18:03:21.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:03:21 smithi059 ceph-mon[98847]: pgmap v543: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.9 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 125 op/s 2023-12-11T18:03:23.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:03:23 smithi059 ceph-mon[98847]: pgmap v544: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.9 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 KiB/s rd, 878 KiB/s wr, 119 op/s 2023-12-11T18:03:24.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:03:23 smithi115 ceph-mon[102748]: pgmap v544: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.9 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 KiB/s rd, 878 KiB/s wr, 119 op/s 2023-12-11T18:03:26.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:03:26 smithi115 ceph-mon[102748]: pgmap v545: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.9 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 760 KiB/s rd, 893 KiB/s wr, 117 op/s 2023-12-11T18:03:26.657 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:03:26 smithi059 ceph-mon[98847]: pgmap v545: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.9 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 760 KiB/s rd, 893 KiB/s wr, 117 op/s 2023-12-11T18:03:28.534 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:03:28 smithi115 ceph-mon[102748]: pgmap v546: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.8 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 757 KiB/s rd, 897 KiB/s wr, 115 op/s 2023-12-11T18:03:28.659 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:03:28 smithi059 ceph-mon[98847]: pgmap v546: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.8 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 757 KiB/s rd, 897 KiB/s wr, 115 op/s 2023-12-11T18:03:30.656 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:03:30 smithi059 ceph-mon[98847]: pgmap v547: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.8 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 743 KiB/s rd, 706 KiB/s wr, 101 op/s 2023-12-11T18:03:30.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:03:30 smithi115 ceph-mon[102748]: pgmap v547: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.8 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 743 KiB/s rd, 706 KiB/s wr, 101 op/s 2023-12-11T18:03:32.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:03:32 smithi059 ceph-mon[98847]: pgmap v548: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.8 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 611 KiB/s rd, 531 KiB/s wr, 94 op/s 2023-12-11T18:03:32.782 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:03:32 smithi115 ceph-mon[102748]: pgmap v548: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.8 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 611 KiB/s rd, 531 KiB/s wr, 94 op/s 2023-12-11T18:03:34.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:03:34 smithi059 ceph-mon[98847]: pgmap v549: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.8 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 830 KiB/s rd, 857 KiB/s wr, 100 op/s 2023-12-11T18:03:34.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:03:34 smithi115 ceph-mon[102748]: pgmap v549: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.8 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 830 KiB/s rd, 857 KiB/s wr, 100 op/s 2023-12-11T18:03:36.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:03:36 smithi059 ceph-mon[98847]: pgmap v550: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.8 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 889 KiB/s rd, 936 KiB/s wr, 103 op/s 2023-12-11T18:03:36.786 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:03:36 smithi115 ceph-mon[102748]: pgmap v550: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.8 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 889 KiB/s rd, 936 KiB/s wr, 103 op/s 2023-12-11T18:03:38.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:03:38 smithi059 ceph-mon[98847]: pgmap v551: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.8 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 818 KiB/s rd, 651 KiB/s wr, 111 op/s 2023-12-11T18:03:38.785 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:03:38 smithi115 ceph-mon[102748]: pgmap v551: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.8 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 818 KiB/s rd, 651 KiB/s wr, 111 op/s 2023-12-11T18:03:40.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:03:40 smithi115 ceph-mon[102748]: pgmap v552: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.8 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 817 KiB/s rd, 649 KiB/s wr, 104 op/s 2023-12-11T18:03:40.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:03:40 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T18:03:40.910 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:03:40 smithi059 ceph-mon[98847]: pgmap v552: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.8 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 817 KiB/s rd, 649 KiB/s wr, 104 op/s 2023-12-11T18:03:40.910 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:03:40 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T18:03:41.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:03:41 smithi115 ceph-mon[102748]: pgmap v553: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.8 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 768 KiB/s rd, 755 KiB/s wr, 102 op/s 2023-12-11T18:03:41.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:03:41 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T18:03:41.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:03:41 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T18:03:41.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:03:41 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T18:03:41.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:03:41 smithi059 ceph-mon[98847]: pgmap v553: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.8 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 768 KiB/s rd, 755 KiB/s wr, 102 op/s 2023-12-11T18:03:41.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:03:41 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T18:03:41.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:03:41 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T18:03:41.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:03:41 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T18:03:43.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:03:43 smithi059 ceph-mon[98847]: pgmap v554: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.8 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 890 KiB/s rd, 885 KiB/s wr, 106 op/s 2023-12-11T18:03:44.032 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:03:43 smithi115 ceph-mon[102748]: pgmap v554: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.8 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 890 KiB/s rd, 885 KiB/s wr, 106 op/s 2023-12-11T18:03:46.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:03:46 smithi115 ceph-mon[102748]: pgmap v555: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.8 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 676 KiB/s rd, 564 KiB/s wr, 96 op/s 2023-12-11T18:03:46.658 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:03:46 smithi059 ceph-mon[98847]: pgmap v555: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.8 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 676 KiB/s rd, 564 KiB/s wr, 96 op/s 2023-12-11T18:03:47.961 INFO:tasks.workunit:Stopping ['suites/blogbench.sh'] on client.0... 2023-12-11T18:03:47.962 DEBUG:teuthology.orchestra.run.smithi059:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2023-12-11T18:03:48.286 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:03:48 smithi059 ceph-mon[98847]: pgmap v556: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.8 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 810 KiB/s rd, 698 KiB/s wr, 97 op/s 2023-12-11T18:03:48.464 DEBUG:teuthology.parallel:result is None 2023-12-11T18:03:48.464 DEBUG:teuthology.orchestra.run.smithi059:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2023-12-11T18:03:48.631 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2023-12-11T18:03:48.632 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2023-12-11T18:03:48.645 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2023-12-11T18:03:48.645 INFO:teuthology.task.exec:Executing custom commands... 2023-12-11T18:03:48.646 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi059.front.sepia.ceph.com 2023-12-11T18:03:48.646 DEBUG:teuthology.orchestra.run.smithi059:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2023-12-11T18:03:48.784 DEBUG:teuthology.orchestra.run.smithi059:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2023-12-11T18:03:48.784 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:03:48 smithi115 ceph-mon[102748]: pgmap v556: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.8 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 810 KiB/s rd, 698 KiB/s wr, 97 op/s 2023-12-11T18:03:48.914 DEBUG:teuthology.orchestra.run.smithi059:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2023-12-11T18:03:49.187 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:49.184+0000 7f707b5cc700 1 -- 172.21.15.59:0/2213568850 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7074109820 msgr2=0x7f70741151f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T18:03:49.187 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:49.184+0000 7f707b5cc700 1 --2- 172.21.15.59:0/2213568850 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7074109820 0x7f70741151f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f706c00b0a0 tx=0x7f706c00b3b0 comp rx=0 tx=0).stop 2023-12-11T18:03:49.188 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:49.184+0000 7f707b5cc700 1 -- 172.21.15.59:0/2213568850 shutdown_connections 2023-12-11T18:03:49.188 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:49.184+0000 7f707b5cc700 1 --2- 172.21.15.59:0/2213568850 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7074109820 0x7f70741151f0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:03:49.188 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:49.184+0000 7f707b5cc700 1 --2- 172.21.15.59:0/2213568850 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f707410d360 0x7f707410d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:03:49.188 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:49.184+0000 7f707b5cc700 1 --2- 172.21.15.59:0/2213568850 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7074108e60 0x7f70741092e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:03:49.188 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:49.184+0000 7f707b5cc700 1 -- 172.21.15.59:0/2213568850 >> 172.21.15.59:0/2213568850 conn(0x7f707406c5a0 msgr2=0x7f707406c9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T18:03:49.188 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:49.184+0000 7f707b5cc700 1 -- 172.21.15.59:0/2213568850 shutdown_connections 2023-12-11T18:03:49.188 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:49.184+0000 7f707b5cc700 1 -- 172.21.15.59:0/2213568850 wait complete. 2023-12-11T18:03:49.193 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:49.185+0000 7f707b5cc700 1 Processor -- start 2023-12-11T18:03:49.193 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:49.185+0000 7f707b5cc700 1 -- start start 2023-12-11T18:03:49.193 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:49.186+0000 7f707b5cc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7074108e60 0x7f70741131e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T18:03:49.193 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:49.186+0000 7f707b5cc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7074109820 0x7f7074113740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T18:03:49.193 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:49.186+0000 7f707b5cc700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f707410d360 0x7f70741a28c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T18:03:49.193 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:49.186+0000 7f707b5cc700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70741a2e00 con 0x7f7074109820 2023-12-11T18:03:49.193 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:49.186+0000 7f707b5cc700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70741a2f70 con 0x7f707410d360 2023-12-11T18:03:49.193 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:49.186+0000 7f707b5cc700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70741a3130 con 0x7f7074108e60 2023-12-11T18:03:49.194 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:49.186+0000 7f7079368700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7074108e60 0x7f70741131e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T18:03:49.194 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:49.186+0000 7f7079368700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7074108e60 0x7f70741131e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.59:40126/0 (socket says 172.21.15.59:40126) 2023-12-11T18:03:49.194 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:49.186+0000 7f7079368700 1 -- 172.21.15.59:0/1179153122 learned_addr learned my addr 172.21.15.59:0/1179153122 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T18:03:49.194 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:49.190+0000 7f7079368700 1 -- 172.21.15.59:0/1179153122 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f707410d360 msgr2=0x7f70741a28c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-12-11T18:03:49.194 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:49.190+0000 7f7079368700 1 --2- 172.21.15.59:0/1179153122 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f707410d360 0x7f70741a28c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:03:49.194 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:49.190+0000 7f7079368700 1 -- 172.21.15.59:0/1179153122 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7074109820 msgr2=0x7f7074113740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T18:03:49.194 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:49.190+0000 7f7079368700 1 --2- 172.21.15.59:0/1179153122 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7074109820 0x7f7074113740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:03:49.194 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:49.190+0000 7f7079368700 1 -- 172.21.15.59:0/1179153122 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f706c009d00 con 0x7f7074108e60 2023-12-11T18:03:49.194 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:49.190+0000 7f7079368700 1 --2- 172.21.15.59:0/1179153122 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7074108e60 0x7f70741131e0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f707000d950 tx=0x7f707000dd10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T18:03:49.195 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:49.190+0000 7f706a7fc700 1 -- 172.21.15.59:0/1179153122 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7070009b40 con 0x7f7074108e60 2023-12-11T18:03:49.195 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:49.190+0000 7f707b5cc700 1 -- 172.21.15.59:0/1179153122 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70741a3410 con 0x7f7074108e60 2023-12-11T18:03:49.195 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:49.190+0000 7f706a7fc700 1 -- 172.21.15.59:0/1179153122 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7070009ca0 con 0x7f7074108e60 2023-12-11T18:03:49.195 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:49.190+0000 7f706a7fc700 1 -- 172.21.15.59:0/1179153122 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f707000f620 con 0x7f7074108e60 2023-12-11T18:03:49.195 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:49.190+0000 7f707b5cc700 1 -- 172.21.15.59:0/1179153122 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70741aae30 con 0x7f7074108e60 2023-12-11T18:03:49.195 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:49.192+0000 7f707b5cc700 1 -- 172.21.15.59:0/1179153122 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f707406d9e0 con 0x7f7074108e60 2023-12-11T18:03:49.206 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:49.202+0000 7f706a7fc700 1 -- 172.21.15.59:0/1179153122 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 28) v1 ==== 136137+0+0 (secure 0 0 0) 0x7f7070010460 con 0x7f7074108e60 2023-12-11T18:03:49.207 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:49.203+0000 7f706a7fc700 1 --2- 172.21.15.59:0/1179153122 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7f70600a16b0 0x7f70600a3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T18:03:49.207 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:49.203+0000 7f706a7fc700 1 -- 172.21.15.59:0/1179153122 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 9803+0+0 (secure 0 0 0) 0x7f70700cd920 con 0x7f7074108e60 2023-12-11T18:03:49.207 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:49.203+0000 7f7078b67700 1 --2- 172.21.15.59:0/1179153122 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7f70600a16b0 0x7f70600a3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T18:03:49.209 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:49.206+0000 7f7078b67700 1 --2- 172.21.15.59:0/1179153122 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7f70600a16b0 0x7f70600a3b70 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f7064005950 tx=0x7f70640058e0 comp rx=0 tx=0).ready entity=mgr.15414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T18:03:49.221 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:49.218+0000 7f706a7fc700 1 -- 172.21.15.59:0/1179153122 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f7070096740 con 0x7f7074108e60 2023-12-11T18:03:49.626 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:49.619+0000 7f707b5cc700 1 -- 172.21.15.59:0/1179153122 --> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) v1 -- 0x7f7074119cf0 con 0x7f70600a16b0 2023-12-11T18:03:49.732 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:49.729+0000 7f706a7fc700 1 -- 172.21.15.59:0/1179153122 <== mgr.15414 v2:172.21.15.59:6800/4126604994 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+335 (secure 0 0 0) 0x7f7074119cf0 con 0x7f70600a16b0 2023-12-11T18:03:49.736 INFO:teuthology.orchestra.run.smithi059.stdout:{"fs": "cephfs", "subvol": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2023-12-11T00:00:00", "created": "2023-12-11T17:46:18", "first": "2023-12-11T17:47:00", "last": "2023-12-11T18:03:00", "last_pruned": "2023-12-11T18:03:01", "created_count": 15, "pruned_count": 9, "active": true} 2023-12-11T18:03:49.745 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:49.742+0000 7f707b5cc700 1 -- 172.21.15.59:0/1179153122 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7f70600a16b0 msgr2=0x7f70600a3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T18:03:49.745 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:49.742+0000 7f707b5cc700 1 --2- 172.21.15.59:0/1179153122 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7f70600a16b0 0x7f70600a3b70 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f7064005950 tx=0x7f70640058e0 comp rx=0 tx=0).stop 2023-12-11T18:03:49.745 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:49.742+0000 7f707b5cc700 1 -- 172.21.15.59:0/1179153122 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7074108e60 msgr2=0x7f70741131e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T18:03:49.745 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:49.742+0000 7f707b5cc700 1 --2- 172.21.15.59:0/1179153122 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7074108e60 0x7f70741131e0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f707000d950 tx=0x7f707000dd10 comp rx=0 tx=0).stop 2023-12-11T18:03:49.753 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:49.750+0000 7f707b5cc700 1 -- 172.21.15.59:0/1179153122 shutdown_connections 2023-12-11T18:03:49.753 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:49.750+0000 7f707b5cc700 1 --2- 172.21.15.59:0/1179153122 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7f70600a16b0 0x7f70600a3b70 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:03:49.753 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:49.750+0000 7f707b5cc700 1 --2- 172.21.15.59:0/1179153122 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f707410d360 0x7f70741a28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:03:49.753 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:49.750+0000 7f707b5cc700 1 --2- 172.21.15.59:0/1179153122 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7074108e60 0x7f70741131e0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:03:49.753 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:49.750+0000 7f707b5cc700 1 --2- 172.21.15.59:0/1179153122 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7074109820 0x7f7074113740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:03:49.753 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:49.750+0000 7f707b5cc700 1 -- 172.21.15.59:0/1179153122 >> 172.21.15.59:0/1179153122 conn(0x7f707406c5a0 msgr2=0x7f7074116840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T18:03:49.770 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:49.764+0000 7f707b5cc700 1 -- 172.21.15.59:0/1179153122 shutdown_connections 2023-12-11T18:03:49.774 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:49.769+0000 7f707b5cc700 1 -- 172.21.15.59:0/1179153122 wait complete. 2023-12-11T18:03:49.803 DEBUG:teuthology.orchestra.run.smithi059:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2023-12-11T18:03:50.120 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:50.117+0000 7ff594e5a700 1 -- 172.21.15.59:0/934770939 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff59010f1c0 msgr2=0x7ff59010f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T18:03:50.120 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:50.117+0000 7ff594e5a700 1 --2- 172.21.15.59:0/934770939 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff59010f1c0 0x7ff59010f640 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7ff5780098f0 tx=0x7ff578009c00 comp rx=0 tx=0).stop 2023-12-11T18:03:50.133 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:50.129+0000 7ff594e5a700 1 -- 172.21.15.59:0/934770939 shutdown_connections 2023-12-11T18:03:50.133 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:50.129+0000 7ff594e5a700 1 --2- 172.21.15.59:0/934770939 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff59010fb80 0x7ff5901085d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:03:50.133 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:50.129+0000 7ff594e5a700 1 --2- 172.21.15.59:0/934770939 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff59010d360 0x7ff59010d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:03:50.133 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:50.129+0000 7ff594e5a700 1 --2- 172.21.15.59:0/934770939 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff59010f1c0 0x7ff59010f640 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:03:50.133 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:50.129+0000 7ff594e5a700 1 -- 172.21.15.59:0/934770939 >> 172.21.15.59:0/934770939 conn(0x7ff59006c5a0 msgr2=0x7ff59006c9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T18:03:50.143 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:50.140+0000 7ff594e5a700 1 -- 172.21.15.59:0/934770939 shutdown_connections 2023-12-11T18:03:50.151 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:50.147+0000 7ff594e5a700 1 -- 172.21.15.59:0/934770939 wait complete. 2023-12-11T18:03:50.153 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:50.148+0000 7ff594e5a700 1 Processor -- start 2023-12-11T18:03:50.154 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:50.150+0000 7ff594e5a700 1 -- start start 2023-12-11T18:03:50.154 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:50.150+0000 7ff594e5a700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff59010d360 0x7ff590119330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T18:03:50.154 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:50.150+0000 7ff594e5a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff59010f1c0 0x7ff590119870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T18:03:50.154 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:50.151+0000 7ff594e5a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff59010fb80 0x7ff59011dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T18:03:50.154 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:50.151+0000 7ff594e5a700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff59011e1e0 con 0x7ff59010f1c0 2023-12-11T18:03:50.154 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:50.151+0000 7ff594e5a700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff59011e350 con 0x7ff59010d360 2023-12-11T18:03:50.154 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:50.151+0000 7ff594e5a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff59011e510 con 0x7ff59010fb80 2023-12-11T18:03:50.158 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:50.155+0000 7ff58e59c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff59010d360 0x7ff590119330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T18:03:50.158 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:50.155+0000 7ff58e59c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff59010d360 0x7ff590119330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.59:60050/0 (socket says 172.21.15.59:60050) 2023-12-11T18:03:50.158 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:50.155+0000 7ff58e59c700 1 -- 172.21.15.59:0/2689533220 learned_addr learned my addr 172.21.15.59:0/2689533220 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T18:03:50.170 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:50.164+0000 7ff58e59c700 1 -- 172.21.15.59:0/2689533220 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff59010fb80 msgr2=0x7ff59011dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T18:03:50.170 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:50.164+0000 7ff58e59c700 1 --2- 172.21.15.59:0/2689533220 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff59010fb80 0x7ff59011dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:03:50.170 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:50.164+0000 7ff58e59c700 1 -- 172.21.15.59:0/2689533220 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff59010f1c0 msgr2=0x7ff590119870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T18:03:50.170 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:50.164+0000 7ff58e59c700 1 --2- 172.21.15.59:0/2689533220 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff59010f1c0 0x7ff590119870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:03:50.170 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:50.164+0000 7ff58e59c700 1 -- 172.21.15.59:0/2689533220 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff578009520 con 0x7ff59010d360 2023-12-11T18:03:50.171 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:50.168+0000 7ff58e59c700 1 --2- 172.21.15.59:0/2689533220 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff59010d360 0x7ff590119330 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff580007fa0 tx=0x7ff58000d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T18:03:50.177 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:50.174+0000 7ff5877fe700 1 -- 172.21.15.59:0/2689533220 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff58000de60 con 0x7ff59010d360 2023-12-11T18:03:50.178 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:50.174+0000 7ff5877fe700 1 -- 172.21.15.59:0/2689533220 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff58000f040 con 0x7ff59010d360 2023-12-11T18:03:50.178 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:50.174+0000 7ff5877fe700 1 -- 172.21.15.59:0/2689533220 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff580014720 con 0x7ff59010d360 2023-12-11T18:03:50.178 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:50.174+0000 7ff594e5a700 1 -- 172.21.15.59:0/2689533220 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff59011e7f0 con 0x7ff59010d360 2023-12-11T18:03:50.178 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:50.174+0000 7ff594e5a700 1 -- 172.21.15.59:0/2689533220 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff590124060 con 0x7ff59010d360 2023-12-11T18:03:50.182 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:50.177+0000 7ff594e5a700 1 -- 172.21.15.59:0/2689533220 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff59011ea10 con 0x7ff59010d360 2023-12-11T18:03:50.183 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:50.179+0000 7ff5877fe700 1 -- 172.21.15.59:0/2689533220 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 28) v1 ==== 136137+0+0 (secure 0 0 0) 0x7ff580015a50 con 0x7ff59010d360 2023-12-11T18:03:50.183 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:50.179+0000 7ff5877fe700 1 --2- 172.21.15.59:0/2689533220 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7ff57c0a17c0 0x7ff57c0a3c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T18:03:50.183 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:50.180+0000 7ff5877fe700 1 -- 172.21.15.59:0/2689533220 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 9803+0+0 (secure 0 0 0) 0x7ff5800cd170 con 0x7ff59010d360 2023-12-11T18:03:50.184 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:50.181+0000 7ff58dd9b700 1 --2- 172.21.15.59:0/2689533220 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7ff57c0a17c0 0x7ff57c0a3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T18:03:50.185 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:50.182+0000 7ff58dd9b700 1 --2- 172.21.15.59:0/2689533220 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7ff57c0a17c0 0x7ff57c0a3c80 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff57800b2e0 tx=0x7ff5780058e0 comp rx=0 tx=0).ready entity=mgr.15414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T18:03:50.202 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:50.196+0000 7ff5877fe700 1 -- 172.21.15.59:0/2689533220 <== mon.2 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7ff580095f30 con 0x7ff59010d360 2023-12-11T18:03:50.613 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:50.604+0000 7ff594e5a700 1 -- 172.21.15.59:0/2689533220 --> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) v1 -- 0x7ff59011a5e0 con 0x7ff57c0a17c0 2023-12-11T18:03:50.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:03:50 smithi059 ceph-mon[98847]: pgmap v557: 129 pgs: 56 active+clean+snaptrim_wait, 18 active+clean+snaptrim, 55 active+clean; 1.8 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 638 KiB/s rd, 632 KiB/s wr, 86 op/s 2023-12-11T18:03:50.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:03:50 smithi059 ceph-mon[98847]: osdmap e118: 12 total, 12 up, 12 in 2023-12-11T18:03:50.715 INFO:teuthology.orchestra.run.smithi059.stdout:/ 1m 6m3h 2023-12-11T18:03:50.716 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:50.711+0000 7ff5877fe700 1 -- 172.21.15.59:0/2689533220 <== mgr.15414 v2:172.21.15.59:6800/4126604994 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+10 (secure 0 0 0) 0x7ff59011a5e0 con 0x7ff57c0a17c0 2023-12-11T18:03:50.722 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:50.718+0000 7ff594e5a700 1 -- 172.21.15.59:0/2689533220 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7ff57c0a17c0 msgr2=0x7ff57c0a3c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T18:03:50.722 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:50.718+0000 7ff594e5a700 1 --2- 172.21.15.59:0/2689533220 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7ff57c0a17c0 0x7ff57c0a3c80 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff57800b2e0 tx=0x7ff5780058e0 comp rx=0 tx=0).stop 2023-12-11T18:03:50.722 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:50.718+0000 7ff594e5a700 1 -- 172.21.15.59:0/2689533220 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff59010d360 msgr2=0x7ff590119330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T18:03:50.722 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:50.718+0000 7ff594e5a700 1 --2- 172.21.15.59:0/2689533220 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff59010d360 0x7ff590119330 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff580007fa0 tx=0x7ff58000d3b0 comp rx=0 tx=0).stop 2023-12-11T18:03:50.728 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:50.725+0000 7ff594e5a700 1 -- 172.21.15.59:0/2689533220 shutdown_connections 2023-12-11T18:03:50.728 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:50.725+0000 7ff594e5a700 1 --2- 172.21.15.59:0/2689533220 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7ff57c0a17c0 0x7ff57c0a3c80 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:03:50.728 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:50.725+0000 7ff594e5a700 1 --2- 172.21.15.59:0/2689533220 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff59010fb80 0x7ff59011dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:03:50.728 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:50.725+0000 7ff594e5a700 1 --2- 172.21.15.59:0/2689533220 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff59010d360 0x7ff590119330 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:03:50.728 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:50.725+0000 7ff594e5a700 1 --2- 172.21.15.59:0/2689533220 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff59010f1c0 0x7ff590119870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:03:50.728 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:50.725+0000 7ff594e5a700 1 -- 172.21.15.59:0/2689533220 >> 172.21.15.59:0/2689533220 conn(0x7ff59006c5a0 msgr2=0x7ff590071e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T18:03:50.736 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:50.733+0000 7ff594e5a700 1 -- 172.21.15.59:0/2689533220 shutdown_connections 2023-12-11T18:03:50.741 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:50.735+0000 7ff594e5a700 1 -- 172.21.15.59:0/2689533220 wait complete. 2023-12-11T18:03:50.766 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2023-12-11T18:03:50.778 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-11T18:03:50.782 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:03:50 smithi115 ceph-mon[102748]: pgmap v557: 129 pgs: 56 active+clean+snaptrim_wait, 18 active+clean+snaptrim, 55 active+clean; 1.8 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 638 KiB/s rd, 632 KiB/s wr, 86 op/s 2023-12-11T18:03:50.782 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:03:50 smithi115 ceph-mon[102748]: osdmap e118: 12 total, 12 up, 12 in 2023-12-11T18:03:51.030 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:51.027+0000 7f9fec9d2700 1 -- 172.21.15.59:0/1781065954 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9fe81085d0 msgr2=0x7f9fe8108a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T18:03:51.031 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:51.027+0000 7f9fec9d2700 1 --2- 172.21.15.59:0/1781065954 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9fe81085d0 0x7f9fe8108a50 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f9fd80098f0 tx=0x7f9fd8009c00 comp rx=0 tx=0).stop 2023-12-11T18:03:51.043 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:51.034+0000 7f9fec9d2700 1 -- 172.21.15.59:0/1781065954 shutdown_connections 2023-12-11T18:03:51.043 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:51.034+0000 7f9fec9d2700 1 --2- 172.21.15.59:0/1781065954 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9fe8109180 0x7f9fe811cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:03:51.043 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:51.034+0000 7f9fec9d2700 1 --2- 172.21.15.59:0/1781065954 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9fe810fbd0 0x7f9fe810ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:03:51.043 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:51.034+0000 7f9fec9d2700 1 --2- 172.21.15.59:0/1781065954 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9fe81085d0 0x7f9fe8108a50 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:03:51.043 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:51.034+0000 7f9fec9d2700 1 -- 172.21.15.59:0/1781065954 >> 172.21.15.59:0/1781065954 conn(0x7f9fe806c540 msgr2=0x7f9fe806c950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T18:03:51.047 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:51.043+0000 7f9fec9d2700 1 -- 172.21.15.59:0/1781065954 shutdown_connections 2023-12-11T18:03:51.047 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:51.044+0000 7f9fec9d2700 1 -- 172.21.15.59:0/1781065954 wait complete. 2023-12-11T18:03:51.048 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:51.045+0000 7f9fec9d2700 1 Processor -- start 2023-12-11T18:03:51.048 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:51.045+0000 7f9fec9d2700 1 -- start start 2023-12-11T18:03:51.048 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:51.045+0000 7f9fec9d2700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9fe81085d0 0x7f9fe81ac500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T18:03:51.048 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:51.045+0000 7f9fec9d2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9fe8109180 0x7f9fe81aca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T18:03:51.048 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:51.045+0000 7f9fec9d2700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9fe810fbd0 0x7f9fe81a6630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T18:03:51.049 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:51.045+0000 7f9fec9d2700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe81a6c90 con 0x7f9fe81085d0 2023-12-11T18:03:51.049 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:51.045+0000 7f9fec9d2700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe81a6e00 con 0x7f9fe810fbd0 2023-12-11T18:03:51.049 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:51.045+0000 7f9fec9d2700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe81a6fc0 con 0x7f9fe8109180 2023-12-11T18:03:51.055 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:51.047+0000 7f9fe659c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9fe81085d0 0x7f9fe81ac500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T18:03:51.055 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:51.047+0000 7f9fe659c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9fe81085d0 0x7f9fe81ac500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35334/0 (socket says 172.21.15.59:35334) 2023-12-11T18:03:51.056 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:51.047+0000 7f9fe659c700 1 -- 172.21.15.59:0/1050425088 learned_addr learned my addr 172.21.15.59:0/1050425088 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T18:03:51.056 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:51.047+0000 7f9fe6d9d700 1 --2- 172.21.15.59:0/1050425088 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9fe810fbd0 0x7f9fe81a6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T18:03:51.056 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:51.047+0000 7f9fe659c700 1 -- 172.21.15.59:0/1050425088 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9fe8109180 msgr2=0x7f9fe81aca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T18:03:51.056 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:51.047+0000 7f9fe659c700 1 --2- 172.21.15.59:0/1050425088 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9fe8109180 0x7f9fe81aca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:03:51.056 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:51.047+0000 7f9fe659c700 1 -- 172.21.15.59:0/1050425088 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9fe810fbd0 msgr2=0x7f9fe81a6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T18:03:51.056 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:51.047+0000 7f9fe659c700 1 --2- 172.21.15.59:0/1050425088 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9fe810fbd0 0x7f9fe81a6630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:03:51.056 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:51.047+0000 7f9fe659c700 1 -- 172.21.15.59:0/1050425088 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fd8009520 con 0x7f9fe81085d0 2023-12-11T18:03:51.056 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:51.047+0000 7f9fe6d9d700 1 --2- 172.21.15.59:0/1050425088 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9fe810fbd0 0x7f9fe81a6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T18:03:51.056 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:51.048+0000 7f9fe659c700 1 --2- 172.21.15.59:0/1050425088 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9fe81085d0 0x7f9fe81ac500 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc00b550 tx=0x7f9fdc00b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T18:03:51.056 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:51.048+0000 7f9fd77fe700 1 -- 172.21.15.59:0/1050425088 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fdc011830 con 0x7f9fe81085d0 2023-12-11T18:03:51.056 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:51.048+0000 7f9fec9d2700 1 -- 172.21.15.59:0/1050425088 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fe81a72a0 con 0x7f9fe81085d0 2023-12-11T18:03:51.056 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:51.048+0000 7f9fd77fe700 1 -- 172.21.15.59:0/1050425088 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9fdc011e50 con 0x7f9fe81085d0 2023-12-11T18:03:51.056 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:51.048+0000 7f9fd77fe700 1 -- 172.21.15.59:0/1050425088 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fdc00f2e0 con 0x7f9fe81085d0 2023-12-11T18:03:51.056 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:51.048+0000 7f9fec9d2700 1 -- 172.21.15.59:0/1050425088 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fe81107f0 con 0x7f9fe81085d0 2023-12-11T18:03:51.057 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:51.052+0000 7f9fd77fe700 1 -- 172.21.15.59:0/1050425088 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 28) v1 ==== 136137+0+0 (secure 0 0 0) 0x7f9fdc011990 con 0x7f9fe81085d0 2023-12-11T18:03:51.057 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:51.053+0000 7f9fd77fe700 1 --2- 172.21.15.59:0/1050425088 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7f9fd00b26c0 0x7f9fd00b4b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T18:03:51.057 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:51.053+0000 7f9fd77fe700 1 -- 172.21.15.59:0/1050425088 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 9727+0+0 (secure 0 0 0) 0x7f9fdc0ccf70 con 0x7f9fe81085d0 2023-12-11T18:03:51.057 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:51.054+0000 7f9fec9d2700 1 -- 172.21.15.59:0/1050425088 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fe810ce30 con 0x7f9fe81085d0 2023-12-11T18:03:51.059 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:51.056+0000 7f9fe5d9b700 1 --2- 172.21.15.59:0/1050425088 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7f9fd00b26c0 0x7f9fd00b4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T18:03:51.060 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:51.057+0000 7f9fe5d9b700 1 --2- 172.21.15.59:0/1050425088 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7f9fd00b26c0 0x7f9fd00b4b80 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9fd8009810 tx=0x7f9fd80058e0 comp rx=0 tx=0).ready entity=mgr.15414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T18:03:51.068 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:51.065+0000 7f9fd77fe700 1 -- 172.21.15.59:0/1050425088 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f9fdc095ef0 con 0x7f9fe81085d0 2023-12-11T18:03:51.439 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:51.435+0000 7f9fec9d2700 1 -- 172.21.15.59:0/1050425088 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9fe8068260 con 0x7f9fe81085d0 2023-12-11T18:03:51.441 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:51.438+0000 7f9fd77fe700 1 -- 172.21.15.59:0/1050425088 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v118) v1 ==== 74+0+25734 (secure 0 0 0) 0x7f9fdc015030 con 0x7f9fe81085d0 2023-12-11T18:03:51.451 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-12-11T18:03:51.451 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":118,"fsid":"d0a07eaa-984b-11ee-95a3-87774f69a715","created":"2023-12-11T17:37:14.526539+0000","modified":"2023-12-11T18:03:50.211469+0000","last_up_change":"2023-12-11T17:58:28.975845+0000","last_in_change":"2023-12-11T17:43:21.014888+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":30,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-11T17:40:26.596029+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-11T17:44:18.586290+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-11T17:44:19.432952+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":25,"snap_epoch":117,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-11T17:44:23.097308+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":25,"snap_epoch":117,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3c0b3218-4b2b-40f8-8bef-ed8cd5633607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":1040759398},{"type":"v1","addr":"172.21.15.59:6803","nonce":1040759398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":1040759398},{"type":"v1","addr":"172.21.15.59:6805","nonce":1040759398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":1040759398},{"type":"v1","addr":"172.21.15.59:6809","nonce":1040759398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":1040759398},{"type":"v1","addr":"172.21.15.59:6807","nonce":1040759398}]},"public_addr":"172.21.15.59:6803/1040759398","cluster_addr":"172.21.15.59:6805/1040759398","heartbeat_back_addr":"172.21.15.59:6809/1040759398","heartbeat_front_addr":"172.21.15.59:6807/1040759398","state":["exists","up"]},{"osd":1,"uuid":"2ead7578-765b-4f5a-a72c-863731ed69e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":2776354245},{"type":"v1","addr":"172.21.15.115:6801","nonce":2776354245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":2776354245},{"type":"v1","addr":"172.21.15.115:6803","nonce":2776354245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":2776354245},{"type":"v1","addr":"172.21.15.115:6807","nonce":2776354245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":2776354245},{"type":"v1","addr":"172.21.15.115:6805","nonce":2776354245}]},"public_addr":"172.21.15.115:6801/2776354245","cluster_addr":"172.21.15.115:6803/2776354245","heartbeat_back_addr":"172.21.15.115:6807/2776354245","heartbeat_front_addr":"172.21.15.115:6805/2776354245","state":["exists","up"]},{"osd":2,"uuid":"8cb1d7ec-53cc-4672-b0b6-70d73797c44f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":18,"last_clean_end":97,"up_from":102,"up_thru":104,"down_at":98,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1449149834},{"type":"v1","addr":"172.21.15.181:6810","nonce":1449149834}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":1449149834},{"type":"v1","addr":"172.21.15.181:6814","nonce":1449149834}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":1449149834},{"type":"v1","addr":"172.21.15.181:6823","nonce":1449149834}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1449149834},{"type":"v1","addr":"172.21.15.181:6818","nonce":1449149834}]},"public_addr":"172.21.15.181:6810/1449149834","cluster_addr":"172.21.15.181:6814/1449149834","heartbeat_back_addr":"172.21.15.181:6823/1449149834","heartbeat_front_addr":"172.21.15.181:6818/1449149834","state":["exists","up"]},{"osd":3,"uuid":"5d64d9af-831b-4d0a-8ace-a452169351d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":219909995},{"type":"v1","addr":"172.21.15.59:6811","nonce":219909995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":219909995},{"type":"v1","addr":"172.21.15.59:6813","nonce":219909995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":219909995},{"type":"v1","addr":"172.21.15.59:6817","nonce":219909995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":219909995},{"type":"v1","addr":"172.21.15.59:6815","nonce":219909995}]},"public_addr":"172.21.15.59:6811/219909995","cluster_addr":"172.21.15.59:6813/219909995","heartbeat_back_addr":"172.21.15.59:6817/219909995","heartbeat_front_addr":"172.21.15.59:6815/219909995","state":["exists","up"]},{"osd":4,"uuid":"ef8fad28-1db5-4cba-b5f2-10e54e8e1549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":1021473826},{"type":"v1","addr":"172.21.15.115:6809","nonce":1021473826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":1021473826},{"type":"v1","addr":"172.21.15.115:6811","nonce":1021473826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":1021473826},{"type":"v1","addr":"172.21.15.115:6815","nonce":1021473826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":1021473826},{"type":"v1","addr":"172.21.15.115:6813","nonce":1021473826}]},"public_addr":"172.21.15.115:6809/1021473826","cluster_addr":"172.21.15.115:6811/1021473826","heartbeat_back_addr":"172.21.15.115:6815/1021473826","heartbeat_front_addr":"172.21.15.115:6813/1021473826","state":["exists","up"]},{"osd":5,"uuid":"4233a512-9e8f-4db8-aed2-6c3af706e589","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":37,"last_clean_end":97,"up_from":104,"up_thru":104,"down_at":98,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6809","nonce":3565629391},{"type":"v1","addr":"172.21.15.181:6811","nonce":3565629391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6813","nonce":3565629391},{"type":"v1","addr":"172.21.15.181:6815","nonce":3565629391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":3565629391},{"type":"v1","addr":"172.21.15.181:6822","nonce":3565629391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":3565629391},{"type":"v1","addr":"172.21.15.181:6819","nonce":3565629391}]},"public_addr":"172.21.15.181:6811/3565629391","cluster_addr":"172.21.15.181:6815/3565629391","heartbeat_back_addr":"172.21.15.181:6822/3565629391","heartbeat_front_addr":"172.21.15.181:6819/3565629391","state":["exists","up"]},{"osd":6,"uuid":"8faab332-016e-4c77-8370-8bef95fc6f34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":815374853},{"type":"v1","addr":"172.21.15.59:6819","nonce":815374853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":815374853},{"type":"v1","addr":"172.21.15.59:6821","nonce":815374853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":815374853},{"type":"v1","addr":"172.21.15.59:6825","nonce":815374853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":815374853},{"type":"v1","addr":"172.21.15.59:6823","nonce":815374853}]},"public_addr":"172.21.15.59:6819/815374853","cluster_addr":"172.21.15.59:6821/815374853","heartbeat_back_addr":"172.21.15.59:6825/815374853","heartbeat_front_addr":"172.21.15.59:6823/815374853","state":["exists","up"]},{"osd":7,"uuid":"f20f470a-0d40-4ed3-8e0e-fd58675cf143","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":1314971870},{"type":"v1","addr":"172.21.15.115:6817","nonce":1314971870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":1314971870},{"type":"v1","addr":"172.21.15.115:6819","nonce":1314971870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":1314971870},{"type":"v1","addr":"172.21.15.115:6823","nonce":1314971870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":1314971870},{"type":"v1","addr":"172.21.15.115:6821","nonce":1314971870}]},"public_addr":"172.21.15.115:6817/1314971870","cluster_addr":"172.21.15.115:6819/1314971870","heartbeat_back_addr":"172.21.15.115:6823/1314971870","heartbeat_front_addr":"172.21.15.115:6821/1314971870","state":["exists","up"]},{"osd":8,"uuid":"eb657d32-1a95-4fd7-a293-d0a970837fc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":57,"last_clean_end":97,"up_from":103,"up_thru":104,"down_at":98,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":3914011605},{"type":"v1","addr":"172.21.15.181:6833","nonce":3914011605}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3914011605},{"type":"v1","addr":"172.21.15.181:6835","nonce":3914011605}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":3914011605},{"type":"v1","addr":"172.21.15.181:6839","nonce":3914011605}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":3914011605},{"type":"v1","addr":"172.21.15.181:6837","nonce":3914011605}]},"public_addr":"172.21.15.181:6833/3914011605","cluster_addr":"172.21.15.181:6835/3914011605","heartbeat_back_addr":"172.21.15.181:6839/3914011605","heartbeat_front_addr":"172.21.15.181:6837/3914011605","state":["exists","up"]},{"osd":9,"uuid":"8f7717d4-b6f2-4ec1-9b58-62feb0c9a146","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":1163095677},{"type":"v1","addr":"172.21.15.59:6827","nonce":1163095677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":1163095677},{"type":"v1","addr":"172.21.15.59:6829","nonce":1163095677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":1163095677},{"type":"v1","addr":"172.21.15.59:6833","nonce":1163095677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":1163095677},{"type":"v1","addr":"172.21.15.59:6831","nonce":1163095677}]},"public_addr":"172.21.15.59:6827/1163095677","cluster_addr":"172.21.15.59:6829/1163095677","heartbeat_back_addr":"172.21.15.59:6833/1163095677","heartbeat_front_addr":"172.21.15.59:6831/1163095677","state":["exists","up"]},{"osd":10,"uuid":"b75c179a-875d-43bc-b4c1-e8eb80fb64fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":2199610269},{"type":"v1","addr":"172.21.15.115:6825","nonce":2199610269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":2199610269},{"type":"v1","addr":"172.21.15.115:6827","nonce":2199610269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":2199610269},{"type":"v1","addr":"172.21.15.115:6831","nonce":2199610269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":2199610269},{"type":"v1","addr":"172.21.15.115:6829","nonce":2199610269}]},"public_addr":"172.21.15.115:6825/2199610269","cluster_addr":"172.21.15.115:6827/2199610269","heartbeat_back_addr":"172.21.15.115:6831/2199610269","heartbeat_front_addr":"172.21.15.115:6829/2199610269","state":["exists","up"]},{"osd":11,"uuid":"5e141dfe-8a00-4bbc-8204-2dc9be24e18d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":77,"last_clean_end":97,"up_from":102,"up_thru":104,"down_at":98,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":540621004},{"type":"v1","addr":"172.21.15.181:6825","nonce":540621004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":540621004},{"type":"v1","addr":"172.21.15.181:6827","nonce":540621004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":540621004},{"type":"v1","addr":"172.21.15.181:6831","nonce":540621004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":540621004},{"type":"v1","addr":"172.21.15.181:6829","nonce":540621004}]},"public_addr":"172.21.15.181:6825/540621004","cluster_addr":"172.21.15.181:6827/540621004","heartbeat_back_addr":"172.21.15.181:6831/540621004","heartbeat_front_addr":"172.21.15.181:6829/540621004","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:39:42.687084+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:40:03.903448+0000","dead_epoch":0},{"osd":2,"down_stamp":"2023-12-11T17:56:22.121381+0000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:40:24.899908+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:40:46.600251+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:41:07.690037+0000","dead_epoch":0},{"osd":5,"down_stamp":"2023-12-11T17:56:22.121381+0000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:41:28.488855+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:41:49.459274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:42:10.863881+0000","dead_epoch":0},{"osd":8,"down_stamp":"2023-12-11T17:56:22.121381+0000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:42:32.248391+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:42:54.112265+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:43:15.634872+0000","dead_epoch":0},{"osd":11,"down_stamp":"2023-12-11T17:56:22.121381+0000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:43:36.922534+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":7}]},{"pgid":"3.28","mappings":[{"from":5,"to":6},{"from":3,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.37","mappings":[{"from":1,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"4.6","mappings":[{"from":9,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:6839/2214384806":"2023-12-12T17:57:14.937455+0000","172.21.15.59:6801/3627728353":"2023-12-12T17:46:17.753068+0000","172.21.15.59:0/2192742819":"2023-12-12T17:37:44.773925+0000","172.21.15.181:6838/2214384806":"2023-12-12T17:57:14.937455+0000","172.21.15.59:0/2859415380":"2023-12-12T17:38:14.037922+0000","172.21.15.59:0/644616907":"2023-12-12T17:46:17.753068+0000","172.21.15.59:0/3853462899":"2023-12-12T17:37:44.773925+0000","172.21.15.59:0/4159306605":"2023-12-12T17:46:17.753068+0000","172.21.15.59:6800/3627728353":"2023-12-12T17:46:17.753068+0000","172.21.15.59:0/3570767650":"2023-12-12T17:37:44.773925+0000","172.21.15.59:6800/1672530462":"2023-12-12T17:37:44.773925+0000","172.21.15.59:6801/1259916482":"2023-12-12T17:38:14.037922+0000","172.21.15.59:0/1053944547":"2023-12-12T17:38:14.037922+0000","172.21.15.59:6801/1672530462":"2023-12-12T17:37:44.773925+0000","172.21.15.59:0/2683787475":"2023-12-12T17:46:17.753068+0000","172.21.15.59:0/17457774":"2023-12-12T17:38:14.037922+0000","172.21.15.59:6800/1259916482":"2023-12-12T17:38:14.037922+0000","172.21.15.59:0/2112420116":"2023-12-12T17:46:17.753068+0000","172.21.15.59:0/1373005911":"2023-12-12T17:46:17.753068+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":12,"length":1},{"begin":21,"length":1},{"begin":23,"length":1},{"begin":25,"length":1}]},{"pool":4,"snaps":[{"begin":10,"length":1},{"begin":12,"length":1},{"begin":21,"length":1},{"begin":23,"length":1},{"begin":25,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":10,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-11T18:03:51.454 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:51.451+0000 7f9fec9d2700 1 -- 172.21.15.59:0/1050425088 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7f9fd00b26c0 msgr2=0x7f9fd00b4b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T18:03:51.454 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:51.451+0000 7f9fec9d2700 1 --2- 172.21.15.59:0/1050425088 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7f9fd00b26c0 0x7f9fd00b4b80 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9fd8009810 tx=0x7f9fd80058e0 comp rx=0 tx=0).stop 2023-12-11T18:03:51.454 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:51.451+0000 7f9fec9d2700 1 -- 172.21.15.59:0/1050425088 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9fe81085d0 msgr2=0x7f9fe81ac500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T18:03:51.454 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:51.451+0000 7f9fec9d2700 1 --2- 172.21.15.59:0/1050425088 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9fe81085d0 0x7f9fe81ac500 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc00b550 tx=0x7f9fdc00b860 comp rx=0 tx=0).stop 2023-12-11T18:03:51.454 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:51.451+0000 7f9fec9d2700 1 -- 172.21.15.59:0/1050425088 shutdown_connections 2023-12-11T18:03:51.454 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:51.451+0000 7f9fec9d2700 1 --2- 172.21.15.59:0/1050425088 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7f9fd00b26c0 0x7f9fd00b4b80 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:03:51.454 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:51.451+0000 7f9fec9d2700 1 --2- 172.21.15.59:0/1050425088 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9fe810fbd0 0x7f9fe81a6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:03:51.455 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:51.451+0000 7f9fec9d2700 1 --2- 172.21.15.59:0/1050425088 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9fe81085d0 0x7f9fe81ac500 secure :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc00b550 tx=0x7f9fdc00b860 comp rx=0 tx=0).stop 2023-12-11T18:03:51.455 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:51.451+0000 7f9fec9d2700 1 --2- 172.21.15.59:0/1050425088 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9fe8109180 0x7f9fe81aca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:03:51.455 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:51.451+0000 7f9fec9d2700 1 -- 172.21.15.59:0/1050425088 >> 172.21.15.59:0/1050425088 conn(0x7f9fe806c540 msgr2=0x7f9fe80700d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T18:03:51.465 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:51.455+0000 7f9fec9d2700 1 -- 172.21.15.59:0/1050425088 shutdown_connections 2023-12-11T18:03:51.472 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:51.461+0000 7f9fec9d2700 1 -- 172.21.15.59:0/1050425088 wait complete. 2023-12-11T18:03:51.510 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-11T17:40:26.596029+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-11T17:44:18.586290+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-11T17:44:19.432952+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '117', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 25, 'snap_epoch': 117, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-12-11T17:44:23.097308+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '117', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 25, 'snap_epoch': 117, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-12-11T18:03:51.511 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-11T18:03:51.607 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:03:51 smithi059 ceph-mon[98847]: from='client.34126 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T18:03:51.786 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:03:51 smithi115 ceph-mon[102748]: from='client.34126 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T18:03:51.803 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:51.796+0000 7f622341e700 1 -- 172.21.15.59:0/4278089807 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f621c114720 msgr2=0x7f621c116b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T18:03:51.803 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:51.796+0000 7f622341e700 1 --2- 172.21.15.59:0/4278089807 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f621c114720 0x7f621c116b10 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f621400b0d0 tx=0x7f621400b3e0 comp rx=0 tx=0).stop 2023-12-11T18:03:51.804 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:51.801+0000 7f622341e700 1 -- 172.21.15.59:0/4278089807 shutdown_connections 2023-12-11T18:03:51.804 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:51.801+0000 7f622341e700 1 --2- 172.21.15.59:0/4278089807 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f621c114720 0x7f621c116b10 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:03:51.805 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:51.801+0000 7f622341e700 1 --2- 172.21.15.59:0/4278089807 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f621c108700 0x7f621c108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:03:51.805 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:51.801+0000 7f622341e700 1 --2- 172.21.15.59:0/4278089807 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f621c109020 0x7f621c1141e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:03:51.805 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:51.801+0000 7f622341e700 1 -- 172.21.15.59:0/4278089807 >> 172.21.15.59:0/4278089807 conn(0x7f621c06c610 msgr2=0x7f621c06ca20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T18:03:51.805 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:51.802+0000 7f622341e700 1 -- 172.21.15.59:0/4278089807 shutdown_connections 2023-12-11T18:03:51.808 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:51.805+0000 7f622341e700 1 -- 172.21.15.59:0/4278089807 wait complete. 2023-12-11T18:03:51.808 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:51.805+0000 7f622341e700 1 Processor -- start 2023-12-11T18:03:51.809 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:51.806+0000 7f622341e700 1 -- start start 2023-12-11T18:03:51.809 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:51.806+0000 7f622341e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f621c108700 0x7f621c1a0340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T18:03:51.809 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:51.806+0000 7f622341e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f621c109020 0x7f621c1a0880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T18:03:51.809 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:51.806+0000 7f622341e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f621c114720 0x7f621c1a4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T18:03:51.809 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:51.806+0000 7f622341e700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f621c1a5210 con 0x7f621c108700 2023-12-11T18:03:51.809 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:51.806+0000 7f622341e700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f621c1a5380 con 0x7f621c109020 2023-12-11T18:03:51.809 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:51.806+0000 7f622341e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f621c1a5540 con 0x7f621c114720 2023-12-11T18:03:51.810 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:51.806+0000 7f62211ba700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f621c108700 0x7f621c1a0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T18:03:51.810 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:51.806+0000 7f62211ba700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f621c108700 0x7f621c1a0340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35348/0 (socket says 172.21.15.59:35348) 2023-12-11T18:03:51.811 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:51.806+0000 7f62211ba700 1 -- 172.21.15.59:0/4145198894 learned_addr learned my addr 172.21.15.59:0/4145198894 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T18:03:51.811 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:51.807+0000 7f62209b9700 1 --2- 172.21.15.59:0/4145198894 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f621c109020 0x7f621c1a0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T18:03:51.811 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:51.807+0000 7f62211ba700 1 -- 172.21.15.59:0/4145198894 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f621c114720 msgr2=0x7f621c1a4bd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-11T18:03:51.811 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:51.807+0000 7f62211ba700 1 --2- 172.21.15.59:0/4145198894 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f621c114720 0x7f621c1a4bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:03:51.811 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:51.807+0000 7f62211ba700 1 -- 172.21.15.59:0/4145198894 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f621c109020 msgr2=0x7f621c1a0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T18:03:51.811 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:51.807+0000 7f62211ba700 1 --2- 172.21.15.59:0/4145198894 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f621c109020 0x7f621c1a0880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:03:51.811 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:51.807+0000 7f62211ba700 1 -- 172.21.15.59:0/4145198894 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6214009d00 con 0x7f621c108700 2023-12-11T18:03:51.811 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:51.807+0000 7f62211ba700 1 --2- 172.21.15.59:0/4145198894 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f621c108700 0x7f621c1a0340 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f620c00b530 tx=0x7f620c00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T18:03:51.811 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:51.807+0000 7f62127fc700 1 -- 172.21.15.59:0/4145198894 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f620c011830 con 0x7f621c108700 2023-12-11T18:03:51.812 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:51.808+0000 7f622341e700 1 -- 172.21.15.59:0/4145198894 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f621c1a5820 con 0x7f621c108700 2023-12-11T18:03:51.812 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:51.808+0000 7f62127fc700 1 -- 172.21.15.59:0/4145198894 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f620c011e50 con 0x7f621c108700 2023-12-11T18:03:51.812 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:51.808+0000 7f62127fc700 1 -- 172.21.15.59:0/4145198894 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f620c00f2e0 con 0x7f621c108700 2023-12-11T18:03:51.812 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:51.808+0000 7f622341e700 1 -- 172.21.15.59:0/4145198894 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f621c1ad060 con 0x7f621c108700 2023-12-11T18:03:51.814 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:51.811+0000 7f62127fc700 1 -- 172.21.15.59:0/4145198894 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 28) v1 ==== 136137+0+0 (secure 0 0 0) 0x7f620c010380 con 0x7f621c108700 2023-12-11T18:03:51.815 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:51.811+0000 7f62127fc700 1 --2- 172.21.15.59:0/4145198894 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7f62080a1670 0x7f62080a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T18:03:51.815 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:51.812+0000 7f622341e700 1 -- 172.21.15.59:0/4145198894 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f621c10f930 con 0x7f621c108700 2023-12-11T18:03:51.815 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:51.812+0000 7f62209b9700 1 --2- 172.21.15.59:0/4145198894 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7f62080a1670 0x7f62080a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T18:03:51.815 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:51.812+0000 7f62127fc700 1 -- 172.21.15.59:0/4145198894 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 9727+0+0 (secure 0 0 0) 0x7f620c0cf170 con 0x7f621c108700 2023-12-11T18:03:51.815 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:51.812+0000 7f62209b9700 1 --2- 172.21.15.59:0/4145198894 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7f62080a1670 0x7f62080a3b30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f621c1a1840 tx=0x7f6218009190 comp rx=0 tx=0).ready entity=mgr.15414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T18:03:51.826 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:51.823+0000 7f62127fc700 1 -- 172.21.15.59:0/4145198894 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f620c0d0060 con 0x7f621c108700 2023-12-11T18:03:52.180 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:52.176+0000 7f622341e700 1 -- 172.21.15.59:0/4145198894 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f621c068340 con 0x7f621c108700 2023-12-11T18:03:52.180 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:52.177+0000 7f62127fc700 1 -- 172.21.15.59:0/4145198894 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v118) v1 ==== 93+0+10 (secure 0 0 0) 0x7f620c015070 con 0x7f621c108700 2023-12-11T18:03:52.180 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2023-12-11T18:03:52.189 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:52.185+0000 7f622341e700 1 -- 172.21.15.59:0/4145198894 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7f62080a1670 msgr2=0x7f62080a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T18:03:52.189 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:52.185+0000 7f622341e700 1 --2- 172.21.15.59:0/4145198894 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7f62080a1670 0x7f62080a3b30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f621c1a1840 tx=0x7f6218009190 comp rx=0 tx=0).stop 2023-12-11T18:03:52.189 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:52.185+0000 7f622341e700 1 -- 172.21.15.59:0/4145198894 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f621c108700 msgr2=0x7f621c1a0340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T18:03:52.190 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:52.185+0000 7f622341e700 1 --2- 172.21.15.59:0/4145198894 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f621c108700 0x7f621c1a0340 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f620c00b530 tx=0x7f620c00b840 comp rx=0 tx=0).stop 2023-12-11T18:03:52.194 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:52.190+0000 7f622341e700 1 -- 172.21.15.59:0/4145198894 shutdown_connections 2023-12-11T18:03:52.194 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:52.190+0000 7f622341e700 1 --2- 172.21.15.59:0/4145198894 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7f62080a1670 0x7f62080a3b30 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:03:52.194 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:52.190+0000 7f622341e700 1 --2- 172.21.15.59:0/4145198894 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f621c114720 0x7f621c1a4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:03:52.194 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:52.190+0000 7f622341e700 1 --2- 172.21.15.59:0/4145198894 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f621c108700 0x7f621c1a0340 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:03:52.195 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:52.190+0000 7f622341e700 1 --2- 172.21.15.59:0/4145198894 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f621c109020 0x7f621c1a0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:03:52.195 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:52.190+0000 7f622341e700 1 -- 172.21.15.59:0/4145198894 >> 172.21.15.59:0/4145198894 conn(0x7f621c06c610 msgr2=0x7f621c070200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T18:03:52.196 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:52.192+0000 7f622341e700 1 -- 172.21.15.59:0/4145198894 shutdown_connections 2023-12-11T18:03:52.196 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:52.193+0000 7f622341e700 1 -- 172.21.15.59:0/4145198894 wait complete. 2023-12-11T18:03:52.232 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-11T18:03:52.512 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:52.508+0000 7f519ee1c700 1 -- 172.21.15.59:0/866690042 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f519811adb0 msgr2=0x7f519811d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T18:03:52.512 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:52.508+0000 7f519ee1c700 1 --2- 172.21.15.59:0/866690042 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f519811adb0 0x7f519811d1a0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f51880099c0 tx=0x7f5188009cd0 comp rx=0 tx=0).stop 2023-12-11T18:03:52.512 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:52.509+0000 7f519ee1c700 1 -- 172.21.15.59:0/866690042 shutdown_connections 2023-12-11T18:03:52.512 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:52.509+0000 7f519ee1c700 1 --2- 172.21.15.59:0/866690042 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f519811adb0 0x7f519811d1a0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:03:52.512 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:52.509+0000 7f519ee1c700 1 --2- 172.21.15.59:0/866690042 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f519810fd60 0x7f51981085d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:03:52.512 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:52.509+0000 7f519ee1c700 1 --2- 172.21.15.59:0/866690042 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5198108b10 0x7f5198108f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:03:52.513 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:52.509+0000 7f519ee1c700 1 -- 172.21.15.59:0/866690042 >> 172.21.15.59:0/866690042 conn(0x7f519806c610 msgr2=0x7f519806ca20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T18:03:52.514 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:52.511+0000 7f519ee1c700 1 -- 172.21.15.59:0/866690042 shutdown_connections 2023-12-11T18:03:52.517 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:52.514+0000 7f519ee1c700 1 -- 172.21.15.59:0/866690042 wait complete. 2023-12-11T18:03:52.523 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:52.515+0000 7f519ee1c700 1 Processor -- start 2023-12-11T18:03:52.523 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:52.515+0000 7f519ee1c700 1 -- start start 2023-12-11T18:03:52.523 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:52.515+0000 7f519ee1c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5198108b10 0x7f51981ac550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T18:03:52.523 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:52.515+0000 7f519ee1c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f519810fd60 0x7f51981aca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T18:03:52.523 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:52.515+0000 7f519ee1c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f519811adb0 0x7f51981a6680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T18:03:52.523 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:52.515+0000 7f519ee1c700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51981a6ce0 con 0x7f519811adb0 2023-12-11T18:03:52.523 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:52.515+0000 7f519ee1c700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51981a6e50 con 0x7f519810fd60 2023-12-11T18:03:52.524 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:52.515+0000 7f519ee1c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51981a7010 con 0x7f5198108b10 2023-12-11T18:03:52.524 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:52.516+0000 7f519cbb8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5198108b10 0x7f51981ac550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T18:03:52.524 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:52.516+0000 7f519cbb8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5198108b10 0x7f51981ac550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.59:40176/0 (socket says 172.21.15.59:40176) 2023-12-11T18:03:52.524 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:52.516+0000 7f519cbb8700 1 -- 172.21.15.59:0/4069492138 learned_addr learned my addr 172.21.15.59:0/4069492138 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T18:03:52.524 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:52.516+0000 7f519cbb8700 1 -- 172.21.15.59:0/4069492138 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f519810fd60 msgr2=0x7f51981aca90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-11T18:03:52.524 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:52.516+0000 7f519cbb8700 1 --2- 172.21.15.59:0/4069492138 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f519810fd60 0x7f51981aca90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:03:52.524 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:52.516+0000 7f519cbb8700 1 -- 172.21.15.59:0/4069492138 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f519811adb0 msgr2=0x7f51981a6680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T18:03:52.524 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:52.516+0000 7f519cbb8700 1 --2- 172.21.15.59:0/4069492138 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f519811adb0 0x7f51981a6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:03:52.524 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:52.516+0000 7f519cbb8700 1 -- 172.21.15.59:0/4069492138 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51880095f0 con 0x7f5198108b10 2023-12-11T18:03:52.524 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:52.517+0000 7f519cbb8700 1 --2- 172.21.15.59:0/4069492138 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5198108b10 0x7f51981ac550 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f5190009fa0 tx=0x7f519000bde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T18:03:52.524 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:52.518+0000 7f5195ffb700 1 -- 172.21.15.59:0/4069492138 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f519002ea40 con 0x7f5198108b10 2023-12-11T18:03:52.524 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:52.518+0000 7f5195ffb700 1 -- 172.21.15.59:0/4069492138 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f519002eba0 con 0x7f5198108b10 2023-12-11T18:03:52.524 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:52.518+0000 7f5195ffb700 1 -- 172.21.15.59:0/4069492138 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51900075b0 con 0x7f5198108b10 2023-12-11T18:03:52.524 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:52.518+0000 7f519ee1c700 1 -- 172.21.15.59:0/4069492138 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51981a72f0 con 0x7f5198108b10 2023-12-11T18:03:52.524 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:52.518+0000 7f519ee1c700 1 -- 172.21.15.59:0/4069492138 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5198114790 con 0x7f5198108b10 2023-12-11T18:03:52.530 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:52.522+0000 7f519ee1c700 1 -- 172.21.15.59:0/4069492138 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51840052f0 con 0x7f5198108b10 2023-12-11T18:03:52.530 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:52.525+0000 7f5195ffb700 1 -- 172.21.15.59:0/4069492138 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 28) v1 ==== 136137+0+0 (secure 0 0 0) 0x7f5190036440 con 0x7f5198108b10 2023-12-11T18:03:52.530 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:52.526+0000 7f5195ffb700 1 --2- 172.21.15.59:0/4069492138 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7f51800a1780 0x7f51800a3c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T18:03:52.530 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:52.526+0000 7f5195ffb700 1 -- 172.21.15.59:0/4069492138 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 9727+0+0 (secure 0 0 0) 0x7f51900ee970 con 0x7f5198108b10 2023-12-11T18:03:52.530 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:52.526+0000 7f5197fff700 1 --2- 172.21.15.59:0/4069492138 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7f51800a1780 0x7f51800a3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T18:03:52.530 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:52.527+0000 7f5197fff700 1 --2- 172.21.15.59:0/4069492138 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7f51800a1780 0x7f51800a3c40 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f51981a7d90 tx=0x7f518c00b3f0 comp rx=0 tx=0).ready entity=mgr.15414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T18:03:52.546 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:52.543+0000 7f5195ffb700 1 -- 172.21.15.59:0/4069492138 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f51900712f0 con 0x7f5198108b10 2023-12-11T18:03:52.658 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:03:52 smithi059 ceph-mon[98847]: pgmap v559: 129 pgs: 56 active+clean+snaptrim_wait, 17 active+clean+snaptrim, 56 active+clean; 1.8 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 719 KiB/s rd, 723 KiB/s wr, 90 op/s 2023-12-11T18:03:52.658 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:03:52 smithi059 ceph-mon[98847]: from='client.25289 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T18:03:52.658 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:03:52 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/1050425088' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-11T18:03:52.658 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:03:52 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/4145198894' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-11T18:03:52.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:03:52 smithi115 ceph-mon[102748]: pgmap v559: 129 pgs: 56 active+clean+snaptrim_wait, 17 active+clean+snaptrim, 56 active+clean; 1.8 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 719 KiB/s rd, 723 KiB/s wr, 90 op/s 2023-12-11T18:03:52.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:03:52 smithi115 ceph-mon[102748]: from='client.25289 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-12-11T18:03:52.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:03:52 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/1050425088' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-11T18:03:52.784 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:03:52 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/4145198894' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-11T18:03:52.911 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:52.908+0000 7f519ee1c700 1 -- 172.21.15.59:0/4069492138 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5184005b20 con 0x7f5198108b10 2023-12-11T18:03:52.911 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:52.908+0000 7f5195ffb700 1 -- 172.21.15.59:0/4069492138 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v118) v1 ==== 104+0+11 (secure 0 0 0) 0x7f51900b78f0 con 0x7f5198108b10 2023-12-11T18:03:52.922 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2023-12-11T18:03:52.925 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:52.922+0000 7f519ee1c700 1 -- 172.21.15.59:0/4069492138 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7f51800a1780 msgr2=0x7f51800a3c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T18:03:52.925 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:52.922+0000 7f519ee1c700 1 --2- 172.21.15.59:0/4069492138 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7f51800a1780 0x7f51800a3c40 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f51981a7d90 tx=0x7f518c00b3f0 comp rx=0 tx=0).stop 2023-12-11T18:03:52.925 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:52.922+0000 7f519ee1c700 1 -- 172.21.15.59:0/4069492138 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5198108b10 msgr2=0x7f51981ac550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T18:03:52.925 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:52.922+0000 7f519ee1c700 1 --2- 172.21.15.59:0/4069492138 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5198108b10 0x7f51981ac550 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f5190009fa0 tx=0x7f519000bde0 comp rx=0 tx=0).stop 2023-12-11T18:03:52.925 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:52.922+0000 7f519ee1c700 1 -- 172.21.15.59:0/4069492138 shutdown_connections 2023-12-11T18:03:52.926 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:52.922+0000 7f519ee1c700 1 --2- 172.21.15.59:0/4069492138 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7f51800a1780 0x7f51800a3c40 secure :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f51981a7d90 tx=0x7f518c00b3f0 comp rx=0 tx=0).stop 2023-12-11T18:03:52.926 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:52.922+0000 7f519ee1c700 1 --2- 172.21.15.59:0/4069492138 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f519811adb0 0x7f51981a6680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:03:52.926 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:52.922+0000 7f519ee1c700 1 --2- 172.21.15.59:0/4069492138 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5198108b10 0x7f51981ac550 secure :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f5190009fa0 tx=0x7f519000bde0 comp rx=0 tx=0).stop 2023-12-11T18:03:52.926 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:52.923+0000 7f519ee1c700 1 --2- 172.21.15.59:0/4069492138 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f519810fd60 0x7f51981aca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:03:52.926 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:52.923+0000 7f519ee1c700 1 -- 172.21.15.59:0/4069492138 >> 172.21.15.59:0/4069492138 conn(0x7f519806c610 msgr2=0x7f5198118da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T18:03:52.932 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:52.929+0000 7f519ee1c700 1 -- 172.21.15.59:0/4069492138 shutdown_connections 2023-12-11T18:03:52.932 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:52.929+0000 7f519ee1c700 1 -- 172.21.15.59:0/4069492138 wait complete. 2023-12-11T18:03:52.957 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-11T18:03:53.208 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:53.204+0000 7f42f1c2f700 1 -- 172.21.15.59:0/4138714449 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f42ec10fd60 msgr2=0x7f42ec1085d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T18:03:53.208 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:53.204+0000 7f42f1c2f700 1 --2- 172.21.15.59:0/4138714449 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f42ec10fd60 0x7f42ec1085d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f42dc01a9a0 tx=0x7f42dc01acb0 comp rx=0 tx=0).stop 2023-12-11T18:03:53.214 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:53.211+0000 7f42f1c2f700 1 -- 172.21.15.59:0/4138714449 shutdown_connections 2023-12-11T18:03:53.214 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:53.211+0000 7f42f1c2f700 1 --2- 172.21.15.59:0/4138714449 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f42ec11adb0 0x7f42ec11d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:03:53.214 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:53.211+0000 7f42f1c2f700 1 --2- 172.21.15.59:0/4138714449 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f42ec10fd60 0x7f42ec1085d0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:03:53.214 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:53.211+0000 7f42f1c2f700 1 --2- 172.21.15.59:0/4138714449 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f42ec108b10 0x7f42ec108f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:03:53.214 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:53.211+0000 7f42f1c2f700 1 -- 172.21.15.59:0/4138714449 >> 172.21.15.59:0/4138714449 conn(0x7f42ec06c610 msgr2=0x7f42ec06ca20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T18:03:53.221 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:53.212+0000 7f42f1c2f700 1 -- 172.21.15.59:0/4138714449 shutdown_connections 2023-12-11T18:03:53.231 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:53.224+0000 7f42f1c2f700 1 -- 172.21.15.59:0/4138714449 wait complete. 2023-12-11T18:03:53.231 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:53.224+0000 7f42f1c2f700 1 Processor -- start 2023-12-11T18:03:53.234 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:53.230+0000 7f42f1c2f700 1 -- start start 2023-12-11T18:03:53.234 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:53.230+0000 7f42f1c2f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f42ec108b10 0x7f42ec1165a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T18:03:53.234 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:53.230+0000 7f42f1c2f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f42ec10fd60 0x7f42ec116ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T18:03:53.234 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:53.230+0000 7f42f1c2f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f42ec11adb0 0x7f42ec1a2a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T18:03:53.234 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:53.230+0000 7f42f1c2f700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42ec1a3060 con 0x7f42ec108b10 2023-12-11T18:03:53.234 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:53.230+0000 7f42f1c2f700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42ec1a31a0 con 0x7f42ec11adb0 2023-12-11T18:03:53.234 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:53.230+0000 7f42f1c2f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42ec1a3360 con 0x7f42ec10fd60 2023-12-11T18:03:53.239 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:53.235+0000 7f42ebfff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f42ec11adb0 0x7f42ec1a2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T18:03:53.239 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:53.235+0000 7f42ebfff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f42ec11adb0 0x7f42ec1a2a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.59:60098/0 (socket says 172.21.15.59:60098) 2023-12-11T18:03:53.239 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:53.235+0000 7f42ebfff700 1 -- 172.21.15.59:0/903332900 learned_addr learned my addr 172.21.15.59:0/903332900 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T18:03:53.239 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:53.236+0000 7f42ebfff700 1 -- 172.21.15.59:0/903332900 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f42ec10fd60 msgr2=0x7f42ec116ae0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-12-11T18:03:53.239 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:53.236+0000 7f42ebfff700 1 --2- 172.21.15.59:0/903332900 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f42ec10fd60 0x7f42ec116ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:03:53.239 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:53.236+0000 7f42ebfff700 1 -- 172.21.15.59:0/903332900 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f42ec108b10 msgr2=0x7f42ec1165a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-11T18:03:53.239 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:53.236+0000 7f42ebfff700 1 --2- 172.21.15.59:0/903332900 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f42ec108b10 0x7f42ec1165a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:03:53.239 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:53.236+0000 7f42ebfff700 1 -- 172.21.15.59:0/903332900 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42dc01a600 con 0x7f42ec11adb0 2023-12-11T18:03:53.240 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:53.236+0000 7f42ebfff700 1 --2- 172.21.15.59:0/903332900 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f42ec11adb0 0x7f42ec1a2a00 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f42e400b970 tx=0x7f42e400bd30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T18:03:53.240 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:53.236+0000 7f42e8ff9700 1 -- 172.21.15.59:0/903332900 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42e400c9b0 con 0x7f42ec11adb0 2023-12-11T18:03:53.240 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:53.236+0000 7f42f1c2f700 1 -- 172.21.15.59:0/903332900 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42ec1a3640 con 0x7f42ec11adb0 2023-12-11T18:03:53.240 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:53.236+0000 7f42f1c2f700 1 -- 172.21.15.59:0/903332900 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42ec1aada0 con 0x7f42ec11adb0 2023-12-11T18:03:53.240 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:53.237+0000 7f42e8ff9700 1 -- 172.21.15.59:0/903332900 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f42e4014440 con 0x7f42ec11adb0 2023-12-11T18:03:53.240 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:53.237+0000 7f42e8ff9700 1 -- 172.21.15.59:0/903332900 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42e4007590 con 0x7f42ec11adb0 2023-12-11T18:03:53.241 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:53.237+0000 7f42f1c2f700 1 -- 172.21.15.59:0/903332900 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42ec06da90 con 0x7f42ec11adb0 2023-12-11T18:03:53.242 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:53.239+0000 7f42e8ff9700 1 -- 172.21.15.59:0/903332900 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 28) v1 ==== 136137+0+0 (secure 0 0 0) 0x7f42e40077b0 con 0x7f42ec11adb0 2023-12-11T18:03:53.243 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:53.240+0000 7f42e8ff9700 1 --2- 172.21.15.59:0/903332900 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7f42d40a1670 0x7f42d40a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T18:03:53.243 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:53.240+0000 7f42e8ff9700 1 -- 172.21.15.59:0/903332900 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 9727+0+0 (secure 0 0 0) 0x7f42e40cd390 con 0x7f42ec11adb0 2023-12-11T18:03:53.246 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:53.243+0000 7f42e8ff9700 1 -- 172.21.15.59:0/903332900 <== mon.2 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f42e4096310 con 0x7f42ec11adb0 2023-12-11T18:03:53.249 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:53.246+0000 7f42eb7fe700 1 --2- 172.21.15.59:0/903332900 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7f42d40a1670 0x7f42d40a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T18:03:53.250 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:53.247+0000 7f42eb7fe700 1 --2- 172.21.15.59:0/903332900 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7f42d40a1670 0x7f42d40a3b30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f42dc000c00 tx=0x7f42dc005fb0 comp rx=0 tx=0).ready entity=mgr.15414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T18:03:53.363 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:03:53 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/4069492138' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-11T18:03:53.622 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:53.614+0000 7f42f1c2f700 1 -- 172.21.15.59:0/903332900 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f42ec068310 con 0x7f42ec11adb0 2023-12-11T18:03:53.622 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:53.617+0000 7f42e8ff9700 1 -- 172.21.15.59:0/903332900 <== mon.2 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v118) v1 ==== 100+0+11 (secure 0 0 0) 0x7f42ec068310 con 0x7f42ec11adb0 2023-12-11T18:03:53.628 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2023-12-11T18:03:53.643 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:53.636+0000 7f42f1c2f700 1 -- 172.21.15.59:0/903332900 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7f42d40a1670 msgr2=0x7f42d40a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T18:03:53.643 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:53.636+0000 7f42f1c2f700 1 --2- 172.21.15.59:0/903332900 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7f42d40a1670 0x7f42d40a3b30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f42dc000c00 tx=0x7f42dc005fb0 comp rx=0 tx=0).stop 2023-12-11T18:03:53.643 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:53.636+0000 7f42f1c2f700 1 -- 172.21.15.59:0/903332900 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f42ec11adb0 msgr2=0x7f42ec1a2a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T18:03:53.643 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:53.636+0000 7f42f1c2f700 1 --2- 172.21.15.59:0/903332900 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f42ec11adb0 0x7f42ec1a2a00 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f42e400b970 tx=0x7f42e400bd30 comp rx=0 tx=0).stop 2023-12-11T18:03:53.655 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:53.649+0000 7f42f1c2f700 1 -- 172.21.15.59:0/903332900 shutdown_connections 2023-12-11T18:03:53.655 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:53.649+0000 7f42f1c2f700 1 --2- 172.21.15.59:0/903332900 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7f42d40a1670 0x7f42d40a3b30 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:03:53.655 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:53.649+0000 7f42f1c2f700 1 --2- 172.21.15.59:0/903332900 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f42ec11adb0 0x7f42ec1a2a00 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:03:53.655 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:53.649+0000 7f42f1c2f700 1 --2- 172.21.15.59:0/903332900 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f42ec108b10 0x7f42ec1165a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:03:53.655 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:53.649+0000 7f42f1c2f700 1 --2- 172.21.15.59:0/903332900 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f42ec10fd60 0x7f42ec116ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:03:53.655 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:53.649+0000 7f42f1c2f700 1 -- 172.21.15.59:0/903332900 >> 172.21.15.59:0/903332900 conn(0x7f42ec06c610 msgr2=0x7f42ec11b190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T18:03:53.659 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:53.656+0000 7f42f1c2f700 1 -- 172.21.15.59:0/903332900 shutdown_connections 2023-12-11T18:03:53.667 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:53.661+0000 7f42f1c2f700 1 -- 172.21.15.59:0/903332900 wait complete. 2023-12-11T18:03:53.706 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-12-11T18:03:53.782 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:03:53 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/4069492138' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-11T18:03:53.986 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:53.983+0000 7f55fb535700 1 -- 172.21.15.59:0/3756787160 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f55f41064a0 msgr2=0x7f55f4106880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T18:03:53.986 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:53.983+0000 7f55fb535700 1 --2- 172.21.15.59:0/3756787160 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f55f41064a0 0x7f55f4106880 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f55f0009990 tx=0x7f55f0009ca0 comp rx=0 tx=0).stop 2023-12-11T18:03:53.986 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:53.983+0000 7f55fb535700 1 -- 172.21.15.59:0/3756787160 shutdown_connections 2023-12-11T18:03:53.987 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:53.983+0000 7f55fb535700 1 --2- 172.21.15.59:0/3756787160 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55f4100de0 0x7f55f4069a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:03:53.987 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:53.983+0000 7f55fb535700 1 --2- 172.21.15.59:0/3756787160 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f55f41064a0 0x7f55f4106880 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:03:53.987 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:53.983+0000 7f55fb535700 1 --2- 172.21.15.59:0/3756787160 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f55f4100420 0x7f55f41008a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:03:53.987 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:53.983+0000 7f55fb535700 1 -- 172.21.15.59:0/3756787160 >> 172.21.15.59:0/3756787160 conn(0x7f55f4077570 msgr2=0x7f55f4077980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T18:03:53.987 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:53.983+0000 7f55fb535700 1 -- 172.21.15.59:0/3756787160 shutdown_connections 2023-12-11T18:03:53.987 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:53.983+0000 7f55fb535700 1 -- 172.21.15.59:0/3756787160 wait complete. 2023-12-11T18:03:53.987 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:53.984+0000 7f55fb535700 1 Processor -- start 2023-12-11T18:03:53.988 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:53.984+0000 7f55fb535700 1 -- start start 2023-12-11T18:03:53.988 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:53.984+0000 7f55fb535700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55f4100420 0x7f55f410f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T18:03:53.988 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:53.984+0000 7f55fb535700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f55f4100de0 0x7f55f410faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T18:03:53.988 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:53.984+0000 7f55fb535700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f55f41064a0 0x7f55f4110030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T18:03:53.988 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:53.984+0000 7f55fb535700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55f4071d10 con 0x7f55f4100de0 2023-12-11T18:03:53.988 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:53.985+0000 7f55fb535700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55f4071e50 con 0x7f55f41064a0 2023-12-11T18:03:53.988 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:53.985+0000 7f55fb535700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55f4072010 con 0x7f55f4100420 2023-12-11T18:03:53.988 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:53.985+0000 7f55f8ad0700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f55f4100de0 0x7f55f410faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T18:03:53.989 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:53.985+0000 7f55f8ad0700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f55f4100de0 0x7f55f410faf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35382/0 (socket says 172.21.15.59:35382) 2023-12-11T18:03:53.989 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:53.985+0000 7f55f8ad0700 1 -- 172.21.15.59:0/2396342680 learned_addr learned my addr 172.21.15.59:0/2396342680 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T18:03:53.989 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:53.985+0000 7f55f9ad2700 1 --2- 172.21.15.59:0/2396342680 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f55f41064a0 0x7f55f4110030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T18:03:53.991 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:53.988+0000 7f55f92d1700 1 --2- 172.21.15.59:0/2396342680 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55f4100420 0x7f55f410f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T18:03:53.991 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:53.988+0000 7f55f9ad2700 1 -- 172.21.15.59:0/2396342680 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55f4100420 msgr2=0x7f55f410f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T18:03:53.991 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:53.988+0000 7f55f9ad2700 1 --2- 172.21.15.59:0/2396342680 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55f4100420 0x7f55f410f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:03:53.991 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:53.988+0000 7f55f9ad2700 1 -- 172.21.15.59:0/2396342680 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f55f4100de0 msgr2=0x7f55f410faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T18:03:53.991 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:53.988+0000 7f55f9ad2700 1 --2- 172.21.15.59:0/2396342680 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f55f4100de0 0x7f55f410faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:03:53.991 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:53.988+0000 7f55f9ad2700 1 -- 172.21.15.59:0/2396342680 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55f00095f0 con 0x7f55f41064a0 2023-12-11T18:03:53.992 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:53.988+0000 7f55f9ad2700 1 --2- 172.21.15.59:0/2396342680 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f55f41064a0 0x7f55f4110030 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f55ec00be20 tx=0x7f55ec009ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T18:03:53.995 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:53.992+0000 7f55e67fc700 1 -- 172.21.15.59:0/2396342680 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55ec00cc10 con 0x7f55f41064a0 2023-12-11T18:03:53.995 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:53.992+0000 7f55e67fc700 1 -- 172.21.15.59:0/2396342680 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f55ec00cd70 con 0x7f55f41064a0 2023-12-11T18:03:53.995 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:53.992+0000 7f55e67fc700 1 -- 172.21.15.59:0/2396342680 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55ec0126c0 con 0x7f55f41064a0 2023-12-11T18:03:53.998 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:53.995+0000 7f55fb535700 1 -- 172.21.15.59:0/2396342680 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55f40722f0 con 0x7f55f41064a0 2023-12-11T18:03:53.998 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:53.995+0000 7f55fb535700 1 -- 172.21.15.59:0/2396342680 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55f4072780 con 0x7f55f41064a0 2023-12-11T18:03:53.999 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:53.996+0000 7f55fb535700 1 -- 172.21.15.59:0/2396342680 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55f4101b40 con 0x7f55f41064a0 2023-12-11T18:03:54.012 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:54.004+0000 7f55e67fc700 1 -- 172.21.15.59:0/2396342680 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 28) v1 ==== 136137+0+0 (secure 0 0 0) 0x7f55ec012820 con 0x7f55f41064a0 2023-12-11T18:03:54.012 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:54.005+0000 7f55e67fc700 1 --2- 172.21.15.59:0/2396342680 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7f55e00a16f0 0x7f55e00a3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T18:03:54.012 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:54.005+0000 7f55e67fc700 1 -- 172.21.15.59:0/2396342680 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 9727+0+0 (secure 0 0 0) 0x7f55ec006e40 con 0x7f55f41064a0 2023-12-11T18:03:54.012 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:54.007+0000 7f55f92d1700 1 --2- 172.21.15.59:0/2396342680 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7f55e00a16f0 0x7f55e00a3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T18:03:54.012 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:54.007+0000 7f55f92d1700 1 --2- 172.21.15.59:0/2396342680 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7f55e00a16f0 0x7f55e00a3bb0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f55f000b5c0 tx=0x7f55f0005fd0 comp rx=0 tx=0).ready entity=mgr.15414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T18:03:54.015 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:54.012+0000 7f55e67fc700 1 -- 172.21.15.59:0/2396342680 <== mon.2 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f55ec096190 con 0x7f55f41064a0 2023-12-11T18:03:54.370 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:54.367+0000 7f55fb535700 1 -- 172.21.15.59:0/2396342680 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f55f4110b30 con 0x7f55f41064a0 2023-12-11T18:03:54.374 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:54.371+0000 7f55e67fc700 1 -- 172.21.15.59:0/2396342680 <== mon.2 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v118) v1 ==== 103+0+11 (secure 0 0 0) 0x7f55ec0190a0 con 0x7f55f41064a0 2023-12-11T18:03:54.378 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2023-12-11T18:03:54.396 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:54.393+0000 7f55fb535700 1 -- 172.21.15.59:0/2396342680 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7f55e00a16f0 msgr2=0x7f55e00a3bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T18:03:54.396 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:54.393+0000 7f55fb535700 1 --2- 172.21.15.59:0/2396342680 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7f55e00a16f0 0x7f55e00a3bb0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f55f000b5c0 tx=0x7f55f0005fd0 comp rx=0 tx=0).stop 2023-12-11T18:03:54.396 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:54.393+0000 7f55fb535700 1 -- 172.21.15.59:0/2396342680 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f55f41064a0 msgr2=0x7f55f4110030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T18:03:54.396 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:54.393+0000 7f55fb535700 1 --2- 172.21.15.59:0/2396342680 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f55f41064a0 0x7f55f4110030 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f55ec00be20 tx=0x7f55ec009ed0 comp rx=0 tx=0).stop 2023-12-11T18:03:54.396 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:54.393+0000 7f55fb535700 1 -- 172.21.15.59:0/2396342680 shutdown_connections 2023-12-11T18:03:54.396 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:54.393+0000 7f55fb535700 1 --2- 172.21.15.59:0/2396342680 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7f55e00a16f0 0x7f55e00a3bb0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:03:54.396 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:54.393+0000 7f55fb535700 1 --2- 172.21.15.59:0/2396342680 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f55f41064a0 0x7f55f4110030 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:03:54.396 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:54.393+0000 7f55fb535700 1 --2- 172.21.15.59:0/2396342680 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55f4100420 0x7f55f410f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:03:54.396 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:54.393+0000 7f55fb535700 1 --2- 172.21.15.59:0/2396342680 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f55f4100de0 0x7f55f410faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:03:54.396 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:54.393+0000 7f55fb535700 1 -- 172.21.15.59:0/2396342680 >> 172.21.15.59:0/2396342680 conn(0x7f55f4077570 msgr2=0x7f55f40ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T18:03:54.402 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:54.399+0000 7f55fb535700 1 -- 172.21.15.59:0/2396342680 shutdown_connections 2023-12-11T18:03:54.402 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:54.399+0000 7f55fb535700 1 -- 172.21.15.59:0/2396342680 wait complete. 2023-12-11T18:03:54.426 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2023-12-11T18:03:54.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:03:54 smithi059 ceph-mon[98847]: pgmap v560: 129 pgs: 48 active+clean+snaptrim_wait, 16 active+clean+snaptrim, 65 active+clean; 1.8 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 566 KiB/s rd, 567 KiB/s wr, 90 op/s 2023-12-11T18:03:54.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:03:54 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/903332900' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-11T18:03:54.668 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:54.665+0000 7faa11739700 1 -- 172.21.15.59:0/18475556 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faa0c1085d0 msgr2=0x7faa0c108a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T18:03:54.668 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:54.665+0000 7faa11739700 1 --2- 172.21.15.59:0/18475556 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faa0c1085d0 0x7faa0c108a50 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7faa0400b0a0 tx=0x7faa0400b3b0 comp rx=0 tx=0).stop 2023-12-11T18:03:54.668 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:54.665+0000 7faa11739700 1 -- 172.21.15.59:0/18475556 shutdown_connections 2023-12-11T18:03:54.668 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:54.665+0000 7faa11739700 1 --2- 172.21.15.59:0/18475556 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa0c108f90 0x7faa0c114d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:03:54.668 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:54.665+0000 7faa11739700 1 --2- 172.21.15.59:0/18475556 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa0c10ceb0 0x7faa0c10d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:03:54.668 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:54.665+0000 7faa11739700 1 --2- 172.21.15.59:0/18475556 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faa0c1085d0 0x7faa0c108a50 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:03:54.669 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:54.665+0000 7faa11739700 1 -- 172.21.15.59:0/18475556 >> 172.21.15.59:0/18475556 conn(0x7faa0c06c990 msgr2=0x7faa0c06cda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T18:03:54.678 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:54.675+0000 7faa11739700 1 -- 172.21.15.59:0/18475556 shutdown_connections 2023-12-11T18:03:54.685 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:54.681+0000 7faa11739700 1 -- 172.21.15.59:0/18475556 wait complete. 2023-12-11T18:03:54.688 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:54.685+0000 7faa11739700 1 Processor -- start 2023-12-11T18:03:54.690 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:54.687+0000 7faa11739700 1 -- start start 2023-12-11T18:03:54.690 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:54.687+0000 7faa11739700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa0c1085d0 0x7faa0c11dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T18:03:54.691 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:54.687+0000 7faa11739700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faa0c108f90 0x7faa0c11e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T18:03:54.691 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:54.687+0000 7faa11739700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa0c10ceb0 0x7faa0c117f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T18:03:54.691 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:54.687+0000 7faa11739700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa0c1186f0 con 0x7faa0c1085d0 2023-12-11T18:03:54.691 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:54.687+0000 7faa11739700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa0c118860 con 0x7faa0c108f90 2023-12-11T18:03:54.691 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:54.687+0000 7faa11739700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa0c118a20 con 0x7faa0c10ceb0 2023-12-11T18:03:54.702 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:54.688+0000 7faa0a7fc700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faa0c108f90 0x7faa0c11e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T18:03:54.702 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:54.688+0000 7faa0affd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa0c1085d0 0x7faa0c11dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T18:03:54.702 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:54.688+0000 7faa0a7fc700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faa0c108f90 0x7faa0c11e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.59:60120/0 (socket says 172.21.15.59:60120) 2023-12-11T18:03:54.702 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:54.688+0000 7faa0a7fc700 1 -- 172.21.15.59:0/2003391227 learned_addr learned my addr 172.21.15.59:0/2003391227 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T18:03:54.702 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:54.688+0000 7faa0affd700 1 -- 172.21.15.59:0/2003391227 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa0c10ceb0 msgr2=0x7faa0c117f20 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-12-11T18:03:54.702 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:54.688+0000 7faa0affd700 1 --2- 172.21.15.59:0/2003391227 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa0c10ceb0 0x7faa0c117f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:03:54.702 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:54.688+0000 7faa0affd700 1 -- 172.21.15.59:0/2003391227 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faa0c108f90 msgr2=0x7faa0c11e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T18:03:54.702 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:54.688+0000 7faa0affd700 1 --2- 172.21.15.59:0/2003391227 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faa0c108f90 0x7faa0c11e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:03:54.702 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:54.688+0000 7faa0affd700 1 -- 172.21.15.59:0/2003391227 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa04009d00 con 0x7faa0c1085d0 2023-12-11T18:03:54.702 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:54.689+0000 7faa0affd700 1 --2- 172.21.15.59:0/2003391227 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa0c1085d0 0x7faa0c11dd10 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fa9fc006fd0 tx=0x7fa9fc00b730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T18:03:54.702 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:54.689+0000 7fa9f3fff700 1 -- 172.21.15.59:0/2003391227 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9fc011830 con 0x7faa0c1085d0 2023-12-11T18:03:54.702 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:54.689+0000 7faa11739700 1 -- 172.21.15.59:0/2003391227 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa0c1bb8c0 con 0x7faa0c1085d0 2023-12-11T18:03:54.702 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:54.689+0000 7fa9f3fff700 1 -- 172.21.15.59:0/2003391227 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa9fc011e50 con 0x7faa0c1085d0 2023-12-11T18:03:54.703 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:54.689+0000 7fa9f3fff700 1 -- 172.21.15.59:0/2003391227 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9fc010e40 con 0x7faa0c1085d0 2023-12-11T18:03:54.703 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:54.689+0000 7faa11739700 1 -- 172.21.15.59:0/2003391227 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa0c1bbce0 con 0x7faa0c1085d0 2023-12-11T18:03:54.703 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:54.691+0000 7faa11739700 1 -- 172.21.15.59:0/2003391227 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa0c110470 con 0x7faa0c1085d0 2023-12-11T18:03:54.703 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:54.692+0000 7fa9f3fff700 1 -- 172.21.15.59:0/2003391227 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 28) v1 ==== 136137+0+0 (secure 0 0 0) 0x7fa9fc01a400 con 0x7faa0c1085d0 2023-12-11T18:03:54.703 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:54.693+0000 7fa9f3fff700 1 --2- 172.21.15.59:0/2003391227 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7fa9f40a1620 0x7fa9f40a3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T18:03:54.703 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:54.693+0000 7fa9f3fff700 1 -- 172.21.15.59:0/2003391227 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 9727+0+0 (secure 0 0 0) 0x7fa9fc0ccf10 con 0x7faa0c1085d0 2023-12-11T18:03:54.703 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:54.695+0000 7faa0a7fc700 1 --2- 172.21.15.59:0/2003391227 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7fa9f40a1620 0x7fa9f40a3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T18:03:54.703 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:54.696+0000 7faa0a7fc700 1 --2- 172.21.15.59:0/2003391227 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7fa9f40a1620 0x7fa9f40a3ae0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7faa04009610 tx=0x7faa0400b700 comp rx=0 tx=0).ready entity=mgr.15414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T18:03:54.714 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:54.709+0000 7fa9f3fff700 1 -- 172.21.15.59:0/2003391227 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fa9fc096e10 con 0x7faa0c1085d0 2023-12-11T18:03:54.782 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:03:54 smithi115 ceph-mon[102748]: pgmap v560: 129 pgs: 48 active+clean+snaptrim_wait, 16 active+clean+snaptrim, 65 active+clean; 1.8 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 566 KiB/s rd, 567 KiB/s wr, 90 op/s 2023-12-11T18:03:54.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:03:54 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/903332900' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-11T18:03:55.197 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:55.194+0000 7faa11739700 1 -- 172.21.15.59:0/2003391227 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) v1 -- 0x7faa0c1192e0 con 0x7faa0c1085d0 2023-12-11T18:03:55.212 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:55.208+0000 7fa9f3fff700 1 -- 172.21.15.59:0/2003391227 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v28) v1 ==== 81+0+734410 (secure 0 0 0) 0x7fa9fc015090 con 0x7faa0c1085d0 2023-12-11T18:03:55.224 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-12-11T18:03:55.224 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2023-12-11T18:03:55.224 INFO:teuthology.orchestra.run.smithi059.stdout: "epoch": 28, 2023-12-11T18:03:55.225 INFO:teuthology.orchestra.run.smithi059.stdout: "active_gid": 15414, 2023-12-11T18:03:55.225 INFO:teuthology.orchestra.run.smithi059.stdout: "active_name": "x", 2023-12-11T18:03:55.225 INFO:teuthology.orchestra.run.smithi059.stdout: "active_addrs": { 2023-12-11T18:03:55.225 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2023-12-11T18:03:55.225 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-12-11T18:03:55.225 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2023-12-11T18:03:55.225 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6800", 2023-12-11T18:03:55.225 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4126604994 2023-12-11T18:03:55.225 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.225 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-12-11T18:03:55.225 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2023-12-11T18:03:55.225 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6801", 2023-12-11T18:03:55.225 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4126604994 2023-12-11T18:03:55.226 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:55.226 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2023-12-11T18:03:55.226 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.226 INFO:teuthology.orchestra.run.smithi059.stdout: "active_addr": "172.21.15.59:6801/4126604994", 2023-12-11T18:03:55.226 INFO:teuthology.orchestra.run.smithi059.stdout: "active_change": "2023-12-11T17:46:17.753350+0000", 2023-12-11T18:03:55.226 INFO:teuthology.orchestra.run.smithi059.stdout: "active_mgr_features": 4540138322906710015, 2023-12-11T18:03:55.226 INFO:teuthology.orchestra.run.smithi059.stdout: "available": true, 2023-12-11T18:03:55.226 INFO:teuthology.orchestra.run.smithi059.stdout: "standbys": [ 2023-12-11T18:03:55.226 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-12-11T18:03:55.226 INFO:teuthology.orchestra.run.smithi059.stdout: "gid": 15525, 2023-12-11T18:03:55.226 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "z", 2023-12-11T18:03:55.226 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr_features": 4540138322906710015, 2023-12-11T18:03:55.226 INFO:teuthology.orchestra.run.smithi059.stdout: "available_modules": [ 2023-12-11T18:03:55.226 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-12-11T18:03:55.227 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "alerts", 2023-12-11T18:03:55.227 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2023-12-11T18:03:55.227 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2023-12-11T18:03:55.227 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2023-12-11T18:03:55.227 INFO:teuthology.orchestra.run.smithi059.stdout: "interval": { 2023-12-11T18:03:55.227 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "interval", 2023-12-11T18:03:55.227 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2023-12-11T18:03:55.227 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.227 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.227 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "60", 2023-12-11T18:03:55.228 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.228 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.228 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.228 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "How frequently to reexamine health status", 2023-12-11T18:03:55.228 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.228 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.228 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.228 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.228 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2023-12-11T18:03:55.228 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2023-12-11T18:03:55.228 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.228 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.228 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.228 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.229 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.229 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.229 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:55.229 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:55.229 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:55.229 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:55.229 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:55.229 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:55.229 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:55.229 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:55.229 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.229 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.229 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.229 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.229 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.230 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2023-12-11T18:03:55.230 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2023-12-11T18:03:55.230 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.230 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.230 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.230 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.230 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.230 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.230 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.230 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.230 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.230 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.231 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.231 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.231 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2023-12-11T18:03:55.231 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2023-12-11T18:03:55.231 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.231 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.231 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.231 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2023-12-11T18:03:55.231 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.231 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.231 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:55.231 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:55.231 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:55.231 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:55.231 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:55.232 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:55.232 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:55.232 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:55.232 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.232 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.232 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.232 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.232 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.232 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2023-12-11T18:03:55.232 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2023-12-11T18:03:55.232 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.232 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.232 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.232 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.232 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.232 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.233 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.233 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.233 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.233 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.233 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.233 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.233 INFO:teuthology.orchestra.run.smithi059.stdout: "smtp_destination": { 2023-12-11T18:03:55.233 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "smtp_destination", 2023-12-11T18:03:55.233 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.233 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.233 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.233 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.233 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.233 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.233 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.234 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Email address to send alerts to", 2023-12-11T18:03:55.234 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.234 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.234 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.234 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.234 INFO:teuthology.orchestra.run.smithi059.stdout: "smtp_from_name": { 2023-12-11T18:03:55.234 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "smtp_from_name", 2023-12-11T18:03:55.234 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.234 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.234 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.234 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "Ceph", 2023-12-11T18:03:55.234 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.234 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.234 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.235 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Email From: name", 2023-12-11T18:03:55.235 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.235 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.235 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.235 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.235 INFO:teuthology.orchestra.run.smithi059.stdout: "smtp_host": { 2023-12-11T18:03:55.235 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "smtp_host", 2023-12-11T18:03:55.235 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.235 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.235 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.235 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.235 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.235 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.235 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.235 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "SMTP server", 2023-12-11T18:03:55.236 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.236 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.236 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.236 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.236 INFO:teuthology.orchestra.run.smithi059.stdout: "smtp_password": { 2023-12-11T18:03:55.236 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "smtp_password", 2023-12-11T18:03:55.236 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.236 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.236 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.236 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.236 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.236 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.236 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.236 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Password to authenticate with", 2023-12-11T18:03:55.236 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.237 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.237 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.237 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.237 INFO:teuthology.orchestra.run.smithi059.stdout: "smtp_port": { 2023-12-11T18:03:55.237 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "smtp_port", 2023-12-11T18:03:55.237 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2023-12-11T18:03:55.237 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.237 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.237 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "465", 2023-12-11T18:03:55.237 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.237 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.237 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.237 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "SMTP port", 2023-12-11T18:03:55.237 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.238 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.238 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.238 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.238 INFO:teuthology.orchestra.run.smithi059.stdout: "smtp_sender": { 2023-12-11T18:03:55.238 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "smtp_sender", 2023-12-11T18:03:55.238 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.238 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.238 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.238 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.238 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.238 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.238 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.239 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "SMTP envelope sender", 2023-12-11T18:03:55.239 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.239 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.239 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.239 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.239 INFO:teuthology.orchestra.run.smithi059.stdout: "smtp_ssl": { 2023-12-11T18:03:55.239 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "smtp_ssl", 2023-12-11T18:03:55.239 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.239 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.239 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.239 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2023-12-11T18:03:55.239 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.239 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.239 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.239 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Use SSL to connect to SMTP server", 2023-12-11T18:03:55.240 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.240 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.240 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.240 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.240 INFO:teuthology.orchestra.run.smithi059.stdout: "smtp_user": { 2023-12-11T18:03:55.240 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "smtp_user", 2023-12-11T18:03:55.240 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.240 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.240 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.240 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.240 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.240 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.240 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.240 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "User to authenticate as", 2023-12-11T18:03:55.240 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.240 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.241 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.241 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:55.241 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:55.241 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.241 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-12-11T18:03:55.241 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "balancer", 2023-12-11T18:03:55.241 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2023-12-11T18:03:55.241 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2023-12-11T18:03:55.241 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2023-12-11T18:03:55.241 INFO:teuthology.orchestra.run.smithi059.stdout: "active": { 2023-12-11T18:03:55.241 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "active", 2023-12-11T18:03:55.241 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.241 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.241 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.241 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2023-12-11T18:03:55.242 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.242 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.242 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.242 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "automatically balance PGs across cluster", 2023-12-11T18:03:55.242 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.242 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.242 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.242 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.242 INFO:teuthology.orchestra.run.smithi059.stdout: "begin_time": { 2023-12-11T18:03:55.242 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "begin_time", 2023-12-11T18:03:55.242 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.242 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.243 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.243 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0000", 2023-12-11T18:03:55.243 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.243 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.243 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.243 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "beginning time of day to automatically balance", 2023-12-11T18:03:55.243 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-12-11T18:03:55.243 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.243 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.243 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.243 INFO:teuthology.orchestra.run.smithi059.stdout: "begin_weekday": { 2023-12-11T18:03:55.243 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "begin_weekday", 2023-12-11T18:03:55.244 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "uint", 2023-12-11T18:03:55.244 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.244 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.244 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2023-12-11T18:03:55.244 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "0", 2023-12-11T18:03:55.244 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "6", 2023-12-11T18:03:55.244 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.244 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-12-11T18:03:55.244 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-12-11T18:03:55.244 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.244 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.244 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.244 INFO:teuthology.orchestra.run.smithi059.stdout: "crush_compat_max_iterations": { 2023-12-11T18:03:55.244 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "crush_compat_max_iterations", 2023-12-11T18:03:55.244 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "uint", 2023-12-11T18:03:55.245 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.245 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.245 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "25", 2023-12-11T18:03:55.245 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "1", 2023-12-11T18:03:55.245 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "250", 2023-12-11T18:03:55.245 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.245 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-12-11T18:03:55.245 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.245 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.245 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.245 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.245 INFO:teuthology.orchestra.run.smithi059.stdout: "crush_compat_metrics": { 2023-12-11T18:03:55.245 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "crush_compat_metrics", 2023-12-11T18:03:55.245 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.245 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.245 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.246 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "pgs,objects,bytes", 2023-12-11T18:03:55.246 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.246 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.246 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.246 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-12-11T18:03:55.246 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-12-11T18:03:55.246 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.246 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.246 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.246 INFO:teuthology.orchestra.run.smithi059.stdout: "crush_compat_step": { 2023-12-11T18:03:55.246 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "crush_compat_step", 2023-12-11T18:03:55.246 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "float", 2023-12-11T18:03:55.246 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.246 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.247 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0.5", 2023-12-11T18:03:55.247 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "0.001", 2023-12-11T18:03:55.247 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "0.999", 2023-12-11T18:03:55.247 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.247 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "aggressiveness of optimization", 2023-12-11T18:03:55.247 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-12-11T18:03:55.247 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.247 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.247 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.247 INFO:teuthology.orchestra.run.smithi059.stdout: "end_time": { 2023-12-11T18:03:55.247 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "end_time", 2023-12-11T18:03:55.247 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.247 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.247 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.247 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "2359", 2023-12-11T18:03:55.248 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.248 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.248 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.248 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "ending time of day to automatically balance", 2023-12-11T18:03:55.248 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-12-11T18:03:55.248 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.248 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.248 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.248 INFO:teuthology.orchestra.run.smithi059.stdout: "end_weekday": { 2023-12-11T18:03:55.248 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "end_weekday", 2023-12-11T18:03:55.248 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "uint", 2023-12-11T18:03:55.248 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.249 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.249 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2023-12-11T18:03:55.249 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "0", 2023-12-11T18:03:55.249 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "6", 2023-12-11T18:03:55.249 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.249 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-12-11T18:03:55.249 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-12-11T18:03:55.249 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.249 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.249 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.249 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2023-12-11T18:03:55.249 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2023-12-11T18:03:55.249 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.249 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.249 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.249 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.249 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.250 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.250 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:55.250 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:55.250 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:55.250 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:55.250 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:55.250 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:55.250 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:55.250 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:55.250 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.250 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.250 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.250 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.250 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.250 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2023-12-11T18:03:55.251 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2023-12-11T18:03:55.251 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.251 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.251 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.251 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.251 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.251 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.251 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.251 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.251 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.251 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.251 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.251 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.251 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2023-12-11T18:03:55.251 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2023-12-11T18:03:55.251 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.252 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.252 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.252 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2023-12-11T18:03:55.252 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.252 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.252 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:55.252 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:55.252 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:55.252 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:55.252 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:55.252 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:55.252 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:55.252 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:55.252 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.252 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.253 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.253 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.253 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.253 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2023-12-11T18:03:55.253 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2023-12-11T18:03:55.253 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.253 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.253 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.253 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.253 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.253 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.253 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.253 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.253 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.253 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.253 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.254 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.254 INFO:teuthology.orchestra.run.smithi059.stdout: "min_score": { 2023-12-11T18:03:55.254 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "min_score", 2023-12-11T18:03:55.254 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "float", 2023-12-11T18:03:55.254 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.254 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.254 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2023-12-11T18:03:55.254 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.254 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.254 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.254 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-12-11T18:03:55.254 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.254 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.254 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.255 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.255 INFO:teuthology.orchestra.run.smithi059.stdout: "mode": { 2023-12-11T18:03:55.255 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "mode", 2023-12-11T18:03:55.255 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.255 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.255 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.255 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "upmap", 2023-12-11T18:03:55.255 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.255 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.255 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:55.255 INFO:teuthology.orchestra.run.smithi059.stdout: "crush-compat", 2023-12-11T18:03:55.255 INFO:teuthology.orchestra.run.smithi059.stdout: "none", 2023-12-11T18:03:55.255 INFO:teuthology.orchestra.run.smithi059.stdout: "upmap" 2023-12-11T18:03:55.255 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:55.255 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Balancer mode", 2023-12-11T18:03:55.255 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.256 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.256 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.256 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.256 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_ids": { 2023-12-11T18:03:55.256 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "pool_ids", 2023-12-11T18:03:55.256 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.256 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.256 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.256 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.256 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.256 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.256 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.256 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-12-11T18:03:55.256 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.256 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.256 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.257 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.257 INFO:teuthology.orchestra.run.smithi059.stdout: "sleep_interval": { 2023-12-11T18:03:55.257 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sleep_interval", 2023-12-11T18:03:55.257 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2023-12-11T18:03:55.257 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.257 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.257 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "60", 2023-12-11T18:03:55.257 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.257 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.257 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.257 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-12-11T18:03:55.257 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.257 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.257 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.257 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.258 INFO:teuthology.orchestra.run.smithi059.stdout: "upmap_max_deviation": { 2023-12-11T18:03:55.258 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "upmap_max_deviation", 2023-12-11T18:03:55.258 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2023-12-11T18:03:55.258 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.258 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.258 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "5", 2023-12-11T18:03:55.258 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "1", 2023-12-11T18:03:55.258 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.258 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.258 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "deviation below which no optimization is attempted", 2023-12-11T18:03:55.258 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-12-11T18:03:55.259 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.259 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.259 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.259 INFO:teuthology.orchestra.run.smithi059.stdout: "upmap_max_optimizations": { 2023-12-11T18:03:55.259 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "upmap_max_optimizations", 2023-12-11T18:03:55.259 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "uint", 2023-12-11T18:03:55.259 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.259 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.259 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "10", 2023-12-11T18:03:55.259 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.259 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.259 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.259 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-12-11T18:03:55.259 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.259 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.259 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.260 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:55.260 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:55.260 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.260 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-12-11T18:03:55.260 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "cephadm", 2023-12-11T18:03:55.260 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2023-12-11T18:03:55.260 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2023-12-11T18:03:55.260 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2023-12-11T18:03:55.260 INFO:teuthology.orchestra.run.smithi059.stdout: "agent_down_multiplier": { 2023-12-11T18:03:55.260 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "agent_down_multiplier", 2023-12-11T18:03:55.260 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "float", 2023-12-11T18:03:55.260 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.260 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.260 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "3.0", 2023-12-11T18:03:55.260 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.261 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.261 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.261 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-12-11T18:03:55.261 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.261 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.261 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.261 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.261 INFO:teuthology.orchestra.run.smithi059.stdout: "agent_refresh_rate": { 2023-12-11T18:03:55.261 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "agent_refresh_rate", 2023-12-11T18:03:55.261 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2023-12-11T18:03:55.261 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.261 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.261 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "20", 2023-12-11T18:03:55.261 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.261 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.261 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.262 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-12-11T18:03:55.262 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.262 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.262 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.262 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.262 INFO:teuthology.orchestra.run.smithi059.stdout: "agent_starting_port": { 2023-12-11T18:03:55.262 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "agent_starting_port", 2023-12-11T18:03:55.262 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2023-12-11T18:03:55.262 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.262 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.262 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "4721", 2023-12-11T18:03:55.262 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.262 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.262 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.263 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-12-11T18:03:55.263 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.263 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.263 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.263 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.263 INFO:teuthology.orchestra.run.smithi059.stdout: "allow_ptrace": { 2023-12-11T18:03:55.263 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "allow_ptrace", 2023-12-11T18:03:55.263 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.263 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.263 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.263 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.263 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.263 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.263 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.264 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-12-11T18:03:55.264 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-12-11T18:03:55.264 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.264 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.264 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.264 INFO:teuthology.orchestra.run.smithi059.stdout: "autotune_interval": { 2023-12-11T18:03:55.264 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "autotune_interval", 2023-12-11T18:03:55.264 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2023-12-11T18:03:55.264 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.264 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.264 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "600", 2023-12-11T18:03:55.264 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.264 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.264 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.264 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "how frequently to autotune daemon memory", 2023-12-11T18:03:55.264 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.265 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.265 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.265 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.265 INFO:teuthology.orchestra.run.smithi059.stdout: "autotune_memory_target_ratio": { 2023-12-11T18:03:55.265 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "autotune_memory_target_ratio", 2023-12-11T18:03:55.265 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "float", 2023-12-11T18:03:55.265 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.265 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.265 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0.7", 2023-12-11T18:03:55.265 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.265 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.265 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.265 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-12-11T18:03:55.265 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.265 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.265 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.266 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.266 INFO:teuthology.orchestra.run.smithi059.stdout: "cephadm_log_destination": { 2023-12-11T18:03:55.266 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "cephadm_log_destination", 2023-12-11T18:03:55.266 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.266 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.266 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.266 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.266 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.266 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.266 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:55.266 INFO:teuthology.orchestra.run.smithi059.stdout: "file", 2023-12-11T18:03:55.266 INFO:teuthology.orchestra.run.smithi059.stdout: "file,syslog", 2023-12-11T18:03:55.266 INFO:teuthology.orchestra.run.smithi059.stdout: "syslog" 2023-12-11T18:03:55.266 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:55.266 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Destination for cephadm command's persistent logging", 2023-12-11T18:03:55.267 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.267 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.267 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.267 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.267 INFO:teuthology.orchestra.run.smithi059.stdout: "cgroups_split": { 2023-12-11T18:03:55.267 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "cgroups_split", 2023-12-11T18:03:55.267 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.267 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.267 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.267 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2023-12-11T18:03:55.267 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.267 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.267 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.267 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-12-11T18:03:55.267 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.267 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.268 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.268 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.268 INFO:teuthology.orchestra.run.smithi059.stdout: "config_checks_enabled": { 2023-12-11T18:03:55.268 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "config_checks_enabled", 2023-12-11T18:03:55.268 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.268 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.268 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.268 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.268 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.268 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.268 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.268 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-12-11T18:03:55.269 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.269 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.269 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.269 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.269 INFO:teuthology.orchestra.run.smithi059.stdout: "config_dashboard": { 2023-12-11T18:03:55.269 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "config_dashboard", 2023-12-11T18:03:55.269 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.269 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.269 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.269 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2023-12-11T18:03:55.270 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.270 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.270 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.270 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-12-11T18:03:55.270 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.270 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.270 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.270 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.270 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_alertmanager": { 2023-12-11T18:03:55.270 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_alertmanager", 2023-12-11T18:03:55.270 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.270 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.270 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.270 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-12-11T18:03:55.271 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.271 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.271 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.271 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Prometheus container image", 2023-12-11T18:03:55.271 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.271 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.271 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.271 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.271 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_base": { 2023-12-11T18:03:55.271 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_base", 2023-12-11T18:03:55.271 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.271 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.271 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.272 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/ceph/ceph", 2023-12-11T18:03:55.272 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.272 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.272 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.272 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Container image name, without the tag", 2023-12-11T18:03:55.272 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.272 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.272 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.272 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.272 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_elasticsearch": { 2023-12-11T18:03:55.272 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_elasticsearch", 2023-12-11T18:03:55.272 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.272 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.272 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.273 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-12-11T18:03:55.273 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.273 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.273 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.273 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "elasticsearch container image", 2023-12-11T18:03:55.273 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.273 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.273 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.273 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.273 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_grafana": { 2023-12-11T18:03:55.273 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_grafana", 2023-12-11T18:03:55.273 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.273 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.273 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.273 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-12-11T18:03:55.273 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.274 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.274 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.274 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Prometheus container image", 2023-12-11T18:03:55.274 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.274 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.274 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.274 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.274 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_haproxy": { 2023-12-11T18:03:55.274 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_haproxy", 2023-12-11T18:03:55.274 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.274 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.274 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.274 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-12-11T18:03:55.274 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.275 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.275 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.275 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "HAproxy container image", 2023-12-11T18:03:55.275 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.275 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.275 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.275 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.275 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_jaeger_agent": { 2023-12-11T18:03:55.275 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_jaeger_agent", 2023-12-11T18:03:55.275 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.275 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.276 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.276 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-12-11T18:03:55.276 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.276 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.276 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.276 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Jaeger agent container image", 2023-12-11T18:03:55.276 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.276 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.276 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.276 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.276 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_jaeger_collector": { 2023-12-11T18:03:55.276 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_jaeger_collector", 2023-12-11T18:03:55.276 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.276 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.276 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.276 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-12-11T18:03:55.277 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.277 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.277 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.277 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Jaeger collector container image", 2023-12-11T18:03:55.277 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.277 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.277 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.277 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.277 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_jaeger_query": { 2023-12-11T18:03:55.277 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_jaeger_query", 2023-12-11T18:03:55.277 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.277 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.277 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.277 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-12-11T18:03:55.277 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.277 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.278 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.278 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Jaeger query container image", 2023-12-11T18:03:55.278 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.278 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.278 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.278 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.278 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_keepalived": { 2023-12-11T18:03:55.278 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_keepalived", 2023-12-11T18:03:55.278 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.278 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.278 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.278 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-12-11T18:03:55.278 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.278 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.279 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.279 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Keepalived container image", 2023-12-11T18:03:55.279 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.279 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.279 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.279 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.279 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_loki": { 2023-12-11T18:03:55.279 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_loki", 2023-12-11T18:03:55.281 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.281 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.281 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.281 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-12-11T18:03:55.281 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.282 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.282 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.282 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Loki container image", 2023-12-11T18:03:55.282 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.282 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.282 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.282 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.282 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_node_exporter": { 2023-12-11T18:03:55.282 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_node_exporter", 2023-12-11T18:03:55.282 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.282 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.282 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.282 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-12-11T18:03:55.283 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.283 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.283 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.283 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Prometheus container image", 2023-12-11T18:03:55.283 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.283 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.283 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.283 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.283 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_nvmeof": { 2023-12-11T18:03:55.283 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_nvmeof", 2023-12-11T18:03:55.283 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.283 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.283 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.283 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/ceph/nvmeof:latest", 2023-12-11T18:03:55.283 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.284 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.284 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.284 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Nvme-of container image", 2023-12-11T18:03:55.284 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.284 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.284 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.284 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.284 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_prometheus": { 2023-12-11T18:03:55.284 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_prometheus", 2023-12-11T18:03:55.284 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.284 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.284 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.284 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-12-11T18:03:55.285 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.285 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.285 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.285 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Prometheus container image", 2023-12-11T18:03:55.285 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.285 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.285 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.285 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.285 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_promtail": { 2023-12-11T18:03:55.285 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_promtail", 2023-12-11T18:03:55.285 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.285 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.285 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.285 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-12-11T18:03:55.286 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.286 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.286 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.286 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Promtail container image", 2023-12-11T18:03:55.286 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.286 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.286 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.286 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.286 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_snmp_gateway": { 2023-12-11T18:03:55.286 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_snmp_gateway", 2023-12-11T18:03:55.286 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.286 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.286 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.287 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-12-11T18:03:55.287 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.287 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.287 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.287 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "SNMP Gateway container image", 2023-12-11T18:03:55.287 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.287 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.287 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.287 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.287 INFO:teuthology.orchestra.run.smithi059.stdout: "container_init": { 2023-12-11T18:03:55.287 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_init", 2023-12-11T18:03:55.287 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.287 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.287 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.287 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2023-12-11T18:03:55.288 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.288 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.288 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.288 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Run podman/docker with `--init`", 2023-12-11T18:03:55.288 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.288 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.288 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.288 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.288 INFO:teuthology.orchestra.run.smithi059.stdout: "daemon_cache_timeout": { 2023-12-11T18:03:55.288 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "daemon_cache_timeout", 2023-12-11T18:03:55.288 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2023-12-11T18:03:55.288 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.288 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.288 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "600", 2023-12-11T18:03:55.288 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.288 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.289 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.289 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-12-11T18:03:55.289 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.289 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.289 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.289 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.289 INFO:teuthology.orchestra.run.smithi059.stdout: "default_cephadm_command_timeout": { 2023-12-11T18:03:55.289 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "default_cephadm_command_timeout", 2023-12-11T18:03:55.289 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2023-12-11T18:03:55.289 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.289 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.289 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "900", 2023-12-11T18:03:55.289 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.289 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.290 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.290 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-12-11T18:03:55.290 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.290 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.290 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.290 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.290 INFO:teuthology.orchestra.run.smithi059.stdout: "default_registry": { 2023-12-11T18:03:55.290 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "default_registry", 2023-12-11T18:03:55.290 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.290 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.290 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.290 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "docker.io", 2023-12-11T18:03:55.290 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.290 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.290 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.290 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-12-11T18:03:55.291 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.291 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.291 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.291 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.291 INFO:teuthology.orchestra.run.smithi059.stdout: "device_cache_timeout": { 2023-12-11T18:03:55.291 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "device_cache_timeout", 2023-12-11T18:03:55.291 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2023-12-11T18:03:55.291 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.291 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.291 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "1800", 2023-12-11T18:03:55.291 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.291 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.291 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.292 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "seconds to cache device inventory", 2023-12-11T18:03:55.292 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.292 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.292 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.292 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.292 INFO:teuthology.orchestra.run.smithi059.stdout: "device_enhanced_scan": { 2023-12-11T18:03:55.292 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "device_enhanced_scan", 2023-12-11T18:03:55.292 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.292 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.292 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.292 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.292 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.293 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.293 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.293 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Use libstoragemgmt during device scans", 2023-12-11T18:03:55.293 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.293 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.293 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.293 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.293 INFO:teuthology.orchestra.run.smithi059.stdout: "facts_cache_timeout": { 2023-12-11T18:03:55.293 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "facts_cache_timeout", 2023-12-11T18:03:55.293 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2023-12-11T18:03:55.293 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.293 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.293 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "60", 2023-12-11T18:03:55.293 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.293 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.294 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.294 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "seconds to cache host facts data", 2023-12-11T18:03:55.294 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.294 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.294 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.294 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.294 INFO:teuthology.orchestra.run.smithi059.stdout: "host_check_interval": { 2023-12-11T18:03:55.294 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "host_check_interval", 2023-12-11T18:03:55.294 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2023-12-11T18:03:55.294 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.294 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.294 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "600", 2023-12-11T18:03:55.294 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.294 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.294 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.295 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "how frequently to perform a host check", 2023-12-11T18:03:55.295 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.295 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.295 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.295 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.295 INFO:teuthology.orchestra.run.smithi059.stdout: "inventory_list_all": { 2023-12-11T18:03:55.295 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "inventory_list_all", 2023-12-11T18:03:55.295 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.295 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.295 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.295 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.295 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.295 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.295 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.295 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2023-12-11T18:03:55.296 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.296 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.296 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.296 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.296 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2023-12-11T18:03:55.296 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2023-12-11T18:03:55.296 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.296 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.296 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.296 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.296 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.296 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.296 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:55.296 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:55.296 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:55.297 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:55.297 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:55.297 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:55.297 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:55.297 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:55.297 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.297 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.297 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.297 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.297 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.297 INFO:teuthology.orchestra.run.smithi059.stdout: "log_refresh_metadata": { 2023-12-11T18:03:55.297 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_refresh_metadata", 2023-12-11T18:03:55.297 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.297 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.297 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.298 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.298 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.298 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.298 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.298 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-12-11T18:03:55.298 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.298 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.298 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.298 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.298 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2023-12-11T18:03:55.298 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2023-12-11T18:03:55.298 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.298 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.299 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.299 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2023-12-11T18:03:55.299 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.299 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.299 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.299 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-12-11T18:03:55.299 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.299 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.299 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.299 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.299 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2023-12-11T18:03:55.299 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2023-12-11T18:03:55.299 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.299 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.299 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.299 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2023-12-11T18:03:55.299 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.300 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.300 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:55.300 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:55.300 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:55.300 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:55.300 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:55.300 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:55.300 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:55.300 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:55.300 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.300 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.300 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.300 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.300 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.300 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2023-12-11T18:03:55.300 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2023-12-11T18:03:55.301 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.301 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.301 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.301 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.301 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.301 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.301 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.301 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.301 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.301 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.301 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.301 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.301 INFO:teuthology.orchestra.run.smithi059.stdout: "manage_etc_ceph_ceph_conf": { 2023-12-11T18:03:55.301 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-12-11T18:03:55.302 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.302 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.302 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.302 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.302 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.302 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.302 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.302 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-12-11T18:03:55.302 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.302 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.303 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.303 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.303 INFO:teuthology.orchestra.run.smithi059.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-12-11T18:03:55.303 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-12-11T18:03:55.303 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.303 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.303 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.303 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "*", 2023-12-11T18:03:55.303 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.303 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.303 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.303 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-12-11T18:03:55.303 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.304 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.304 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.304 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.304 INFO:teuthology.orchestra.run.smithi059.stdout: "max_count_per_host": { 2023-12-11T18:03:55.304 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "max_count_per_host", 2023-12-11T18:03:55.304 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2023-12-11T18:03:55.304 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.304 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.304 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "10", 2023-12-11T18:03:55.304 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.304 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.304 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.304 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "max number of daemons per service per host", 2023-12-11T18:03:55.304 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.305 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.305 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.305 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.305 INFO:teuthology.orchestra.run.smithi059.stdout: "max_osd_draining_count": { 2023-12-11T18:03:55.305 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "max_osd_draining_count", 2023-12-11T18:03:55.305 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2023-12-11T18:03:55.305 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.305 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.305 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "10", 2023-12-11T18:03:55.305 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.305 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.305 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.306 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-12-11T18:03:55.306 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.306 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.306 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.306 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.306 INFO:teuthology.orchestra.run.smithi059.stdout: "migration_current": { 2023-12-11T18:03:55.306 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "migration_current", 2023-12-11T18:03:55.306 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2023-12-11T18:03:55.306 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.306 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.306 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.306 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.306 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.306 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.306 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "internal - do not modify", 2023-12-11T18:03:55.307 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.307 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.307 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.307 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.307 INFO:teuthology.orchestra.run.smithi059.stdout: "mode": { 2023-12-11T18:03:55.307 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "mode", 2023-12-11T18:03:55.307 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.307 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.307 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.307 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "root", 2023-12-11T18:03:55.307 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.307 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.307 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:55.307 INFO:teuthology.orchestra.run.smithi059.stdout: "cephadm-package", 2023-12-11T18:03:55.307 INFO:teuthology.orchestra.run.smithi059.stdout: "root" 2023-12-11T18:03:55.308 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:55.308 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "mode for remote execution of cephadm", 2023-12-11T18:03:55.308 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.308 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.308 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.308 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.308 INFO:teuthology.orchestra.run.smithi059.stdout: "prometheus_alerts_path": { 2023-12-11T18:03:55.308 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "prometheus_alerts_path", 2023-12-11T18:03:55.308 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.308 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.308 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.308 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-12-11T18:03:55.308 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.308 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.308 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.309 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-12-11T18:03:55.309 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.309 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.309 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.309 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.309 INFO:teuthology.orchestra.run.smithi059.stdout: "registry_insecure": { 2023-12-11T18:03:55.309 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "registry_insecure", 2023-12-11T18:03:55.309 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.309 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.309 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.309 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.310 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.310 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.310 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.310 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-12-11T18:03:55.310 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.310 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.310 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.310 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.310 INFO:teuthology.orchestra.run.smithi059.stdout: "registry_password": { 2023-12-11T18:03:55.310 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "registry_password", 2023-12-11T18:03:55.310 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.310 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.310 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.310 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.310 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.310 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.311 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.311 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-12-11T18:03:55.311 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.311 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.311 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.311 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.311 INFO:teuthology.orchestra.run.smithi059.stdout: "registry_url": { 2023-12-11T18:03:55.311 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "registry_url", 2023-12-11T18:03:55.311 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.311 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.311 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.311 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.311 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.311 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.311 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.311 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-12-11T18:03:55.312 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.312 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.312 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.312 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.312 INFO:teuthology.orchestra.run.smithi059.stdout: "registry_username": { 2023-12-11T18:03:55.312 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "registry_username", 2023-12-11T18:03:55.312 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.312 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.312 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.312 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.312 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.312 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.312 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.313 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-12-11T18:03:55.313 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.313 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.313 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.313 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.313 INFO:teuthology.orchestra.run.smithi059.stdout: "secure_monitoring_stack": { 2023-12-11T18:03:55.313 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "secure_monitoring_stack", 2023-12-11T18:03:55.313 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.313 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.313 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.313 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.313 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.314 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.314 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.314 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-12-11T18:03:55.314 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.314 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.314 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.314 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.314 INFO:teuthology.orchestra.run.smithi059.stdout: "service_discovery_port": { 2023-12-11T18:03:55.314 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "service_discovery_port", 2023-12-11T18:03:55.314 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2023-12-11T18:03:55.314 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.314 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.314 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "8765", 2023-12-11T18:03:55.314 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.315 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.315 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.315 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "cephadm service discovery port", 2023-12-11T18:03:55.315 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.315 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.315 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.315 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.315 INFO:teuthology.orchestra.run.smithi059.stdout: "ssh_config_file": { 2023-12-11T18:03:55.315 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ssh_config_file", 2023-12-11T18:03:55.315 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.315 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.315 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.315 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.315 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.315 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.315 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.316 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-12-11T18:03:55.316 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.316 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.316 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.316 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.316 INFO:teuthology.orchestra.run.smithi059.stdout: "use_agent": { 2023-12-11T18:03:55.316 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "use_agent", 2023-12-11T18:03:55.316 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.316 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.316 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.316 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.316 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.316 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.316 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.316 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-12-11T18:03:55.316 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.317 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.317 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.317 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.317 INFO:teuthology.orchestra.run.smithi059.stdout: "use_repo_digest": { 2023-12-11T18:03:55.317 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "use_repo_digest", 2023-12-11T18:03:55.317 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.317 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.317 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.317 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2023-12-11T18:03:55.317 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.317 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.317 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.317 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-12-11T18:03:55.317 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.317 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.318 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.318 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.318 INFO:teuthology.orchestra.run.smithi059.stdout: "warn_on_failed_host_check": { 2023-12-11T18:03:55.318 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "warn_on_failed_host_check", 2023-12-11T18:03:55.318 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.318 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.318 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.318 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2023-12-11T18:03:55.318 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.318 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.318 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.318 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "raise a health warning if the host check fails", 2023-12-11T18:03:55.318 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.318 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.319 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.319 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.319 INFO:teuthology.orchestra.run.smithi059.stdout: "warn_on_stray_daemons": { 2023-12-11T18:03:55.319 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "warn_on_stray_daemons", 2023-12-11T18:03:55.319 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.319 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.319 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.319 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2023-12-11T18:03:55.319 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.319 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.319 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.319 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-12-11T18:03:55.319 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.319 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.319 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.319 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.320 INFO:teuthology.orchestra.run.smithi059.stdout: "warn_on_stray_hosts": { 2023-12-11T18:03:55.320 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "warn_on_stray_hosts", 2023-12-11T18:03:55.320 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.320 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.320 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.320 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2023-12-11T18:03:55.320 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.320 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.320 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.320 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-12-11T18:03:55.320 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.320 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.321 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.321 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:55.321 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:55.321 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.321 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-12-11T18:03:55.321 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "crash", 2023-12-11T18:03:55.321 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2023-12-11T18:03:55.321 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2023-12-11T18:03:55.321 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2023-12-11T18:03:55.321 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2023-12-11T18:03:55.321 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2023-12-11T18:03:55.321 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.321 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.321 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.321 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.321 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.322 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.322 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:55.322 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:55.322 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:55.322 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:55.322 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:55.322 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:55.322 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:55.322 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:55.322 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.322 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.322 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.322 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.322 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.323 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2023-12-11T18:03:55.323 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2023-12-11T18:03:55.323 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.323 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.323 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.323 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.323 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.323 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.323 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.323 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.323 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.323 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.324 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.324 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.324 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2023-12-11T18:03:55.324 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2023-12-11T18:03:55.324 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.324 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.324 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.324 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2023-12-11T18:03:55.324 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.324 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.324 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:55.324 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:55.324 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:55.324 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:55.324 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:55.325 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:55.325 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:55.325 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:55.325 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.325 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.325 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.325 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.325 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.325 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2023-12-11T18:03:55.325 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2023-12-11T18:03:55.325 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.325 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.325 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.326 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.326 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.326 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.326 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.326 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.326 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.326 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.326 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.326 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.326 INFO:teuthology.orchestra.run.smithi059.stdout: "retain_interval": { 2023-12-11T18:03:55.326 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "retain_interval", 2023-12-11T18:03:55.326 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2023-12-11T18:03:55.326 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.326 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.326 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "31536000", 2023-12-11T18:03:55.327 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.327 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.327 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.327 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "how long to retain crashes before pruning them", 2023-12-11T18:03:55.327 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.327 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.327 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.327 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.327 INFO:teuthology.orchestra.run.smithi059.stdout: "warn_recent_interval": { 2023-12-11T18:03:55.327 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "warn_recent_interval", 2023-12-11T18:03:55.327 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2023-12-11T18:03:55.327 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.327 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.327 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "1209600", 2023-12-11T18:03:55.327 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.328 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.328 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.328 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "time interval in which to warn about recent crashes", 2023-12-11T18:03:55.328 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.328 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.328 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.328 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:55.328 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:55.328 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.328 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-12-11T18:03:55.328 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "dashboard", 2023-12-11T18:03:55.328 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2023-12-11T18:03:55.328 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2023-12-11T18:03:55.328 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2023-12-11T18:03:55.328 INFO:teuthology.orchestra.run.smithi059.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-12-11T18:03:55.329 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-12-11T18:03:55.329 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2023-12-11T18:03:55.329 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.329 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.329 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "10", 2023-12-11T18:03:55.329 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.329 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.329 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.329 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.329 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.329 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.329 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.329 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.329 INFO:teuthology.orchestra.run.smithi059.stdout: "ALERTMANAGER_API_HOST": { 2023-12-11T18:03:55.329 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ALERTMANAGER_API_HOST", 2023-12-11T18:03:55.329 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.330 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.330 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.330 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.330 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.330 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.330 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.330 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.330 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.330 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.330 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.330 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.330 INFO:teuthology.orchestra.run.smithi059.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-12-11T18:03:55.330 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-12-11T18:03:55.330 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.331 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.331 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.331 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2023-12-11T18:03:55.331 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.331 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.331 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.331 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.331 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.331 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.331 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.331 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.331 INFO:teuthology.orchestra.run.smithi059.stdout: "AUDIT_API_ENABLED": { 2023-12-11T18:03:55.331 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "AUDIT_API_ENABLED", 2023-12-11T18:03:55.331 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.331 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.332 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.332 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.332 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.332 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.332 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.332 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.332 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.332 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.332 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.332 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.332 INFO:teuthology.orchestra.run.smithi059.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-12-11T18:03:55.332 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-12-11T18:03:55.332 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.332 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.332 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.332 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2023-12-11T18:03:55.333 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.333 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.333 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.333 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.333 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.333 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.333 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.333 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.333 INFO:teuthology.orchestra.run.smithi059.stdout: "ENABLE_BROWSABLE_API": { 2023-12-11T18:03:55.333 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ENABLE_BROWSABLE_API", 2023-12-11T18:03:55.333 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.333 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.334 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.334 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2023-12-11T18:03:55.334 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.334 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.334 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.334 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.334 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.334 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.334 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.334 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.334 INFO:teuthology.orchestra.run.smithi059.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-12-11T18:03:55.334 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-12-11T18:03:55.334 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.334 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.335 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.335 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2023-12-11T18:03:55.335 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.335 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.335 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.335 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.335 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.335 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.335 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.335 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.335 INFO:teuthology.orchestra.run.smithi059.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-12-11T18:03:55.335 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-12-11T18:03:55.335 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.335 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.335 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.336 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2023-12-11T18:03:55.336 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.336 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.336 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.336 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.336 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.336 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.336 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.336 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.336 INFO:teuthology.orchestra.run.smithi059.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-12-11T18:03:55.336 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-12-11T18:03:55.336 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.336 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.336 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.336 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2023-12-11T18:03:55.336 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.337 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.337 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.337 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.337 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.337 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.337 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.337 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.337 INFO:teuthology.orchestra.run.smithi059.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-12-11T18:03:55.337 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-12-11T18:03:55.337 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.337 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.337 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.337 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2023-12-11T18:03:55.337 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.337 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.338 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.338 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.338 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.338 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.338 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.338 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.338 INFO:teuthology.orchestra.run.smithi059.stdout: "FEATURE_TOGGLE_NFS": { 2023-12-11T18:03:55.338 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-12-11T18:03:55.338 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.338 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.338 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.338 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2023-12-11T18:03:55.338 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.338 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.339 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.339 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.339 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.339 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.339 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.339 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.339 INFO:teuthology.orchestra.run.smithi059.stdout: "FEATURE_TOGGLE_RBD": { 2023-12-11T18:03:55.339 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-12-11T18:03:55.339 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.339 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.339 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.339 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2023-12-11T18:03:55.339 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.339 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.339 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.339 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.340 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.340 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.340 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.340 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.340 INFO:teuthology.orchestra.run.smithi059.stdout: "FEATURE_TOGGLE_RGW": { 2023-12-11T18:03:55.340 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-12-11T18:03:55.340 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.340 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.340 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.340 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2023-12-11T18:03:55.340 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.340 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.340 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.340 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.340 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.340 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.340 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.341 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.341 INFO:teuthology.orchestra.run.smithi059.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-12-11T18:03:55.341 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-12-11T18:03:55.341 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.341 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.341 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.341 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.341 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.341 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.341 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.341 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.341 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.341 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.341 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.341 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.342 INFO:teuthology.orchestra.run.smithi059.stdout: "GRAFANA_API_PASSWORD": { 2023-12-11T18:03:55.342 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "GRAFANA_API_PASSWORD", 2023-12-11T18:03:55.342 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.342 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.342 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.342 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "admin", 2023-12-11T18:03:55.342 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.342 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.342 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.342 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.342 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.342 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.342 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.343 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.343 INFO:teuthology.orchestra.run.smithi059.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-12-11T18:03:55.343 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-12-11T18:03:55.343 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.343 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.343 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.343 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2023-12-11T18:03:55.343 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.343 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.343 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.343 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.343 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.344 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.344 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.344 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.344 INFO:teuthology.orchestra.run.smithi059.stdout: "GRAFANA_API_URL": { 2023-12-11T18:03:55.344 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "GRAFANA_API_URL", 2023-12-11T18:03:55.344 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.344 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.344 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.344 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.344 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.344 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.345 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.345 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.345 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.345 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.345 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.345 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.345 INFO:teuthology.orchestra.run.smithi059.stdout: "GRAFANA_API_USERNAME": { 2023-12-11T18:03:55.345 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "GRAFANA_API_USERNAME", 2023-12-11T18:03:55.345 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.346 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.346 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.346 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "admin", 2023-12-11T18:03:55.346 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.346 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.346 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.347 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.347 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.347 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.347 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.347 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.347 INFO:teuthology.orchestra.run.smithi059.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-12-11T18:03:55.347 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-12-11T18:03:55.347 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.347 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.347 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.347 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.347 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.348 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.348 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.348 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.348 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.348 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.348 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.348 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.348 INFO:teuthology.orchestra.run.smithi059.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-12-11T18:03:55.348 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-12-11T18:03:55.348 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.348 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.348 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.348 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.348 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.349 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.349 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.349 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.349 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.349 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.349 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.349 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.349 INFO:teuthology.orchestra.run.smithi059.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-12-11T18:03:55.349 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-12-11T18:03:55.349 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.349 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.349 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.349 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2023-12-11T18:03:55.349 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.349 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.350 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.350 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.350 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.350 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.350 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.350 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.350 INFO:teuthology.orchestra.run.smithi059.stdout: "ISSUE_TRACKER_API_KEY": { 2023-12-11T18:03:55.350 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-12-11T18:03:55.350 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.350 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.350 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.350 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.350 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.350 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.351 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.351 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.351 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.351 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.351 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.351 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.351 INFO:teuthology.orchestra.run.smithi059.stdout: "PROMETHEUS_API_HOST": { 2023-12-11T18:03:55.351 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PROMETHEUS_API_HOST", 2023-12-11T18:03:55.351 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.351 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.351 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.351 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.351 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.351 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.351 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.351 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.352 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.352 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.352 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.352 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.352 INFO:teuthology.orchestra.run.smithi059.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-12-11T18:03:55.352 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-12-11T18:03:55.352 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.352 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.352 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.352 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2023-12-11T18:03:55.352 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.353 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.353 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.353 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.353 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.353 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.353 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.353 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.353 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-12-11T18:03:55.353 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-12-11T18:03:55.353 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.353 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.353 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.353 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.354 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.354 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.354 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.354 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.354 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.354 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.354 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.354 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.354 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-12-11T18:03:55.355 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-12-11T18:03:55.355 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.355 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.355 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.355 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.355 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.355 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.355 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.355 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.355 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.355 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.355 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.355 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.355 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-12-11T18:03:55.356 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-12-11T18:03:55.356 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.356 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.356 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.356 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2023-12-11T18:03:55.356 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.356 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.356 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.356 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.356 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.356 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.356 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.356 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.357 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-12-11T18:03:55.357 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-12-11T18:03:55.357 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.357 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.357 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.357 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2023-12-11T18:03:55.357 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.357 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.357 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.357 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.357 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.357 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.357 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.357 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.357 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-12-11T18:03:55.358 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-12-11T18:03:55.358 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.358 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.358 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.358 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.358 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.358 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.358 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.358 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.358 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.358 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.358 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.358 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.358 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-12-11T18:03:55.359 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-12-11T18:03:55.359 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.359 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.359 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.359 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.359 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.359 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.359 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.359 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.359 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.359 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.359 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.359 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.359 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-12-11T18:03:55.360 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-12-11T18:03:55.360 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.360 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.360 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.360 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.360 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.360 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.360 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.360 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.360 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.360 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.360 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.360 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.360 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_ENABLED": { 2023-12-11T18:03:55.361 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_ENABLED", 2023-12-11T18:03:55.361 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.361 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.361 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.361 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2023-12-11T18:03:55.361 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.361 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.361 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.361 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.361 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.361 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.361 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.361 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.361 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-12-11T18:03:55.361 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-12-11T18:03:55.362 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.362 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.362 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.362 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-12-11T18:03:55.362 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.362 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.362 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.362 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.362 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.362 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.362 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.362 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.362 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-12-11T18:03:55.363 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-12-11T18:03:55.363 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2023-12-11T18:03:55.363 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.363 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.363 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "10", 2023-12-11T18:03:55.363 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.363 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.363 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.363 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.363 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.363 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.363 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.363 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.363 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-12-11T18:03:55.364 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-12-11T18:03:55.364 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2023-12-11T18:03:55.364 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.364 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.364 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "8", 2023-12-11T18:03:55.364 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.364 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.364 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.364 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.364 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.364 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.364 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.364 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.364 INFO:teuthology.orchestra.run.smithi059.stdout: "REST_REQUESTS_TIMEOUT": { 2023-12-11T18:03:55.364 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-12-11T18:03:55.365 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2023-12-11T18:03:55.365 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.365 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.365 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "45", 2023-12-11T18:03:55.365 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.365 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.365 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.365 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.365 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.365 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.365 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.366 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.366 INFO:teuthology.orchestra.run.smithi059.stdout: "RGW_API_ACCESS_KEY": { 2023-12-11T18:03:55.366 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "RGW_API_ACCESS_KEY", 2023-12-11T18:03:55.366 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.366 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.366 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.366 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.366 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.366 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.366 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.366 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.367 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.367 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.367 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.367 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.367 INFO:teuthology.orchestra.run.smithi059.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-12-11T18:03:55.367 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-12-11T18:03:55.367 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.367 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.367 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.367 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "admin", 2023-12-11T18:03:55.367 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.367 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.367 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.367 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.367 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.368 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.368 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.368 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.368 INFO:teuthology.orchestra.run.smithi059.stdout: "RGW_API_SECRET_KEY": { 2023-12-11T18:03:55.368 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "RGW_API_SECRET_KEY", 2023-12-11T18:03:55.368 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.368 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.368 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.368 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.368 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.368 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.368 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.368 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.368 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.368 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.369 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.369 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.369 INFO:teuthology.orchestra.run.smithi059.stdout: "RGW_API_SSL_VERIFY": { 2023-12-11T18:03:55.369 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "RGW_API_SSL_VERIFY", 2023-12-11T18:03:55.369 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.369 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.369 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.369 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2023-12-11T18:03:55.369 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.369 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.369 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.369 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.370 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.370 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.370 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.370 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.370 INFO:teuthology.orchestra.run.smithi059.stdout: "UNSAFE_TLS_v1_2": { 2023-12-11T18:03:55.370 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "UNSAFE_TLS_v1_2", 2023-12-11T18:03:55.370 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.370 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.370 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.370 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.370 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.370 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.370 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.371 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.371 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.371 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.371 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.371 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.371 INFO:teuthology.orchestra.run.smithi059.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-12-11T18:03:55.371 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-12-11T18:03:55.371 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2023-12-11T18:03:55.371 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.371 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.371 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2023-12-11T18:03:55.371 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.371 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.372 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.372 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.372 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.372 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.372 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.372 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.372 INFO:teuthology.orchestra.run.smithi059.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-12-11T18:03:55.372 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-12-11T18:03:55.372 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2023-12-11T18:03:55.372 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.372 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.372 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "10", 2023-12-11T18:03:55.372 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.372 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.373 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.373 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.373 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.373 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.373 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.373 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.373 INFO:teuthology.orchestra.run.smithi059.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-12-11T18:03:55.373 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-12-11T18:03:55.373 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2023-12-11T18:03:55.373 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.373 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.373 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "5", 2023-12-11T18:03:55.373 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.373 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.373 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.374 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.374 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.374 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.374 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.374 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.374 INFO:teuthology.orchestra.run.smithi059.stdout: "cross_origin_url": { 2023-12-11T18:03:55.374 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "cross_origin_url", 2023-12-11T18:03:55.374 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.374 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.374 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.374 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.374 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.374 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.375 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.375 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.375 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.375 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.375 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.375 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.375 INFO:teuthology.orchestra.run.smithi059.stdout: "crt_file": { 2023-12-11T18:03:55.375 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "crt_file", 2023-12-11T18:03:55.375 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.375 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.375 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.376 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.376 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.376 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.376 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.376 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.376 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.376 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.376 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.376 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.376 INFO:teuthology.orchestra.run.smithi059.stdout: "debug": { 2023-12-11T18:03:55.376 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "debug", 2023-12-11T18:03:55.376 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.376 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.376 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.376 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.377 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.377 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.377 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.377 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Enable/disable debug options", 2023-12-11T18:03:55.377 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.377 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.377 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.377 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.377 INFO:teuthology.orchestra.run.smithi059.stdout: "jwt_token_ttl": { 2023-12-11T18:03:55.377 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "jwt_token_ttl", 2023-12-11T18:03:55.377 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2023-12-11T18:03:55.377 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.377 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.377 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "28800", 2023-12-11T18:03:55.377 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.377 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.377 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.378 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.378 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.378 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.378 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.378 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.378 INFO:teuthology.orchestra.run.smithi059.stdout: "key_file": { 2023-12-11T18:03:55.378 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "key_file", 2023-12-11T18:03:55.378 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.378 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.378 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.378 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.378 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.378 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.378 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.379 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.379 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.379 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.379 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.379 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.379 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2023-12-11T18:03:55.379 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2023-12-11T18:03:55.379 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.379 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.379 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.379 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.379 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.379 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.379 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:55.379 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:55.379 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:55.380 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:55.380 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:55.380 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:55.380 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:55.380 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:55.380 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.380 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.380 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.380 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.380 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.380 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2023-12-11T18:03:55.380 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2023-12-11T18:03:55.381 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.381 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.381 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.381 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.381 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.381 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.381 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.381 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.381 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.381 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.381 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.381 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.381 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2023-12-11T18:03:55.381 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2023-12-11T18:03:55.381 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.381 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.382 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.382 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2023-12-11T18:03:55.382 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.382 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.382 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:55.382 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:55.382 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:55.382 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:55.382 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:55.382 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:55.382 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:55.382 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:55.383 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.383 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.383 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.383 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.383 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.383 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2023-12-11T18:03:55.383 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2023-12-11T18:03:55.383 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.383 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.383 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.383 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.383 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.383 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.383 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.383 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.384 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.384 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.384 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.384 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.384 INFO:teuthology.orchestra.run.smithi059.stdout: "motd": { 2023-12-11T18:03:55.384 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "motd", 2023-12-11T18:03:55.384 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.384 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.384 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.384 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.384 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.384 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.384 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.384 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "The message of the day", 2023-12-11T18:03:55.384 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.385 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.385 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.385 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.385 INFO:teuthology.orchestra.run.smithi059.stdout: "redirect_resolve_ip_addr": { 2023-12-11T18:03:55.385 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "redirect_resolve_ip_addr", 2023-12-11T18:03:55.385 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.385 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.385 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.385 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.385 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.385 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.385 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.385 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.385 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.385 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.385 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.386 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.386 INFO:teuthology.orchestra.run.smithi059.stdout: "server_addr": { 2023-12-11T18:03:55.386 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "server_addr", 2023-12-11T18:03:55.386 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.386 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.386 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.386 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "::", 2023-12-11T18:03:55.386 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.386 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.386 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.387 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.387 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.387 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.387 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.387 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.387 INFO:teuthology.orchestra.run.smithi059.stdout: "server_port": { 2023-12-11T18:03:55.387 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "server_port", 2023-12-11T18:03:55.387 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2023-12-11T18:03:55.387 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.387 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.387 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "8080", 2023-12-11T18:03:55.387 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.387 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.387 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.387 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.388 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.388 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.388 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.388 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.388 INFO:teuthology.orchestra.run.smithi059.stdout: "ssl": { 2023-12-11T18:03:55.388 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ssl", 2023-12-11T18:03:55.388 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.388 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.388 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.388 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2023-12-11T18:03:55.388 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.388 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.388 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.388 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.388 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.389 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.389 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.389 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.389 INFO:teuthology.orchestra.run.smithi059.stdout: "ssl_server_port": { 2023-12-11T18:03:55.389 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ssl_server_port", 2023-12-11T18:03:55.389 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2023-12-11T18:03:55.389 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.389 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.389 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "8443", 2023-12-11T18:03:55.389 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.389 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.389 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.389 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.389 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.389 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.390 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.390 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.390 INFO:teuthology.orchestra.run.smithi059.stdout: "standby_behaviour": { 2023-12-11T18:03:55.390 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "standby_behaviour", 2023-12-11T18:03:55.390 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.390 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.390 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.390 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "redirect", 2023-12-11T18:03:55.390 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.390 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.390 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:55.390 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:55.390 INFO:teuthology.orchestra.run.smithi059.stdout: "redirect" 2023-12-11T18:03:55.390 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:55.391 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.391 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.391 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.391 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.391 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.391 INFO:teuthology.orchestra.run.smithi059.stdout: "standby_error_status_code": { 2023-12-11T18:03:55.391 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "standby_error_status_code", 2023-12-11T18:03:55.391 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2023-12-11T18:03:55.391 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.391 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.391 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "500", 2023-12-11T18:03:55.391 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "400", 2023-12-11T18:03:55.391 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "599", 2023-12-11T18:03:55.391 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.391 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.392 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.392 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.392 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.392 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.392 INFO:teuthology.orchestra.run.smithi059.stdout: "url_prefix": { 2023-12-11T18:03:55.392 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "url_prefix", 2023-12-11T18:03:55.392 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.392 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.392 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.392 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.392 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.392 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.392 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.392 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.393 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.393 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.393 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.393 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:55.393 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:55.393 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.393 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-12-11T18:03:55.393 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "devicehealth", 2023-12-11T18:03:55.393 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2023-12-11T18:03:55.393 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2023-12-11T18:03:55.393 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2023-12-11T18:03:55.393 INFO:teuthology.orchestra.run.smithi059.stdout: "enable_monitoring": { 2023-12-11T18:03:55.393 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "enable_monitoring", 2023-12-11T18:03:55.393 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.393 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.394 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.394 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2023-12-11T18:03:55.394 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.394 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.394 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.394 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "monitor device health metrics", 2023-12-11T18:03:55.394 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.394 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.394 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.394 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.394 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2023-12-11T18:03:55.395 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2023-12-11T18:03:55.395 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.395 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.395 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.395 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.395 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.395 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.395 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:55.395 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:55.395 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:55.395 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:55.395 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:55.396 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:55.396 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:55.396 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:55.396 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.396 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.396 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.396 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.396 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.396 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2023-12-11T18:03:55.396 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2023-12-11T18:03:55.397 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.397 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.397 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.397 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.397 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.397 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.397 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.397 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.397 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.397 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.397 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.397 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.397 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2023-12-11T18:03:55.397 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2023-12-11T18:03:55.398 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.398 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.398 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.398 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2023-12-11T18:03:55.398 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.398 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.398 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:55.398 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:55.398 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:55.398 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:55.398 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:55.398 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:55.398 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:55.399 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:55.399 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.399 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.399 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.399 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.399 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.399 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2023-12-11T18:03:55.399 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2023-12-11T18:03:55.399 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.399 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.399 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.399 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.399 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.399 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.399 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.400 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.400 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.400 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.400 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.400 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.400 INFO:teuthology.orchestra.run.smithi059.stdout: "mark_out_threshold": { 2023-12-11T18:03:55.400 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "mark_out_threshold", 2023-12-11T18:03:55.400 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2023-12-11T18:03:55.400 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.400 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.400 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "2419200", 2023-12-11T18:03:55.400 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.400 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.400 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.400 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-12-11T18:03:55.401 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.401 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.401 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.401 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.401 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_name": { 2023-12-11T18:03:55.401 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "pool_name", 2023-12-11T18:03:55.401 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.401 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.401 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.401 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "device_health_metrics", 2023-12-11T18:03:55.401 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.401 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.401 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.402 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "name of pool in which to store device health metrics", 2023-12-11T18:03:55.402 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.402 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.402 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.402 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.402 INFO:teuthology.orchestra.run.smithi059.stdout: "retention_period": { 2023-12-11T18:03:55.402 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "retention_period", 2023-12-11T18:03:55.402 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2023-12-11T18:03:55.402 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.402 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.402 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "15552000", 2023-12-11T18:03:55.403 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.403 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.403 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.403 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "how long to retain device health metrics", 2023-12-11T18:03:55.403 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.403 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.403 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.403 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.403 INFO:teuthology.orchestra.run.smithi059.stdout: "scrape_frequency": { 2023-12-11T18:03:55.403 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "scrape_frequency", 2023-12-11T18:03:55.403 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2023-12-11T18:03:55.403 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.403 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.403 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "86400", 2023-12-11T18:03:55.404 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.404 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.404 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.404 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "how frequently to scrape device health metrics", 2023-12-11T18:03:55.404 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.404 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.404 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.404 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.404 INFO:teuthology.orchestra.run.smithi059.stdout: "self_heal": { 2023-12-11T18:03:55.404 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "self_heal", 2023-12-11T18:03:55.404 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.404 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.405 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.405 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2023-12-11T18:03:55.405 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.405 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.405 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.405 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-12-11T18:03:55.405 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.405 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.405 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.405 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.405 INFO:teuthology.orchestra.run.smithi059.stdout: "sleep_interval": { 2023-12-11T18:03:55.405 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sleep_interval", 2023-12-11T18:03:55.406 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2023-12-11T18:03:55.406 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.406 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.406 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "600", 2023-12-11T18:03:55.406 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.406 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.406 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.406 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "how frequently to wake up and check device health", 2023-12-11T18:03:55.406 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.406 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.407 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.407 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.407 INFO:teuthology.orchestra.run.smithi059.stdout: "warn_threshold": { 2023-12-11T18:03:55.407 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "warn_threshold", 2023-12-11T18:03:55.407 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2023-12-11T18:03:55.407 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.407 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.407 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "7257600", 2023-12-11T18:03:55.407 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.407 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.408 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.408 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-12-11T18:03:55.408 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.408 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.408 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.408 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:55.408 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:55.408 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.408 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-12-11T18:03:55.408 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "diskprediction_local", 2023-12-11T18:03:55.408 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2023-12-11T18:03:55.408 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2023-12-11T18:03:55.408 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2023-12-11T18:03:55.409 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2023-12-11T18:03:55.409 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2023-12-11T18:03:55.409 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.409 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.409 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.409 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.409 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.409 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.409 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:55.409 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:55.409 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:55.409 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:55.409 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:55.410 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:55.410 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:55.410 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:55.410 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.410 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.410 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.410 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.410 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.410 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2023-12-11T18:03:55.410 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2023-12-11T18:03:55.410 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.410 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.411 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.411 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.411 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.411 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.411 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.411 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.411 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.411 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.411 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.411 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.411 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2023-12-11T18:03:55.411 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2023-12-11T18:03:55.411 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.411 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.411 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.412 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2023-12-11T18:03:55.412 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.412 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.412 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:55.412 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:55.412 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:55.412 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:55.412 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:55.412 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:55.412 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:55.412 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:55.412 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.412 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.412 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.413 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.413 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.413 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2023-12-11T18:03:55.413 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2023-12-11T18:03:55.413 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.413 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.413 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.413 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.413 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.413 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.413 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.413 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.413 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.413 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.413 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.413 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.414 INFO:teuthology.orchestra.run.smithi059.stdout: "predict_interval": { 2023-12-11T18:03:55.414 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "predict_interval", 2023-12-11T18:03:55.414 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.414 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.414 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.414 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "86400", 2023-12-11T18:03:55.414 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.414 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.414 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.414 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.414 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.414 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.414 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.415 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.415 INFO:teuthology.orchestra.run.smithi059.stdout: "predictor_model": { 2023-12-11T18:03:55.415 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "predictor_model", 2023-12-11T18:03:55.415 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.415 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.415 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.415 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "prophetstor", 2023-12-11T18:03:55.415 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.415 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.415 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.415 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.415 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.416 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.416 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.416 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.416 INFO:teuthology.orchestra.run.smithi059.stdout: "sleep_interval": { 2023-12-11T18:03:55.416 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sleep_interval", 2023-12-11T18:03:55.416 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.416 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.416 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.416 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "600", 2023-12-11T18:03:55.416 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.416 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.416 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.416 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.416 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.417 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.417 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.417 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:55.417 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:55.417 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.417 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-12-11T18:03:55.417 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "influx", 2023-12-11T18:03:55.417 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": false, 2023-12-11T18:03:55.417 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "influxdb python module not found", 2023-12-11T18:03:55.417 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2023-12-11T18:03:55.417 INFO:teuthology.orchestra.run.smithi059.stdout: "batch_size": { 2023-12-11T18:03:55.417 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "batch_size", 2023-12-11T18:03:55.418 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2023-12-11T18:03:55.418 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.418 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.418 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "5000", 2023-12-11T18:03:55.418 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.418 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.418 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.418 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-12-11T18:03:55.418 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.418 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.419 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.419 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.419 INFO:teuthology.orchestra.run.smithi059.stdout: "database": { 2023-12-11T18:03:55.419 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "database", 2023-12-11T18:03:55.419 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.419 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.419 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.419 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "ceph", 2023-12-11T18:03:55.419 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.419 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.419 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.419 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-12-11T18:03:55.419 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.419 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.420 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.420 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.420 INFO:teuthology.orchestra.run.smithi059.stdout: "hostname": { 2023-12-11T18:03:55.420 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "hostname", 2023-12-11T18:03:55.420 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.420 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.420 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.420 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.420 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.420 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.420 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.420 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "InfluxDB server hostname", 2023-12-11T18:03:55.420 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.420 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.421 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.421 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.421 INFO:teuthology.orchestra.run.smithi059.stdout: "interval": { 2023-12-11T18:03:55.421 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "interval", 2023-12-11T18:03:55.421 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2023-12-11T18:03:55.421 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.421 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.421 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "30", 2023-12-11T18:03:55.421 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "5", 2023-12-11T18:03:55.421 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.421 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.421 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-12-11T18:03:55.422 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.422 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.422 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.422 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.422 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2023-12-11T18:03:55.422 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2023-12-11T18:03:55.422 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.422 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.422 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.422 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.422 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.423 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.423 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:55.423 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:55.423 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:55.423 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:55.423 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:55.423 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:55.423 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:55.423 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:55.423 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.423 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.423 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.423 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.423 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.424 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2023-12-11T18:03:55.424 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2023-12-11T18:03:55.424 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.424 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.424 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.424 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.424 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.424 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.424 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.424 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.424 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.424 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.424 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.424 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.424 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2023-12-11T18:03:55.425 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2023-12-11T18:03:55.425 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.425 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.425 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.425 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2023-12-11T18:03:55.425 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.425 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.425 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:55.425 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:55.425 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:55.425 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:55.425 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:55.425 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:55.425 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:55.426 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:55.426 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.426 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.426 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.426 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.426 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.426 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2023-12-11T18:03:55.426 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2023-12-11T18:03:55.426 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.426 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.426 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.427 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.427 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.427 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.427 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.427 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.427 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.427 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.427 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.427 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.427 INFO:teuthology.orchestra.run.smithi059.stdout: "password": { 2023-12-11T18:03:55.427 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "password", 2023-12-11T18:03:55.427 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.427 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.427 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.427 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.428 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.428 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.428 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.428 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "password of InfluxDB server user", 2023-12-11T18:03:55.428 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.428 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.428 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.428 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.428 INFO:teuthology.orchestra.run.smithi059.stdout: "port": { 2023-12-11T18:03:55.428 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "port", 2023-12-11T18:03:55.428 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2023-12-11T18:03:55.429 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.429 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.429 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "8086", 2023-12-11T18:03:55.429 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.429 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.429 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.429 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "InfluxDB server port", 2023-12-11T18:03:55.429 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.429 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.429 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.429 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.429 INFO:teuthology.orchestra.run.smithi059.stdout: "ssl": { 2023-12-11T18:03:55.429 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ssl", 2023-12-11T18:03:55.429 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.429 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.430 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.430 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "false", 2023-12-11T18:03:55.430 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.430 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.430 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.430 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-12-11T18:03:55.430 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.430 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.430 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.430 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.431 INFO:teuthology.orchestra.run.smithi059.stdout: "threads": { 2023-12-11T18:03:55.431 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "threads", 2023-12-11T18:03:55.431 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2023-12-11T18:03:55.431 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.431 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.431 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "5", 2023-12-11T18:03:55.431 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "1", 2023-12-11T18:03:55.431 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "32", 2023-12-11T18:03:55.431 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.431 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-12-11T18:03:55.431 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.431 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.431 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.431 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.431 INFO:teuthology.orchestra.run.smithi059.stdout: "username": { 2023-12-11T18:03:55.432 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "username", 2023-12-11T18:03:55.432 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.432 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.432 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.432 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.432 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.432 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.432 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.432 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "username of InfluxDB server user", 2023-12-11T18:03:55.432 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.432 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.432 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.432 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.432 INFO:teuthology.orchestra.run.smithi059.stdout: "verify_ssl": { 2023-12-11T18:03:55.433 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "verify_ssl", 2023-12-11T18:03:55.433 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.433 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.433 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.433 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "true", 2023-12-11T18:03:55.433 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.433 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.433 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.433 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-12-11T18:03:55.433 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.433 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.433 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.434 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:55.434 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:55.434 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.434 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-12-11T18:03:55.434 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "insights", 2023-12-11T18:03:55.434 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2023-12-11T18:03:55.434 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2023-12-11T18:03:55.434 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2023-12-11T18:03:55.434 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2023-12-11T18:03:55.434 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2023-12-11T18:03:55.434 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.434 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.435 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.435 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.435 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.435 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.435 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:55.435 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:55.435 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:55.435 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:55.435 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:55.435 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:55.435 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:55.435 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:55.435 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.435 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.435 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.436 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.436 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.436 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2023-12-11T18:03:55.436 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2023-12-11T18:03:55.436 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.436 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.436 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.436 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.436 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.436 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.436 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.436 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.436 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.437 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.437 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.437 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.437 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2023-12-11T18:03:55.437 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2023-12-11T18:03:55.437 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.437 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.437 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.437 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2023-12-11T18:03:55.437 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.437 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.438 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:55.438 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:55.438 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:55.438 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:55.438 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:55.438 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:55.438 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:55.438 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:55.438 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.438 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.438 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.438 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.438 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.439 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2023-12-11T18:03:55.439 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2023-12-11T18:03:55.439 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.439 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.439 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.439 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.439 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.439 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.439 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.439 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.439 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.439 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.439 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.439 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:55.440 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:55.440 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.440 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-12-11T18:03:55.440 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "iostat", 2023-12-11T18:03:55.440 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2023-12-11T18:03:55.440 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2023-12-11T18:03:55.440 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2023-12-11T18:03:55.440 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2023-12-11T18:03:55.440 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2023-12-11T18:03:55.440 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.440 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.440 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.440 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.440 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.441 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.441 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:55.441 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:55.441 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:55.441 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:55.441 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:55.441 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:55.441 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:55.441 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:55.441 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.441 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.441 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.441 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.441 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.442 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2023-12-11T18:03:55.442 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2023-12-11T18:03:55.442 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.442 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.442 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.442 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.442 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.442 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.442 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.442 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.442 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.442 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.443 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.443 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.443 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2023-12-11T18:03:55.443 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2023-12-11T18:03:55.443 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.443 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.443 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.443 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2023-12-11T18:03:55.443 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.443 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.443 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:55.443 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:55.443 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:55.444 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:55.444 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:55.444 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:55.444 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:55.444 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:55.444 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.444 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.444 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.444 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.444 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.444 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2023-12-11T18:03:55.445 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2023-12-11T18:03:55.445 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.445 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.445 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.445 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.445 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.445 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.445 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.445 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.445 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.445 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.445 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.445 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:55.445 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:55.446 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.446 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-12-11T18:03:55.446 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "k8sevents", 2023-12-11T18:03:55.446 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2023-12-11T18:03:55.446 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2023-12-11T18:03:55.446 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2023-12-11T18:03:55.446 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_event_retention_days": { 2023-12-11T18:03:55.446 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ceph_event_retention_days", 2023-12-11T18:03:55.446 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2023-12-11T18:03:55.446 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.447 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.447 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "7", 2023-12-11T18:03:55.447 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.447 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.447 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.447 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Days to hold ceph event information within local cache", 2023-12-11T18:03:55.447 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.447 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.447 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.447 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.447 INFO:teuthology.orchestra.run.smithi059.stdout: "config_check_secs": { 2023-12-11T18:03:55.448 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "config_check_secs", 2023-12-11T18:03:55.448 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2023-12-11T18:03:55.448 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.448 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.448 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "10", 2023-12-11T18:03:55.448 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "10", 2023-12-11T18:03:55.448 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.448 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.448 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-12-11T18:03:55.448 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.448 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.448 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.448 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.449 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2023-12-11T18:03:55.449 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2023-12-11T18:03:55.449 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.449 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.449 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.449 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.449 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.449 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.449 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:55.449 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:55.449 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:55.449 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:55.449 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:55.449 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:55.450 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:55.450 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:55.450 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.450 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.450 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.450 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.450 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.450 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2023-12-11T18:03:55.450 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2023-12-11T18:03:55.450 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.450 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.450 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.450 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.451 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.451 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.451 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.451 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.451 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.451 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.451 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.451 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.451 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2023-12-11T18:03:55.451 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2023-12-11T18:03:55.451 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.451 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.451 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.451 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2023-12-11T18:03:55.451 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.452 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.452 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:55.452 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:55.452 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:55.452 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:55.452 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:55.452 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:55.452 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:55.452 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:55.452 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.452 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.452 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.452 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.452 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.453 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2023-12-11T18:03:55.453 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2023-12-11T18:03:55.453 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.453 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.453 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.453 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.453 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.453 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.453 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.453 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.453 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.453 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.453 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.453 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:55.454 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:55.454 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.454 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-12-11T18:03:55.454 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "localpool", 2023-12-11T18:03:55.454 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2023-12-11T18:03:55.454 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2023-12-11T18:03:55.454 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2023-12-11T18:03:55.454 INFO:teuthology.orchestra.run.smithi059.stdout: "failure_domain": { 2023-12-11T18:03:55.454 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "failure_domain", 2023-12-11T18:03:55.454 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.454 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.454 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.454 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "host", 2023-12-11T18:03:55.454 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.455 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.455 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.455 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "failure domain for any created local pool", 2023-12-11T18:03:55.455 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-12-11T18:03:55.455 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.455 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.455 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.455 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2023-12-11T18:03:55.455 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2023-12-11T18:03:55.455 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.455 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.456 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.456 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.456 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.456 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.456 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:55.456 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:55.456 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:55.456 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:55.456 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:55.456 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:55.456 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:55.456 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:55.456 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.456 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.457 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.457 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.457 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.457 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2023-12-11T18:03:55.457 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2023-12-11T18:03:55.457 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.457 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.457 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.457 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.457 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.457 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.457 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.458 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.458 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.458 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.458 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.458 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.458 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2023-12-11T18:03:55.458 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2023-12-11T18:03:55.458 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.458 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.458 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.458 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2023-12-11T18:03:55.458 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.458 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.459 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:55.459 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:55.459 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:55.459 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:55.459 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:55.459 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:55.459 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:55.459 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:55.459 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.459 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.460 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.460 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.460 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.460 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2023-12-11T18:03:55.460 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2023-12-11T18:03:55.460 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.460 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.460 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.460 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.460 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.460 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.460 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.460 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.460 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.461 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.461 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.461 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.461 INFO:teuthology.orchestra.run.smithi059.stdout: "min_size": { 2023-12-11T18:03:55.461 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "min_size", 2023-12-11T18:03:55.461 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2023-12-11T18:03:55.461 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.461 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.461 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.461 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.461 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.461 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.461 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "default min_size for any created local pool", 2023-12-11T18:03:55.461 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-12-11T18:03:55.461 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.462 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.462 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.462 INFO:teuthology.orchestra.run.smithi059.stdout: "num_rep": { 2023-12-11T18:03:55.462 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "num_rep", 2023-12-11T18:03:55.462 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2023-12-11T18:03:55.462 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.462 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.462 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "3", 2023-12-11T18:03:55.462 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.462 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.462 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.462 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "default replica count for any created local pool", 2023-12-11T18:03:55.462 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.463 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.463 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.463 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.463 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num": { 2023-12-11T18:03:55.463 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "pg_num", 2023-12-11T18:03:55.463 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2023-12-11T18:03:55.463 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.463 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.463 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "128", 2023-12-11T18:03:55.463 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.463 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.463 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.463 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "default pg_num for any created local pool", 2023-12-11T18:03:55.463 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.464 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.464 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.464 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.464 INFO:teuthology.orchestra.run.smithi059.stdout: "prefix": { 2023-12-11T18:03:55.464 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "prefix", 2023-12-11T18:03:55.464 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.464 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.464 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.464 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.464 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.464 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.464 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.464 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "name prefix for any created local pool", 2023-12-11T18:03:55.464 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.464 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.465 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.465 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.465 INFO:teuthology.orchestra.run.smithi059.stdout: "subtree": { 2023-12-11T18:03:55.465 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "subtree", 2023-12-11T18:03:55.465 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.465 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.465 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.465 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "rack", 2023-12-11T18:03:55.465 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.465 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.465 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.465 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "CRUSH level for which to create a local pool", 2023-12-11T18:03:55.465 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-12-11T18:03:55.465 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.466 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.466 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:55.466 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:55.466 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.466 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-12-11T18:03:55.466 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "mds_autoscaler", 2023-12-11T18:03:55.466 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2023-12-11T18:03:55.466 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2023-12-11T18:03:55.466 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2023-12-11T18:03:55.466 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2023-12-11T18:03:55.466 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2023-12-11T18:03:55.466 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.467 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.467 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.467 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.467 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.467 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.467 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:55.467 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:55.467 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:55.467 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:55.467 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:55.467 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:55.467 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:55.467 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:55.467 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.467 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.467 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.468 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.468 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.468 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2023-12-11T18:03:55.468 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2023-12-11T18:03:55.468 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.468 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.468 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.468 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.468 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.468 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.468 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.468 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.468 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.468 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.468 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.468 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.469 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2023-12-11T18:03:55.469 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2023-12-11T18:03:55.469 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.469 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.469 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.469 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2023-12-11T18:03:55.469 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.469 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.469 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:55.469 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:55.469 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:55.469 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:55.469 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:55.470 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:55.470 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:55.470 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:55.470 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.470 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.470 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.470 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.470 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.470 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2023-12-11T18:03:55.471 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2023-12-11T18:03:55.471 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.471 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.471 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.471 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.471 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.471 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.471 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.471 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.471 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.471 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.471 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.471 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:55.471 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:55.472 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.472 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-12-11T18:03:55.472 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "mirroring", 2023-12-11T18:03:55.472 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2023-12-11T18:03:55.472 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2023-12-11T18:03:55.472 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2023-12-11T18:03:55.472 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2023-12-11T18:03:55.472 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2023-12-11T18:03:55.472 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.472 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.472 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.472 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.473 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.473 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.473 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:55.473 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:55.473 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:55.473 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:55.473 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:55.473 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:55.473 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:55.473 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:55.473 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.473 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.473 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.473 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.474 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.474 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2023-12-11T18:03:55.474 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2023-12-11T18:03:55.474 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.474 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.474 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.474 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.474 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.474 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.474 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.474 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.474 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.474 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.474 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.475 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.475 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2023-12-11T18:03:55.475 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2023-12-11T18:03:55.475 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.475 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.475 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.475 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2023-12-11T18:03:55.475 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.475 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.475 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:55.475 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:55.475 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:55.475 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:55.475 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:55.475 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:55.476 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:55.476 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:55.476 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.476 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.476 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.476 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.476 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.476 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2023-12-11T18:03:55.476 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2023-12-11T18:03:55.476 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.476 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.476 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.476 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.476 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.477 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.477 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.477 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.477 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.477 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.477 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.477 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:55.477 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:55.477 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.477 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-12-11T18:03:55.477 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "nfs", 2023-12-11T18:03:55.477 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2023-12-11T18:03:55.477 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2023-12-11T18:03:55.477 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2023-12-11T18:03:55.477 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2023-12-11T18:03:55.478 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2023-12-11T18:03:55.478 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.478 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.478 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.478 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.478 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.478 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.478 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:55.478 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:55.478 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:55.478 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:55.478 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:55.478 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:55.479 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:55.479 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:55.479 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.479 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.479 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.479 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.479 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.479 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2023-12-11T18:03:55.479 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2023-12-11T18:03:55.479 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.479 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.479 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.479 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.480 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.480 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.480 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.480 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.480 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.480 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.480 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.480 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.480 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2023-12-11T18:03:55.480 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2023-12-11T18:03:55.480 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.480 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.481 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.481 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2023-12-11T18:03:55.481 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.481 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.481 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:55.481 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:55.481 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:55.481 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:55.481 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:55.481 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:55.481 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:55.481 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:55.481 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.482 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.482 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.482 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.482 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.482 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2023-12-11T18:03:55.482 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2023-12-11T18:03:55.482 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.482 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.482 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.482 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.482 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.482 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.482 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.483 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.483 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.483 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.483 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.483 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:55.483 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:55.483 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.483 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-12-11T18:03:55.483 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "orchestrator", 2023-12-11T18:03:55.483 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2023-12-11T18:03:55.483 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2023-12-11T18:03:55.483 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2023-12-11T18:03:55.483 INFO:teuthology.orchestra.run.smithi059.stdout: "fail_fs": { 2023-12-11T18:03:55.483 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "fail_fs", 2023-12-11T18:03:55.484 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.484 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.484 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.484 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.484 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.484 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.484 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.484 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-12-11T18:03:55.484 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.484 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.484 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.484 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.485 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2023-12-11T18:03:55.485 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2023-12-11T18:03:55.485 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.485 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.485 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.485 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.485 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.485 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.485 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:55.485 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:55.485 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:55.485 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:55.485 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:55.485 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:55.485 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:55.486 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:55.486 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.486 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.486 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.486 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.486 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.486 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2023-12-11T18:03:55.486 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2023-12-11T18:03:55.486 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.486 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.486 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.486 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.486 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.487 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.487 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.487 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.487 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.487 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.487 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.487 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.487 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2023-12-11T18:03:55.487 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2023-12-11T18:03:55.487 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.487 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.487 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.487 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2023-12-11T18:03:55.487 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.487 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.488 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:55.488 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:55.488 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:55.488 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:55.488 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:55.488 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:55.488 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:55.488 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:55.488 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.488 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.488 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.488 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.488 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.489 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2023-12-11T18:03:55.489 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2023-12-11T18:03:55.489 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.489 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.489 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.489 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.489 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.489 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.489 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.489 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.489 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.489 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.489 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.489 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.490 INFO:teuthology.orchestra.run.smithi059.stdout: "orchestrator": { 2023-12-11T18:03:55.490 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "orchestrator", 2023-12-11T18:03:55.490 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.490 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.490 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.490 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.490 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.490 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.490 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:55.491 INFO:teuthology.orchestra.run.smithi059.stdout: "cephadm", 2023-12-11T18:03:55.491 INFO:teuthology.orchestra.run.smithi059.stdout: "rook", 2023-12-11T18:03:55.491 INFO:teuthology.orchestra.run.smithi059.stdout: "test_orchestrator" 2023-12-11T18:03:55.491 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:55.491 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Orchestrator backend", 2023-12-11T18:03:55.491 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.491 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.491 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.491 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:55.491 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:55.491 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.491 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-12-11T18:03:55.491 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "osd_perf_query", 2023-12-11T18:03:55.491 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2023-12-11T18:03:55.491 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2023-12-11T18:03:55.492 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2023-12-11T18:03:55.492 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2023-12-11T18:03:55.492 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2023-12-11T18:03:55.492 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.492 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.492 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.492 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.492 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.492 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.492 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:55.492 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:55.492 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:55.492 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:55.492 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:55.493 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:55.493 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:55.493 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:55.493 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.493 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.493 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.493 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.493 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.493 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2023-12-11T18:03:55.493 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2023-12-11T18:03:55.493 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.493 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.493 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.493 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.494 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.494 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.494 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.494 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.494 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.494 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.494 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.494 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.494 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2023-12-11T18:03:55.494 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2023-12-11T18:03:55.494 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.494 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.494 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.495 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2023-12-11T18:03:55.495 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.495 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.495 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:55.495 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:55.495 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:55.495 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:55.495 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:55.495 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:55.495 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:55.495 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:55.495 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.496 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.496 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.496 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.496 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.496 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2023-12-11T18:03:55.496 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2023-12-11T18:03:55.496 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.496 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.496 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.496 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.497 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.497 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.497 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.497 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.497 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.497 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.497 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.497 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:55.497 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:55.497 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.497 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-12-11T18:03:55.497 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "osd_support", 2023-12-11T18:03:55.498 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2023-12-11T18:03:55.498 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2023-12-11T18:03:55.498 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2023-12-11T18:03:55.498 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2023-12-11T18:03:55.498 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2023-12-11T18:03:55.498 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.498 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.498 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.498 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.498 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.498 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.499 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:55.499 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:55.499 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:55.499 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:55.499 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:55.499 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:55.499 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:55.499 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:55.499 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.499 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.499 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.500 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.500 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.500 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2023-12-11T18:03:55.500 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2023-12-11T18:03:55.500 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.500 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.500 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.500 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.500 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.501 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.501 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.501 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.501 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.501 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.501 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.501 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.501 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2023-12-11T18:03:55.501 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2023-12-11T18:03:55.501 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.501 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.501 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.501 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2023-12-11T18:03:55.502 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.502 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.502 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:55.502 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:55.502 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:55.502 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:55.502 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:55.502 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:55.502 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:55.502 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:55.502 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.502 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.502 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.503 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.503 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.503 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2023-12-11T18:03:55.503 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2023-12-11T18:03:55.503 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.503 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.503 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.503 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.503 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.503 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.503 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.503 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.503 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.504 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.504 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.504 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:55.504 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:55.504 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.504 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-12-11T18:03:55.504 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "pg_autoscaler", 2023-12-11T18:03:55.504 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2023-12-11T18:03:55.504 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2023-12-11T18:03:55.504 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2023-12-11T18:03:55.504 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2023-12-11T18:03:55.505 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2023-12-11T18:03:55.505 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.505 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.505 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.505 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.505 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.505 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.505 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:55.505 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:55.505 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:55.505 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:55.506 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:55.506 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:55.506 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:55.506 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:55.506 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.507 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.507 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.507 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.507 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.507 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2023-12-11T18:03:55.507 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2023-12-11T18:03:55.507 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.508 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.508 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.508 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.508 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.508 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.508 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.508 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.509 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.509 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.509 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.509 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.509 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2023-12-11T18:03:55.509 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2023-12-11T18:03:55.509 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.510 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.510 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.510 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2023-12-11T18:03:55.510 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.510 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.510 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:55.510 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:55.510 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:55.511 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:55.511 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:55.511 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:55.511 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:55.511 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:55.511 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.511 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.511 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.511 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.511 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.511 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2023-12-11T18:03:55.511 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2023-12-11T18:03:55.511 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.511 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.512 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.512 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.512 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.512 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.512 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.512 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.512 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.512 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.512 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.512 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.512 INFO:teuthology.orchestra.run.smithi059.stdout: "sleep_interval": { 2023-12-11T18:03:55.512 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sleep_interval", 2023-12-11T18:03:55.512 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2023-12-11T18:03:55.512 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.513 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.513 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "60", 2023-12-11T18:03:55.513 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.513 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.513 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.513 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.513 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.513 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.513 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.513 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.513 INFO:teuthology.orchestra.run.smithi059.stdout: "threshold": { 2023-12-11T18:03:55.513 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "threshold", 2023-12-11T18:03:55.513 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "float", 2023-12-11T18:03:55.513 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.513 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.513 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "3.0", 2023-12-11T18:03:55.514 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "1.0", 2023-12-11T18:03:55.514 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.514 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.514 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "scaling threshold", 2023-12-11T18:03:55.514 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-12-11T18:03:55.514 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.514 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.514 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:55.514 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:55.514 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.514 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-12-11T18:03:55.514 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "progress", 2023-12-11T18:03:55.515 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2023-12-11T18:03:55.515 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2023-12-11T18:03:55.515 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2023-12-11T18:03:55.515 INFO:teuthology.orchestra.run.smithi059.stdout: "allow_pg_recovery_event": { 2023-12-11T18:03:55.515 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "allow_pg_recovery_event", 2023-12-11T18:03:55.515 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.515 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.515 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.515 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.515 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.515 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.516 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.516 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "allow the module to show pg recovery progress", 2023-12-11T18:03:55.516 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.516 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.516 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.516 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.516 INFO:teuthology.orchestra.run.smithi059.stdout: "enabled": { 2023-12-11T18:03:55.516 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "enabled", 2023-12-11T18:03:55.516 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.516 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.516 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.516 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2023-12-11T18:03:55.516 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.517 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.517 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.517 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.517 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.517 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.517 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.517 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.517 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2023-12-11T18:03:55.517 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2023-12-11T18:03:55.517 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.517 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.517 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.517 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.517 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.517 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.517 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:55.518 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:55.518 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:55.518 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:55.518 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:55.518 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:55.518 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:55.518 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:55.518 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.518 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.518 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.518 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.519 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.519 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2023-12-11T18:03:55.519 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2023-12-11T18:03:55.519 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.519 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.519 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.519 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.519 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.519 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.519 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.519 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.519 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.519 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.519 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.520 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.520 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2023-12-11T18:03:55.520 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2023-12-11T18:03:55.520 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.520 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.520 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.520 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2023-12-11T18:03:55.520 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.520 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.520 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:55.520 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:55.520 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:55.520 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:55.520 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:55.520 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:55.520 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:55.521 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:55.521 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.521 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.521 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.521 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.521 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.521 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2023-12-11T18:03:55.521 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2023-12-11T18:03:55.521 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.521 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.521 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.521 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.522 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.522 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.522 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.522 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.522 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.522 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.522 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.522 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.522 INFO:teuthology.orchestra.run.smithi059.stdout: "max_completed_events": { 2023-12-11T18:03:55.522 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "max_completed_events", 2023-12-11T18:03:55.522 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2023-12-11T18:03:55.522 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.523 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.523 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "50", 2023-12-11T18:03:55.523 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.523 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.523 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.523 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "number of past completed events to remember", 2023-12-11T18:03:55.523 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.523 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.523 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.523 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.523 INFO:teuthology.orchestra.run.smithi059.stdout: "sleep_interval": { 2023-12-11T18:03:55.523 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sleep_interval", 2023-12-11T18:03:55.523 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2023-12-11T18:03:55.523 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.523 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.524 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "5", 2023-12-11T18:03:55.524 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.524 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.524 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.524 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "how long the module is going to sleep", 2023-12-11T18:03:55.524 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.524 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.524 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.524 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:55.524 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:55.524 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.524 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-12-11T18:03:55.524 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "prometheus", 2023-12-11T18:03:55.524 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2023-12-11T18:03:55.524 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2023-12-11T18:03:55.524 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2023-12-11T18:03:55.525 INFO:teuthology.orchestra.run.smithi059.stdout: "cache": { 2023-12-11T18:03:55.525 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "cache", 2023-12-11T18:03:55.525 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.525 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.525 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.525 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2023-12-11T18:03:55.525 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.525 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.525 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.525 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.525 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.525 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.525 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.525 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.525 INFO:teuthology.orchestra.run.smithi059.stdout: "exclude_perf_counters": { 2023-12-11T18:03:55.526 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "exclude_perf_counters", 2023-12-11T18:03:55.526 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.526 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.526 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.526 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2023-12-11T18:03:55.526 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.526 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.526 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.526 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-12-11T18:03:55.526 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-12-11T18:03:55.526 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.526 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.526 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.527 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2023-12-11T18:03:55.527 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2023-12-11T18:03:55.527 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.527 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.527 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.527 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.527 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.527 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.527 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:55.527 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:55.527 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:55.527 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:55.527 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:55.528 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:55.528 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:55.528 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:55.528 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.528 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.528 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.528 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.528 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.528 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2023-12-11T18:03:55.528 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2023-12-11T18:03:55.528 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.528 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.528 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.528 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.529 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.529 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.529 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.529 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.529 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.529 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.529 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.529 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.529 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2023-12-11T18:03:55.529 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2023-12-11T18:03:55.529 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.529 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.529 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.529 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2023-12-11T18:03:55.530 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.530 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.530 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:55.530 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:55.530 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:55.530 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:55.530 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:55.530 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:55.530 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:55.530 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:55.530 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.530 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.530 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.530 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.531 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.531 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2023-12-11T18:03:55.531 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2023-12-11T18:03:55.531 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.531 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.531 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.531 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.531 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.531 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.531 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.531 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.531 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.531 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.531 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.532 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.532 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_stats_pools": { 2023-12-11T18:03:55.532 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rbd_stats_pools", 2023-12-11T18:03:55.532 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.532 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.532 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.532 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.532 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.532 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.532 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.532 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.532 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.532 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.532 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.533 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.533 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_stats_pools_refresh_interval": { 2023-12-11T18:03:55.533 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-12-11T18:03:55.533 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2023-12-11T18:03:55.533 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.533 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.533 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "300", 2023-12-11T18:03:55.533 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.533 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.533 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.533 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.533 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.534 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.534 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.534 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.534 INFO:teuthology.orchestra.run.smithi059.stdout: "scrape_interval": { 2023-12-11T18:03:55.534 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "scrape_interval", 2023-12-11T18:03:55.534 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "float", 2023-12-11T18:03:55.534 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.534 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.534 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "15.0", 2023-12-11T18:03:55.534 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.534 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.534 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.534 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.534 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.535 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.535 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.535 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.535 INFO:teuthology.orchestra.run.smithi059.stdout: "server_addr": { 2023-12-11T18:03:55.535 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "server_addr", 2023-12-11T18:03:55.535 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.535 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.535 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.535 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "::", 2023-12-11T18:03:55.535 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.535 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.535 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.535 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-12-11T18:03:55.535 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.535 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.535 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.536 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.536 INFO:teuthology.orchestra.run.smithi059.stdout: "server_port": { 2023-12-11T18:03:55.536 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "server_port", 2023-12-11T18:03:55.536 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2023-12-11T18:03:55.536 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.536 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.536 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "9283", 2023-12-11T18:03:55.536 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.536 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.536 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.536 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-12-11T18:03:55.536 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.536 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.536 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.536 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.536 INFO:teuthology.orchestra.run.smithi059.stdout: "stale_cache_strategy": { 2023-12-11T18:03:55.536 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "stale_cache_strategy", 2023-12-11T18:03:55.537 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.537 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.537 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.537 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "log", 2023-12-11T18:03:55.537 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.537 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.537 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.537 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.537 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.537 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.537 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.537 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.537 INFO:teuthology.orchestra.run.smithi059.stdout: "standby_behaviour": { 2023-12-11T18:03:55.537 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "standby_behaviour", 2023-12-11T18:03:55.537 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.537 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.538 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.538 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "default", 2023-12-11T18:03:55.538 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.538 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.538 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:55.538 INFO:teuthology.orchestra.run.smithi059.stdout: "default", 2023-12-11T18:03:55.538 INFO:teuthology.orchestra.run.smithi059.stdout: "error" 2023-12-11T18:03:55.538 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:55.538 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.538 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.538 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.538 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.538 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.538 INFO:teuthology.orchestra.run.smithi059.stdout: "standby_error_status_code": { 2023-12-11T18:03:55.538 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "standby_error_status_code", 2023-12-11T18:03:55.539 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2023-12-11T18:03:55.539 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.539 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.539 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "500", 2023-12-11T18:03:55.539 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "400", 2023-12-11T18:03:55.539 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "599", 2023-12-11T18:03:55.539 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.539 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.539 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.539 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.539 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.539 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:55.539 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:55.539 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.539 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-12-11T18:03:55.540 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rbd_support", 2023-12-11T18:03:55.540 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2023-12-11T18:03:55.540 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2023-12-11T18:03:55.540 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2023-12-11T18:03:55.540 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2023-12-11T18:03:55.540 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2023-12-11T18:03:55.540 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.540 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.540 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.540 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.540 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.540 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.540 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:55.540 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:55.540 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:55.541 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:55.541 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:55.541 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:55.541 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:55.541 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:55.541 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.541 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.541 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.541 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.541 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.541 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2023-12-11T18:03:55.541 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2023-12-11T18:03:55.541 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.541 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.541 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.541 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.541 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.542 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.542 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.542 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.542 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.542 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.542 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.542 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.542 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2023-12-11T18:03:55.542 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2023-12-11T18:03:55.542 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.542 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.542 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.543 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2023-12-11T18:03:55.543 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.543 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.543 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:55.543 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:55.543 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:55.543 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:55.543 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:55.543 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:55.543 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:55.543 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:55.543 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.543 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.543 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.543 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.544 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.544 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2023-12-11T18:03:55.544 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2023-12-11T18:03:55.544 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.544 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.544 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.544 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.544 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.544 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.544 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.544 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.544 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.545 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.545 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.545 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.545 INFO:teuthology.orchestra.run.smithi059.stdout: "max_concurrent_snap_create": { 2023-12-11T18:03:55.545 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "max_concurrent_snap_create", 2023-12-11T18:03:55.545 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2023-12-11T18:03:55.545 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.545 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.545 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "10", 2023-12-11T18:03:55.545 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.545 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.545 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.545 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.545 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.546 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.546 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.546 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.546 INFO:teuthology.orchestra.run.smithi059.stdout: "mirror_snapshot_schedule": { 2023-12-11T18:03:55.546 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "mirror_snapshot_schedule", 2023-12-11T18:03:55.546 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.546 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.546 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.546 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.546 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.546 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.546 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.546 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.546 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.547 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.547 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.547 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.547 INFO:teuthology.orchestra.run.smithi059.stdout: "trash_purge_schedule": { 2023-12-11T18:03:55.547 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "trash_purge_schedule", 2023-12-11T18:03:55.547 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.547 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.547 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.547 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.547 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.547 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.547 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.547 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.547 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.548 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.548 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.548 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:55.548 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:55.548 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.548 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-12-11T18:03:55.548 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "restful", 2023-12-11T18:03:55.548 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2023-12-11T18:03:55.548 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2023-12-11T18:03:55.548 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2023-12-11T18:03:55.548 INFO:teuthology.orchestra.run.smithi059.stdout: "enable_auth": { 2023-12-11T18:03:55.548 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "enable_auth", 2023-12-11T18:03:55.548 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.548 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.548 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.548 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2023-12-11T18:03:55.548 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.549 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.549 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.549 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.549 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.549 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.549 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.549 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.549 INFO:teuthology.orchestra.run.smithi059.stdout: "key_file": { 2023-12-11T18:03:55.549 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "key_file", 2023-12-11T18:03:55.549 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.549 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.549 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.549 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.549 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.549 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.549 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.550 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.550 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.550 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.550 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.550 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.550 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2023-12-11T18:03:55.550 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2023-12-11T18:03:55.550 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.550 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.550 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.550 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.550 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.550 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.550 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:55.551 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:55.551 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:55.551 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:55.551 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:55.551 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:55.551 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:55.551 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:55.551 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.551 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.551 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.551 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.551 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.551 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2023-12-11T18:03:55.551 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2023-12-11T18:03:55.551 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.551 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.551 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.552 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.552 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.552 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.552 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.552 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.552 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.552 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.552 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.552 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.552 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2023-12-11T18:03:55.552 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2023-12-11T18:03:55.552 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.552 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.552 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.552 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2023-12-11T18:03:55.553 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.553 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.553 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:55.553 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:55.553 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:55.553 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:55.553 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:55.553 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:55.553 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:55.553 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:55.553 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.554 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.554 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.554 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.554 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.554 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2023-12-11T18:03:55.554 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2023-12-11T18:03:55.554 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.554 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.554 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.554 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.554 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.554 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.554 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.555 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.555 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.555 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.555 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.555 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.555 INFO:teuthology.orchestra.run.smithi059.stdout: "server_addr": { 2023-12-11T18:03:55.555 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "server_addr", 2023-12-11T18:03:55.555 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.555 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.555 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.555 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.555 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.555 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.555 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.555 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.556 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.556 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.556 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.556 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.556 INFO:teuthology.orchestra.run.smithi059.stdout: "server_port": { 2023-12-11T18:03:55.556 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "server_port", 2023-12-11T18:03:55.556 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.556 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.556 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.556 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.556 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.556 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.556 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.556 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.557 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.557 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.557 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.557 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:55.557 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:55.557 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.557 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-12-11T18:03:55.557 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rgw", 2023-12-11T18:03:55.557 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2023-12-11T18:03:55.557 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2023-12-11T18:03:55.557 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2023-12-11T18:03:55.557 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2023-12-11T18:03:55.557 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2023-12-11T18:03:55.557 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.557 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.558 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.558 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.558 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.558 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.558 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:55.558 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:55.558 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:55.558 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:55.558 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:55.558 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:55.558 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:55.558 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:55.558 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.559 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.559 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.559 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.559 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.559 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2023-12-11T18:03:55.559 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2023-12-11T18:03:55.559 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.559 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.559 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.559 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.559 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.559 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.559 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.559 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.559 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.559 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.559 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.560 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.560 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2023-12-11T18:03:55.560 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2023-12-11T18:03:55.560 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.560 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.560 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.560 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2023-12-11T18:03:55.560 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.560 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.560 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:55.560 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:55.560 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:55.560 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:55.560 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:55.560 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:55.560 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:55.561 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:55.561 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.561 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.561 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.561 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.561 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.561 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2023-12-11T18:03:55.561 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2023-12-11T18:03:55.561 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.561 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.561 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.561 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.561 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.561 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.562 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.562 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.562 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.562 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.562 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.562 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:55.562 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:55.562 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.562 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-12-11T18:03:55.562 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rook", 2023-12-11T18:03:55.562 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2023-12-11T18:03:55.562 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2023-12-11T18:03:55.563 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2023-12-11T18:03:55.563 INFO:teuthology.orchestra.run.smithi059.stdout: "drive_group_interval": { 2023-12-11T18:03:55.563 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "drive_group_interval", 2023-12-11T18:03:55.563 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "float", 2023-12-11T18:03:55.563 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.563 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.563 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "300.0", 2023-12-11T18:03:55.563 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.563 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.563 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.563 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-12-11T18:03:55.563 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.563 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.563 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.564 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.564 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2023-12-11T18:03:55.564 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2023-12-11T18:03:55.564 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.564 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.564 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.564 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.564 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.564 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.564 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:55.564 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:55.564 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:55.564 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:55.564 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:55.564 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:55.565 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:55.565 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:55.565 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.565 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.565 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.565 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.565 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.565 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2023-12-11T18:03:55.565 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2023-12-11T18:03:55.565 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.565 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.565 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.565 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.565 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.565 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.566 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.566 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.566 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.566 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.566 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.566 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.566 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2023-12-11T18:03:55.566 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2023-12-11T18:03:55.566 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.566 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.566 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.566 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2023-12-11T18:03:55.566 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.566 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.566 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:55.567 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:55.567 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:55.567 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:55.567 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:55.567 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:55.567 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:55.567 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:55.567 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.567 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.567 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.567 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.567 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.567 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2023-12-11T18:03:55.567 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2023-12-11T18:03:55.567 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.568 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.568 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.568 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.568 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.568 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.568 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.568 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.568 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.568 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.568 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.568 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.568 INFO:teuthology.orchestra.run.smithi059.stdout: "storage_class": { 2023-12-11T18:03:55.568 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "storage_class", 2023-12-11T18:03:55.568 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.568 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.568 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.569 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "local", 2023-12-11T18:03:55.569 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.569 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.569 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.569 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-12-11T18:03:55.569 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.569 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.569 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.569 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:55.569 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:55.569 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.569 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-12-11T18:03:55.569 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "selftest", 2023-12-11T18:03:55.569 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2023-12-11T18:03:55.569 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2023-12-11T18:03:55.569 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2023-12-11T18:03:55.570 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2023-12-11T18:03:55.570 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2023-12-11T18:03:55.570 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.570 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.570 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.570 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.570 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.570 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.570 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:55.570 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:55.570 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:55.570 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:55.570 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:55.571 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:55.571 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:55.571 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:55.571 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.571 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.571 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.571 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.571 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.571 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2023-12-11T18:03:55.571 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2023-12-11T18:03:55.571 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.571 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.571 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.571 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.572 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.572 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.572 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.572 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.572 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.572 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.572 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.572 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.572 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2023-12-11T18:03:55.572 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2023-12-11T18:03:55.572 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.572 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.573 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.573 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2023-12-11T18:03:55.573 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.573 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.573 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:55.573 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:55.573 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:55.573 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:55.573 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:55.573 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:55.573 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:55.573 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:55.573 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.573 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.573 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.573 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.574 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.574 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2023-12-11T18:03:55.574 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2023-12-11T18:03:55.574 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.574 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.574 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.574 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.574 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.574 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.574 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.574 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.574 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.574 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.574 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.575 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.575 INFO:teuthology.orchestra.run.smithi059.stdout: "roption1": { 2023-12-11T18:03:55.575 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "roption1", 2023-12-11T18:03:55.575 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.575 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.575 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.575 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.575 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.575 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.575 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.575 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.575 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.575 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.575 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.575 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.575 INFO:teuthology.orchestra.run.smithi059.stdout: "roption2": { 2023-12-11T18:03:55.576 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "roption2", 2023-12-11T18:03:55.576 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.576 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.576 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.576 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "xyz", 2023-12-11T18:03:55.576 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.576 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.576 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.576 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.576 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.576 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.576 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.576 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.576 INFO:teuthology.orchestra.run.smithi059.stdout: "rwoption1": { 2023-12-11T18:03:55.577 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rwoption1", 2023-12-11T18:03:55.577 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.577 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.577 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.577 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.577 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.577 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.577 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.577 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.577 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.577 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.577 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.577 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.577 INFO:teuthology.orchestra.run.smithi059.stdout: "rwoption2": { 2023-12-11T18:03:55.577 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rwoption2", 2023-12-11T18:03:55.577 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2023-12-11T18:03:55.578 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.578 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.578 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.578 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.578 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.578 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.578 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.578 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.578 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.578 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.578 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.578 INFO:teuthology.orchestra.run.smithi059.stdout: "rwoption3": { 2023-12-11T18:03:55.578 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rwoption3", 2023-12-11T18:03:55.578 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "float", 2023-12-11T18:03:55.578 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.578 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.579 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.579 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.579 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.579 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.579 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.579 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.579 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.579 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.579 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.579 INFO:teuthology.orchestra.run.smithi059.stdout: "rwoption4": { 2023-12-11T18:03:55.579 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rwoption4", 2023-12-11T18:03:55.579 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.579 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.579 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.579 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.580 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.580 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.580 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.580 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.580 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.580 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.580 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.580 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.580 INFO:teuthology.orchestra.run.smithi059.stdout: "rwoption5": { 2023-12-11T18:03:55.580 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rwoption5", 2023-12-11T18:03:55.580 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.580 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.580 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.580 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.580 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.580 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.581 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.581 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.581 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.581 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.581 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.581 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.581 INFO:teuthology.orchestra.run.smithi059.stdout: "rwoption6": { 2023-12-11T18:03:55.581 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rwoption6", 2023-12-11T18:03:55.581 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.581 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.581 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.581 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2023-12-11T18:03:55.581 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.581 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.581 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.581 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.582 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.582 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.582 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.582 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.582 INFO:teuthology.orchestra.run.smithi059.stdout: "rwoption7": { 2023-12-11T18:03:55.582 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rwoption7", 2023-12-11T18:03:55.582 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2023-12-11T18:03:55.582 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.582 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.582 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.582 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "1", 2023-12-11T18:03:55.582 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "42", 2023-12-11T18:03:55.582 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.582 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.582 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.583 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.583 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.583 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.583 INFO:teuthology.orchestra.run.smithi059.stdout: "testkey": { 2023-12-11T18:03:55.583 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "testkey", 2023-12-11T18:03:55.583 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.583 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.583 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.583 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.583 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.583 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.583 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.583 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.584 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.584 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.584 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.584 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.584 INFO:teuthology.orchestra.run.smithi059.stdout: "testlkey": { 2023-12-11T18:03:55.584 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "testlkey", 2023-12-11T18:03:55.584 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.584 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.584 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.584 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.584 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.584 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.584 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.584 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.584 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.584 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.584 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.585 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.585 INFO:teuthology.orchestra.run.smithi059.stdout: "testnewline": { 2023-12-11T18:03:55.585 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "testnewline", 2023-12-11T18:03:55.585 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.585 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.585 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.585 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.585 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.585 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.585 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.585 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.585 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.585 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.585 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.585 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:55.585 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:55.586 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.586 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-12-11T18:03:55.586 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "snap_schedule", 2023-12-11T18:03:55.586 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2023-12-11T18:03:55.586 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2023-12-11T18:03:55.586 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2023-12-11T18:03:55.586 INFO:teuthology.orchestra.run.smithi059.stdout: "allow_m_granularity": { 2023-12-11T18:03:55.586 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "allow_m_granularity", 2023-12-11T18:03:55.586 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.586 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.586 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.586 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.586 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.586 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.586 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.586 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "allow minute scheduled snapshots", 2023-12-11T18:03:55.587 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.587 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.587 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.587 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.587 INFO:teuthology.orchestra.run.smithi059.stdout: "dump_on_update": { 2023-12-11T18:03:55.587 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "dump_on_update", 2023-12-11T18:03:55.587 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.587 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.587 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.587 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.587 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.587 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.587 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.587 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "dump database to debug log on update", 2023-12-11T18:03:55.587 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.588 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.588 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.588 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.588 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2023-12-11T18:03:55.588 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2023-12-11T18:03:55.588 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.588 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.588 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.588 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.588 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.588 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.588 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:55.588 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:55.588 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:55.588 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:55.588 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:55.589 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:55.589 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:55.589 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:55.589 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.589 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.589 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.589 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.589 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.589 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2023-12-11T18:03:55.589 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2023-12-11T18:03:55.589 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.589 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.589 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.589 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.589 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.589 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.589 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.590 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.590 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.590 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.590 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.590 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.590 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2023-12-11T18:03:55.590 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2023-12-11T18:03:55.590 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.590 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.590 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.590 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2023-12-11T18:03:55.590 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.590 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.590 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:55.590 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:55.591 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:55.591 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:55.591 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:55.591 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:55.591 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:55.591 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:55.591 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.591 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.591 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.591 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.591 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.591 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2023-12-11T18:03:55.591 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2023-12-11T18:03:55.592 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.592 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.592 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.592 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.592 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.592 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.592 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.592 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.592 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.592 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.592 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.592 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:55.592 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:55.592 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.592 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-12-11T18:03:55.593 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "stats", 2023-12-11T18:03:55.593 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2023-12-11T18:03:55.593 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2023-12-11T18:03:55.593 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2023-12-11T18:03:55.593 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2023-12-11T18:03:55.593 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2023-12-11T18:03:55.593 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.593 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.593 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.593 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.593 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.593 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.593 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:55.594 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:55.594 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:55.594 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:55.594 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:55.594 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:55.594 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:55.594 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:55.594 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.594 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.594 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.594 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.594 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.594 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2023-12-11T18:03:55.595 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2023-12-11T18:03:55.595 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.595 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.595 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.595 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.595 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.595 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.595 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.595 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.595 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.595 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.595 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.595 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.595 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2023-12-11T18:03:55.596 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2023-12-11T18:03:55.596 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.596 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.596 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.596 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2023-12-11T18:03:55.596 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.596 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.596 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:55.596 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:55.596 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:55.596 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:55.596 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:55.596 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:55.596 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:55.596 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:55.597 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.597 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.597 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.597 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.597 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.597 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2023-12-11T18:03:55.597 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2023-12-11T18:03:55.597 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.597 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.597 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.598 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.598 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.598 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.598 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.598 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.598 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.598 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.598 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.598 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:55.598 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:55.598 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.598 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-12-11T18:03:55.598 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "status", 2023-12-11T18:03:55.598 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2023-12-11T18:03:55.599 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2023-12-11T18:03:55.599 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2023-12-11T18:03:55.599 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2023-12-11T18:03:55.599 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2023-12-11T18:03:55.599 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.599 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.599 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.599 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.599 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.599 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.599 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:55.599 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:55.599 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:55.599 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:55.599 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:55.600 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:55.600 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:55.600 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:55.600 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.600 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.600 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.600 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.600 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.600 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2023-12-11T18:03:55.600 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2023-12-11T18:03:55.600 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.600 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.600 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.600 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.600 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.601 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.601 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.601 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.601 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.601 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.601 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.601 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.601 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2023-12-11T18:03:55.601 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2023-12-11T18:03:55.601 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.601 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.601 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.601 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2023-12-11T18:03:55.601 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.601 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.601 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:55.602 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:55.602 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:55.602 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:55.602 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:55.602 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:55.602 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:55.602 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:55.602 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.602 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.602 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.602 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.602 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.602 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2023-12-11T18:03:55.602 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2023-12-11T18:03:55.602 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.603 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.603 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.603 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.603 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.603 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.603 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.603 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.603 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.603 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.603 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.603 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:55.603 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:55.603 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.603 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-12-11T18:03:55.604 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "telegraf", 2023-12-11T18:03:55.604 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2023-12-11T18:03:55.604 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2023-12-11T18:03:55.604 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2023-12-11T18:03:55.604 INFO:teuthology.orchestra.run.smithi059.stdout: "address": { 2023-12-11T18:03:55.604 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "address", 2023-12-11T18:03:55.604 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.604 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.604 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.604 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-12-11T18:03:55.604 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.604 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.604 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.605 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.605 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.605 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.605 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.605 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.605 INFO:teuthology.orchestra.run.smithi059.stdout: "interval": { 2023-12-11T18:03:55.605 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "interval", 2023-12-11T18:03:55.605 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2023-12-11T18:03:55.605 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.605 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.605 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "15", 2023-12-11T18:03:55.605 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.605 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.606 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.606 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.606 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.606 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.606 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.606 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.606 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2023-12-11T18:03:55.606 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2023-12-11T18:03:55.606 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.606 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.606 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.606 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.606 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.606 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.607 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:55.607 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:55.607 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:55.607 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:55.607 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:55.607 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:55.607 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:55.607 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:55.607 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.607 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.607 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.607 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.607 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.607 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2023-12-11T18:03:55.608 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2023-12-11T18:03:55.608 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.608 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.608 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.608 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.608 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.608 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.608 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.608 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.608 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.608 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.608 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.608 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.608 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2023-12-11T18:03:55.608 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2023-12-11T18:03:55.608 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.609 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.609 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.609 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2023-12-11T18:03:55.609 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.609 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.609 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:55.609 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:55.609 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:55.609 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:55.609 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:55.609 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:55.609 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:55.609 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:55.609 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.609 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.609 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.609 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.610 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.610 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2023-12-11T18:03:55.610 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2023-12-11T18:03:55.610 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.610 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.610 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.610 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.610 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.610 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.610 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.610 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.610 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.610 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.610 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.610 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:55.611 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:55.611 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.611 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-12-11T18:03:55.611 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "telemetry", 2023-12-11T18:03:55.611 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2023-12-11T18:03:55.611 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2023-12-11T18:03:55.611 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2023-12-11T18:03:55.611 INFO:teuthology.orchestra.run.smithi059.stdout: "channel_basic": { 2023-12-11T18:03:55.611 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "channel_basic", 2023-12-11T18:03:55.611 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.611 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.611 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.611 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2023-12-11T18:03:55.611 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.611 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.611 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.612 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Share basic cluster information (size, version)", 2023-12-11T18:03:55.612 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.612 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.612 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.612 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.612 INFO:teuthology.orchestra.run.smithi059.stdout: "channel_crash": { 2023-12-11T18:03:55.612 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "channel_crash", 2023-12-11T18:03:55.612 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.612 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.612 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.612 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2023-12-11T18:03:55.612 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.612 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.612 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.612 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-12-11T18:03:55.613 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.613 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.613 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.613 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.613 INFO:teuthology.orchestra.run.smithi059.stdout: "channel_device": { 2023-12-11T18:03:55.613 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "channel_device", 2023-12-11T18:03:55.613 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.613 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.613 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.613 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2023-12-11T18:03:55.613 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.613 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.613 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.613 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-12-11T18:03:55.614 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.614 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.614 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.614 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.614 INFO:teuthology.orchestra.run.smithi059.stdout: "channel_ident": { 2023-12-11T18:03:55.614 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "channel_ident", 2023-12-11T18:03:55.614 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.614 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.614 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.614 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.614 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.614 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.614 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.615 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-12-11T18:03:55.615 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.615 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.615 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.615 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.615 INFO:teuthology.orchestra.run.smithi059.stdout: "channel_perf": { 2023-12-11T18:03:55.615 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "channel_perf", 2023-12-11T18:03:55.615 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.615 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.615 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.615 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.615 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.615 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.615 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.615 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Share various performance metrics of a cluster", 2023-12-11T18:03:55.616 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.616 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.616 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.616 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.616 INFO:teuthology.orchestra.run.smithi059.stdout: "contact": { 2023-12-11T18:03:55.616 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "contact", 2023-12-11T18:03:55.616 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.616 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.616 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.616 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.616 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.616 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.616 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.617 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.617 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.617 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.617 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.617 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.617 INFO:teuthology.orchestra.run.smithi059.stdout: "description": { 2023-12-11T18:03:55.617 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "description", 2023-12-11T18:03:55.617 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.617 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.617 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.617 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.617 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.617 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.618 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.618 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.618 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.618 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.618 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.618 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.618 INFO:teuthology.orchestra.run.smithi059.stdout: "device_url": { 2023-12-11T18:03:55.618 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "device_url", 2023-12-11T18:03:55.618 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.618 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.618 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.618 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-12-11T18:03:55.618 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.618 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.618 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.619 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.619 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.619 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.619 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.619 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.619 INFO:teuthology.orchestra.run.smithi059.stdout: "enabled": { 2023-12-11T18:03:55.619 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "enabled", 2023-12-11T18:03:55.619 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.619 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.619 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.619 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.619 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.619 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.619 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.620 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.620 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.620 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.620 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.620 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.620 INFO:teuthology.orchestra.run.smithi059.stdout: "interval": { 2023-12-11T18:03:55.620 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "interval", 2023-12-11T18:03:55.620 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2023-12-11T18:03:55.620 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.620 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.620 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "24", 2023-12-11T18:03:55.620 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "8", 2023-12-11T18:03:55.620 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.620 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.621 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.621 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.621 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.621 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.621 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.621 INFO:teuthology.orchestra.run.smithi059.stdout: "last_opt_revision": { 2023-12-11T18:03:55.621 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "last_opt_revision", 2023-12-11T18:03:55.621 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2023-12-11T18:03:55.621 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.621 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.621 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "1", 2023-12-11T18:03:55.621 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.621 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.622 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.622 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.622 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.622 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.622 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.622 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.622 INFO:teuthology.orchestra.run.smithi059.stdout: "leaderboard": { 2023-12-11T18:03:55.622 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "leaderboard", 2023-12-11T18:03:55.622 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.622 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.622 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.623 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.623 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.623 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.623 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.623 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.623 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.623 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.623 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.623 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.623 INFO:teuthology.orchestra.run.smithi059.stdout: "leaderboard_description": { 2023-12-11T18:03:55.623 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "leaderboard_description", 2023-12-11T18:03:55.623 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.623 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.623 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.623 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.624 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.624 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.624 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.624 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.624 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.624 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.624 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.624 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.624 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2023-12-11T18:03:55.624 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2023-12-11T18:03:55.624 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.624 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.624 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.624 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.624 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.624 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.625 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:55.625 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:55.625 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:55.625 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:55.625 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:55.625 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:55.625 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:55.625 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:55.625 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.625 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.625 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.625 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.625 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.625 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2023-12-11T18:03:55.625 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2023-12-11T18:03:55.626 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.626 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.626 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.626 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.626 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.626 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.626 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.626 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.626 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.626 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.626 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.626 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.626 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2023-12-11T18:03:55.627 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2023-12-11T18:03:55.627 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.627 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.627 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.627 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2023-12-11T18:03:55.627 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.627 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.627 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:55.627 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:55.627 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:55.627 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:55.628 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:55.628 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:55.628 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:55.628 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:55.628 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.628 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.628 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.628 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.628 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.628 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2023-12-11T18:03:55.628 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2023-12-11T18:03:55.628 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.628 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.628 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.628 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.629 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.629 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.629 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.629 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.629 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.629 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.629 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.629 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.629 INFO:teuthology.orchestra.run.smithi059.stdout: "organization": { 2023-12-11T18:03:55.629 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "organization", 2023-12-11T18:03:55.629 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.630 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.630 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.630 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.630 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.630 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.630 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.630 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.630 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.630 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.630 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.630 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.630 INFO:teuthology.orchestra.run.smithi059.stdout: "proxy": { 2023-12-11T18:03:55.630 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "proxy", 2023-12-11T18:03:55.631 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.631 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.631 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.631 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.631 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.631 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.631 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.631 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.631 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.631 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.631 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.631 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.631 INFO:teuthology.orchestra.run.smithi059.stdout: "url": { 2023-12-11T18:03:55.632 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "url", 2023-12-11T18:03:55.632 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.632 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.632 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.632 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-12-11T18:03:55.632 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.632 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.632 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.632 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.632 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.632 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.632 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.632 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:55.633 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:55.633 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.633 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-12-11T18:03:55.633 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "test_orchestrator", 2023-12-11T18:03:55.633 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2023-12-11T18:03:55.633 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2023-12-11T18:03:55.633 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2023-12-11T18:03:55.633 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2023-12-11T18:03:55.633 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2023-12-11T18:03:55.633 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.633 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.633 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.633 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.633 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.634 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.634 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:55.634 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:55.634 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:55.634 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:55.634 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:55.634 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:55.634 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:55.634 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:55.634 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.634 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.635 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.635 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.635 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.635 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2023-12-11T18:03:55.635 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2023-12-11T18:03:55.635 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.635 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.635 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.635 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.635 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.635 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.635 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.635 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.635 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.636 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.636 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.636 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.636 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2023-12-11T18:03:55.636 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2023-12-11T18:03:55.636 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.636 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.636 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.636 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2023-12-11T18:03:55.636 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.636 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.636 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:55.636 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:55.636 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:55.637 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:55.637 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:55.637 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:55.637 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:55.637 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:55.637 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.637 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.637 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.637 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.637 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.637 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2023-12-11T18:03:55.637 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2023-12-11T18:03:55.637 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.638 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.638 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.638 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.638 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.638 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.638 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.638 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.638 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.638 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.638 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.638 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:55.639 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:55.639 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.639 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-12-11T18:03:55.639 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "volumes", 2023-12-11T18:03:55.639 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2023-12-11T18:03:55.639 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2023-12-11T18:03:55.639 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2023-12-11T18:03:55.639 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2023-12-11T18:03:55.639 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2023-12-11T18:03:55.639 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.639 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.639 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.639 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.639 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.640 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.640 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:55.640 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:55.640 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:55.640 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:55.640 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:55.640 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:55.640 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:55.640 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:55.640 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.640 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.640 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.640 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.640 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.641 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2023-12-11T18:03:55.641 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2023-12-11T18:03:55.641 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.641 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.641 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.641 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.641 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.641 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.641 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.641 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.641 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.641 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.641 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.641 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.642 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2023-12-11T18:03:55.642 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2023-12-11T18:03:55.642 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.642 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.642 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.642 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2023-12-11T18:03:55.642 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.642 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.642 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:55.642 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:55.642 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:55.642 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:55.642 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:55.643 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:55.643 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:55.643 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:55.643 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.643 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.643 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.643 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.643 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.643 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2023-12-11T18:03:55.643 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2023-12-11T18:03:55.643 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.643 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.643 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.644 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.644 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.644 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.644 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.644 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.644 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.644 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.644 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.644 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.644 INFO:teuthology.orchestra.run.smithi059.stdout: "max_concurrent_clones": { 2023-12-11T18:03:55.644 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "max_concurrent_clones", 2023-12-11T18:03:55.644 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2023-12-11T18:03:55.645 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.645 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.645 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "4", 2023-12-11T18:03:55.645 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.645 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.645 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.645 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Number of asynchronous cloner threads", 2023-12-11T18:03:55.645 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.645 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.645 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.645 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.645 INFO:teuthology.orchestra.run.smithi059.stdout: "periodic_async_work": { 2023-12-11T18:03:55.645 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "periodic_async_work", 2023-12-11T18:03:55.645 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.646 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.646 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.646 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.646 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.646 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.646 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.646 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Periodically check for async work", 2023-12-11T18:03:55.646 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.646 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.646 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.646 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.646 INFO:teuthology.orchestra.run.smithi059.stdout: "snapshot_clone_delay": { 2023-12-11T18:03:55.647 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "snapshot_clone_delay", 2023-12-11T18:03:55.647 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2023-12-11T18:03:55.647 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.647 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.647 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2023-12-11T18:03:55.647 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.647 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.647 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.647 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-12-11T18:03:55.647 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.647 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.647 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.647 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:55.647 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:55.647 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.648 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-12-11T18:03:55.648 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "zabbix", 2023-12-11T18:03:55.648 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2023-12-11T18:03:55.648 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2023-12-11T18:03:55.648 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2023-12-11T18:03:55.648 INFO:teuthology.orchestra.run.smithi059.stdout: "discovery_interval": { 2023-12-11T18:03:55.648 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "discovery_interval", 2023-12-11T18:03:55.648 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "uint", 2023-12-11T18:03:55.648 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.648 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.648 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "100", 2023-12-11T18:03:55.648 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.648 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.648 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.649 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.649 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.649 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.649 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.649 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.649 INFO:teuthology.orchestra.run.smithi059.stdout: "identifier": { 2023-12-11T18:03:55.649 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "identifier", 2023-12-11T18:03:55.649 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.649 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.649 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.649 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.649 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.649 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.650 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.650 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.650 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.650 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.650 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.650 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.650 INFO:teuthology.orchestra.run.smithi059.stdout: "interval": { 2023-12-11T18:03:55.650 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "interval", 2023-12-11T18:03:55.650 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2023-12-11T18:03:55.650 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.650 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.650 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "60", 2023-12-11T18:03:55.650 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.651 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.651 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.651 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.651 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.651 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.651 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.651 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.651 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2023-12-11T18:03:55.651 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2023-12-11T18:03:55.651 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.651 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.651 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.651 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.651 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.651 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.652 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:55.652 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:55.652 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:55.652 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:55.652 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:55.652 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:55.652 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:55.652 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:55.652 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.652 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.652 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.652 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.652 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.652 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2023-12-11T18:03:55.653 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2023-12-11T18:03:55.653 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.653 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.653 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.653 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.653 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.653 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.653 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.653 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.653 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.653 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.653 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.653 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.653 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2023-12-11T18:03:55.653 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2023-12-11T18:03:55.654 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.654 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.654 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.654 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2023-12-11T18:03:55.654 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.654 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.654 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:55.654 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:55.654 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:55.654 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:55.654 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:55.654 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:55.654 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:55.655 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:55.655 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.655 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.655 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.655 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.655 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.655 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2023-12-11T18:03:55.655 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2023-12-11T18:03:55.655 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.655 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.655 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.655 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.655 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.655 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.656 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.656 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.656 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.656 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.656 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.656 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.656 INFO:teuthology.orchestra.run.smithi059.stdout: "zabbix_host": { 2023-12-11T18:03:55.656 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "zabbix_host", 2023-12-11T18:03:55.656 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.656 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.656 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.656 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.656 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.656 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.657 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.657 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.657 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.657 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.657 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.657 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.657 INFO:teuthology.orchestra.run.smithi059.stdout: "zabbix_port": { 2023-12-11T18:03:55.657 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "zabbix_port", 2023-12-11T18:03:55.657 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2023-12-11T18:03:55.657 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.657 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.658 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "10051", 2023-12-11T18:03:55.658 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.658 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.658 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.658 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.658 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.658 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.658 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.658 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.658 INFO:teuthology.orchestra.run.smithi059.stdout: "zabbix_sender": { 2023-12-11T18:03:55.658 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "zabbix_sender", 2023-12-11T18:03:55.658 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.658 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.659 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.659 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-12-11T18:03:55.659 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.659 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.659 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.659 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.659 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.659 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.659 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.659 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:55.659 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:55.659 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:55.659 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2023-12-11T18:03:55.659 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.659 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-12-11T18:03:55.660 INFO:teuthology.orchestra.run.smithi059.stdout: "gid": 25145, 2023-12-11T18:03:55.660 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "y", 2023-12-11T18:03:55.660 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr_features": 4540138322906710015, 2023-12-11T18:03:55.660 INFO:teuthology.orchestra.run.smithi059.stdout: "available_modules": [ 2023-12-11T18:03:55.660 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-12-11T18:03:55.660 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "alerts", 2023-12-11T18:03:55.660 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2023-12-11T18:03:55.660 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2023-12-11T18:03:55.660 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2023-12-11T18:03:55.660 INFO:teuthology.orchestra.run.smithi059.stdout: "interval": { 2023-12-11T18:03:55.660 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "interval", 2023-12-11T18:03:55.660 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2023-12-11T18:03:55.660 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.660 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.660 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "60", 2023-12-11T18:03:55.660 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.661 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.661 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.661 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "How frequently to reexamine health status", 2023-12-11T18:03:55.661 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.661 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.661 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.661 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.661 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2023-12-11T18:03:55.661 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2023-12-11T18:03:55.661 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.661 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.661 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.661 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.662 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.662 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.662 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:55.662 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:55.662 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:55.662 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:55.662 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:55.662 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:55.662 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:55.662 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:55.662 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.662 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.662 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.663 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.663 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.663 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2023-12-11T18:03:55.663 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2023-12-11T18:03:55.663 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.663 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.663 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.663 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.663 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.663 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.663 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.663 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.663 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.664 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.664 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.664 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.664 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2023-12-11T18:03:55.664 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2023-12-11T18:03:55.664 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.664 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.664 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.664 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2023-12-11T18:03:55.664 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.664 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.664 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:55.665 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:55.665 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:55.665 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:55.665 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:55.665 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:55.665 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:55.665 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:55.665 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.665 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.665 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.665 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.665 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.665 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2023-12-11T18:03:55.666 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2023-12-11T18:03:55.666 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.666 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.666 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.666 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.666 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.666 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.666 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.666 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.666 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.666 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.666 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.667 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.667 INFO:teuthology.orchestra.run.smithi059.stdout: "smtp_destination": { 2023-12-11T18:03:55.667 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "smtp_destination", 2023-12-11T18:03:55.667 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.667 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.667 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.667 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.667 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.667 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.667 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.667 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Email address to send alerts to", 2023-12-11T18:03:55.667 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.667 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.667 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.668 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.668 INFO:teuthology.orchestra.run.smithi059.stdout: "smtp_from_name": { 2023-12-11T18:03:55.668 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "smtp_from_name", 2023-12-11T18:03:55.668 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.668 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.668 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.668 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "Ceph", 2023-12-11T18:03:55.668 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.668 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.668 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.668 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Email From: name", 2023-12-11T18:03:55.668 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.668 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.668 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.669 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.669 INFO:teuthology.orchestra.run.smithi059.stdout: "smtp_host": { 2023-12-11T18:03:55.669 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "smtp_host", 2023-12-11T18:03:55.669 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.669 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.669 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.669 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.669 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.669 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.669 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.669 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "SMTP server", 2023-12-11T18:03:55.669 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.669 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.669 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.670 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.670 INFO:teuthology.orchestra.run.smithi059.stdout: "smtp_password": { 2023-12-11T18:03:55.670 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "smtp_password", 2023-12-11T18:03:55.670 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.670 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.670 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.670 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.670 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.670 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.670 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.670 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Password to authenticate with", 2023-12-11T18:03:55.670 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.671 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.671 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.671 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.671 INFO:teuthology.orchestra.run.smithi059.stdout: "smtp_port": { 2023-12-11T18:03:55.671 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "smtp_port", 2023-12-11T18:03:55.671 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2023-12-11T18:03:55.671 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.671 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.671 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "465", 2023-12-11T18:03:55.671 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.672 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.672 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.672 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "SMTP port", 2023-12-11T18:03:55.672 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.672 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.672 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.672 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.672 INFO:teuthology.orchestra.run.smithi059.stdout: "smtp_sender": { 2023-12-11T18:03:55.672 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "smtp_sender", 2023-12-11T18:03:55.672 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.672 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.672 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.672 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.672 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.673 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.673 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.673 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "SMTP envelope sender", 2023-12-11T18:03:55.673 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.673 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.673 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.673 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.673 INFO:teuthology.orchestra.run.smithi059.stdout: "smtp_ssl": { 2023-12-11T18:03:55.673 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "smtp_ssl", 2023-12-11T18:03:55.673 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.673 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.673 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.674 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2023-12-11T18:03:55.674 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.674 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.674 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.674 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Use SSL to connect to SMTP server", 2023-12-11T18:03:55.674 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.674 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.674 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.674 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.674 INFO:teuthology.orchestra.run.smithi059.stdout: "smtp_user": { 2023-12-11T18:03:55.674 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "smtp_user", 2023-12-11T18:03:55.674 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.674 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.674 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.675 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.675 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.675 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.675 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.675 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "User to authenticate as", 2023-12-11T18:03:55.675 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.675 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.675 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.675 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:55.675 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:55.675 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.675 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-12-11T18:03:55.675 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "balancer", 2023-12-11T18:03:55.675 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2023-12-11T18:03:55.676 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2023-12-11T18:03:55.676 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2023-12-11T18:03:55.676 INFO:teuthology.orchestra.run.smithi059.stdout: "active": { 2023-12-11T18:03:55.676 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "active", 2023-12-11T18:03:55.676 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.676 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.676 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.676 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2023-12-11T18:03:55.676 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.676 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.676 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.676 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "automatically balance PGs across cluster", 2023-12-11T18:03:55.676 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.676 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.676 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.677 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.677 INFO:teuthology.orchestra.run.smithi059.stdout: "begin_time": { 2023-12-11T18:03:55.677 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "begin_time", 2023-12-11T18:03:55.677 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.677 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.677 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.677 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0000", 2023-12-11T18:03:55.677 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.677 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.677 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.677 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "beginning time of day to automatically balance", 2023-12-11T18:03:55.677 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-12-11T18:03:55.678 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.678 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.678 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.678 INFO:teuthology.orchestra.run.smithi059.stdout: "begin_weekday": { 2023-12-11T18:03:55.678 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "begin_weekday", 2023-12-11T18:03:55.678 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "uint", 2023-12-11T18:03:55.678 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.678 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.678 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2023-12-11T18:03:55.678 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "0", 2023-12-11T18:03:55.678 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "6", 2023-12-11T18:03:55.679 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.679 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-12-11T18:03:55.679 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-12-11T18:03:55.679 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.679 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.679 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.679 INFO:teuthology.orchestra.run.smithi059.stdout: "crush_compat_max_iterations": { 2023-12-11T18:03:55.679 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "crush_compat_max_iterations", 2023-12-11T18:03:55.679 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "uint", 2023-12-11T18:03:55.679 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.679 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.679 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "25", 2023-12-11T18:03:55.679 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "1", 2023-12-11T18:03:55.679 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "250", 2023-12-11T18:03:55.680 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.680 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-12-11T18:03:55.680 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.680 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.680 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.680 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.680 INFO:teuthology.orchestra.run.smithi059.stdout: "crush_compat_metrics": { 2023-12-11T18:03:55.680 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "crush_compat_metrics", 2023-12-11T18:03:55.680 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.680 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.680 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.680 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "pgs,objects,bytes", 2023-12-11T18:03:55.680 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.680 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.681 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.681 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-12-11T18:03:55.681 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-12-11T18:03:55.681 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.681 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.681 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.681 INFO:teuthology.orchestra.run.smithi059.stdout: "crush_compat_step": { 2023-12-11T18:03:55.681 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "crush_compat_step", 2023-12-11T18:03:55.681 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "float", 2023-12-11T18:03:55.681 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.681 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.681 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0.5", 2023-12-11T18:03:55.681 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "0.001", 2023-12-11T18:03:55.681 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "0.999", 2023-12-11T18:03:55.681 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.682 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "aggressiveness of optimization", 2023-12-11T18:03:55.682 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-12-11T18:03:55.682 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.682 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.682 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.682 INFO:teuthology.orchestra.run.smithi059.stdout: "end_time": { 2023-12-11T18:03:55.682 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "end_time", 2023-12-11T18:03:55.682 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.682 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.682 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.682 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "2359", 2023-12-11T18:03:55.682 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.683 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.683 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.683 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "ending time of day to automatically balance", 2023-12-11T18:03:55.683 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-12-11T18:03:55.683 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.683 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.683 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.683 INFO:teuthology.orchestra.run.smithi059.stdout: "end_weekday": { 2023-12-11T18:03:55.683 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "end_weekday", 2023-12-11T18:03:55.683 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "uint", 2023-12-11T18:03:55.683 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.683 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.683 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2023-12-11T18:03:55.683 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "0", 2023-12-11T18:03:55.684 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "6", 2023-12-11T18:03:55.684 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.684 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-12-11T18:03:55.684 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-12-11T18:03:55.684 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.684 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.685 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.685 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2023-12-11T18:03:55.685 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2023-12-11T18:03:55.685 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.685 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.685 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.686 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.686 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.686 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.686 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:55.686 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:55.686 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:55.686 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:55.686 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:55.686 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:55.687 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:55.687 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:55.687 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.687 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.687 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.687 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.688 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.688 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2023-12-11T18:03:55.688 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2023-12-11T18:03:55.688 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.688 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.688 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.689 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.689 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.689 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.689 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.689 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.690 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.690 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.690 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.690 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.690 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2023-12-11T18:03:55.693 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2023-12-11T18:03:55.693 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.693 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.693 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.694 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2023-12-11T18:03:55.694 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.694 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.694 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:55.694 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:55.694 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:55.694 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:55.694 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:55.694 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:55.694 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:55.694 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:55.694 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.695 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.695 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.695 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.695 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.695 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2023-12-11T18:03:55.695 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2023-12-11T18:03:55.695 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.695 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.695 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.695 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.695 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.695 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.695 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.695 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.695 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.696 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.696 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.696 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.696 INFO:teuthology.orchestra.run.smithi059.stdout: "min_score": { 2023-12-11T18:03:55.696 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "min_score", 2023-12-11T18:03:55.696 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "float", 2023-12-11T18:03:55.696 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.696 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.696 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2023-12-11T18:03:55.696 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.696 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.696 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.696 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-12-11T18:03:55.697 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.697 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.697 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.697 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.697 INFO:teuthology.orchestra.run.smithi059.stdout: "mode": { 2023-12-11T18:03:55.697 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "mode", 2023-12-11T18:03:55.697 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.697 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.697 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.697 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "upmap", 2023-12-11T18:03:55.697 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.698 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.698 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:55.698 INFO:teuthology.orchestra.run.smithi059.stdout: "crush-compat", 2023-12-11T18:03:55.698 INFO:teuthology.orchestra.run.smithi059.stdout: "none", 2023-12-11T18:03:55.698 INFO:teuthology.orchestra.run.smithi059.stdout: "upmap" 2023-12-11T18:03:55.698 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:55.698 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Balancer mode", 2023-12-11T18:03:55.699 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.699 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.699 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.699 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.699 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_ids": { 2023-12-11T18:03:55.699 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "pool_ids", 2023-12-11T18:03:55.699 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.699 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.699 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.699 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.699 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.699 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.699 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.699 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-12-11T18:03:55.700 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.700 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.700 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.700 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.700 INFO:teuthology.orchestra.run.smithi059.stdout: "sleep_interval": { 2023-12-11T18:03:55.700 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sleep_interval", 2023-12-11T18:03:55.700 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2023-12-11T18:03:55.700 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.700 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.700 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "60", 2023-12-11T18:03:55.700 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.700 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.700 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.700 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-12-11T18:03:55.701 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.701 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.701 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.701 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.701 INFO:teuthology.orchestra.run.smithi059.stdout: "upmap_max_deviation": { 2023-12-11T18:03:55.701 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "upmap_max_deviation", 2023-12-11T18:03:55.701 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2023-12-11T18:03:55.701 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.701 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.701 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "5", 2023-12-11T18:03:55.701 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "1", 2023-12-11T18:03:55.701 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.701 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.702 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "deviation below which no optimization is attempted", 2023-12-11T18:03:55.702 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-12-11T18:03:55.702 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.702 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.702 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.702 INFO:teuthology.orchestra.run.smithi059.stdout: "upmap_max_optimizations": { 2023-12-11T18:03:55.702 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "upmap_max_optimizations", 2023-12-11T18:03:55.702 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "uint", 2023-12-11T18:03:55.702 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.702 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.702 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "10", 2023-12-11T18:03:55.702 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.703 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.703 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.703 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-12-11T18:03:55.703 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.703 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.703 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.703 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:55.703 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:55.703 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.703 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-12-11T18:03:55.703 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "cephadm", 2023-12-11T18:03:55.703 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2023-12-11T18:03:55.703 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2023-12-11T18:03:55.704 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2023-12-11T18:03:55.704 INFO:teuthology.orchestra.run.smithi059.stdout: "agent_down_multiplier": { 2023-12-11T18:03:55.704 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "agent_down_multiplier", 2023-12-11T18:03:55.704 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "float", 2023-12-11T18:03:55.704 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.704 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.704 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "3.0", 2023-12-11T18:03:55.704 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.704 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.704 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.704 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-12-11T18:03:55.704 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.705 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.705 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.705 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.705 INFO:teuthology.orchestra.run.smithi059.stdout: "agent_refresh_rate": { 2023-12-11T18:03:55.705 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "agent_refresh_rate", 2023-12-11T18:03:55.705 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2023-12-11T18:03:55.705 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.705 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.705 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "20", 2023-12-11T18:03:55.706 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.706 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.706 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.706 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-12-11T18:03:55.706 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.706 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.706 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.706 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.706 INFO:teuthology.orchestra.run.smithi059.stdout: "agent_starting_port": { 2023-12-11T18:03:55.706 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "agent_starting_port", 2023-12-11T18:03:55.706 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2023-12-11T18:03:55.706 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.706 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.707 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "4721", 2023-12-11T18:03:55.707 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.707 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.707 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.707 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-12-11T18:03:55.707 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.707 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.707 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.707 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.707 INFO:teuthology.orchestra.run.smithi059.stdout: "allow_ptrace": { 2023-12-11T18:03:55.707 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "allow_ptrace", 2023-12-11T18:03:55.707 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.707 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.707 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.708 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.708 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.708 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.708 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.708 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-12-11T18:03:55.708 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-12-11T18:03:55.708 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.708 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.708 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.708 INFO:teuthology.orchestra.run.smithi059.stdout: "autotune_interval": { 2023-12-11T18:03:55.708 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "autotune_interval", 2023-12-11T18:03:55.708 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2023-12-11T18:03:55.708 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.708 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.709 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "600", 2023-12-11T18:03:55.709 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.709 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.709 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.709 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "how frequently to autotune daemon memory", 2023-12-11T18:03:55.709 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.709 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.709 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.709 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.709 INFO:teuthology.orchestra.run.smithi059.stdout: "autotune_memory_target_ratio": { 2023-12-11T18:03:55.709 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "autotune_memory_target_ratio", 2023-12-11T18:03:55.709 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "float", 2023-12-11T18:03:55.709 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.709 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.710 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0.7", 2023-12-11T18:03:55.710 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.710 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.710 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.710 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-12-11T18:03:55.710 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.710 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.710 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.710 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.710 INFO:teuthology.orchestra.run.smithi059.stdout: "cephadm_log_destination": { 2023-12-11T18:03:55.710 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "cephadm_log_destination", 2023-12-11T18:03:55.710 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.710 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.710 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.711 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.711 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.711 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.711 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:55.711 INFO:teuthology.orchestra.run.smithi059.stdout: "file", 2023-12-11T18:03:55.711 INFO:teuthology.orchestra.run.smithi059.stdout: "file,syslog", 2023-12-11T18:03:55.711 INFO:teuthology.orchestra.run.smithi059.stdout: "syslog" 2023-12-11T18:03:55.711 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:55.711 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Destination for cephadm command's persistent logging", 2023-12-11T18:03:55.711 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.711 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.711 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.711 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.712 INFO:teuthology.orchestra.run.smithi059.stdout: "cgroups_split": { 2023-12-11T18:03:55.712 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "cgroups_split", 2023-12-11T18:03:55.712 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.712 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.712 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.712 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2023-12-11T18:03:55.712 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.712 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.712 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.712 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-12-11T18:03:55.712 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.712 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.712 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.712 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.713 INFO:teuthology.orchestra.run.smithi059.stdout: "config_checks_enabled": { 2023-12-11T18:03:55.713 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "config_checks_enabled", 2023-12-11T18:03:55.713 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.713 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.713 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.713 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.713 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.713 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.713 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.713 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-12-11T18:03:55.713 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.714 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.714 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.714 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.714 INFO:teuthology.orchestra.run.smithi059.stdout: "config_dashboard": { 2023-12-11T18:03:55.714 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "config_dashboard", 2023-12-11T18:03:55.714 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.715 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.715 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.715 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2023-12-11T18:03:55.715 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.715 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.715 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.715 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-12-11T18:03:55.715 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.715 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.716 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.716 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.716 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_alertmanager": { 2023-12-11T18:03:55.716 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_alertmanager", 2023-12-11T18:03:55.716 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.716 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.716 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.716 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-12-11T18:03:55.716 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.716 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.716 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.716 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Prometheus container image", 2023-12-11T18:03:55.716 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.716 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.717 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.717 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.717 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_base": { 2023-12-11T18:03:55.717 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_base", 2023-12-11T18:03:55.717 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.717 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.717 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.717 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/ceph/ceph", 2023-12-11T18:03:55.717 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.717 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.717 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.717 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Container image name, without the tag", 2023-12-11T18:03:55.717 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.717 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.717 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.718 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.718 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_elasticsearch": { 2023-12-11T18:03:55.718 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_elasticsearch", 2023-12-11T18:03:55.718 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.718 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.718 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.718 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-12-11T18:03:55.718 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.718 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.718 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.718 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "elasticsearch container image", 2023-12-11T18:03:55.718 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.718 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.719 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.719 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.719 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_grafana": { 2023-12-11T18:03:55.719 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_grafana", 2023-12-11T18:03:55.719 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.719 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.719 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.719 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-12-11T18:03:55.719 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.719 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.719 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.720 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Prometheus container image", 2023-12-11T18:03:55.720 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.720 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.720 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.720 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.720 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_haproxy": { 2023-12-11T18:03:55.720 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_haproxy", 2023-12-11T18:03:55.721 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.721 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.721 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.721 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-12-11T18:03:55.721 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.721 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.721 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.722 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "HAproxy container image", 2023-12-11T18:03:55.722 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.722 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.722 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.722 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.722 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_jaeger_agent": { 2023-12-11T18:03:55.723 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_jaeger_agent", 2023-12-11T18:03:55.723 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.723 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.723 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.723 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-12-11T18:03:55.723 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.724 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.724 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.724 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Jaeger agent container image", 2023-12-11T18:03:55.724 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.724 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.724 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.724 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.725 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_jaeger_collector": { 2023-12-11T18:03:55.725 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_jaeger_collector", 2023-12-11T18:03:55.725 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.725 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.725 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.725 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-12-11T18:03:55.725 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.725 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.726 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.726 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Jaeger collector container image", 2023-12-11T18:03:55.726 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.726 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.726 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.726 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.727 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_jaeger_query": { 2023-12-11T18:03:55.727 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_jaeger_query", 2023-12-11T18:03:55.727 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.727 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.727 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.727 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-12-11T18:03:55.728 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.728 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.728 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.728 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Jaeger query container image", 2023-12-11T18:03:55.728 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.728 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.728 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.729 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.729 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_keepalived": { 2023-12-11T18:03:55.729 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_keepalived", 2023-12-11T18:03:55.729 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.729 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.729 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.729 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-12-11T18:03:55.729 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.729 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.729 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.730 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Keepalived container image", 2023-12-11T18:03:55.730 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.730 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.730 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.730 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.730 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_loki": { 2023-12-11T18:03:55.730 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_loki", 2023-12-11T18:03:55.730 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.730 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.730 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.730 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-12-11T18:03:55.730 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.730 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.731 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.731 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Loki container image", 2023-12-11T18:03:55.731 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.731 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.731 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.731 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.731 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_node_exporter": { 2023-12-11T18:03:55.731 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_node_exporter", 2023-12-11T18:03:55.731 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.731 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.731 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.731 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-12-11T18:03:55.731 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.731 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.732 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.732 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Prometheus container image", 2023-12-11T18:03:55.732 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.732 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.732 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.732 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.732 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_nvmeof": { 2023-12-11T18:03:55.732 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_nvmeof", 2023-12-11T18:03:55.732 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.732 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.732 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.732 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/ceph/nvmeof:latest", 2023-12-11T18:03:55.733 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.733 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.733 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.733 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Nvme-of container image", 2023-12-11T18:03:55.733 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.733 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.733 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.733 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.733 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_prometheus": { 2023-12-11T18:03:55.733 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_prometheus", 2023-12-11T18:03:55.734 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.734 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.734 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.734 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-12-11T18:03:55.734 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.734 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.734 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.734 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Prometheus container image", 2023-12-11T18:03:55.734 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.734 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.734 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.735 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.735 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_promtail": { 2023-12-11T18:03:55.735 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_promtail", 2023-12-11T18:03:55.735 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.735 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.735 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.735 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-12-11T18:03:55.735 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.735 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.735 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.735 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Promtail container image", 2023-12-11T18:03:55.735 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.735 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.735 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.735 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.736 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_snmp_gateway": { 2023-12-11T18:03:55.736 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_snmp_gateway", 2023-12-11T18:03:55.736 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.736 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.736 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.736 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-12-11T18:03:55.736 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.736 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.736 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.736 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "SNMP Gateway container image", 2023-12-11T18:03:55.736 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.736 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.737 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.737 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.737 INFO:teuthology.orchestra.run.smithi059.stdout: "container_init": { 2023-12-11T18:03:55.737 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_init", 2023-12-11T18:03:55.737 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.737 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.737 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.737 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2023-12-11T18:03:55.737 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.737 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.737 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.737 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Run podman/docker with `--init`", 2023-12-11T18:03:55.737 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.737 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.738 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.738 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.738 INFO:teuthology.orchestra.run.smithi059.stdout: "daemon_cache_timeout": { 2023-12-11T18:03:55.738 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "daemon_cache_timeout", 2023-12-11T18:03:55.738 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2023-12-11T18:03:55.738 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.738 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.738 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "600", 2023-12-11T18:03:55.738 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.738 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.738 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.738 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-12-11T18:03:55.738 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.739 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.739 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.739 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.739 INFO:teuthology.orchestra.run.smithi059.stdout: "default_cephadm_command_timeout": { 2023-12-11T18:03:55.739 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "default_cephadm_command_timeout", 2023-12-11T18:03:55.739 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2023-12-11T18:03:55.739 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.739 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.739 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "900", 2023-12-11T18:03:55.739 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.739 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.739 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.739 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-12-11T18:03:55.740 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.740 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.740 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.740 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.740 INFO:teuthology.orchestra.run.smithi059.stdout: "default_registry": { 2023-12-11T18:03:55.740 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "default_registry", 2023-12-11T18:03:55.740 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.740 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.740 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.740 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "docker.io", 2023-12-11T18:03:55.740 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.740 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.740 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.740 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-12-11T18:03:55.741 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.741 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.741 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.741 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.741 INFO:teuthology.orchestra.run.smithi059.stdout: "device_cache_timeout": { 2023-12-11T18:03:55.741 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "device_cache_timeout", 2023-12-11T18:03:55.741 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2023-12-11T18:03:55.741 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.741 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.741 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "1800", 2023-12-11T18:03:55.741 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.741 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.741 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.741 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "seconds to cache device inventory", 2023-12-11T18:03:55.742 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.742 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.742 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.742 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.742 INFO:teuthology.orchestra.run.smithi059.stdout: "device_enhanced_scan": { 2023-12-11T18:03:55.742 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "device_enhanced_scan", 2023-12-11T18:03:55.742 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.742 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.742 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.742 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.742 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.742 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.742 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.743 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Use libstoragemgmt during device scans", 2023-12-11T18:03:55.743 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.743 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.743 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.743 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.743 INFO:teuthology.orchestra.run.smithi059.stdout: "facts_cache_timeout": { 2023-12-11T18:03:55.743 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "facts_cache_timeout", 2023-12-11T18:03:55.743 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2023-12-11T18:03:55.743 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.743 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.743 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "60", 2023-12-11T18:03:55.743 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.743 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.744 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.744 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "seconds to cache host facts data", 2023-12-11T18:03:55.744 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.744 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.744 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.744 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.744 INFO:teuthology.orchestra.run.smithi059.stdout: "host_check_interval": { 2023-12-11T18:03:55.744 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "host_check_interval", 2023-12-11T18:03:55.744 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2023-12-11T18:03:55.744 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.744 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.744 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "600", 2023-12-11T18:03:55.745 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.745 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.745 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.745 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "how frequently to perform a host check", 2023-12-11T18:03:55.745 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.745 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.745 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.745 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.745 INFO:teuthology.orchestra.run.smithi059.stdout: "inventory_list_all": { 2023-12-11T18:03:55.745 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "inventory_list_all", 2023-12-11T18:03:55.745 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.745 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.745 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.745 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.746 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.746 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.746 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.746 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2023-12-11T18:03:55.746 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.746 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.746 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.746 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.746 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2023-12-11T18:03:55.746 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2023-12-11T18:03:55.746 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.746 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.746 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.747 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.747 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.747 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.747 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:55.747 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:55.747 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:55.747 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:55.747 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:55.747 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:55.747 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:55.747 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:55.747 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.747 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.747 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.748 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.748 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.748 INFO:teuthology.orchestra.run.smithi059.stdout: "log_refresh_metadata": { 2023-12-11T18:03:55.748 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_refresh_metadata", 2023-12-11T18:03:55.748 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.748 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.748 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.748 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.748 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.748 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.748 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.748 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-12-11T18:03:55.748 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.748 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.749 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.749 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.749 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2023-12-11T18:03:55.749 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2023-12-11T18:03:55.749 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.749 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.749 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.749 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2023-12-11T18:03:55.749 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.749 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.749 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.749 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-12-11T18:03:55.749 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.749 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.749 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.749 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.750 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2023-12-11T18:03:55.750 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2023-12-11T18:03:55.750 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.750 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.750 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.750 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2023-12-11T18:03:55.750 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.750 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.750 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:55.750 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:55.750 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:55.750 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:55.750 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:55.751 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:55.751 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:55.751 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:55.751 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.751 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.751 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.751 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.751 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.751 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2023-12-11T18:03:55.751 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2023-12-11T18:03:55.751 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.751 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.751 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.752 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.752 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.752 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.752 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.752 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.752 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.752 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.752 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.752 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.752 INFO:teuthology.orchestra.run.smithi059.stdout: "manage_etc_ceph_ceph_conf": { 2023-12-11T18:03:55.752 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-12-11T18:03:55.752 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.752 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.753 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.753 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.753 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.753 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.753 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.753 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-12-11T18:03:55.753 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.753 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.753 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.753 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.753 INFO:teuthology.orchestra.run.smithi059.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-12-11T18:03:55.753 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-12-11T18:03:55.753 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.753 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.754 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.754 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "*", 2023-12-11T18:03:55.754 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.754 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.754 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.754 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-12-11T18:03:55.754 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.754 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.754 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.754 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.754 INFO:teuthology.orchestra.run.smithi059.stdout: "max_count_per_host": { 2023-12-11T18:03:55.754 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "max_count_per_host", 2023-12-11T18:03:55.754 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2023-12-11T18:03:55.755 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.755 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.755 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "10", 2023-12-11T18:03:55.755 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.755 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.755 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.755 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "max number of daemons per service per host", 2023-12-11T18:03:55.755 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.755 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.755 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.755 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.755 INFO:teuthology.orchestra.run.smithi059.stdout: "max_osd_draining_count": { 2023-12-11T18:03:55.755 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "max_osd_draining_count", 2023-12-11T18:03:55.755 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2023-12-11T18:03:55.756 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.756 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.756 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "10", 2023-12-11T18:03:55.756 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.756 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.756 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.756 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-12-11T18:03:55.756 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.756 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.756 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.756 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.756 INFO:teuthology.orchestra.run.smithi059.stdout: "migration_current": { 2023-12-11T18:03:55.756 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "migration_current", 2023-12-11T18:03:55.756 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2023-12-11T18:03:55.757 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.757 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.757 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.757 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.757 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.757 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.757 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "internal - do not modify", 2023-12-11T18:03:55.757 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.757 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.757 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.757 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.757 INFO:teuthology.orchestra.run.smithi059.stdout: "mode": { 2023-12-11T18:03:55.757 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "mode", 2023-12-11T18:03:55.757 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.757 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.758 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.758 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "root", 2023-12-11T18:03:55.758 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.758 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.758 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:55.758 INFO:teuthology.orchestra.run.smithi059.stdout: "cephadm-package", 2023-12-11T18:03:55.758 INFO:teuthology.orchestra.run.smithi059.stdout: "root" 2023-12-11T18:03:55.759 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:55.759 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "mode for remote execution of cephadm", 2023-12-11T18:03:55.759 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.759 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.759 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.759 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.759 INFO:teuthology.orchestra.run.smithi059.stdout: "prometheus_alerts_path": { 2023-12-11T18:03:55.760 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "prometheus_alerts_path", 2023-12-11T18:03:55.760 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.760 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.760 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.760 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-12-11T18:03:55.760 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.761 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.761 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.761 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-12-11T18:03:55.761 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.761 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.761 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.761 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.762 INFO:teuthology.orchestra.run.smithi059.stdout: "registry_insecure": { 2023-12-11T18:03:55.762 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "registry_insecure", 2023-12-11T18:03:55.762 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.762 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.762 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.762 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.762 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.763 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.763 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.763 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-12-11T18:03:55.763 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.763 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.763 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.763 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.763 INFO:teuthology.orchestra.run.smithi059.stdout: "registry_password": { 2023-12-11T18:03:55.764 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "registry_password", 2023-12-11T18:03:55.764 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.764 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.764 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.764 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.764 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.764 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.764 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.764 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-12-11T18:03:55.764 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.765 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.765 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.765 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.765 INFO:teuthology.orchestra.run.smithi059.stdout: "registry_url": { 2023-12-11T18:03:55.765 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "registry_url", 2023-12-11T18:03:55.765 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.765 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.765 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.765 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.765 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.765 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.765 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.765 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-12-11T18:03:55.765 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.766 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.766 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.766 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.766 INFO:teuthology.orchestra.run.smithi059.stdout: "registry_username": { 2023-12-11T18:03:55.766 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "registry_username", 2023-12-11T18:03:55.766 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.766 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.766 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.766 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.766 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.767 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.767 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.767 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-12-11T18:03:55.767 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.767 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.767 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.767 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.767 INFO:teuthology.orchestra.run.smithi059.stdout: "secure_monitoring_stack": { 2023-12-11T18:03:55.767 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "secure_monitoring_stack", 2023-12-11T18:03:55.768 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.768 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.768 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.768 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.768 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.768 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.768 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.768 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-12-11T18:03:55.769 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.769 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.769 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.769 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.769 INFO:teuthology.orchestra.run.smithi059.stdout: "service_discovery_port": { 2023-12-11T18:03:55.769 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "service_discovery_port", 2023-12-11T18:03:55.769 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2023-12-11T18:03:55.769 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.770 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.770 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "8765", 2023-12-11T18:03:55.770 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.770 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.770 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.770 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "cephadm service discovery port", 2023-12-11T18:03:55.770 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.770 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.770 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.770 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.771 INFO:teuthology.orchestra.run.smithi059.stdout: "ssh_config_file": { 2023-12-11T18:03:55.771 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ssh_config_file", 2023-12-11T18:03:55.771 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.771 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.771 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.771 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.771 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.771 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.771 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.772 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-12-11T18:03:55.772 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.772 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.772 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.772 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.772 INFO:teuthology.orchestra.run.smithi059.stdout: "use_agent": { 2023-12-11T18:03:55.772 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "use_agent", 2023-12-11T18:03:55.772 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.773 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.773 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.773 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.773 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.773 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.773 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.773 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-12-11T18:03:55.773 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.773 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.774 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.774 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.774 INFO:teuthology.orchestra.run.smithi059.stdout: "use_repo_digest": { 2023-12-11T18:03:55.774 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "use_repo_digest", 2023-12-11T18:03:55.774 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.774 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.774 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.775 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2023-12-11T18:03:55.775 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.775 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.775 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.775 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-12-11T18:03:55.775 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.775 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.775 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.776 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.776 INFO:teuthology.orchestra.run.smithi059.stdout: "warn_on_failed_host_check": { 2023-12-11T18:03:55.776 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "warn_on_failed_host_check", 2023-12-11T18:03:55.776 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.776 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.776 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.776 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2023-12-11T18:03:55.776 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.777 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.777 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.777 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "raise a health warning if the host check fails", 2023-12-11T18:03:55.777 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.777 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.777 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.777 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.777 INFO:teuthology.orchestra.run.smithi059.stdout: "warn_on_stray_daemons": { 2023-12-11T18:03:55.777 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "warn_on_stray_daemons", 2023-12-11T18:03:55.778 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.778 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.778 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.778 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2023-12-11T18:03:55.778 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.778 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.778 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.778 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-12-11T18:03:55.779 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.779 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.779 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.779 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.779 INFO:teuthology.orchestra.run.smithi059.stdout: "warn_on_stray_hosts": { 2023-12-11T18:03:55.779 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "warn_on_stray_hosts", 2023-12-11T18:03:55.779 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.779 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.780 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.780 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2023-12-11T18:03:55.780 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.780 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.780 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.780 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-12-11T18:03:55.780 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.780 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.780 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.781 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:55.781 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:55.781 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.781 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-12-11T18:03:55.781 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "crash", 2023-12-11T18:03:55.781 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2023-12-11T18:03:55.781 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2023-12-11T18:03:55.781 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2023-12-11T18:03:55.781 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2023-12-11T18:03:55.782 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2023-12-11T18:03:55.782 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.782 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.782 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.782 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.782 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.782 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.782 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:55.782 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:55.782 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:55.782 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:55.783 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:55.783 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:55.783 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:55.783 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:55.783 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.783 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.783 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.783 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.783 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.784 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2023-12-11T18:03:55.784 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2023-12-11T18:03:55.784 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.784 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.784 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.784 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.784 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.784 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.784 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.784 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.784 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.784 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.784 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.784 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.784 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2023-12-11T18:03:55.785 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2023-12-11T18:03:55.785 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.785 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.785 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.785 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2023-12-11T18:03:55.785 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.785 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.785 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:55.785 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:55.785 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:55.785 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:55.785 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:55.785 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:55.786 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:55.786 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:55.786 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.786 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.786 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.786 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.786 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.786 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2023-12-11T18:03:55.786 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2023-12-11T18:03:55.786 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.786 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.786 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.786 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.787 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.787 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.787 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.787 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.787 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.787 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.787 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.787 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.787 INFO:teuthology.orchestra.run.smithi059.stdout: "retain_interval": { 2023-12-11T18:03:55.787 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "retain_interval", 2023-12-11T18:03:55.787 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2023-12-11T18:03:55.787 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.787 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.787 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "31536000", 2023-12-11T18:03:55.787 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.788 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.788 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.788 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "how long to retain crashes before pruning them", 2023-12-11T18:03:55.788 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.788 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.788 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.788 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.788 INFO:teuthology.orchestra.run.smithi059.stdout: "warn_recent_interval": { 2023-12-11T18:03:55.788 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "warn_recent_interval", 2023-12-11T18:03:55.788 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2023-12-11T18:03:55.788 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.788 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.788 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "1209600", 2023-12-11T18:03:55.788 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.789 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.789 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.789 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "time interval in which to warn about recent crashes", 2023-12-11T18:03:55.789 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.789 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.789 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.789 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:55.789 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:55.789 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.789 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-12-11T18:03:55.789 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "dashboard", 2023-12-11T18:03:55.789 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2023-12-11T18:03:55.789 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2023-12-11T18:03:55.789 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2023-12-11T18:03:55.790 INFO:teuthology.orchestra.run.smithi059.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-12-11T18:03:55.790 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-12-11T18:03:55.790 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2023-12-11T18:03:55.790 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.790 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.790 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "10", 2023-12-11T18:03:55.790 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.790 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.790 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.790 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.790 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.790 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.790 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.791 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.791 INFO:teuthology.orchestra.run.smithi059.stdout: "ALERTMANAGER_API_HOST": { 2023-12-11T18:03:55.791 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ALERTMANAGER_API_HOST", 2023-12-11T18:03:55.791 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.791 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.791 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.791 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.791 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.791 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.791 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.791 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.791 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.791 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.791 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.792 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.792 INFO:teuthology.orchestra.run.smithi059.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-12-11T18:03:55.792 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-12-11T18:03:55.792 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.792 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.792 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.792 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2023-12-11T18:03:55.792 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.792 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.792 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.792 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.792 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.792 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.792 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.792 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.793 INFO:teuthology.orchestra.run.smithi059.stdout: "AUDIT_API_ENABLED": { 2023-12-11T18:03:55.793 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "AUDIT_API_ENABLED", 2023-12-11T18:03:55.793 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.793 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.793 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.793 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.793 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.793 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.793 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.793 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.793 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.793 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.793 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.793 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.794 INFO:teuthology.orchestra.run.smithi059.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-12-11T18:03:55.794 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-12-11T18:03:55.794 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.794 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.794 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.794 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2023-12-11T18:03:55.794 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.794 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.794 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.794 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.794 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.795 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.795 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.795 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.795 INFO:teuthology.orchestra.run.smithi059.stdout: "ENABLE_BROWSABLE_API": { 2023-12-11T18:03:55.795 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ENABLE_BROWSABLE_API", 2023-12-11T18:03:55.795 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.795 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.795 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.795 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2023-12-11T18:03:55.795 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.795 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.795 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.795 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.795 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.796 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.796 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.796 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.796 INFO:teuthology.orchestra.run.smithi059.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-12-11T18:03:55.796 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-12-11T18:03:55.796 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.796 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.796 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.796 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2023-12-11T18:03:55.796 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.796 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.796 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.796 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.796 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.796 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.797 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.797 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.797 INFO:teuthology.orchestra.run.smithi059.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-12-11T18:03:55.797 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-12-11T18:03:55.797 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.797 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.797 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.797 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2023-12-11T18:03:55.797 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.797 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.797 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.797 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.797 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.797 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.798 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.798 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.798 INFO:teuthology.orchestra.run.smithi059.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-12-11T18:03:55.798 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-12-11T18:03:55.798 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.798 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.798 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.798 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2023-12-11T18:03:55.798 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.798 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.798 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.798 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.798 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.798 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.799 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.799 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.799 INFO:teuthology.orchestra.run.smithi059.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-12-11T18:03:55.799 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-12-11T18:03:55.799 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.799 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.799 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.799 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2023-12-11T18:03:55.799 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.799 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.799 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.799 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.799 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.799 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.800 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.800 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.800 INFO:teuthology.orchestra.run.smithi059.stdout: "FEATURE_TOGGLE_NFS": { 2023-12-11T18:03:55.800 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-12-11T18:03:55.800 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.800 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.800 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.800 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2023-12-11T18:03:55.800 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.800 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.800 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.800 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.800 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.800 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.800 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.801 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.801 INFO:teuthology.orchestra.run.smithi059.stdout: "FEATURE_TOGGLE_RBD": { 2023-12-11T18:03:55.801 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-12-11T18:03:55.801 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.801 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.801 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.801 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2023-12-11T18:03:55.801 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.801 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.801 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.801 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.801 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.802 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.802 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.802 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.802 INFO:teuthology.orchestra.run.smithi059.stdout: "FEATURE_TOGGLE_RGW": { 2023-12-11T18:03:55.802 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-12-11T18:03:55.802 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.802 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.803 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.803 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2023-12-11T18:03:55.803 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.803 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.803 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.803 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.803 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.804 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.804 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.804 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.804 INFO:teuthology.orchestra.run.smithi059.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-12-11T18:03:55.804 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-12-11T18:03:55.804 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.805 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.805 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.805 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.805 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.805 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.805 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.806 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.806 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.806 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.806 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.806 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.806 INFO:teuthology.orchestra.run.smithi059.stdout: "GRAFANA_API_PASSWORD": { 2023-12-11T18:03:55.806 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "GRAFANA_API_PASSWORD", 2023-12-11T18:03:55.807 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.807 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.807 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.807 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "admin", 2023-12-11T18:03:55.807 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.807 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.807 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.807 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.808 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.808 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.808 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.808 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.808 INFO:teuthology.orchestra.run.smithi059.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-12-11T18:03:55.808 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-12-11T18:03:55.809 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.809 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.809 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.809 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2023-12-11T18:03:55.809 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.809 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.809 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.810 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.810 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.810 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.810 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.810 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.811 INFO:teuthology.orchestra.run.smithi059.stdout: "GRAFANA_API_URL": { 2023-12-11T18:03:55.811 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "GRAFANA_API_URL", 2023-12-11T18:03:55.811 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.811 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.811 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.811 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.811 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.811 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.811 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.812 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.812 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.812 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.812 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.812 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.812 INFO:teuthology.orchestra.run.smithi059.stdout: "GRAFANA_API_USERNAME": { 2023-12-11T18:03:55.812 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "GRAFANA_API_USERNAME", 2023-12-11T18:03:55.812 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.812 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.812 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.812 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "admin", 2023-12-11T18:03:55.812 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.812 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.813 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.813 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.813 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.813 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.813 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.813 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.813 INFO:teuthology.orchestra.run.smithi059.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-12-11T18:03:55.813 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-12-11T18:03:55.813 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.813 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.813 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.813 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.813 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.814 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.814 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.814 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.814 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.814 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.814 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.814 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.814 INFO:teuthology.orchestra.run.smithi059.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-12-11T18:03:55.814 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-12-11T18:03:55.814 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.814 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.814 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.815 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.815 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.815 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.815 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.815 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.815 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.815 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.815 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.815 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.815 INFO:teuthology.orchestra.run.smithi059.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-12-11T18:03:55.815 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-12-11T18:03:55.815 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.815 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.815 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.816 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2023-12-11T18:03:55.816 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.816 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.816 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.816 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.816 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.816 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.816 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.816 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.816 INFO:teuthology.orchestra.run.smithi059.stdout: "ISSUE_TRACKER_API_KEY": { 2023-12-11T18:03:55.816 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-12-11T18:03:55.816 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.816 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.816 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.816 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.817 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.817 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.817 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.817 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.817 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.817 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.817 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.817 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.817 INFO:teuthology.orchestra.run.smithi059.stdout: "PROMETHEUS_API_HOST": { 2023-12-11T18:03:55.817 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PROMETHEUS_API_HOST", 2023-12-11T18:03:55.817 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.817 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.817 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.817 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.817 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.818 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.818 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.818 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.818 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.818 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.818 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.818 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.818 INFO:teuthology.orchestra.run.smithi059.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-12-11T18:03:55.818 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-12-11T18:03:55.818 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.818 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.818 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.818 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2023-12-11T18:03:55.819 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.819 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.819 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.819 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.819 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.819 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.819 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.819 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.819 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-12-11T18:03:55.819 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-12-11T18:03:55.819 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.819 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.819 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.820 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.820 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.820 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.820 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.820 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.820 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.820 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.820 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.820 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.820 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-12-11T18:03:55.820 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-12-11T18:03:55.820 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.820 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.820 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.820 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.821 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.821 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.821 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.821 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.821 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.821 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.821 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.821 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.821 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-12-11T18:03:55.821 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-12-11T18:03:55.821 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.821 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.821 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.821 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2023-12-11T18:03:55.821 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.822 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.822 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.822 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.822 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.822 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.822 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.822 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.822 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-12-11T18:03:55.822 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-12-11T18:03:55.822 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.822 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.822 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.822 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2023-12-11T18:03:55.823 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.823 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.823 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.823 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.823 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.823 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.823 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.823 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.823 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-12-11T18:03:55.823 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-12-11T18:03:55.823 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.823 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.824 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.824 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.824 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.824 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.824 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.824 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.824 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.824 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.824 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.825 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.825 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-12-11T18:03:55.825 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-12-11T18:03:55.825 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.825 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.825 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.825 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.825 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.825 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.825 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.825 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.825 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.825 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.825 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.826 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.826 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-12-11T18:03:55.826 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-12-11T18:03:55.826 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.826 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.826 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.826 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.826 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.826 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.826 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.826 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.826 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.826 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.827 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.827 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.827 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_ENABLED": { 2023-12-11T18:03:55.827 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_ENABLED", 2023-12-11T18:03:55.827 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.827 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.827 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.827 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2023-12-11T18:03:55.827 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.827 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.827 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.827 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.827 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.827 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.828 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.828 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.828 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-12-11T18:03:55.828 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-12-11T18:03:55.828 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.828 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.828 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.828 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-12-11T18:03:55.828 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.828 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.828 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.828 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.828 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.828 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.829 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.829 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.829 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-12-11T18:03:55.829 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-12-11T18:03:55.829 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2023-12-11T18:03:55.829 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.829 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.829 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "10", 2023-12-11T18:03:55.829 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.829 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.829 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.829 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.829 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.830 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.830 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.830 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.830 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-12-11T18:03:55.830 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-12-11T18:03:55.830 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2023-12-11T18:03:55.830 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.830 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.830 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "8", 2023-12-11T18:03:55.830 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.830 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.831 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.831 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.831 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.831 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.831 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.831 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.831 INFO:teuthology.orchestra.run.smithi059.stdout: "REST_REQUESTS_TIMEOUT": { 2023-12-11T18:03:55.831 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-12-11T18:03:55.831 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2023-12-11T18:03:55.831 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.831 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.831 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "45", 2023-12-11T18:03:55.831 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.831 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.832 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.832 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.832 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.832 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.832 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.832 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.832 INFO:teuthology.orchestra.run.smithi059.stdout: "RGW_API_ACCESS_KEY": { 2023-12-11T18:03:55.832 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "RGW_API_ACCESS_KEY", 2023-12-11T18:03:55.832 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.832 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.832 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.832 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.832 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.832 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.833 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.833 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.833 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.833 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.833 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.833 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.833 INFO:teuthology.orchestra.run.smithi059.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-12-11T18:03:55.833 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-12-11T18:03:55.833 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.833 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.833 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.834 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "admin", 2023-12-11T18:03:55.834 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.834 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.834 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.834 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.834 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.834 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.834 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.834 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.834 INFO:teuthology.orchestra.run.smithi059.stdout: "RGW_API_SECRET_KEY": { 2023-12-11T18:03:55.834 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "RGW_API_SECRET_KEY", 2023-12-11T18:03:55.834 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.834 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.835 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.835 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.835 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.835 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.835 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.835 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.835 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.835 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.835 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.835 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.835 INFO:teuthology.orchestra.run.smithi059.stdout: "RGW_API_SSL_VERIFY": { 2023-12-11T18:03:55.835 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "RGW_API_SSL_VERIFY", 2023-12-11T18:03:55.835 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.835 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.836 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.836 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2023-12-11T18:03:55.836 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.836 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.836 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.836 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.836 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.836 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.836 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.836 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.836 INFO:teuthology.orchestra.run.smithi059.stdout: "UNSAFE_TLS_v1_2": { 2023-12-11T18:03:55.836 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "UNSAFE_TLS_v1_2", 2023-12-11T18:03:55.836 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.836 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.837 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.837 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.837 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.837 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.837 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.837 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.837 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.837 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.837 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.837 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.837 INFO:teuthology.orchestra.run.smithi059.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-12-11T18:03:55.837 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-12-11T18:03:55.837 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2023-12-11T18:03:55.837 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.837 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.838 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2023-12-11T18:03:55.838 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.838 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.838 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.838 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.838 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.838 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.838 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.838 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.838 INFO:teuthology.orchestra.run.smithi059.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-12-11T18:03:55.838 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-12-11T18:03:55.838 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2023-12-11T18:03:55.839 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.839 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.839 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "10", 2023-12-11T18:03:55.839 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.839 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.839 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.839 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.839 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.839 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.839 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.839 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.839 INFO:teuthology.orchestra.run.smithi059.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-12-11T18:03:55.839 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-12-11T18:03:55.839 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2023-12-11T18:03:55.840 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.840 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.840 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "5", 2023-12-11T18:03:55.840 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.840 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.840 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.840 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.840 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.840 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.840 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.840 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.840 INFO:teuthology.orchestra.run.smithi059.stdout: "cross_origin_url": { 2023-12-11T18:03:55.840 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "cross_origin_url", 2023-12-11T18:03:55.840 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.840 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.840 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.841 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.841 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.841 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.841 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.841 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.841 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.841 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.841 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.841 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.841 INFO:teuthology.orchestra.run.smithi059.stdout: "crt_file": { 2023-12-11T18:03:55.841 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "crt_file", 2023-12-11T18:03:55.841 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.841 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.841 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.841 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.842 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.842 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.842 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.842 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.842 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.842 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.842 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.842 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.842 INFO:teuthology.orchestra.run.smithi059.stdout: "debug": { 2023-12-11T18:03:55.842 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "debug", 2023-12-11T18:03:55.842 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.842 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.843 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.843 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.843 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.843 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.843 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.843 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Enable/disable debug options", 2023-12-11T18:03:55.843 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.843 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.843 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.843 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.843 INFO:teuthology.orchestra.run.smithi059.stdout: "jwt_token_ttl": { 2023-12-11T18:03:55.843 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "jwt_token_ttl", 2023-12-11T18:03:55.843 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2023-12-11T18:03:55.844 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.844 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.844 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "28800", 2023-12-11T18:03:55.844 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.844 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.844 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.844 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.844 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.844 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.844 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.844 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.844 INFO:teuthology.orchestra.run.smithi059.stdout: "key_file": { 2023-12-11T18:03:55.844 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "key_file", 2023-12-11T18:03:55.844 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.845 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.845 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.845 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.845 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.845 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.845 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.845 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.845 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.845 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.845 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.845 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.845 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2023-12-11T18:03:55.845 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2023-12-11T18:03:55.846 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.846 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.846 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.846 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.846 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.846 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.846 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:55.846 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:55.846 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:55.846 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:55.846 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:55.846 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:55.846 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:55.847 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:55.847 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.847 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.847 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.847 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.847 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.847 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2023-12-11T18:03:55.847 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2023-12-11T18:03:55.847 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.847 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.847 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.847 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.847 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.847 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.847 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.848 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.848 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.848 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.848 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.848 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.848 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2023-12-11T18:03:55.848 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2023-12-11T18:03:55.848 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.848 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.848 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.848 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2023-12-11T18:03:55.848 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.848 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.848 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:55.849 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:55.849 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:55.849 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:55.849 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:55.849 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:55.849 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:55.849 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:55.849 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.849 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.849 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.849 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.849 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.849 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2023-12-11T18:03:55.849 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2023-12-11T18:03:55.849 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.849 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.849 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.850 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.850 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.850 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.850 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.850 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.850 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.850 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.850 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.850 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.850 INFO:teuthology.orchestra.run.smithi059.stdout: "motd": { 2023-12-11T18:03:55.850 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "motd", 2023-12-11T18:03:55.850 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.850 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.850 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.851 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.851 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.851 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.851 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.851 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "The message of the day", 2023-12-11T18:03:55.851 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.851 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.851 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.851 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.851 INFO:teuthology.orchestra.run.smithi059.stdout: "redirect_resolve_ip_addr": { 2023-12-11T18:03:55.851 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "redirect_resolve_ip_addr", 2023-12-11T18:03:55.851 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.851 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.851 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.851 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.852 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.852 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.852 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.852 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.852 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.852 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.852 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.852 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.852 INFO:teuthology.orchestra.run.smithi059.stdout: "server_addr": { 2023-12-11T18:03:55.852 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "server_addr", 2023-12-11T18:03:55.852 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.852 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.852 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.852 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "::", 2023-12-11T18:03:55.852 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.853 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.853 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.853 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.853 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.853 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.853 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.853 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.853 INFO:teuthology.orchestra.run.smithi059.stdout: "server_port": { 2023-12-11T18:03:55.853 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "server_port", 2023-12-11T18:03:55.853 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2023-12-11T18:03:55.853 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.853 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.853 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "8080", 2023-12-11T18:03:55.853 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.854 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.854 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.854 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.854 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.854 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.854 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.854 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.854 INFO:teuthology.orchestra.run.smithi059.stdout: "ssl": { 2023-12-11T18:03:55.854 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ssl", 2023-12-11T18:03:55.854 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.854 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.854 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.854 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2023-12-11T18:03:55.855 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.855 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.855 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.855 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.855 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.855 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.855 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.855 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.855 INFO:teuthology.orchestra.run.smithi059.stdout: "ssl_server_port": { 2023-12-11T18:03:55.855 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ssl_server_port", 2023-12-11T18:03:55.855 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2023-12-11T18:03:55.855 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.855 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.855 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "8443", 2023-12-11T18:03:55.855 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.856 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.856 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.856 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.856 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.856 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.856 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.856 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.856 INFO:teuthology.orchestra.run.smithi059.stdout: "standby_behaviour": { 2023-12-11T18:03:55.856 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "standby_behaviour", 2023-12-11T18:03:55.856 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.856 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.856 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.856 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "redirect", 2023-12-11T18:03:55.857 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.857 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.857 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:55.857 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:55.857 INFO:teuthology.orchestra.run.smithi059.stdout: "redirect" 2023-12-11T18:03:55.857 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:55.857 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.857 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.857 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.857 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.857 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.857 INFO:teuthology.orchestra.run.smithi059.stdout: "standby_error_status_code": { 2023-12-11T18:03:55.857 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "standby_error_status_code", 2023-12-11T18:03:55.857 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2023-12-11T18:03:55.857 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.858 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.858 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "500", 2023-12-11T18:03:55.858 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "400", 2023-12-11T18:03:55.858 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "599", 2023-12-11T18:03:55.858 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.858 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.858 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.858 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.858 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.858 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.858 INFO:teuthology.orchestra.run.smithi059.stdout: "url_prefix": { 2023-12-11T18:03:55.858 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "url_prefix", 2023-12-11T18:03:55.858 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.858 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.859 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.859 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.859 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.859 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.859 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.859 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.859 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.859 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.859 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.859 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:55.859 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:55.859 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.860 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-12-11T18:03:55.860 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "devicehealth", 2023-12-11T18:03:55.860 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2023-12-11T18:03:55.860 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2023-12-11T18:03:55.860 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2023-12-11T18:03:55.860 INFO:teuthology.orchestra.run.smithi059.stdout: "enable_monitoring": { 2023-12-11T18:03:55.860 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "enable_monitoring", 2023-12-11T18:03:55.860 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.860 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.860 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.860 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2023-12-11T18:03:55.860 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.860 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.860 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.861 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "monitor device health metrics", 2023-12-11T18:03:55.861 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.861 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.861 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.861 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.861 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2023-12-11T18:03:55.861 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2023-12-11T18:03:55.861 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.861 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.861 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.861 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.861 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.861 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.861 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:55.862 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:55.862 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:55.862 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:55.862 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:55.862 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:55.862 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:55.862 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:55.862 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.862 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.862 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.862 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.862 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.862 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2023-12-11T18:03:55.863 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2023-12-11T18:03:55.863 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.863 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.863 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.863 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.863 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.863 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.863 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.863 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.863 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.863 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.863 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.863 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.863 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2023-12-11T18:03:55.863 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2023-12-11T18:03:55.864 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.864 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.864 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.864 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2023-12-11T18:03:55.864 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.864 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.864 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:55.864 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:55.864 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:55.864 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:55.864 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:55.864 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:55.864 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:55.864 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:55.865 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.865 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.865 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.865 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.865 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.865 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2023-12-11T18:03:55.865 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2023-12-11T18:03:55.865 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.865 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.865 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.865 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.865 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.865 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.865 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.866 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.866 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.866 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.866 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.866 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.866 INFO:teuthology.orchestra.run.smithi059.stdout: "mark_out_threshold": { 2023-12-11T18:03:55.866 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "mark_out_threshold", 2023-12-11T18:03:55.866 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2023-12-11T18:03:55.866 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.866 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.866 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "2419200", 2023-12-11T18:03:55.866 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.866 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.866 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.867 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-12-11T18:03:55.867 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.867 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.867 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.867 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.867 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_name": { 2023-12-11T18:03:55.867 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "pool_name", 2023-12-11T18:03:55.867 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.867 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.867 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.867 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "device_health_metrics", 2023-12-11T18:03:55.867 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.867 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.867 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.868 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "name of pool in which to store device health metrics", 2023-12-11T18:03:55.868 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.868 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.868 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.868 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.868 INFO:teuthology.orchestra.run.smithi059.stdout: "retention_period": { 2023-12-11T18:03:55.868 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "retention_period", 2023-12-11T18:03:55.868 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2023-12-11T18:03:55.868 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.868 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.868 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "15552000", 2023-12-11T18:03:55.868 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.868 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.868 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.868 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "how long to retain device health metrics", 2023-12-11T18:03:55.868 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.869 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.869 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.869 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.869 INFO:teuthology.orchestra.run.smithi059.stdout: "scrape_frequency": { 2023-12-11T18:03:55.869 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "scrape_frequency", 2023-12-11T18:03:55.869 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2023-12-11T18:03:55.869 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.869 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.869 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "86400", 2023-12-11T18:03:55.869 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.869 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.869 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.869 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "how frequently to scrape device health metrics", 2023-12-11T18:03:55.869 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.869 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.870 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.870 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.870 INFO:teuthology.orchestra.run.smithi059.stdout: "self_heal": { 2023-12-11T18:03:55.870 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "self_heal", 2023-12-11T18:03:55.870 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.870 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.870 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.870 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2023-12-11T18:03:55.870 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.870 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.870 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.870 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-12-11T18:03:55.870 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.870 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.871 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.871 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.871 INFO:teuthology.orchestra.run.smithi059.stdout: "sleep_interval": { 2023-12-11T18:03:55.871 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sleep_interval", 2023-12-11T18:03:55.871 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2023-12-11T18:03:55.871 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.871 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.871 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "600", 2023-12-11T18:03:55.871 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.871 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.871 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.871 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "how frequently to wake up and check device health", 2023-12-11T18:03:55.871 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.871 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.871 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.871 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.872 INFO:teuthology.orchestra.run.smithi059.stdout: "warn_threshold": { 2023-12-11T18:03:55.872 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "warn_threshold", 2023-12-11T18:03:55.872 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2023-12-11T18:03:55.872 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.872 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.872 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "7257600", 2023-12-11T18:03:55.872 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.872 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.872 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.872 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-12-11T18:03:55.872 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.872 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.872 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.872 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:55.873 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:55.873 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.873 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-12-11T18:03:55.873 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "diskprediction_local", 2023-12-11T18:03:55.873 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2023-12-11T18:03:55.873 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2023-12-11T18:03:55.873 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2023-12-11T18:03:55.873 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2023-12-11T18:03:55.873 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2023-12-11T18:03:55.873 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.873 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.873 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.873 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.873 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.873 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.873 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:55.874 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:55.874 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:55.874 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:55.874 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:55.874 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:55.874 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:55.874 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:55.874 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.874 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.874 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.874 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.874 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.874 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2023-12-11T18:03:55.874 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2023-12-11T18:03:55.875 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.875 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.875 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.875 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.875 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.875 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.875 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.875 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.875 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.875 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.875 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.875 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.875 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2023-12-11T18:03:55.875 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2023-12-11T18:03:55.876 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.876 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.876 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.876 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2023-12-11T18:03:55.876 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.876 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.876 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:55.876 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:55.876 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:55.876 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:55.876 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:55.876 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:55.876 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:55.876 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:55.876 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.876 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.877 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.877 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.877 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.877 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2023-12-11T18:03:55.877 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2023-12-11T18:03:55.877 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.877 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.877 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.877 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.877 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.877 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.877 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.877 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.877 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.877 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.878 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.878 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.878 INFO:teuthology.orchestra.run.smithi059.stdout: "predict_interval": { 2023-12-11T18:03:55.878 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "predict_interval", 2023-12-11T18:03:55.878 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.878 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.878 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.878 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "86400", 2023-12-11T18:03:55.878 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.878 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.878 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.878 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.878 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.878 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.878 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.879 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.879 INFO:teuthology.orchestra.run.smithi059.stdout: "predictor_model": { 2023-12-11T18:03:55.879 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "predictor_model", 2023-12-11T18:03:55.879 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.879 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.879 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.879 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "prophetstor", 2023-12-11T18:03:55.879 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.879 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.879 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.879 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.879 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.879 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.879 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.879 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.879 INFO:teuthology.orchestra.run.smithi059.stdout: "sleep_interval": { 2023-12-11T18:03:55.880 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sleep_interval", 2023-12-11T18:03:55.880 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.880 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.880 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.880 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "600", 2023-12-11T18:03:55.880 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.880 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.880 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.880 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.880 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.880 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.880 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.880 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:55.880 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:55.880 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.880 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-12-11T18:03:55.880 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "influx", 2023-12-11T18:03:55.881 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": false, 2023-12-11T18:03:55.881 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "influxdb python module not found", 2023-12-11T18:03:55.881 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2023-12-11T18:03:55.881 INFO:teuthology.orchestra.run.smithi059.stdout: "batch_size": { 2023-12-11T18:03:55.881 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "batch_size", 2023-12-11T18:03:55.881 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2023-12-11T18:03:55.881 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.881 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.881 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "5000", 2023-12-11T18:03:55.881 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.881 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.881 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.881 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-12-11T18:03:55.881 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.881 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.881 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.882 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.882 INFO:teuthology.orchestra.run.smithi059.stdout: "database": { 2023-12-11T18:03:55.882 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "database", 2023-12-11T18:03:55.882 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.882 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.882 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.882 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "ceph", 2023-12-11T18:03:55.882 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.882 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.882 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.882 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-12-11T18:03:55.882 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.882 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.882 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.882 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.882 INFO:teuthology.orchestra.run.smithi059.stdout: "hostname": { 2023-12-11T18:03:55.883 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "hostname", 2023-12-11T18:03:55.883 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.883 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.883 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.883 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.883 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.883 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.883 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.883 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "InfluxDB server hostname", 2023-12-11T18:03:55.883 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.883 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.883 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.883 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.883 INFO:teuthology.orchestra.run.smithi059.stdout: "interval": { 2023-12-11T18:03:55.883 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "interval", 2023-12-11T18:03:55.884 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2023-12-11T18:03:55.884 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.884 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.884 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "30", 2023-12-11T18:03:55.884 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "5", 2023-12-11T18:03:55.884 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.884 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.884 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-12-11T18:03:55.884 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.884 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.884 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.884 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.884 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2023-12-11T18:03:55.884 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2023-12-11T18:03:55.884 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.885 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.885 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.885 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.885 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.885 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.885 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:55.885 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:55.885 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:55.885 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:55.885 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:55.885 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:55.885 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:55.885 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:55.885 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.885 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.886 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.886 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.886 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.886 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2023-12-11T18:03:55.886 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2023-12-11T18:03:55.886 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.886 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.886 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.886 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.886 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.886 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.886 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.886 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.886 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.886 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.887 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.887 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.887 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2023-12-11T18:03:55.887 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2023-12-11T18:03:55.887 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.887 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.887 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.887 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2023-12-11T18:03:55.887 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.887 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.887 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:55.887 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:55.887 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:55.887 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:55.887 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:55.887 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:55.888 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:55.888 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:55.888 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.888 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.888 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.888 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.888 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.888 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2023-12-11T18:03:55.888 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2023-12-11T18:03:55.888 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.888 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.888 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.888 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.888 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.888 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.888 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.888 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.889 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.889 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.889 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.889 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.889 INFO:teuthology.orchestra.run.smithi059.stdout: "password": { 2023-12-11T18:03:55.889 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "password", 2023-12-11T18:03:55.889 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.889 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.889 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.889 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.889 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.889 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.889 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.889 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "password of InfluxDB server user", 2023-12-11T18:03:55.889 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.890 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.890 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.890 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.890 INFO:teuthology.orchestra.run.smithi059.stdout: "port": { 2023-12-11T18:03:55.890 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "port", 2023-12-11T18:03:55.890 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2023-12-11T18:03:55.890 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.890 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.890 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "8086", 2023-12-11T18:03:55.890 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.890 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.890 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.890 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "InfluxDB server port", 2023-12-11T18:03:55.890 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.890 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.891 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.891 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.891 INFO:teuthology.orchestra.run.smithi059.stdout: "ssl": { 2023-12-11T18:03:55.891 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ssl", 2023-12-11T18:03:55.891 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.891 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.891 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.891 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "false", 2023-12-11T18:03:55.891 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.891 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.891 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.891 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-12-11T18:03:55.891 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.891 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.891 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.892 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.892 INFO:teuthology.orchestra.run.smithi059.stdout: "threads": { 2023-12-11T18:03:55.892 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "threads", 2023-12-11T18:03:55.892 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2023-12-11T18:03:55.892 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.892 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.892 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "5", 2023-12-11T18:03:55.892 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "1", 2023-12-11T18:03:55.892 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "32", 2023-12-11T18:03:55.892 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.892 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-12-11T18:03:55.892 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.892 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.892 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.893 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.893 INFO:teuthology.orchestra.run.smithi059.stdout: "username": { 2023-12-11T18:03:55.893 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "username", 2023-12-11T18:03:55.893 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.893 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.893 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.893 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.893 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.893 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.893 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.893 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "username of InfluxDB server user", 2023-12-11T18:03:55.893 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.893 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.893 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.893 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.894 INFO:teuthology.orchestra.run.smithi059.stdout: "verify_ssl": { 2023-12-11T18:03:55.894 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "verify_ssl", 2023-12-11T18:03:55.894 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.894 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.894 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.894 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "true", 2023-12-11T18:03:55.894 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.894 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.894 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.897 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-12-11T18:03:55.897 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.897 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.897 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.897 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:55.897 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:55.897 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.897 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-12-11T18:03:55.897 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "insights", 2023-12-11T18:03:55.898 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2023-12-11T18:03:55.898 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2023-12-11T18:03:55.898 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2023-12-11T18:03:55.898 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2023-12-11T18:03:55.898 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2023-12-11T18:03:55.898 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.898 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.898 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.898 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.898 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.898 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.898 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:55.898 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:55.898 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:55.898 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:55.898 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:55.899 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:55.899 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:55.899 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:55.899 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.899 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.899 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.899 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.899 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.899 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2023-12-11T18:03:55.899 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2023-12-11T18:03:55.899 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.899 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.899 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.899 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.899 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.899 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.900 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.900 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.900 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.900 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.900 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.900 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.900 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2023-12-11T18:03:55.900 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2023-12-11T18:03:55.900 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.900 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.900 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.900 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2023-12-11T18:03:55.901 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.901 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.901 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:55.901 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:55.901 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:55.901 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:55.901 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:55.901 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:55.901 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:55.901 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:55.901 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.901 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.901 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.901 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.901 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.901 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2023-12-11T18:03:55.902 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2023-12-11T18:03:55.902 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.902 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.902 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.902 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.902 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.902 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.902 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.902 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.902 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.902 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.902 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.902 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:55.902 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:55.902 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.903 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-12-11T18:03:55.903 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "iostat", 2023-12-11T18:03:55.903 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2023-12-11T18:03:55.903 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2023-12-11T18:03:55.903 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2023-12-11T18:03:55.903 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2023-12-11T18:03:55.903 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2023-12-11T18:03:55.903 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.903 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.903 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.903 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.903 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.903 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.903 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:55.903 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:55.903 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:55.904 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:55.904 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:55.904 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:55.904 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:55.904 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:55.904 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.904 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.904 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.904 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.904 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.904 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2023-12-11T18:03:55.904 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2023-12-11T18:03:55.904 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.904 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.904 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.904 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.905 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.905 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.905 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.905 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.905 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.905 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.905 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.905 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.905 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2023-12-11T18:03:55.905 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2023-12-11T18:03:55.905 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.905 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.905 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.905 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2023-12-11T18:03:55.905 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.905 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.906 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:55.906 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:55.906 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:55.906 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:55.906 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:55.906 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:55.906 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:55.906 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:55.906 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.907 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.907 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.907 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.907 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.907 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2023-12-11T18:03:55.907 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2023-12-11T18:03:55.907 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.907 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.907 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.907 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.907 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.907 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.907 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.908 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.908 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.908 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.908 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.908 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:55.908 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:55.908 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.908 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-12-11T18:03:55.908 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "k8sevents", 2023-12-11T18:03:55.908 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2023-12-11T18:03:55.908 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2023-12-11T18:03:55.908 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2023-12-11T18:03:55.908 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_event_retention_days": { 2023-12-11T18:03:55.908 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ceph_event_retention_days", 2023-12-11T18:03:55.908 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2023-12-11T18:03:55.908 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.909 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.909 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "7", 2023-12-11T18:03:55.909 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.909 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.909 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.909 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Days to hold ceph event information within local cache", 2023-12-11T18:03:55.909 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.909 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.909 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.909 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.909 INFO:teuthology.orchestra.run.smithi059.stdout: "config_check_secs": { 2023-12-11T18:03:55.909 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "config_check_secs", 2023-12-11T18:03:55.909 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2023-12-11T18:03:55.909 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.909 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.909 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "10", 2023-12-11T18:03:55.910 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "10", 2023-12-11T18:03:55.910 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.910 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.910 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-12-11T18:03:55.910 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.910 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.910 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.910 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.910 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2023-12-11T18:03:55.910 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2023-12-11T18:03:55.910 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.910 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.910 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.910 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.910 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.910 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.911 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:55.911 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:55.911 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:55.911 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:55.911 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:55.911 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:55.911 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:55.911 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:55.911 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.911 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.911 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.911 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.911 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.911 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2023-12-11T18:03:55.911 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2023-12-11T18:03:55.912 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.912 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.912 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.912 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.912 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.912 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.912 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.912 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.912 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.912 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.912 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.912 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.912 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2023-12-11T18:03:55.912 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2023-12-11T18:03:55.912 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.913 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.913 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.913 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2023-12-11T18:03:55.913 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.913 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.913 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:55.913 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:55.913 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:55.913 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:55.913 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:55.913 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:55.913 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:55.913 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:55.913 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.913 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.913 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.914 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.914 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.914 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2023-12-11T18:03:55.914 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2023-12-11T18:03:55.914 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.914 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.914 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.914 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.914 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.914 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.914 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.914 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.914 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.914 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.915 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.915 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:55.915 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:55.915 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.915 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-12-11T18:03:55.915 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "localpool", 2023-12-11T18:03:55.915 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2023-12-11T18:03:55.915 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2023-12-11T18:03:55.915 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2023-12-11T18:03:55.915 INFO:teuthology.orchestra.run.smithi059.stdout: "failure_domain": { 2023-12-11T18:03:55.915 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "failure_domain", 2023-12-11T18:03:55.915 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.915 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.915 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.916 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "host", 2023-12-11T18:03:55.916 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.916 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.916 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.916 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "failure domain for any created local pool", 2023-12-11T18:03:55.916 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-12-11T18:03:55.916 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.916 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.916 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.916 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2023-12-11T18:03:55.916 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2023-12-11T18:03:55.916 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.916 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.916 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.917 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.917 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.917 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.917 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:55.917 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:55.917 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:55.917 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:55.917 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:55.917 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:55.917 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:55.917 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:55.917 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.917 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.917 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.918 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.918 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.918 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2023-12-11T18:03:55.918 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2023-12-11T18:03:55.918 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.918 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.918 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.918 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.918 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.918 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.918 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.918 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.918 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.918 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.919 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.919 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.919 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2023-12-11T18:03:55.919 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2023-12-11T18:03:55.919 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.919 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.919 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.919 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2023-12-11T18:03:55.919 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.919 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.919 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:55.919 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:55.919 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:55.920 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:55.920 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:55.920 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:55.920 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:55.920 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:55.920 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.920 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.920 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.920 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.920 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.920 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2023-12-11T18:03:55.920 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2023-12-11T18:03:55.920 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.920 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.921 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.921 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.921 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.921 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.921 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.921 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.921 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.921 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.921 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.921 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.921 INFO:teuthology.orchestra.run.smithi059.stdout: "min_size": { 2023-12-11T18:03:55.921 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "min_size", 2023-12-11T18:03:55.921 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2023-12-11T18:03:55.921 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.921 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.922 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.922 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.922 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.922 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.922 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "default min_size for any created local pool", 2023-12-11T18:03:55.922 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-12-11T18:03:55.922 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.922 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.922 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.922 INFO:teuthology.orchestra.run.smithi059.stdout: "num_rep": { 2023-12-11T18:03:55.922 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "num_rep", 2023-12-11T18:03:55.922 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2023-12-11T18:03:55.922 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.923 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.923 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "3", 2023-12-11T18:03:55.923 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.923 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.923 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.923 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "default replica count for any created local pool", 2023-12-11T18:03:55.923 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.923 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.923 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.923 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.923 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num": { 2023-12-11T18:03:55.923 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "pg_num", 2023-12-11T18:03:55.923 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2023-12-11T18:03:55.923 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.924 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.924 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "128", 2023-12-11T18:03:55.924 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.924 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.924 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.924 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "default pg_num for any created local pool", 2023-12-11T18:03:55.924 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.924 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.924 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.924 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.924 INFO:teuthology.orchestra.run.smithi059.stdout: "prefix": { 2023-12-11T18:03:55.924 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "prefix", 2023-12-11T18:03:55.924 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.924 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.925 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.925 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.925 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.925 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.925 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.925 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "name prefix for any created local pool", 2023-12-11T18:03:55.925 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.925 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.925 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.925 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.925 INFO:teuthology.orchestra.run.smithi059.stdout: "subtree": { 2023-12-11T18:03:55.925 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "subtree", 2023-12-11T18:03:55.925 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.926 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.926 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.926 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "rack", 2023-12-11T18:03:55.926 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.926 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.926 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.926 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "CRUSH level for which to create a local pool", 2023-12-11T18:03:55.926 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-12-11T18:03:55.926 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.926 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.926 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:55.926 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:55.926 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.927 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-12-11T18:03:55.927 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "mds_autoscaler", 2023-12-11T18:03:55.927 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2023-12-11T18:03:55.927 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2023-12-11T18:03:55.927 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2023-12-11T18:03:55.927 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2023-12-11T18:03:55.927 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2023-12-11T18:03:55.927 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.927 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.927 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.927 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.927 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.927 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.927 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:55.927 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:55.927 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:55.928 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:55.928 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:55.928 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:55.928 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:55.928 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:55.928 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.928 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.928 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.928 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.928 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.928 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2023-12-11T18:03:55.928 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2023-12-11T18:03:55.928 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.928 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.928 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.928 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.929 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.929 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.929 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.929 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.929 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.929 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.929 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.929 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.929 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2023-12-11T18:03:55.929 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2023-12-11T18:03:55.929 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.929 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.929 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.929 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2023-12-11T18:03:55.929 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.930 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.930 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:55.930 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:55.930 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:55.930 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:55.930 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:55.930 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:55.930 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:55.930 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:55.930 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.930 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.930 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.931 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.931 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.931 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2023-12-11T18:03:55.931 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2023-12-11T18:03:55.931 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.931 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.931 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.931 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.932 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.932 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.932 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.932 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.932 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.932 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.932 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.932 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:55.932 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:55.932 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.932 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-12-11T18:03:55.932 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "mirroring", 2023-12-11T18:03:55.932 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2023-12-11T18:03:55.932 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2023-12-11T18:03:55.932 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2023-12-11T18:03:55.932 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2023-12-11T18:03:55.933 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2023-12-11T18:03:55.933 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.933 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.933 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.933 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.933 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.933 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.933 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:55.933 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:55.933 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:55.933 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:55.933 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:55.933 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:55.933 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:55.934 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:55.934 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.934 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.934 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.934 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.934 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.934 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2023-12-11T18:03:55.934 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2023-12-11T18:03:55.934 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.934 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.934 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.934 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.935 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.935 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.935 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.935 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.935 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.935 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.935 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.935 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.935 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2023-12-11T18:03:55.935 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2023-12-11T18:03:55.935 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.935 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.935 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.935 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2023-12-11T18:03:55.936 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.936 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.936 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:55.936 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:55.936 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:55.936 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:55.936 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:55.936 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:55.936 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:55.936 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:55.936 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.936 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.936 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.936 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.936 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.936 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2023-12-11T18:03:55.937 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2023-12-11T18:03:55.937 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.937 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.937 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.937 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.937 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.937 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.937 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.937 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.937 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.937 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.937 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.937 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:55.937 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:55.937 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.938 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-12-11T18:03:55.938 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "nfs", 2023-12-11T18:03:55.938 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2023-12-11T18:03:55.938 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2023-12-11T18:03:55.938 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2023-12-11T18:03:55.938 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2023-12-11T18:03:55.938 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2023-12-11T18:03:55.938 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.938 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.938 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.938 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.938 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.939 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.939 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:55.939 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:55.939 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:55.939 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:55.939 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:55.939 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:55.939 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:55.939 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:55.939 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.939 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.939 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.939 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.939 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.940 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2023-12-11T18:03:55.940 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2023-12-11T18:03:55.940 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.940 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.940 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.940 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.940 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.940 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.940 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.940 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.940 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.940 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.940 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.940 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.940 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2023-12-11T18:03:55.940 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2023-12-11T18:03:55.941 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.941 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.941 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.941 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2023-12-11T18:03:55.941 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.941 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.941 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:55.941 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:55.941 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:55.941 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:55.941 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:55.941 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:55.941 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:55.941 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:55.942 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.942 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.942 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.942 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.942 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.942 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2023-12-11T18:03:55.942 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2023-12-11T18:03:55.942 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.942 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.942 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.942 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.942 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.942 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.942 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.943 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.943 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.943 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.943 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.943 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:55.943 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:55.943 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.943 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-12-11T18:03:55.943 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "orchestrator", 2023-12-11T18:03:55.943 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2023-12-11T18:03:55.943 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2023-12-11T18:03:55.943 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2023-12-11T18:03:55.943 INFO:teuthology.orchestra.run.smithi059.stdout: "fail_fs": { 2023-12-11T18:03:55.943 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "fail_fs", 2023-12-11T18:03:55.944 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.944 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.944 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.944 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.944 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.944 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.944 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.944 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-12-11T18:03:55.944 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.944 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.944 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.944 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.944 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2023-12-11T18:03:55.944 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2023-12-11T18:03:55.944 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.944 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.945 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.945 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.945 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.945 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.945 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:55.945 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:55.945 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:55.945 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:55.945 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:55.945 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:55.945 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:55.946 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:55.946 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.946 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.946 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.946 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.946 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.946 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2023-12-11T18:03:55.946 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2023-12-11T18:03:55.946 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.946 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.946 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.946 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.946 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.946 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.947 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.947 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.947 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.947 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.947 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.947 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.947 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2023-12-11T18:03:55.947 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2023-12-11T18:03:55.947 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.947 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.947 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.947 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2023-12-11T18:03:55.947 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.947 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.948 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:55.948 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:55.948 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:55.948 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:55.948 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:55.948 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:55.948 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:55.948 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:55.948 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.948 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.948 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.948 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.948 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.948 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2023-12-11T18:03:55.949 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2023-12-11T18:03:55.949 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.949 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.949 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.949 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.949 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.949 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.949 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.949 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.949 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.949 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.949 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.949 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.949 INFO:teuthology.orchestra.run.smithi059.stdout: "orchestrator": { 2023-12-11T18:03:55.950 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "orchestrator", 2023-12-11T18:03:55.950 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.950 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.950 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.950 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.950 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.950 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.950 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:55.950 INFO:teuthology.orchestra.run.smithi059.stdout: "cephadm", 2023-12-11T18:03:55.950 INFO:teuthology.orchestra.run.smithi059.stdout: "rook", 2023-12-11T18:03:55.950 INFO:teuthology.orchestra.run.smithi059.stdout: "test_orchestrator" 2023-12-11T18:03:55.950 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:55.950 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Orchestrator backend", 2023-12-11T18:03:55.950 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.951 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.951 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.951 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:55.951 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:55.951 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.951 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-12-11T18:03:55.951 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "osd_perf_query", 2023-12-11T18:03:55.951 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2023-12-11T18:03:55.951 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2023-12-11T18:03:55.951 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2023-12-11T18:03:55.951 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2023-12-11T18:03:55.951 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2023-12-11T18:03:55.951 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.951 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.952 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.952 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.952 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.952 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.952 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:55.952 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:55.952 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:55.952 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:55.952 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:55.952 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:55.952 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:55.952 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:55.952 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.952 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.953 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.953 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.953 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.953 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2023-12-11T18:03:55.953 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2023-12-11T18:03:55.953 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.953 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.953 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.953 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.953 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.953 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.953 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.953 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.953 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.954 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.954 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.954 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.954 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2023-12-11T18:03:55.954 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2023-12-11T18:03:55.954 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.954 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.954 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.954 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2023-12-11T18:03:55.954 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.954 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.954 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:55.954 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:55.954 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:55.955 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:55.955 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:55.955 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:55.955 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:55.955 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:55.955 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.955 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.955 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.955 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.955 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.955 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2023-12-11T18:03:55.955 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2023-12-11T18:03:55.955 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.955 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.955 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.955 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.955 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.956 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.956 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.956 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.956 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.956 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.956 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.956 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:55.956 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:55.956 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.956 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-12-11T18:03:55.956 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "osd_support", 2023-12-11T18:03:55.956 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2023-12-11T18:03:55.956 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2023-12-11T18:03:55.956 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2023-12-11T18:03:55.956 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2023-12-11T18:03:55.957 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2023-12-11T18:03:55.957 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.957 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.957 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.957 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.957 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.957 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.957 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:55.957 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:55.957 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:55.957 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:55.957 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:55.957 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:55.957 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:55.957 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:55.958 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.958 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.958 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.958 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.958 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.958 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2023-12-11T18:03:55.958 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2023-12-11T18:03:55.958 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.958 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.958 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.958 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.958 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.958 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.959 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.959 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.959 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.959 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.959 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.959 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.959 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2023-12-11T18:03:55.959 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2023-12-11T18:03:55.959 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.959 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.959 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.959 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2023-12-11T18:03:55.960 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.960 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.960 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:55.960 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:55.960 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:55.960 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:55.960 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:55.960 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:55.960 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:55.960 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:55.960 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.960 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.960 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.960 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.961 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.961 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2023-12-11T18:03:55.961 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2023-12-11T18:03:55.961 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.961 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.961 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.961 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.961 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.961 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.961 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.961 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.961 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.961 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.961 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.961 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:55.962 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:55.962 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.962 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-12-11T18:03:55.962 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "pg_autoscaler", 2023-12-11T18:03:55.962 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2023-12-11T18:03:55.962 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2023-12-11T18:03:55.962 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2023-12-11T18:03:55.962 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2023-12-11T18:03:55.962 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2023-12-11T18:03:55.962 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.962 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.962 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.962 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.962 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.962 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.963 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:55.963 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:55.963 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:55.963 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:55.963 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:55.963 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:55.963 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:55.963 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:55.963 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.963 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.963 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.963 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.963 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.963 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2023-12-11T18:03:55.964 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2023-12-11T18:03:55.964 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.964 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.964 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.964 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.964 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.964 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.964 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.964 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.964 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.964 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.964 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.964 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.964 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2023-12-11T18:03:55.964 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2023-12-11T18:03:55.964 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.965 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.965 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.965 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2023-12-11T18:03:55.965 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.965 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.965 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:55.965 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:55.965 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:55.965 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:55.965 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:55.965 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:55.965 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:55.965 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:55.965 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.965 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.965 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.965 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.966 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.966 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2023-12-11T18:03:55.966 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2023-12-11T18:03:55.966 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.966 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.966 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.966 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.966 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.966 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.966 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.966 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.966 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.966 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.966 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.967 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.967 INFO:teuthology.orchestra.run.smithi059.stdout: "sleep_interval": { 2023-12-11T18:03:55.967 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sleep_interval", 2023-12-11T18:03:55.967 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2023-12-11T18:03:55.967 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.967 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.967 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "60", 2023-12-11T18:03:55.967 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.967 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.967 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.967 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.967 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.967 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.967 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.967 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.967 INFO:teuthology.orchestra.run.smithi059.stdout: "threshold": { 2023-12-11T18:03:55.967 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "threshold", 2023-12-11T18:03:55.968 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "float", 2023-12-11T18:03:55.968 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.968 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.968 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "3.0", 2023-12-11T18:03:55.968 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "1.0", 2023-12-11T18:03:55.968 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.968 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.968 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "scaling threshold", 2023-12-11T18:03:55.968 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-12-11T18:03:55.968 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.968 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.968 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:55.968 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:55.968 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.968 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-12-11T18:03:55.968 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "progress", 2023-12-11T18:03:55.969 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2023-12-11T18:03:55.969 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2023-12-11T18:03:55.969 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2023-12-11T18:03:55.969 INFO:teuthology.orchestra.run.smithi059.stdout: "allow_pg_recovery_event": { 2023-12-11T18:03:55.969 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "allow_pg_recovery_event", 2023-12-11T18:03:55.969 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.969 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.969 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.969 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.969 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.969 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.969 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.969 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "allow the module to show pg recovery progress", 2023-12-11T18:03:55.969 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.969 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.970 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.970 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.970 INFO:teuthology.orchestra.run.smithi059.stdout: "enabled": { 2023-12-11T18:03:55.970 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "enabled", 2023-12-11T18:03:55.970 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.970 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.970 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.970 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2023-12-11T18:03:55.970 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.970 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.970 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.970 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.970 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.970 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.970 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.971 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.971 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2023-12-11T18:03:55.971 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2023-12-11T18:03:55.971 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.971 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.971 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.971 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.971 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.971 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.971 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:55.971 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:55.971 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:55.971 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:55.971 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:55.972 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:55.972 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:55.972 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:55.972 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.972 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.972 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.972 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.972 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.972 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2023-12-11T18:03:55.972 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2023-12-11T18:03:55.972 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.972 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.972 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.972 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.972 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.972 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.973 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.973 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.973 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.973 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.973 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.973 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.973 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2023-12-11T18:03:55.973 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2023-12-11T18:03:55.973 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.973 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.973 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.973 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2023-12-11T18:03:55.973 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.973 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.974 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:55.974 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:55.974 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:55.974 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:55.974 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:55.974 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:55.974 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:55.974 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:55.974 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.974 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.974 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.974 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.974 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.974 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2023-12-11T18:03:55.975 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2023-12-11T18:03:55.975 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.975 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.975 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.975 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.975 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.975 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.975 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.975 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.975 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.975 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.975 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.975 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.975 INFO:teuthology.orchestra.run.smithi059.stdout: "max_completed_events": { 2023-12-11T18:03:55.975 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "max_completed_events", 2023-12-11T18:03:55.976 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2023-12-11T18:03:55.976 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.976 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.976 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "50", 2023-12-11T18:03:55.976 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.976 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.976 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.976 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "number of past completed events to remember", 2023-12-11T18:03:55.976 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.976 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.976 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.976 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.976 INFO:teuthology.orchestra.run.smithi059.stdout: "sleep_interval": { 2023-12-11T18:03:55.976 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sleep_interval", 2023-12-11T18:03:55.976 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2023-12-11T18:03:55.977 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.977 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.977 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "5", 2023-12-11T18:03:55.977 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.977 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.977 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.977 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "how long the module is going to sleep", 2023-12-11T18:03:55.977 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.977 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.977 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.977 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:55.977 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:55.977 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.977 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-12-11T18:03:55.977 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "prometheus", 2023-12-11T18:03:55.977 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2023-12-11T18:03:55.978 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2023-12-11T18:03:55.978 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2023-12-11T18:03:55.978 INFO:teuthology.orchestra.run.smithi059.stdout: "cache": { 2023-12-11T18:03:55.978 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "cache", 2023-12-11T18:03:55.978 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.978 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.978 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.978 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2023-12-11T18:03:55.978 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.978 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.978 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.978 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.978 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.978 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.979 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.979 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.979 INFO:teuthology.orchestra.run.smithi059.stdout: "exclude_perf_counters": { 2023-12-11T18:03:55.979 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "exclude_perf_counters", 2023-12-11T18:03:55.979 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.979 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.979 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.979 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2023-12-11T18:03:55.979 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.979 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.979 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.979 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-12-11T18:03:55.979 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-12-11T18:03:55.979 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.979 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.980 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.980 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2023-12-11T18:03:55.980 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2023-12-11T18:03:55.980 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.980 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.980 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.980 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.980 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.980 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.980 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:55.980 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:55.980 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:55.980 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:55.980 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:55.980 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:55.981 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:55.981 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:55.981 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.981 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.981 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.981 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.981 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.981 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2023-12-11T18:03:55.981 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2023-12-11T18:03:55.981 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.981 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.981 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.981 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.981 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.981 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.982 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.982 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.982 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.982 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.982 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.982 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.982 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2023-12-11T18:03:55.982 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2023-12-11T18:03:55.982 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.982 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.982 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.982 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2023-12-11T18:03:55.982 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.982 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.983 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:55.983 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:55.983 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:55.983 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:55.983 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:55.983 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:55.983 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:55.983 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:55.983 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.983 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.983 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.983 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.983 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.983 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2023-12-11T18:03:55.983 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2023-12-11T18:03:55.984 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.984 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.984 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.984 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.984 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.984 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.984 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.984 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.984 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.984 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.984 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.984 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.984 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_stats_pools": { 2023-12-11T18:03:55.985 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rbd_stats_pools", 2023-12-11T18:03:55.985 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.985 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.985 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.985 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.985 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.985 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.985 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.985 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.986 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.986 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.986 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.986 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.986 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_stats_pools_refresh_interval": { 2023-12-11T18:03:55.986 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-12-11T18:03:55.986 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2023-12-11T18:03:55.986 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.986 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.986 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "300", 2023-12-11T18:03:55.986 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.986 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.987 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.987 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.987 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.987 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.987 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.987 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.987 INFO:teuthology.orchestra.run.smithi059.stdout: "scrape_interval": { 2023-12-11T18:03:55.987 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "scrape_interval", 2023-12-11T18:03:55.987 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "float", 2023-12-11T18:03:55.987 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.987 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.987 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "15.0", 2023-12-11T18:03:55.987 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.987 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.988 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.988 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.988 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.988 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.988 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.988 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.988 INFO:teuthology.orchestra.run.smithi059.stdout: "server_addr": { 2023-12-11T18:03:55.988 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "server_addr", 2023-12-11T18:03:55.988 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.988 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.988 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.988 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "::", 2023-12-11T18:03:55.988 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.988 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.989 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.989 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-12-11T18:03:55.989 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.989 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.989 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.989 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.989 INFO:teuthology.orchestra.run.smithi059.stdout: "server_port": { 2023-12-11T18:03:55.989 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "server_port", 2023-12-11T18:03:55.989 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2023-12-11T18:03:55.989 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.989 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.989 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "9283", 2023-12-11T18:03:55.989 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.989 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.990 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.990 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-12-11T18:03:55.990 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.990 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.990 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.990 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.990 INFO:teuthology.orchestra.run.smithi059.stdout: "stale_cache_strategy": { 2023-12-11T18:03:55.990 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "stale_cache_strategy", 2023-12-11T18:03:55.990 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.990 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.990 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:55.990 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "log", 2023-12-11T18:03:55.990 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.991 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.991 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.991 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.991 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.991 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.991 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.991 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.991 INFO:teuthology.orchestra.run.smithi059.stdout: "standby_behaviour": { 2023-12-11T18:03:55.991 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "standby_behaviour", 2023-12-11T18:03:55.991 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.991 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.991 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.991 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "default", 2023-12-11T18:03:55.991 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.992 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.992 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:55.992 INFO:teuthology.orchestra.run.smithi059.stdout: "default", 2023-12-11T18:03:55.992 INFO:teuthology.orchestra.run.smithi059.stdout: "error" 2023-12-11T18:03:55.992 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:55.992 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.992 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.992 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.992 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.992 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.992 INFO:teuthology.orchestra.run.smithi059.stdout: "standby_error_status_code": { 2023-12-11T18:03:55.992 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "standby_error_status_code", 2023-12-11T18:03:55.992 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2023-12-11T18:03:55.992 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.992 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.993 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "500", 2023-12-11T18:03:55.993 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "400", 2023-12-11T18:03:55.993 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "599", 2023-12-11T18:03:55.993 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.993 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.993 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.993 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.993 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.993 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:55.993 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:55.993 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.993 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-12-11T18:03:55.993 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rbd_support", 2023-12-11T18:03:55.993 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2023-12-11T18:03:55.994 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2023-12-11T18:03:55.994 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2023-12-11T18:03:55.994 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2023-12-11T18:03:55.994 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2023-12-11T18:03:55.994 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.994 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.994 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.994 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:55.994 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.994 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.994 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:55.994 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:55.995 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:55.995 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:55.995 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:55.995 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:55.995 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:55.995 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:55.995 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.995 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.995 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.995 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.995 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.995 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2023-12-11T18:03:55.995 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2023-12-11T18:03:55.995 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.996 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.996 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.996 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.996 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.996 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.996 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.996 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.996 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.996 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.996 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.996 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.996 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2023-12-11T18:03:55.996 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2023-12-11T18:03:55.996 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:55.997 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.997 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.997 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2023-12-11T18:03:55.997 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.997 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.997 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:55.997 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:55.997 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:55.997 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:55.997 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:55.997 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:55.997 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:55.997 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:55.998 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.998 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.998 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.998 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.998 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.998 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2023-12-11T18:03:55.998 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2023-12-11T18:03:55.998 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:55.998 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.998 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:55.998 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:55.999 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:55.999 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:55.999 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:55.999 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:55.999 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:55.999 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:55.999 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:55.999 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:55.999 INFO:teuthology.orchestra.run.smithi059.stdout: "max_concurrent_snap_create": { 2023-12-11T18:03:55.999 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "max_concurrent_snap_create", 2023-12-11T18:03:55.999 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2023-12-11T18:03:55.999 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:55.999 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.000 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "10", 2023-12-11T18:03:56.000 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.000 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.000 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.000 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.000 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.000 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.000 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.000 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.000 INFO:teuthology.orchestra.run.smithi059.stdout: "mirror_snapshot_schedule": { 2023-12-11T18:03:56.000 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "mirror_snapshot_schedule", 2023-12-11T18:03:56.000 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.000 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.000 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.001 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:56.001 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.001 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.001 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.001 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.001 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.001 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.001 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.001 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.001 INFO:teuthology.orchestra.run.smithi059.stdout: "trash_purge_schedule": { 2023-12-11T18:03:56.001 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "trash_purge_schedule", 2023-12-11T18:03:56.002 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.002 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.002 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.002 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:56.002 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.002 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.002 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.002 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.002 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.002 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.002 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.002 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:56.002 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:56.003 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.003 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-12-11T18:03:56.003 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "restful", 2023-12-11T18:03:56.003 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2023-12-11T18:03:56.003 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2023-12-11T18:03:56.003 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2023-12-11T18:03:56.003 INFO:teuthology.orchestra.run.smithi059.stdout: "enable_auth": { 2023-12-11T18:03:56.003 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "enable_auth", 2023-12-11T18:03:56.003 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.003 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.003 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.003 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2023-12-11T18:03:56.003 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.004 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.004 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.004 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.004 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.004 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.004 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.004 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.004 INFO:teuthology.orchestra.run.smithi059.stdout: "key_file": { 2023-12-11T18:03:56.004 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "key_file", 2023-12-11T18:03:56.004 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.004 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.004 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.004 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:56.004 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.005 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.005 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.005 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.005 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.005 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.005 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.005 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.005 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2023-12-11T18:03:56.005 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2023-12-11T18:03:56.005 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.005 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.005 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.005 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:56.005 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.006 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.006 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:56.006 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:56.006 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:56.006 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:56.006 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:56.006 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:56.006 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:56.006 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:56.007 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.007 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.007 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.007 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.007 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.007 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2023-12-11T18:03:56.007 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2023-12-11T18:03:56.007 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.007 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.007 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.008 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:56.008 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.008 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.008 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.008 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.008 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.008 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.008 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.008 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.008 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2023-12-11T18:03:56.008 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2023-12-11T18:03:56.008 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.008 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.008 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.009 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2023-12-11T18:03:56.009 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.009 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.009 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:56.009 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:56.009 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:56.009 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:56.009 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:56.009 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:56.009 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:56.009 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:56.009 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.009 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.009 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.010 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.010 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.010 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2023-12-11T18:03:56.010 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2023-12-11T18:03:56.010 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.010 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.010 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.010 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:56.010 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.010 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.010 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.010 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.010 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.011 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.011 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.011 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.011 INFO:teuthology.orchestra.run.smithi059.stdout: "server_addr": { 2023-12-11T18:03:56.011 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "server_addr", 2023-12-11T18:03:56.011 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.011 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.011 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.011 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:56.011 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.011 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.011 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.011 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.011 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.011 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.012 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.012 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.012 INFO:teuthology.orchestra.run.smithi059.stdout: "server_port": { 2023-12-11T18:03:56.012 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "server_port", 2023-12-11T18:03:56.012 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.012 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.012 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.012 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:56.012 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.012 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.012 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.012 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.012 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.012 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.013 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.013 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:56.013 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:56.013 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.013 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-12-11T18:03:56.013 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rgw", 2023-12-11T18:03:56.013 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2023-12-11T18:03:56.013 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2023-12-11T18:03:56.013 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2023-12-11T18:03:56.013 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2023-12-11T18:03:56.013 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2023-12-11T18:03:56.013 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.013 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.013 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.014 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:56.014 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.014 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.014 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:56.014 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:56.014 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:56.014 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:56.014 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:56.014 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:56.014 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:56.014 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:56.014 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.015 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.015 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.015 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.015 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.015 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2023-12-11T18:03:56.015 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2023-12-11T18:03:56.015 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.015 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.015 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.015 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:56.015 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.015 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.015 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.015 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.016 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.016 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.016 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.016 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.016 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2023-12-11T18:03:56.016 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2023-12-11T18:03:56.016 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.016 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.016 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.016 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2023-12-11T18:03:56.016 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.016 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.016 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:56.016 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:56.017 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:56.017 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:56.017 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:56.017 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:56.017 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:56.017 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:56.017 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.017 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.017 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.017 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.017 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.017 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2023-12-11T18:03:56.017 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2023-12-11T18:03:56.017 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.018 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.018 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.018 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:56.018 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.018 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.018 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.018 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.018 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.018 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.018 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.018 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:56.018 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:56.018 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.019 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-12-11T18:03:56.019 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rook", 2023-12-11T18:03:56.019 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2023-12-11T18:03:56.019 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2023-12-11T18:03:56.019 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2023-12-11T18:03:56.019 INFO:teuthology.orchestra.run.smithi059.stdout: "drive_group_interval": { 2023-12-11T18:03:56.019 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "drive_group_interval", 2023-12-11T18:03:56.019 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "float", 2023-12-11T18:03:56.019 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.019 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.019 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "300.0", 2023-12-11T18:03:56.019 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.019 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.019 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.019 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-12-11T18:03:56.020 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.020 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.020 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.020 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.020 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2023-12-11T18:03:56.020 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2023-12-11T18:03:56.020 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.020 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.020 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.020 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:56.020 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.020 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.021 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:56.021 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:56.021 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:56.021 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:56.021 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:56.021 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:56.021 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:56.021 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:56.021 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.021 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.021 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.021 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.021 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.021 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2023-12-11T18:03:56.022 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2023-12-11T18:03:56.022 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.022 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.022 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.022 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:56.022 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.022 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.022 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.022 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.022 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.022 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.022 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.022 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.022 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2023-12-11T18:03:56.023 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2023-12-11T18:03:56.023 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.023 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.023 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.023 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2023-12-11T18:03:56.023 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.023 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.023 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:56.023 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:56.023 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:56.023 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:56.023 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:56.023 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:56.023 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:56.024 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:56.024 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.024 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.024 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.024 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.024 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.024 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2023-12-11T18:03:56.024 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2023-12-11T18:03:56.024 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.024 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.024 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.024 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:56.024 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.024 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.024 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.025 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.025 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.025 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.025 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.025 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.025 INFO:teuthology.orchestra.run.smithi059.stdout: "storage_class": { 2023-12-11T18:03:56.025 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "storage_class", 2023-12-11T18:03:56.025 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.025 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.025 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.025 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "local", 2023-12-11T18:03:56.025 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.026 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.026 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.026 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-12-11T18:03:56.026 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.026 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.026 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.026 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:56.026 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:56.026 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.026 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-12-11T18:03:56.026 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "selftest", 2023-12-11T18:03:56.026 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2023-12-11T18:03:56.027 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2023-12-11T18:03:56.027 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2023-12-11T18:03:56.027 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2023-12-11T18:03:56.027 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2023-12-11T18:03:56.027 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.027 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.027 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.027 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:56.027 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.027 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.027 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:56.027 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:56.027 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:56.027 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:56.028 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:56.028 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:56.028 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:56.028 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:56.028 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.028 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.028 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.028 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.028 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.028 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2023-12-11T18:03:56.028 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2023-12-11T18:03:56.028 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.028 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.028 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.029 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:56.029 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.029 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.029 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.029 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.029 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.029 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.029 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.029 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.029 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2023-12-11T18:03:56.029 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2023-12-11T18:03:56.029 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.029 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.029 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.030 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2023-12-11T18:03:56.030 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.030 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.030 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:56.030 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:56.030 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:56.030 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:56.030 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:56.030 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:56.030 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:56.030 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:56.030 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.030 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.030 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.031 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.031 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.031 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2023-12-11T18:03:56.031 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2023-12-11T18:03:56.031 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.031 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.031 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.031 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:56.031 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.031 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.031 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.031 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.031 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.032 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.032 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.032 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.032 INFO:teuthology.orchestra.run.smithi059.stdout: "roption1": { 2023-12-11T18:03:56.032 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "roption1", 2023-12-11T18:03:56.032 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.032 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.032 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.032 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:56.032 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.032 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.033 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.033 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.033 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.033 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.033 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.033 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.033 INFO:teuthology.orchestra.run.smithi059.stdout: "roption2": { 2023-12-11T18:03:56.033 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "roption2", 2023-12-11T18:03:56.033 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.033 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.033 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.033 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "xyz", 2023-12-11T18:03:56.033 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.033 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.034 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.034 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.034 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.034 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.034 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.034 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.034 INFO:teuthology.orchestra.run.smithi059.stdout: "rwoption1": { 2023-12-11T18:03:56.034 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rwoption1", 2023-12-11T18:03:56.034 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.034 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.034 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.034 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:56.034 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.035 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.035 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.035 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.035 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.035 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.035 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.035 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.035 INFO:teuthology.orchestra.run.smithi059.stdout: "rwoption2": { 2023-12-11T18:03:56.035 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rwoption2", 2023-12-11T18:03:56.035 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2023-12-11T18:03:56.035 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.035 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.036 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:56.036 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.036 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.036 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.036 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.036 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.036 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.036 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.036 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.036 INFO:teuthology.orchestra.run.smithi059.stdout: "rwoption3": { 2023-12-11T18:03:56.037 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rwoption3", 2023-12-11T18:03:56.037 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "float", 2023-12-11T18:03:56.037 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.037 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.037 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:56.037 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.037 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.037 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.038 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.038 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.038 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.038 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.038 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.038 INFO:teuthology.orchestra.run.smithi059.stdout: "rwoption4": { 2023-12-11T18:03:56.038 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rwoption4", 2023-12-11T18:03:56.038 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.038 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.038 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.038 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:56.038 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.039 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.039 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.039 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.039 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.039 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.039 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.039 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.039 INFO:teuthology.orchestra.run.smithi059.stdout: "rwoption5": { 2023-12-11T18:03:56.039 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rwoption5", 2023-12-11T18:03:56.039 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.039 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.039 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.039 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:56.039 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.039 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.039 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.040 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.040 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.040 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.040 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.040 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.040 INFO:teuthology.orchestra.run.smithi059.stdout: "rwoption6": { 2023-12-11T18:03:56.040 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rwoption6", 2023-12-11T18:03:56.040 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.040 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.040 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.040 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2023-12-11T18:03:56.040 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.040 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.040 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.041 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.041 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.041 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.041 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.041 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.041 INFO:teuthology.orchestra.run.smithi059.stdout: "rwoption7": { 2023-12-11T18:03:56.041 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rwoption7", 2023-12-11T18:03:56.041 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2023-12-11T18:03:56.041 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.041 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.041 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:56.041 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "1", 2023-12-11T18:03:56.041 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "42", 2023-12-11T18:03:56.041 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.041 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.042 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.042 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.042 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.042 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.042 INFO:teuthology.orchestra.run.smithi059.stdout: "testkey": { 2023-12-11T18:03:56.042 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "testkey", 2023-12-11T18:03:56.042 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.042 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.042 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.042 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:56.042 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.042 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.042 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.043 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.043 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.043 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.043 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.043 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.043 INFO:teuthology.orchestra.run.smithi059.stdout: "testlkey": { 2023-12-11T18:03:56.043 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "testlkey", 2023-12-11T18:03:56.043 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.043 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.043 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.043 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:56.043 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.043 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.044 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.044 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.044 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.044 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.044 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.044 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.044 INFO:teuthology.orchestra.run.smithi059.stdout: "testnewline": { 2023-12-11T18:03:56.044 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "testnewline", 2023-12-11T18:03:56.044 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.044 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.044 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.044 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:56.044 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.044 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.044 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.045 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.045 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.045 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.045 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.045 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:56.045 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:56.045 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.045 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-12-11T18:03:56.045 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "snap_schedule", 2023-12-11T18:03:56.045 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2023-12-11T18:03:56.045 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2023-12-11T18:03:56.045 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2023-12-11T18:03:56.045 INFO:teuthology.orchestra.run.smithi059.stdout: "allow_m_granularity": { 2023-12-11T18:03:56.046 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "allow_m_granularity", 2023-12-11T18:03:56.046 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.046 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.046 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.046 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:56.046 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.046 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.046 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.046 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "allow minute scheduled snapshots", 2023-12-11T18:03:56.046 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.046 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.046 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.047 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.047 INFO:teuthology.orchestra.run.smithi059.stdout: "dump_on_update": { 2023-12-11T18:03:56.047 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "dump_on_update", 2023-12-11T18:03:56.047 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.047 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.047 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.047 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:56.047 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.047 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.047 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.047 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "dump database to debug log on update", 2023-12-11T18:03:56.047 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.047 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.047 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.047 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.048 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2023-12-11T18:03:56.048 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2023-12-11T18:03:56.048 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.048 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.048 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.048 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:56.048 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.048 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.048 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:56.048 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:56.048 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:56.048 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:56.048 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:56.048 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:56.049 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:56.049 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:56.049 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.049 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.049 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.049 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.049 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.049 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2023-12-11T18:03:56.049 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2023-12-11T18:03:56.049 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.049 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.049 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.049 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:56.049 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.049 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.050 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.050 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.050 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.050 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.050 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.050 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.050 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2023-12-11T18:03:56.050 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2023-12-11T18:03:56.050 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.050 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.050 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.050 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2023-12-11T18:03:56.050 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.050 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.051 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:56.051 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:56.051 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:56.051 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:56.051 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:56.051 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:56.051 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:56.051 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:56.051 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.051 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.051 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.051 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.051 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.051 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2023-12-11T18:03:56.051 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2023-12-11T18:03:56.052 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.052 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.052 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.052 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:56.052 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.052 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.052 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.052 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.052 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.052 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.052 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.053 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:56.053 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:56.053 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.053 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-12-11T18:03:56.053 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "stats", 2023-12-11T18:03:56.053 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2023-12-11T18:03:56.053 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2023-12-11T18:03:56.053 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2023-12-11T18:03:56.053 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2023-12-11T18:03:56.053 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2023-12-11T18:03:56.053 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.053 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.053 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.053 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:56.053 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.054 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.054 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:56.054 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:56.054 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:56.054 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:56.054 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:56.054 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:56.054 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:56.054 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:56.054 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.054 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.054 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.054 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.054 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.055 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2023-12-11T18:03:56.055 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2023-12-11T18:03:56.055 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.055 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.055 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.055 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:56.055 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.055 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.055 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.055 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.055 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.055 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.055 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.055 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.056 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2023-12-11T18:03:56.056 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2023-12-11T18:03:56.056 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.056 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.056 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.056 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2023-12-11T18:03:56.056 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.056 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.056 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:56.056 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:56.056 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:56.056 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:56.057 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:56.057 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:56.057 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:56.057 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:56.057 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.057 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.057 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.057 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.057 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.057 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2023-12-11T18:03:56.057 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2023-12-11T18:03:56.057 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.057 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.057 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.058 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:56.058 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.058 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.058 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.058 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.058 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.058 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.058 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.058 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:56.058 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:56.058 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.058 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-12-11T18:03:56.059 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "status", 2023-12-11T18:03:56.059 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2023-12-11T18:03:56.059 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2023-12-11T18:03:56.059 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2023-12-11T18:03:56.059 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2023-12-11T18:03:56.059 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2023-12-11T18:03:56.059 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.059 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.059 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.059 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:56.059 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.059 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.059 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:56.059 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:56.060 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:56.060 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:56.060 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:56.060 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:56.060 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:56.060 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:56.060 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.060 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.060 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.060 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.060 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.061 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2023-12-11T18:03:56.061 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2023-12-11T18:03:56.061 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.061 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.061 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.061 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:56.061 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.061 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.061 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.061 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.061 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.061 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.061 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.061 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.062 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2023-12-11T18:03:56.062 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2023-12-11T18:03:56.062 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.062 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.062 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.062 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2023-12-11T18:03:56.062 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.062 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.062 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:56.062 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:56.062 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:56.062 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:56.062 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:56.062 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:56.063 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:56.063 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:56.063 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.063 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.063 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.063 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.063 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.063 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2023-12-11T18:03:56.063 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2023-12-11T18:03:56.063 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.063 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.063 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.063 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:56.064 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.064 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.064 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.064 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.064 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.064 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.064 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.064 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:56.064 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:56.064 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.064 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-12-11T18:03:56.064 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "telegraf", 2023-12-11T18:03:56.064 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2023-12-11T18:03:56.064 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2023-12-11T18:03:56.065 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2023-12-11T18:03:56.065 INFO:teuthology.orchestra.run.smithi059.stdout: "address": { 2023-12-11T18:03:56.065 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "address", 2023-12-11T18:03:56.065 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.065 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.065 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.065 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-12-11T18:03:56.065 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.065 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.065 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.065 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.066 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.066 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.066 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.066 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.066 INFO:teuthology.orchestra.run.smithi059.stdout: "interval": { 2023-12-11T18:03:56.066 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "interval", 2023-12-11T18:03:56.066 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2023-12-11T18:03:56.066 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.066 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.066 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "15", 2023-12-11T18:03:56.066 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.066 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.067 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.067 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.067 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.067 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.067 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.067 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.067 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2023-12-11T18:03:56.067 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2023-12-11T18:03:56.067 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.067 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.067 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.068 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:56.068 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.068 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.068 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:56.068 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:56.068 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:56.068 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:56.068 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:56.068 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:56.068 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:56.068 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:56.068 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.068 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.068 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.069 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.069 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.069 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2023-12-11T18:03:56.069 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2023-12-11T18:03:56.069 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.069 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.069 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.069 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:56.069 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.069 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.069 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.069 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.069 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.069 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.069 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.070 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.070 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2023-12-11T18:03:56.070 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2023-12-11T18:03:56.070 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.070 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.070 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.070 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2023-12-11T18:03:56.070 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.070 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.070 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:56.070 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:56.070 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:56.070 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:56.070 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:56.071 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:56.071 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:56.071 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:56.071 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.071 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.071 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.071 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.071 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.071 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2023-12-11T18:03:56.071 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2023-12-11T18:03:56.071 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.071 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.071 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.071 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:56.071 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.071 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.072 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.072 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.072 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.072 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.072 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.072 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:56.072 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:56.072 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.072 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-12-11T18:03:56.072 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "telemetry", 2023-12-11T18:03:56.072 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2023-12-11T18:03:56.072 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2023-12-11T18:03:56.072 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2023-12-11T18:03:56.072 INFO:teuthology.orchestra.run.smithi059.stdout: "channel_basic": { 2023-12-11T18:03:56.073 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "channel_basic", 2023-12-11T18:03:56.073 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.073 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.073 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.073 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2023-12-11T18:03:56.073 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.073 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.073 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.073 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Share basic cluster information (size, version)", 2023-12-11T18:03:56.073 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.073 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.073 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.073 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.074 INFO:teuthology.orchestra.run.smithi059.stdout: "channel_crash": { 2023-12-11T18:03:56.074 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "channel_crash", 2023-12-11T18:03:56.074 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.074 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.074 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.074 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2023-12-11T18:03:56.074 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.074 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.074 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.074 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-12-11T18:03:56.074 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.074 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.075 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.075 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.075 INFO:teuthology.orchestra.run.smithi059.stdout: "channel_device": { 2023-12-11T18:03:56.075 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "channel_device", 2023-12-11T18:03:56.075 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.075 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.075 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.075 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2023-12-11T18:03:56.075 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.075 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.075 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.075 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-12-11T18:03:56.075 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.075 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.076 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.076 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.076 INFO:teuthology.orchestra.run.smithi059.stdout: "channel_ident": { 2023-12-11T18:03:56.076 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "channel_ident", 2023-12-11T18:03:56.076 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.076 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.076 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.076 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:56.076 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.076 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.076 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.076 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-12-11T18:03:56.076 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.076 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.077 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.077 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.077 INFO:teuthology.orchestra.run.smithi059.stdout: "channel_perf": { 2023-12-11T18:03:56.077 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "channel_perf", 2023-12-11T18:03:56.077 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.077 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.077 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.077 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:56.077 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.077 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.077 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.077 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Share various performance metrics of a cluster", 2023-12-11T18:03:56.077 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.077 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.078 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.078 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.078 INFO:teuthology.orchestra.run.smithi059.stdout: "contact": { 2023-12-11T18:03:56.078 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "contact", 2023-12-11T18:03:56.078 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.078 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.078 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.078 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:56.078 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.078 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.078 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.078 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.078 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.078 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.079 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.079 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.079 INFO:teuthology.orchestra.run.smithi059.stdout: "description": { 2023-12-11T18:03:56.079 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "description", 2023-12-11T18:03:56.079 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.079 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.079 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.079 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:56.079 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.079 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.079 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.079 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.079 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.079 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.080 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.080 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.080 INFO:teuthology.orchestra.run.smithi059.stdout: "device_url": { 2023-12-11T18:03:56.080 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "device_url", 2023-12-11T18:03:56.080 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.080 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.080 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.080 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-12-11T18:03:56.080 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.080 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.080 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.080 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.080 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.080 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.080 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.081 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.081 INFO:teuthology.orchestra.run.smithi059.stdout: "enabled": { 2023-12-11T18:03:56.081 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "enabled", 2023-12-11T18:03:56.081 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.081 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.081 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.081 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:56.081 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.081 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.081 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.081 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.081 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.081 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.081 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.082 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.082 INFO:teuthology.orchestra.run.smithi059.stdout: "interval": { 2023-12-11T18:03:56.082 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "interval", 2023-12-11T18:03:56.082 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2023-12-11T18:03:56.082 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.082 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.082 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "24", 2023-12-11T18:03:56.082 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "8", 2023-12-11T18:03:56.082 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.082 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.082 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.082 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.082 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.082 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.083 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.083 INFO:teuthology.orchestra.run.smithi059.stdout: "last_opt_revision": { 2023-12-11T18:03:56.083 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "last_opt_revision", 2023-12-11T18:03:56.083 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2023-12-11T18:03:56.083 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.083 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.083 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "1", 2023-12-11T18:03:56.083 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.083 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.083 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.083 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.084 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.084 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.084 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.084 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.084 INFO:teuthology.orchestra.run.smithi059.stdout: "leaderboard": { 2023-12-11T18:03:56.084 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "leaderboard", 2023-12-11T18:03:56.084 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.084 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.084 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.084 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:56.084 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.084 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.084 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.085 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.085 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.085 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.085 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.085 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.085 INFO:teuthology.orchestra.run.smithi059.stdout: "leaderboard_description": { 2023-12-11T18:03:56.085 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "leaderboard_description", 2023-12-11T18:03:56.085 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.085 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.085 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.085 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:56.085 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.085 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.085 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.085 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.086 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.086 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.086 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.086 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.086 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2023-12-11T18:03:56.086 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2023-12-11T18:03:56.086 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.086 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.086 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.086 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:56.086 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.086 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.087 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:56.087 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:56.087 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:56.087 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:56.087 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:56.087 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:56.087 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:56.087 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:56.087 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.087 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.087 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.087 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.087 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.087 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2023-12-11T18:03:56.087 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2023-12-11T18:03:56.087 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.088 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.088 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.088 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:56.088 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.088 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.088 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.088 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.088 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.088 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.088 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.088 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.089 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2023-12-11T18:03:56.089 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2023-12-11T18:03:56.089 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.089 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.089 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.089 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2023-12-11T18:03:56.089 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.089 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.089 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:56.089 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:56.089 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:56.089 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:56.089 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:56.090 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:56.090 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:56.090 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:56.090 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.090 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.090 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.090 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.090 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.090 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2023-12-11T18:03:56.090 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2023-12-11T18:03:56.090 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.090 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.090 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.091 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:56.091 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.091 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.091 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.091 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.091 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.091 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.091 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.091 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.091 INFO:teuthology.orchestra.run.smithi059.stdout: "organization": { 2023-12-11T18:03:56.091 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "organization", 2023-12-11T18:03:56.091 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.091 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.091 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.092 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:56.092 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.092 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.092 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.092 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.092 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.092 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.092 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.092 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.092 INFO:teuthology.orchestra.run.smithi059.stdout: "proxy": { 2023-12-11T18:03:56.092 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "proxy", 2023-12-11T18:03:56.092 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.092 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.092 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.092 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:56.093 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.093 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.093 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.093 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.093 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.093 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.093 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.093 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.093 INFO:teuthology.orchestra.run.smithi059.stdout: "url": { 2023-12-11T18:03:56.093 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "url", 2023-12-11T18:03:56.093 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.093 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.093 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.093 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-12-11T18:03:56.094 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.094 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.094 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.094 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.094 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.094 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.094 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.094 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:56.094 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:56.094 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.094 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-12-11T18:03:56.094 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "test_orchestrator", 2023-12-11T18:03:56.095 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2023-12-11T18:03:56.095 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2023-12-11T18:03:56.095 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2023-12-11T18:03:56.095 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2023-12-11T18:03:56.095 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2023-12-11T18:03:56.095 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.095 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.095 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.095 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:56.095 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.095 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.095 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:56.095 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:56.096 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:56.096 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:56.096 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:56.096 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:56.096 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:56.096 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:56.096 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.096 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.097 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.097 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.097 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.097 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2023-12-11T18:03:56.097 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2023-12-11T18:03:56.098 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.098 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.098 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.098 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:56.098 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.098 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.099 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.099 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.099 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.099 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.099 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.100 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.100 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2023-12-11T18:03:56.100 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2023-12-11T18:03:56.100 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.100 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.100 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.100 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2023-12-11T18:03:56.100 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.100 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.100 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:56.100 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:56.100 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:56.100 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:56.100 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:56.101 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:56.101 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:56.101 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:56.101 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.101 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.101 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.101 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.101 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.101 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2023-12-11T18:03:56.101 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2023-12-11T18:03:56.101 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.101 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.101 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.101 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:56.101 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.102 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.102 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.102 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.102 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.102 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.102 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.102 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:56.102 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:56.102 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.102 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-12-11T18:03:56.103 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "volumes", 2023-12-11T18:03:56.103 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2023-12-11T18:03:56.103 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2023-12-11T18:03:56.103 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2023-12-11T18:03:56.103 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2023-12-11T18:03:56.103 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2023-12-11T18:03:56.103 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.103 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.103 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.103 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:56.103 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.103 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.103 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:56.104 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:56.104 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:56.104 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:56.104 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:56.104 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:56.104 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:56.104 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:56.104 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.104 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.104 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.104 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.104 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.104 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2023-12-11T18:03:56.104 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2023-12-11T18:03:56.105 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.105 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.105 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.105 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:56.105 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.105 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.105 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.105 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.105 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.105 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.105 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.105 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.105 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2023-12-11T18:03:56.105 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2023-12-11T18:03:56.105 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.106 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.106 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.106 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2023-12-11T18:03:56.106 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.106 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.106 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:56.106 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:56.106 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:56.106 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:56.106 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:56.106 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:56.106 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:56.107 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:56.107 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.107 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.107 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.107 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.107 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.107 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2023-12-11T18:03:56.107 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2023-12-11T18:03:56.107 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.108 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.108 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.108 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:56.108 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.108 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.108 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.108 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.108 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.108 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.108 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.108 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.109 INFO:teuthology.orchestra.run.smithi059.stdout: "max_concurrent_clones": { 2023-12-11T18:03:56.109 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "max_concurrent_clones", 2023-12-11T18:03:56.109 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2023-12-11T18:03:56.109 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.109 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.109 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "4", 2023-12-11T18:03:56.109 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.109 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.109 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.109 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Number of asynchronous cloner threads", 2023-12-11T18:03:56.109 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.109 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.109 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.109 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.109 INFO:teuthology.orchestra.run.smithi059.stdout: "periodic_async_work": { 2023-12-11T18:03:56.109 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "periodic_async_work", 2023-12-11T18:03:56.110 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.110 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.110 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.110 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:56.110 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.110 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.110 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.110 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Periodically check for async work", 2023-12-11T18:03:56.110 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.110 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.110 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.110 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.110 INFO:teuthology.orchestra.run.smithi059.stdout: "snapshot_clone_delay": { 2023-12-11T18:03:56.111 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "snapshot_clone_delay", 2023-12-11T18:03:56.111 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2023-12-11T18:03:56.111 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.111 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.111 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2023-12-11T18:03:56.111 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.111 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.111 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.111 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-12-11T18:03:56.111 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.111 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.111 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.111 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:56.111 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:56.111 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.111 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-12-11T18:03:56.112 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "zabbix", 2023-12-11T18:03:56.112 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2023-12-11T18:03:56.112 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2023-12-11T18:03:56.112 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2023-12-11T18:03:56.112 INFO:teuthology.orchestra.run.smithi059.stdout: "discovery_interval": { 2023-12-11T18:03:56.112 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "discovery_interval", 2023-12-11T18:03:56.112 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "uint", 2023-12-11T18:03:56.112 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.112 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.112 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "100", 2023-12-11T18:03:56.112 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.112 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.112 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.112 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.113 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.113 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.113 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.113 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.113 INFO:teuthology.orchestra.run.smithi059.stdout: "identifier": { 2023-12-11T18:03:56.113 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "identifier", 2023-12-11T18:03:56.113 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.113 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.113 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.113 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:56.113 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.113 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.113 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.113 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.113 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.114 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.114 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.114 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.114 INFO:teuthology.orchestra.run.smithi059.stdout: "interval": { 2023-12-11T18:03:56.114 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "interval", 2023-12-11T18:03:56.114 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2023-12-11T18:03:56.114 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.114 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.114 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "60", 2023-12-11T18:03:56.114 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.114 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.114 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.114 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.115 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.115 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.115 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.115 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.115 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2023-12-11T18:03:56.115 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2023-12-11T18:03:56.115 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.115 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.115 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.115 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:56.115 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.115 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.115 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:56.115 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:56.115 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:56.116 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:56.116 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:56.116 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:56.116 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:56.116 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:56.116 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.116 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.116 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.116 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.116 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.116 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2023-12-11T18:03:56.116 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2023-12-11T18:03:56.116 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.116 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.116 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.117 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:56.117 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.117 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.117 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.117 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.117 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.117 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.117 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.117 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.117 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2023-12-11T18:03:56.117 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2023-12-11T18:03:56.117 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.117 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.117 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.118 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2023-12-11T18:03:56.118 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.118 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.118 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:56.118 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:56.118 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:56.118 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:56.118 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:56.118 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:56.118 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:56.118 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:56.118 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.118 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.118 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.119 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.119 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.119 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2023-12-11T18:03:56.119 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2023-12-11T18:03:56.119 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.119 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.119 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.119 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:56.119 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.119 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.119 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.119 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.119 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.119 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.120 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.120 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.120 INFO:teuthology.orchestra.run.smithi059.stdout: "zabbix_host": { 2023-12-11T18:03:56.120 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "zabbix_host", 2023-12-11T18:03:56.120 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.120 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.120 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.120 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:56.120 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.120 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.120 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.120 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.120 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.121 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.121 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.121 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.121 INFO:teuthology.orchestra.run.smithi059.stdout: "zabbix_port": { 2023-12-11T18:03:56.121 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "zabbix_port", 2023-12-11T18:03:56.121 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2023-12-11T18:03:56.121 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.121 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.121 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "10051", 2023-12-11T18:03:56.121 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.121 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.121 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.121 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.121 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.121 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.122 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.122 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.122 INFO:teuthology.orchestra.run.smithi059.stdout: "zabbix_sender": { 2023-12-11T18:03:56.122 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "zabbix_sender", 2023-12-11T18:03:56.122 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.122 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.122 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.122 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-12-11T18:03:56.122 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.122 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.122 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.122 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.122 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.122 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.123 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.123 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:56.123 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:56.123 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:56.123 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2023-12-11T18:03:56.123 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:56.123 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:56.123 INFO:teuthology.orchestra.run.smithi059.stdout: "modules": [ 2023-12-11T18:03:56.123 INFO:teuthology.orchestra.run.smithi059.stdout: "cephadm", 2023-12-11T18:03:56.123 INFO:teuthology.orchestra.run.smithi059.stdout: "dashboard", 2023-12-11T18:03:56.123 INFO:teuthology.orchestra.run.smithi059.stdout: "iostat", 2023-12-11T18:03:56.123 INFO:teuthology.orchestra.run.smithi059.stdout: "nfs", 2023-12-11T18:03:56.123 INFO:teuthology.orchestra.run.smithi059.stdout: "restful", 2023-12-11T18:03:56.123 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_schedule" 2023-12-11T18:03:56.123 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:56.124 INFO:teuthology.orchestra.run.smithi059.stdout: "available_modules": [ 2023-12-11T18:03:56.124 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-12-11T18:03:56.124 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "alerts", 2023-12-11T18:03:56.124 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2023-12-11T18:03:56.124 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2023-12-11T18:03:56.124 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2023-12-11T18:03:56.124 INFO:teuthology.orchestra.run.smithi059.stdout: "interval": { 2023-12-11T18:03:56.124 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "interval", 2023-12-11T18:03:56.124 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2023-12-11T18:03:56.124 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.124 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.124 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "60", 2023-12-11T18:03:56.124 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.125 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.125 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.125 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "How frequently to reexamine health status", 2023-12-11T18:03:56.125 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.125 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.125 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.125 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.125 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2023-12-11T18:03:56.125 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2023-12-11T18:03:56.125 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.125 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.125 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.125 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:56.125 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.126 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.126 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:56.126 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:56.126 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:56.126 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:56.126 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:56.126 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:56.126 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:56.126 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:56.126 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.126 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.127 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.127 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.127 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.127 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2023-12-11T18:03:56.127 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2023-12-11T18:03:56.127 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.127 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.127 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.127 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:56.127 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.127 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.127 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.127 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.127 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.128 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.128 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.128 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.128 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2023-12-11T18:03:56.128 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2023-12-11T18:03:56.128 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.128 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.128 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.128 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2023-12-11T18:03:56.128 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.128 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.128 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:56.128 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:56.128 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:56.129 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:56.129 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:56.129 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:56.129 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:56.129 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:56.129 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.129 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.129 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.129 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.129 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.129 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2023-12-11T18:03:56.129 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2023-12-11T18:03:56.129 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.129 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.129 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.130 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:56.130 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.130 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.130 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.130 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.130 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.130 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.130 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.130 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.130 INFO:teuthology.orchestra.run.smithi059.stdout: "smtp_destination": { 2023-12-11T18:03:56.130 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "smtp_destination", 2023-12-11T18:03:56.130 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.130 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.130 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.131 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:56.131 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.131 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.131 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.131 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Email address to send alerts to", 2023-12-11T18:03:56.131 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.131 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.131 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.131 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.131 INFO:teuthology.orchestra.run.smithi059.stdout: "smtp_from_name": { 2023-12-11T18:03:56.131 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "smtp_from_name", 2023-12-11T18:03:56.131 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.131 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.131 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.132 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "Ceph", 2023-12-11T18:03:56.132 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.132 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.132 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.132 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Email From: name", 2023-12-11T18:03:56.132 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.132 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.132 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.132 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.132 INFO:teuthology.orchestra.run.smithi059.stdout: "smtp_host": { 2023-12-11T18:03:56.132 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "smtp_host", 2023-12-11T18:03:56.132 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.132 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.132 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.133 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:56.133 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.133 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.133 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.133 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "SMTP server", 2023-12-11T18:03:56.133 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.133 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.133 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.133 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.133 INFO:teuthology.orchestra.run.smithi059.stdout: "smtp_password": { 2023-12-11T18:03:56.133 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "smtp_password", 2023-12-11T18:03:56.134 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.134 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.134 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.134 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:56.134 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.134 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.134 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.134 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Password to authenticate with", 2023-12-11T18:03:56.134 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.134 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.134 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.134 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.134 INFO:teuthology.orchestra.run.smithi059.stdout: "smtp_port": { 2023-12-11T18:03:56.135 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "smtp_port", 2023-12-11T18:03:56.135 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2023-12-11T18:03:56.135 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.135 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.135 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "465", 2023-12-11T18:03:56.135 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.135 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.135 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.135 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "SMTP port", 2023-12-11T18:03:56.135 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.135 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.135 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.135 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.135 INFO:teuthology.orchestra.run.smithi059.stdout: "smtp_sender": { 2023-12-11T18:03:56.136 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "smtp_sender", 2023-12-11T18:03:56.136 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.136 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.136 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.136 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:56.136 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.136 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.137 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.137 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "SMTP envelope sender", 2023-12-11T18:03:56.137 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.137 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.137 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.137 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.138 INFO:teuthology.orchestra.run.smithi059.stdout: "smtp_ssl": { 2023-12-11T18:03:56.138 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "smtp_ssl", 2023-12-11T18:03:56.138 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.138 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.138 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.138 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2023-12-11T18:03:56.138 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.138 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.138 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.139 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Use SSL to connect to SMTP server", 2023-12-11T18:03:56.139 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.139 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.139 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.139 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.140 INFO:teuthology.orchestra.run.smithi059.stdout: "smtp_user": { 2023-12-11T18:03:56.140 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "smtp_user", 2023-12-11T18:03:56.140 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.140 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.140 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.140 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:56.140 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.140 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.140 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.140 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "User to authenticate as", 2023-12-11T18:03:56.140 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.141 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.141 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.141 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:56.141 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:56.141 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.141 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-12-11T18:03:56.141 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "balancer", 2023-12-11T18:03:56.141 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2023-12-11T18:03:56.141 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2023-12-11T18:03:56.141 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2023-12-11T18:03:56.141 INFO:teuthology.orchestra.run.smithi059.stdout: "active": { 2023-12-11T18:03:56.141 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "active", 2023-12-11T18:03:56.141 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.142 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.142 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.142 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2023-12-11T18:03:56.142 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.142 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.142 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.142 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "automatically balance PGs across cluster", 2023-12-11T18:03:56.142 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.142 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.142 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.142 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.142 INFO:teuthology.orchestra.run.smithi059.stdout: "begin_time": { 2023-12-11T18:03:56.142 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "begin_time", 2023-12-11T18:03:56.143 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.143 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.143 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.143 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0000", 2023-12-11T18:03:56.143 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.143 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.143 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.143 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "beginning time of day to automatically balance", 2023-12-11T18:03:56.143 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-12-11T18:03:56.143 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.143 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.143 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.143 INFO:teuthology.orchestra.run.smithi059.stdout: "begin_weekday": { 2023-12-11T18:03:56.143 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "begin_weekday", 2023-12-11T18:03:56.143 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "uint", 2023-12-11T18:03:56.144 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.144 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.144 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2023-12-11T18:03:56.144 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "0", 2023-12-11T18:03:56.144 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "6", 2023-12-11T18:03:56.144 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.144 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-12-11T18:03:56.144 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-12-11T18:03:56.144 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.144 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.144 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.144 INFO:teuthology.orchestra.run.smithi059.stdout: "crush_compat_max_iterations": { 2023-12-11T18:03:56.144 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "crush_compat_max_iterations", 2023-12-11T18:03:56.145 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "uint", 2023-12-11T18:03:56.145 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.145 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.145 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "25", 2023-12-11T18:03:56.145 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "1", 2023-12-11T18:03:56.145 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "250", 2023-12-11T18:03:56.145 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.145 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-12-11T18:03:56.145 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.145 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.145 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.145 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.145 INFO:teuthology.orchestra.run.smithi059.stdout: "crush_compat_metrics": { 2023-12-11T18:03:56.146 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "crush_compat_metrics", 2023-12-11T18:03:56.146 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.146 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.146 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.146 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "pgs,objects,bytes", 2023-12-11T18:03:56.146 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.146 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.146 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.146 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-12-11T18:03:56.146 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-12-11T18:03:56.146 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.146 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.146 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.147 INFO:teuthology.orchestra.run.smithi059.stdout: "crush_compat_step": { 2023-12-11T18:03:56.147 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "crush_compat_step", 2023-12-11T18:03:56.147 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "float", 2023-12-11T18:03:56.147 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.147 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.147 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0.5", 2023-12-11T18:03:56.147 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "0.001", 2023-12-11T18:03:56.147 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "0.999", 2023-12-11T18:03:56.147 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.147 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "aggressiveness of optimization", 2023-12-11T18:03:56.147 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-12-11T18:03:56.147 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.147 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.147 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.148 INFO:teuthology.orchestra.run.smithi059.stdout: "end_time": { 2023-12-11T18:03:56.148 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "end_time", 2023-12-11T18:03:56.148 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.148 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.148 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.148 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "2359", 2023-12-11T18:03:56.148 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.148 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.148 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.148 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "ending time of day to automatically balance", 2023-12-11T18:03:56.148 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-12-11T18:03:56.148 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.148 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.148 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.148 INFO:teuthology.orchestra.run.smithi059.stdout: "end_weekday": { 2023-12-11T18:03:56.148 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "end_weekday", 2023-12-11T18:03:56.149 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "uint", 2023-12-11T18:03:56.149 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.149 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.149 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2023-12-11T18:03:56.149 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "0", 2023-12-11T18:03:56.149 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "6", 2023-12-11T18:03:56.149 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.149 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-12-11T18:03:56.149 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-12-11T18:03:56.149 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.149 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.149 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.149 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2023-12-11T18:03:56.149 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2023-12-11T18:03:56.149 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.149 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.150 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.150 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:56.150 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.150 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.150 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:56.150 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:56.150 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:56.150 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:56.150 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:56.150 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:56.150 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:56.150 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:56.150 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.150 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.151 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.151 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.151 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.151 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2023-12-11T18:03:56.151 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2023-12-11T18:03:56.151 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.151 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.151 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.151 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:56.151 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.151 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.151 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.151 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.151 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.151 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.152 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.152 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.152 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2023-12-11T18:03:56.152 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2023-12-11T18:03:56.152 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.152 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.152 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.152 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2023-12-11T18:03:56.152 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.152 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.152 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:56.152 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:56.152 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:56.152 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:56.152 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:56.152 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:56.153 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:56.153 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:56.153 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.153 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.153 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.153 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.153 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.153 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2023-12-11T18:03:56.153 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2023-12-11T18:03:56.153 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.153 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.153 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.153 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:56.153 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.153 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.153 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.154 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.154 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.154 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.154 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.154 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.154 INFO:teuthology.orchestra.run.smithi059.stdout: "min_score": { 2023-12-11T18:03:56.154 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "min_score", 2023-12-11T18:03:56.154 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "float", 2023-12-11T18:03:56.154 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.154 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.154 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2023-12-11T18:03:56.154 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.154 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.155 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.155 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-12-11T18:03:56.155 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.155 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.155 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.155 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.155 INFO:teuthology.orchestra.run.smithi059.stdout: "mode": { 2023-12-11T18:03:56.155 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "mode", 2023-12-11T18:03:56.155 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.155 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.155 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.155 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "upmap", 2023-12-11T18:03:56.155 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.155 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.156 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:56.156 INFO:teuthology.orchestra.run.smithi059.stdout: "crush-compat", 2023-12-11T18:03:56.156 INFO:teuthology.orchestra.run.smithi059.stdout: "none", 2023-12-11T18:03:56.156 INFO:teuthology.orchestra.run.smithi059.stdout: "upmap" 2023-12-11T18:03:56.156 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:56.156 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Balancer mode", 2023-12-11T18:03:56.156 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.156 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.156 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.156 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.156 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_ids": { 2023-12-11T18:03:56.156 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "pool_ids", 2023-12-11T18:03:56.156 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.156 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.156 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.156 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:56.157 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.157 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.157 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.157 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-12-11T18:03:56.157 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.157 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.157 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.157 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.157 INFO:teuthology.orchestra.run.smithi059.stdout: "sleep_interval": { 2023-12-11T18:03:56.157 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sleep_interval", 2023-12-11T18:03:56.157 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2023-12-11T18:03:56.157 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.157 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.157 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "60", 2023-12-11T18:03:56.157 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.158 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.158 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.158 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-12-11T18:03:56.158 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.158 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.158 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.158 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.158 INFO:teuthology.orchestra.run.smithi059.stdout: "upmap_max_deviation": { 2023-12-11T18:03:56.158 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "upmap_max_deviation", 2023-12-11T18:03:56.158 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2023-12-11T18:03:56.158 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.158 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.158 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "5", 2023-12-11T18:03:56.159 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "1", 2023-12-11T18:03:56.159 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.159 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.159 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "deviation below which no optimization is attempted", 2023-12-11T18:03:56.159 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-12-11T18:03:56.159 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.159 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.159 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.159 INFO:teuthology.orchestra.run.smithi059.stdout: "upmap_max_optimizations": { 2023-12-11T18:03:56.159 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "upmap_max_optimizations", 2023-12-11T18:03:56.159 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "uint", 2023-12-11T18:03:56.160 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.160 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.160 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "10", 2023-12-11T18:03:56.160 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.160 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.160 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.160 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-12-11T18:03:56.160 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.160 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.160 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.160 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:56.160 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:56.160 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.160 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-12-11T18:03:56.160 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "cephadm", 2023-12-11T18:03:56.160 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2023-12-11T18:03:56.161 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2023-12-11T18:03:56.161 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2023-12-11T18:03:56.161 INFO:teuthology.orchestra.run.smithi059.stdout: "agent_down_multiplier": { 2023-12-11T18:03:56.161 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "agent_down_multiplier", 2023-12-11T18:03:56.161 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "float", 2023-12-11T18:03:56.161 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.161 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.161 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "3.0", 2023-12-11T18:03:56.161 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.161 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.161 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.161 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-12-11T18:03:56.161 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.161 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.161 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.162 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.162 INFO:teuthology.orchestra.run.smithi059.stdout: "agent_refresh_rate": { 2023-12-11T18:03:56.162 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "agent_refresh_rate", 2023-12-11T18:03:56.162 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2023-12-11T18:03:56.162 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.162 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.162 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "20", 2023-12-11T18:03:56.162 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.162 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.162 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.162 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-12-11T18:03:56.162 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.162 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.163 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.163 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.163 INFO:teuthology.orchestra.run.smithi059.stdout: "agent_starting_port": { 2023-12-11T18:03:56.163 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "agent_starting_port", 2023-12-11T18:03:56.163 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2023-12-11T18:03:56.163 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.163 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.163 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "4721", 2023-12-11T18:03:56.163 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.163 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.163 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.163 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-12-11T18:03:56.163 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.163 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.163 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.164 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.164 INFO:teuthology.orchestra.run.smithi059.stdout: "allow_ptrace": { 2023-12-11T18:03:56.164 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "allow_ptrace", 2023-12-11T18:03:56.164 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.164 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.164 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.164 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:56.164 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.164 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.164 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.164 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-12-11T18:03:56.164 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-12-11T18:03:56.164 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.164 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.164 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.164 INFO:teuthology.orchestra.run.smithi059.stdout: "autotune_interval": { 2023-12-11T18:03:56.165 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "autotune_interval", 2023-12-11T18:03:56.165 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2023-12-11T18:03:56.165 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.165 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.165 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "600", 2023-12-11T18:03:56.165 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.166 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.166 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.166 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "how frequently to autotune daemon memory", 2023-12-11T18:03:56.166 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.166 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.166 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.166 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.166 INFO:teuthology.orchestra.run.smithi059.stdout: "autotune_memory_target_ratio": { 2023-12-11T18:03:56.166 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "autotune_memory_target_ratio", 2023-12-11T18:03:56.166 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "float", 2023-12-11T18:03:56.166 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.166 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.166 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0.7", 2023-12-11T18:03:56.166 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.167 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.167 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.167 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-12-11T18:03:56.167 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.167 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.167 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.167 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.167 INFO:teuthology.orchestra.run.smithi059.stdout: "cephadm_log_destination": { 2023-12-11T18:03:56.167 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "cephadm_log_destination", 2023-12-11T18:03:56.167 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.167 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.167 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.167 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:56.167 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.167 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.168 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:56.168 INFO:teuthology.orchestra.run.smithi059.stdout: "file", 2023-12-11T18:03:56.168 INFO:teuthology.orchestra.run.smithi059.stdout: "file,syslog", 2023-12-11T18:03:56.168 INFO:teuthology.orchestra.run.smithi059.stdout: "syslog" 2023-12-11T18:03:56.168 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:56.168 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Destination for cephadm command's persistent logging", 2023-12-11T18:03:56.168 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.168 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.168 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.168 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.168 INFO:teuthology.orchestra.run.smithi059.stdout: "cgroups_split": { 2023-12-11T18:03:56.168 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "cgroups_split", 2023-12-11T18:03:56.168 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.168 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.168 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.168 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2023-12-11T18:03:56.168 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.169 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.169 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.169 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-12-11T18:03:56.169 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.169 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.169 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.169 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.169 INFO:teuthology.orchestra.run.smithi059.stdout: "config_checks_enabled": { 2023-12-11T18:03:56.169 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "config_checks_enabled", 2023-12-11T18:03:56.169 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.169 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.169 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.169 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:56.169 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.169 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.169 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.170 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-12-11T18:03:56.170 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.170 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.170 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.170 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.170 INFO:teuthology.orchestra.run.smithi059.stdout: "config_dashboard": { 2023-12-11T18:03:56.170 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "config_dashboard", 2023-12-11T18:03:56.170 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.170 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.170 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.170 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2023-12-11T18:03:56.170 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.170 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.170 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.171 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-12-11T18:03:56.171 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.171 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.171 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.171 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.171 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_alertmanager": { 2023-12-11T18:03:56.171 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_alertmanager", 2023-12-11T18:03:56.171 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.171 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.171 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.171 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-12-11T18:03:56.171 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.171 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.171 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.171 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Prometheus container image", 2023-12-11T18:03:56.171 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.171 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.172 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.172 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.172 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_base": { 2023-12-11T18:03:56.172 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_base", 2023-12-11T18:03:56.172 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.172 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.172 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.172 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/ceph/ceph", 2023-12-11T18:03:56.172 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.172 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.172 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.172 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Container image name, without the tag", 2023-12-11T18:03:56.172 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.172 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.172 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.172 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.173 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_elasticsearch": { 2023-12-11T18:03:56.173 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_elasticsearch", 2023-12-11T18:03:56.173 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.173 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.173 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.173 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-12-11T18:03:56.173 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.173 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.173 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.173 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "elasticsearch container image", 2023-12-11T18:03:56.173 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.173 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.173 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.173 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.173 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_grafana": { 2023-12-11T18:03:56.174 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_grafana", 2023-12-11T18:03:56.174 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.174 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.174 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.174 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-12-11T18:03:56.174 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.174 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.174 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.174 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Prometheus container image", 2023-12-11T18:03:56.174 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.174 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.174 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.174 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.174 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_haproxy": { 2023-12-11T18:03:56.175 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_haproxy", 2023-12-11T18:03:56.175 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.175 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.175 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.175 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-12-11T18:03:56.175 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.175 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.175 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.175 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "HAproxy container image", 2023-12-11T18:03:56.175 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.175 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.175 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.175 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.175 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_jaeger_agent": { 2023-12-11T18:03:56.175 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_jaeger_agent", 2023-12-11T18:03:56.176 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.176 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.176 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.176 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-12-11T18:03:56.176 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.176 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.176 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.176 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Jaeger agent container image", 2023-12-11T18:03:56.176 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.176 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.176 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.176 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.176 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_jaeger_collector": { 2023-12-11T18:03:56.176 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_jaeger_collector", 2023-12-11T18:03:56.176 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.176 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.177 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.177 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-12-11T18:03:56.177 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.177 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.177 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.177 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Jaeger collector container image", 2023-12-11T18:03:56.177 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.177 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.177 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.177 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.177 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_jaeger_query": { 2023-12-11T18:03:56.177 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_jaeger_query", 2023-12-11T18:03:56.177 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.177 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.177 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.177 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-12-11T18:03:56.178 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.178 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.178 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.178 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Jaeger query container image", 2023-12-11T18:03:56.178 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.178 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.178 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.178 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.178 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_keepalived": { 2023-12-11T18:03:56.178 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_keepalived", 2023-12-11T18:03:56.178 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.178 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.178 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.178 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-12-11T18:03:56.179 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.179 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.179 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.179 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Keepalived container image", 2023-12-11T18:03:56.179 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.179 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.179 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.179 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.179 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_loki": { 2023-12-11T18:03:56.179 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_loki", 2023-12-11T18:03:56.179 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.179 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.179 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.179 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-12-11T18:03:56.180 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.180 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.180 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.180 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Loki container image", 2023-12-11T18:03:56.180 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.180 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.180 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.180 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.180 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_node_exporter": { 2023-12-11T18:03:56.180 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_node_exporter", 2023-12-11T18:03:56.180 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.180 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.180 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.180 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-12-11T18:03:56.180 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.181 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.181 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.181 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Prometheus container image", 2023-12-11T18:03:56.181 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.181 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.181 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.181 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.181 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_nvmeof": { 2023-12-11T18:03:56.181 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_nvmeof", 2023-12-11T18:03:56.181 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.181 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.181 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.181 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/ceph/nvmeof:latest", 2023-12-11T18:03:56.181 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.181 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.181 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.182 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Nvme-of container image", 2023-12-11T18:03:56.182 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.182 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.182 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.182 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.182 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_prometheus": { 2023-12-11T18:03:56.182 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_prometheus", 2023-12-11T18:03:56.182 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.182 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.182 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.182 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-12-11T18:03:56.182 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.182 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.183 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.183 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Prometheus container image", 2023-12-11T18:03:56.183 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.183 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.183 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.183 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.183 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_promtail": { 2023-12-11T18:03:56.183 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_promtail", 2023-12-11T18:03:56.183 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.183 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.183 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.183 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-12-11T18:03:56.184 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.184 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.184 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.184 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Promtail container image", 2023-12-11T18:03:56.184 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.184 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.184 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.184 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.184 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_snmp_gateway": { 2023-12-11T18:03:56.184 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_snmp_gateway", 2023-12-11T18:03:56.184 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.184 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.184 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.184 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-12-11T18:03:56.184 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.185 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.185 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.185 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "SNMP Gateway container image", 2023-12-11T18:03:56.185 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.185 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.185 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.185 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.185 INFO:teuthology.orchestra.run.smithi059.stdout: "container_init": { 2023-12-11T18:03:56.185 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_init", 2023-12-11T18:03:56.185 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.185 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.185 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.186 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2023-12-11T18:03:56.186 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.186 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.186 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.186 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Run podman/docker with `--init`", 2023-12-11T18:03:56.186 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.186 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.186 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.186 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.186 INFO:teuthology.orchestra.run.smithi059.stdout: "daemon_cache_timeout": { 2023-12-11T18:03:56.186 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "daemon_cache_timeout", 2023-12-11T18:03:56.186 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2023-12-11T18:03:56.186 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.187 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.187 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "600", 2023-12-11T18:03:56.187 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.187 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.187 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.187 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-12-11T18:03:56.187 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.187 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.187 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.187 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.187 INFO:teuthology.orchestra.run.smithi059.stdout: "default_cephadm_command_timeout": { 2023-12-11T18:03:56.187 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "default_cephadm_command_timeout", 2023-12-11T18:03:56.187 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2023-12-11T18:03:56.187 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.187 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.188 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "900", 2023-12-11T18:03:56.188 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.188 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.188 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.188 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-12-11T18:03:56.188 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.188 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.188 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.188 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.188 INFO:teuthology.orchestra.run.smithi059.stdout: "default_registry": { 2023-12-11T18:03:56.188 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "default_registry", 2023-12-11T18:03:56.188 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.188 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.188 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.189 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "docker.io", 2023-12-11T18:03:56.189 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.189 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.189 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.189 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-12-11T18:03:56.189 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.189 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.189 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.189 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.189 INFO:teuthology.orchestra.run.smithi059.stdout: "device_cache_timeout": { 2023-12-11T18:03:56.189 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "device_cache_timeout", 2023-12-11T18:03:56.189 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2023-12-11T18:03:56.189 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.189 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.190 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "1800", 2023-12-11T18:03:56.190 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.190 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.190 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.190 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "seconds to cache device inventory", 2023-12-11T18:03:56.190 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.190 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.190 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.190 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.190 INFO:teuthology.orchestra.run.smithi059.stdout: "device_enhanced_scan": { 2023-12-11T18:03:56.190 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "device_enhanced_scan", 2023-12-11T18:03:56.190 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.190 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.190 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.190 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:56.191 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.191 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.191 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.191 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Use libstoragemgmt during device scans", 2023-12-11T18:03:56.191 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.191 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.191 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.191 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.191 INFO:teuthology.orchestra.run.smithi059.stdout: "facts_cache_timeout": { 2023-12-11T18:03:56.191 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "facts_cache_timeout", 2023-12-11T18:03:56.191 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2023-12-11T18:03:56.191 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.191 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.191 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "60", 2023-12-11T18:03:56.191 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.191 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.192 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.192 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "seconds to cache host facts data", 2023-12-11T18:03:56.192 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.192 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.192 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.192 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.192 INFO:teuthology.orchestra.run.smithi059.stdout: "host_check_interval": { 2023-12-11T18:03:56.192 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "host_check_interval", 2023-12-11T18:03:56.192 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2023-12-11T18:03:56.192 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.192 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.192 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "600", 2023-12-11T18:03:56.192 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.192 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.192 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.193 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "how frequently to perform a host check", 2023-12-11T18:03:56.193 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.193 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.193 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.193 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.193 INFO:teuthology.orchestra.run.smithi059.stdout: "inventory_list_all": { 2023-12-11T18:03:56.193 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "inventory_list_all", 2023-12-11T18:03:56.193 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.193 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.193 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.193 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:56.193 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.193 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.193 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.193 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2023-12-11T18:03:56.193 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.194 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.194 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.194 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.194 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2023-12-11T18:03:56.194 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2023-12-11T18:03:56.194 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.194 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.194 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.194 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:56.194 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.194 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.194 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:56.194 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:56.194 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:56.195 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:56.195 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:56.195 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:56.195 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:56.195 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:56.195 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.195 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.195 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.195 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.195 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.195 INFO:teuthology.orchestra.run.smithi059.stdout: "log_refresh_metadata": { 2023-12-11T18:03:56.195 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_refresh_metadata", 2023-12-11T18:03:56.195 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.195 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.196 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.196 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:56.196 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.196 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.196 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.196 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-12-11T18:03:56.196 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.196 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.196 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.196 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.196 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2023-12-11T18:03:56.196 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2023-12-11T18:03:56.196 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.197 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.197 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.197 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2023-12-11T18:03:56.197 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.197 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.197 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.197 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-12-11T18:03:56.197 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.197 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.197 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.197 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.197 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2023-12-11T18:03:56.197 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2023-12-11T18:03:56.197 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.197 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.198 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.198 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2023-12-11T18:03:56.198 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.198 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.198 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:56.198 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:56.198 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:56.198 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:56.198 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:56.198 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:56.198 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:56.198 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:56.198 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.199 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.199 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.199 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.199 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.199 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2023-12-11T18:03:56.199 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2023-12-11T18:03:56.199 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.199 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.199 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.199 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:56.199 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.199 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.199 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.199 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.199 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.199 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.200 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.200 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.200 INFO:teuthology.orchestra.run.smithi059.stdout: "manage_etc_ceph_ceph_conf": { 2023-12-11T18:03:56.200 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-12-11T18:03:56.200 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.200 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.200 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.200 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:56.200 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.200 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.200 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.200 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-12-11T18:03:56.200 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.201 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.201 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.201 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.201 INFO:teuthology.orchestra.run.smithi059.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-12-11T18:03:56.201 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-12-11T18:03:56.201 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.201 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.201 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.201 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "*", 2023-12-11T18:03:56.201 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.201 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.201 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.201 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-12-11T18:03:56.201 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.202 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.202 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.202 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.202 INFO:teuthology.orchestra.run.smithi059.stdout: "max_count_per_host": { 2023-12-11T18:03:56.202 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "max_count_per_host", 2023-12-11T18:03:56.202 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2023-12-11T18:03:56.202 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.202 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.202 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "10", 2023-12-11T18:03:56.202 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.202 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.202 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.202 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "max number of daemons per service per host", 2023-12-11T18:03:56.203 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.203 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.203 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.203 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.203 INFO:teuthology.orchestra.run.smithi059.stdout: "max_osd_draining_count": { 2023-12-11T18:03:56.203 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "max_osd_draining_count", 2023-12-11T18:03:56.203 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2023-12-11T18:03:56.203 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.203 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.203 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "10", 2023-12-11T18:03:56.203 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.203 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.203 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.203 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-12-11T18:03:56.203 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.204 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.204 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.204 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.204 INFO:teuthology.orchestra.run.smithi059.stdout: "migration_current": { 2023-12-11T18:03:56.204 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "migration_current", 2023-12-11T18:03:56.204 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2023-12-11T18:03:56.204 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.204 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.204 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:56.204 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.204 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.204 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.204 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "internal - do not modify", 2023-12-11T18:03:56.204 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.204 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.205 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.205 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.205 INFO:teuthology.orchestra.run.smithi059.stdout: "mode": { 2023-12-11T18:03:56.205 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "mode", 2023-12-11T18:03:56.205 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.205 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.205 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.205 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "root", 2023-12-11T18:03:56.205 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.205 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.205 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:56.205 INFO:teuthology.orchestra.run.smithi059.stdout: "cephadm-package", 2023-12-11T18:03:56.205 INFO:teuthology.orchestra.run.smithi059.stdout: "root" 2023-12-11T18:03:56.205 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:56.205 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "mode for remote execution of cephadm", 2023-12-11T18:03:56.206 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.206 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.206 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.206 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.206 INFO:teuthology.orchestra.run.smithi059.stdout: "prometheus_alerts_path": { 2023-12-11T18:03:56.206 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "prometheus_alerts_path", 2023-12-11T18:03:56.206 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.206 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.206 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.206 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-12-11T18:03:56.206 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.206 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.206 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.206 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-12-11T18:03:56.207 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.207 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.207 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.207 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.207 INFO:teuthology.orchestra.run.smithi059.stdout: "registry_insecure": { 2023-12-11T18:03:56.207 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "registry_insecure", 2023-12-11T18:03:56.207 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.207 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.207 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.207 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:56.207 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.207 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.208 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.208 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-12-11T18:03:56.208 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.208 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.208 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.208 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.208 INFO:teuthology.orchestra.run.smithi059.stdout: "registry_password": { 2023-12-11T18:03:56.208 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "registry_password", 2023-12-11T18:03:56.208 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.208 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.208 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.208 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:56.209 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.209 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.209 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.209 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-12-11T18:03:56.209 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.209 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.209 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.209 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.209 INFO:teuthology.orchestra.run.smithi059.stdout: "registry_url": { 2023-12-11T18:03:56.209 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "registry_url", 2023-12-11T18:03:56.209 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.210 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.210 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.210 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:56.210 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.210 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.210 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.210 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-12-11T18:03:56.210 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.210 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.210 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.210 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.210 INFO:teuthology.orchestra.run.smithi059.stdout: "registry_username": { 2023-12-11T18:03:56.210 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "registry_username", 2023-12-11T18:03:56.210 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.211 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.211 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.211 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:56.211 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.211 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.211 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.211 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-12-11T18:03:56.211 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.211 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.211 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.211 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.211 INFO:teuthology.orchestra.run.smithi059.stdout: "secure_monitoring_stack": { 2023-12-11T18:03:56.211 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "secure_monitoring_stack", 2023-12-11T18:03:56.211 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.211 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.212 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.212 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:56.212 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.212 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.212 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.212 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-12-11T18:03:56.212 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.212 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.212 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.212 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.212 INFO:teuthology.orchestra.run.smithi059.stdout: "service_discovery_port": { 2023-12-11T18:03:56.212 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "service_discovery_port", 2023-12-11T18:03:56.212 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2023-12-11T18:03:56.212 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.213 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.213 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "8765", 2023-12-11T18:03:56.213 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.213 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.213 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.213 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "cephadm service discovery port", 2023-12-11T18:03:56.213 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.213 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.213 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.213 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.213 INFO:teuthology.orchestra.run.smithi059.stdout: "ssh_config_file": { 2023-12-11T18:03:56.213 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ssh_config_file", 2023-12-11T18:03:56.213 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.213 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.213 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.214 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:56.214 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.214 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.214 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.214 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-12-11T18:03:56.214 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.214 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.214 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.214 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.214 INFO:teuthology.orchestra.run.smithi059.stdout: "use_agent": { 2023-12-11T18:03:56.214 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "use_agent", 2023-12-11T18:03:56.214 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.214 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.214 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.215 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:56.215 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.215 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.215 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.215 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-12-11T18:03:56.215 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.215 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.215 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.215 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.215 INFO:teuthology.orchestra.run.smithi059.stdout: "use_repo_digest": { 2023-12-11T18:03:56.215 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "use_repo_digest", 2023-12-11T18:03:56.215 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.215 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.215 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.215 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2023-12-11T18:03:56.215 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.215 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.216 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.216 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-12-11T18:03:56.216 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.216 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.216 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.216 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.216 INFO:teuthology.orchestra.run.smithi059.stdout: "warn_on_failed_host_check": { 2023-12-11T18:03:56.216 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "warn_on_failed_host_check", 2023-12-11T18:03:56.216 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.216 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.216 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.216 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2023-12-11T18:03:56.216 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.216 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.216 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.216 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "raise a health warning if the host check fails", 2023-12-11T18:03:56.217 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.217 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.217 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.217 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.217 INFO:teuthology.orchestra.run.smithi059.stdout: "warn_on_stray_daemons": { 2023-12-11T18:03:56.217 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "warn_on_stray_daemons", 2023-12-11T18:03:56.217 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.217 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.217 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.217 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2023-12-11T18:03:56.217 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.217 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.217 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.217 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-12-11T18:03:56.217 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.218 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.218 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.218 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.218 INFO:teuthology.orchestra.run.smithi059.stdout: "warn_on_stray_hosts": { 2023-12-11T18:03:56.218 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "warn_on_stray_hosts", 2023-12-11T18:03:56.218 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.218 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.218 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.218 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2023-12-11T18:03:56.218 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.218 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.218 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.218 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-12-11T18:03:56.218 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.218 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.219 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.219 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:56.219 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:56.219 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.219 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-12-11T18:03:56.219 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "crash", 2023-12-11T18:03:56.219 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2023-12-11T18:03:56.219 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2023-12-11T18:03:56.219 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2023-12-11T18:03:56.219 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2023-12-11T18:03:56.219 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2023-12-11T18:03:56.220 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.220 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.220 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.220 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:56.220 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.220 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.220 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:56.220 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:56.220 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:56.220 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:56.220 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:56.220 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:56.220 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:56.221 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:56.221 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.221 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.221 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.221 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.221 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.221 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2023-12-11T18:03:56.221 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2023-12-11T18:03:56.221 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.221 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.221 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.221 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:56.221 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.221 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.221 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.222 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.222 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.222 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.222 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.222 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.222 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2023-12-11T18:03:56.222 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2023-12-11T18:03:56.222 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.222 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.222 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.222 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2023-12-11T18:03:56.222 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.222 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.222 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:56.223 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:56.223 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:56.223 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:56.223 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:56.223 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:56.223 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:56.223 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:56.223 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.223 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.223 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.223 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.223 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.223 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2023-12-11T18:03:56.223 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2023-12-11T18:03:56.223 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.224 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.224 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.224 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:56.224 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.224 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.224 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.224 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.224 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.224 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.224 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.224 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.224 INFO:teuthology.orchestra.run.smithi059.stdout: "retain_interval": { 2023-12-11T18:03:56.224 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "retain_interval", 2023-12-11T18:03:56.224 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2023-12-11T18:03:56.225 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.225 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.225 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "31536000", 2023-12-11T18:03:56.225 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.225 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.225 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.225 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "how long to retain crashes before pruning them", 2023-12-11T18:03:56.225 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.225 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.225 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.225 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.225 INFO:teuthology.orchestra.run.smithi059.stdout: "warn_recent_interval": { 2023-12-11T18:03:56.225 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "warn_recent_interval", 2023-12-11T18:03:56.225 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2023-12-11T18:03:56.225 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.226 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.226 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "1209600", 2023-12-11T18:03:56.226 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.226 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.226 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.226 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "time interval in which to warn about recent crashes", 2023-12-11T18:03:56.226 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.226 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.226 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.226 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:56.226 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:56.226 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.226 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-12-11T18:03:56.226 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "dashboard", 2023-12-11T18:03:56.226 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2023-12-11T18:03:56.227 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2023-12-11T18:03:56.227 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2023-12-11T18:03:56.227 INFO:teuthology.orchestra.run.smithi059.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-12-11T18:03:56.227 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-12-11T18:03:56.227 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2023-12-11T18:03:56.227 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.227 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.227 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "10", 2023-12-11T18:03:56.227 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.227 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.227 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.227 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.228 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.228 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.228 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.228 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.228 INFO:teuthology.orchestra.run.smithi059.stdout: "ALERTMANAGER_API_HOST": { 2023-12-11T18:03:56.228 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ALERTMANAGER_API_HOST", 2023-12-11T18:03:56.228 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.228 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.228 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.228 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:56.228 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.228 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.228 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.228 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.229 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.229 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.229 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.229 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.229 INFO:teuthology.orchestra.run.smithi059.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-12-11T18:03:56.229 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-12-11T18:03:56.229 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.229 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.229 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.229 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2023-12-11T18:03:56.229 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.229 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.229 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.229 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.229 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.230 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.230 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.230 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.230 INFO:teuthology.orchestra.run.smithi059.stdout: "AUDIT_API_ENABLED": { 2023-12-11T18:03:56.230 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "AUDIT_API_ENABLED", 2023-12-11T18:03:56.230 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.230 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.230 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.230 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:56.230 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.230 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.230 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.231 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.231 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.231 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.231 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.231 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.231 INFO:teuthology.orchestra.run.smithi059.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-12-11T18:03:56.231 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-12-11T18:03:56.231 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.231 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.231 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.231 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2023-12-11T18:03:56.231 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.231 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.231 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.231 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.231 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.232 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.232 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.232 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.232 INFO:teuthology.orchestra.run.smithi059.stdout: "ENABLE_BROWSABLE_API": { 2023-12-11T18:03:56.232 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ENABLE_BROWSABLE_API", 2023-12-11T18:03:56.232 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.232 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.232 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.232 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2023-12-11T18:03:56.232 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.232 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.232 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.233 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.233 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.233 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.233 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.233 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.233 INFO:teuthology.orchestra.run.smithi059.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-12-11T18:03:56.233 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-12-11T18:03:56.233 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.233 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.233 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.233 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2023-12-11T18:03:56.233 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.233 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.234 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.234 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.234 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.234 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.234 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.234 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.234 INFO:teuthology.orchestra.run.smithi059.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-12-11T18:03:56.234 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-12-11T18:03:56.234 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.234 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.234 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.235 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2023-12-11T18:03:56.235 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.235 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.235 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.235 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.235 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.235 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.235 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.235 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.235 INFO:teuthology.orchestra.run.smithi059.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-12-11T18:03:56.235 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-12-11T18:03:56.235 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.235 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.235 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.235 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2023-12-11T18:03:56.236 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.236 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.236 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.236 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.236 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.236 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.236 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.236 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.236 INFO:teuthology.orchestra.run.smithi059.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-12-11T18:03:56.236 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-12-11T18:03:56.236 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.236 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.236 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.236 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2023-12-11T18:03:56.236 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.237 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.237 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.237 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.237 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.237 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.237 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.237 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.237 INFO:teuthology.orchestra.run.smithi059.stdout: "FEATURE_TOGGLE_NFS": { 2023-12-11T18:03:56.237 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-12-11T18:03:56.237 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.237 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.237 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.237 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2023-12-11T18:03:56.237 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.237 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.238 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.238 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.238 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.238 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.238 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.238 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.238 INFO:teuthology.orchestra.run.smithi059.stdout: "FEATURE_TOGGLE_RBD": { 2023-12-11T18:03:56.238 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-12-11T18:03:56.238 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.238 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.238 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.238 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2023-12-11T18:03:56.238 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.238 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.239 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.239 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.239 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.239 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.239 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.239 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.239 INFO:teuthology.orchestra.run.smithi059.stdout: "FEATURE_TOGGLE_RGW": { 2023-12-11T18:03:56.239 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-12-11T18:03:56.239 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.239 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.239 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.239 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2023-12-11T18:03:56.239 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.239 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.240 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.240 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.240 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.240 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.240 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.240 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.240 INFO:teuthology.orchestra.run.smithi059.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-12-11T18:03:56.240 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-12-11T18:03:56.240 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.240 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.240 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.240 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:56.240 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.240 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.241 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.241 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.241 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.241 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.241 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.241 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.241 INFO:teuthology.orchestra.run.smithi059.stdout: "GRAFANA_API_PASSWORD": { 2023-12-11T18:03:56.241 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "GRAFANA_API_PASSWORD", 2023-12-11T18:03:56.241 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.241 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.241 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.241 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "admin", 2023-12-11T18:03:56.241 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.241 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.241 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.242 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.242 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.242 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.242 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.242 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.242 INFO:teuthology.orchestra.run.smithi059.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-12-11T18:03:56.242 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-12-11T18:03:56.242 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.242 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.242 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.242 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2023-12-11T18:03:56.242 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.242 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.242 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.242 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.243 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.243 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.243 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.243 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.243 INFO:teuthology.orchestra.run.smithi059.stdout: "GRAFANA_API_URL": { 2023-12-11T18:03:56.243 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "GRAFANA_API_URL", 2023-12-11T18:03:56.243 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.243 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.243 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.243 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:56.243 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.243 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.243 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.243 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.243 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.244 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.244 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.244 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.244 INFO:teuthology.orchestra.run.smithi059.stdout: "GRAFANA_API_USERNAME": { 2023-12-11T18:03:56.244 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "GRAFANA_API_USERNAME", 2023-12-11T18:03:56.244 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.244 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.244 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.244 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "admin", 2023-12-11T18:03:56.244 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.244 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.244 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.244 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.244 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.245 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.245 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.245 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.245 INFO:teuthology.orchestra.run.smithi059.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-12-11T18:03:56.245 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-12-11T18:03:56.245 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.245 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.245 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.245 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:56.245 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.245 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.245 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.245 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.246 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.246 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.246 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.246 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.246 INFO:teuthology.orchestra.run.smithi059.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-12-11T18:03:56.246 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-12-11T18:03:56.246 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.246 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.246 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.246 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:56.246 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.246 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.247 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.247 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.247 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.247 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.247 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.247 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.247 INFO:teuthology.orchestra.run.smithi059.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-12-11T18:03:56.247 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-12-11T18:03:56.247 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.247 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.247 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.247 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2023-12-11T18:03:56.247 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.247 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.247 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.248 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.248 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.248 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.248 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.248 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.248 INFO:teuthology.orchestra.run.smithi059.stdout: "ISSUE_TRACKER_API_KEY": { 2023-12-11T18:03:56.248 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-12-11T18:03:56.248 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.248 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.248 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.248 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:56.249 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.249 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.249 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.249 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.249 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.249 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.249 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.249 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.249 INFO:teuthology.orchestra.run.smithi059.stdout: "PROMETHEUS_API_HOST": { 2023-12-11T18:03:56.249 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PROMETHEUS_API_HOST", 2023-12-11T18:03:56.249 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.249 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.249 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.249 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:56.250 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.250 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.250 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.250 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.250 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.250 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.250 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.250 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.250 INFO:teuthology.orchestra.run.smithi059.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-12-11T18:03:56.250 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-12-11T18:03:56.250 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.250 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.250 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.250 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2023-12-11T18:03:56.251 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.251 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.251 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.251 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.251 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.251 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.251 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.251 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.251 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-12-11T18:03:56.251 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-12-11T18:03:56.251 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.251 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.251 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.251 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:56.251 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.252 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.252 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.252 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.252 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.252 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.252 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.252 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.252 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-12-11T18:03:56.252 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-12-11T18:03:56.252 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.252 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.252 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.252 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:56.252 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.252 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.252 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.253 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.253 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.253 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.253 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.253 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.253 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-12-11T18:03:56.253 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-12-11T18:03:56.253 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.253 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.253 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.253 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2023-12-11T18:03:56.253 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.253 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.253 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.254 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.254 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.254 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.254 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.254 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.254 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-12-11T18:03:56.254 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-12-11T18:03:56.254 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.254 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.254 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.254 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2023-12-11T18:03:56.254 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.254 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.254 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.255 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.255 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.255 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.255 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.255 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.255 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-12-11T18:03:56.255 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-12-11T18:03:56.255 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.255 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.255 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.255 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:56.255 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.255 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.255 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.255 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.256 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.256 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.256 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.256 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.256 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-12-11T18:03:56.256 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-12-11T18:03:56.256 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.256 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.256 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.256 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:56.256 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.256 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.256 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.256 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.256 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.257 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.257 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.257 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.257 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-12-11T18:03:56.257 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-12-11T18:03:56.257 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.257 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.257 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.257 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:56.257 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.257 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.257 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.257 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.257 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.258 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.258 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.258 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.258 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_ENABLED": { 2023-12-11T18:03:56.258 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_ENABLED", 2023-12-11T18:03:56.258 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.258 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.258 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.258 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2023-12-11T18:03:56.258 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.258 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.258 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.258 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.258 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.259 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.259 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.259 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.259 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-12-11T18:03:56.259 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-12-11T18:03:56.259 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.259 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.259 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.259 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-12-11T18:03:56.259 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.259 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.259 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.259 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.259 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.259 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.260 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.260 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.260 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-12-11T18:03:56.260 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-12-11T18:03:56.260 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2023-12-11T18:03:56.260 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.260 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.260 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "10", 2023-12-11T18:03:56.260 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.260 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.260 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.260 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.260 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.260 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.261 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.261 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.261 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-12-11T18:03:56.261 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-12-11T18:03:56.261 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2023-12-11T18:03:56.261 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.261 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.261 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "8", 2023-12-11T18:03:56.261 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.261 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.261 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.261 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.261 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.261 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.262 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.262 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.262 INFO:teuthology.orchestra.run.smithi059.stdout: "REST_REQUESTS_TIMEOUT": { 2023-12-11T18:03:56.262 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-12-11T18:03:56.262 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2023-12-11T18:03:56.262 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.262 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.262 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "45", 2023-12-11T18:03:56.262 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.262 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.262 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.262 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.262 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.262 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.263 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.263 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.263 INFO:teuthology.orchestra.run.smithi059.stdout: "RGW_API_ACCESS_KEY": { 2023-12-11T18:03:56.263 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "RGW_API_ACCESS_KEY", 2023-12-11T18:03:56.263 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.263 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.263 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.263 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:56.263 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.263 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.263 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.263 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.263 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.263 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.263 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.264 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.264 INFO:teuthology.orchestra.run.smithi059.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-12-11T18:03:56.264 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-12-11T18:03:56.264 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.264 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.264 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.264 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "admin", 2023-12-11T18:03:56.264 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.264 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.264 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.264 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.264 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.264 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.264 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.265 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.265 INFO:teuthology.orchestra.run.smithi059.stdout: "RGW_API_SECRET_KEY": { 2023-12-11T18:03:56.265 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "RGW_API_SECRET_KEY", 2023-12-11T18:03:56.265 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.265 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.265 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.265 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:56.265 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.265 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.265 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.265 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.265 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.265 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.265 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.265 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.266 INFO:teuthology.orchestra.run.smithi059.stdout: "RGW_API_SSL_VERIFY": { 2023-12-11T18:03:56.266 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "RGW_API_SSL_VERIFY", 2023-12-11T18:03:56.266 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.266 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.266 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.266 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2023-12-11T18:03:56.266 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.266 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.266 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.266 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.266 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.267 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.267 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.267 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.267 INFO:teuthology.orchestra.run.smithi059.stdout: "UNSAFE_TLS_v1_2": { 2023-12-11T18:03:56.267 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "UNSAFE_TLS_v1_2", 2023-12-11T18:03:56.267 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.267 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.267 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.267 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:56.267 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.267 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.267 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.267 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.267 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.268 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.268 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.268 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.268 INFO:teuthology.orchestra.run.smithi059.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-12-11T18:03:56.268 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-12-11T18:03:56.268 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2023-12-11T18:03:56.268 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.268 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.268 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2023-12-11T18:03:56.268 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.268 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.268 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.268 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.268 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.268 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.269 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.269 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.269 INFO:teuthology.orchestra.run.smithi059.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-12-11T18:03:56.269 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-12-11T18:03:56.269 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2023-12-11T18:03:56.269 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.269 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.269 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "10", 2023-12-11T18:03:56.269 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.269 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.269 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.269 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.269 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.269 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.269 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.269 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.270 INFO:teuthology.orchestra.run.smithi059.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-12-11T18:03:56.270 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-12-11T18:03:56.270 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2023-12-11T18:03:56.270 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.270 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.270 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "5", 2023-12-11T18:03:56.270 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.270 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.270 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.270 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.270 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.270 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.270 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.271 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.271 INFO:teuthology.orchestra.run.smithi059.stdout: "cross_origin_url": { 2023-12-11T18:03:56.271 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "cross_origin_url", 2023-12-11T18:03:56.271 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.271 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.271 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.271 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:56.271 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.271 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.271 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.271 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.271 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.271 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.271 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.271 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.272 INFO:teuthology.orchestra.run.smithi059.stdout: "crt_file": { 2023-12-11T18:03:56.272 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "crt_file", 2023-12-11T18:03:56.272 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.272 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.272 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.272 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:56.272 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.272 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.272 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.272 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.272 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.272 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.272 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.272 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.272 INFO:teuthology.orchestra.run.smithi059.stdout: "debug": { 2023-12-11T18:03:56.273 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "debug", 2023-12-11T18:03:56.273 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.273 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.273 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.273 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:56.273 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.273 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.273 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.273 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Enable/disable debug options", 2023-12-11T18:03:56.273 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.273 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.273 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.273 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.273 INFO:teuthology.orchestra.run.smithi059.stdout: "jwt_token_ttl": { 2023-12-11T18:03:56.273 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "jwt_token_ttl", 2023-12-11T18:03:56.274 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2023-12-11T18:03:56.274 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.274 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.274 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "28800", 2023-12-11T18:03:56.274 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.274 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.274 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.274 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.274 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.274 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.274 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.274 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.275 INFO:teuthology.orchestra.run.smithi059.stdout: "key_file": { 2023-12-11T18:03:56.275 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "key_file", 2023-12-11T18:03:56.275 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.275 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.275 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.275 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:56.275 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.275 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.275 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.275 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.275 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.275 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.275 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.275 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.275 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2023-12-11T18:03:56.276 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2023-12-11T18:03:56.276 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.276 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.276 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.276 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:56.276 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.276 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.276 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:56.276 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:56.276 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:56.276 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:56.276 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:56.277 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:56.277 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:56.277 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:56.277 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.277 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.277 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.277 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.277 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.277 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2023-12-11T18:03:56.277 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2023-12-11T18:03:56.277 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.277 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.277 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.277 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:56.277 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.278 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.278 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.278 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.278 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.278 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.278 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.278 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.278 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2023-12-11T18:03:56.278 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2023-12-11T18:03:56.278 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.278 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.278 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.278 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2023-12-11T18:03:56.278 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.278 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.279 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:56.279 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:56.279 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:56.279 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:56.279 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:56.279 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:56.279 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:56.279 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:56.279 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.279 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.279 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.279 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.279 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.279 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2023-12-11T18:03:56.280 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2023-12-11T18:03:56.280 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.280 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.280 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.280 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:56.280 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.280 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.280 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.280 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.280 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.280 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.280 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.280 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.280 INFO:teuthology.orchestra.run.smithi059.stdout: "motd": { 2023-12-11T18:03:56.280 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "motd", 2023-12-11T18:03:56.281 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.281 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.281 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.281 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:56.281 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.281 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.281 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.281 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "The message of the day", 2023-12-11T18:03:56.281 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.281 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.281 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.281 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.281 INFO:teuthology.orchestra.run.smithi059.stdout: "redirect_resolve_ip_addr": { 2023-12-11T18:03:56.281 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "redirect_resolve_ip_addr", 2023-12-11T18:03:56.282 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.282 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.282 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.282 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:56.282 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.282 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.282 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.282 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.282 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.282 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.282 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.282 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.282 INFO:teuthology.orchestra.run.smithi059.stdout: "server_addr": { 2023-12-11T18:03:56.282 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "server_addr", 2023-12-11T18:03:56.282 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.283 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.283 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.283 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "::", 2023-12-11T18:03:56.283 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.283 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.283 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.283 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.283 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.283 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.283 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.283 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.283 INFO:teuthology.orchestra.run.smithi059.stdout: "server_port": { 2023-12-11T18:03:56.284 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "server_port", 2023-12-11T18:03:56.284 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2023-12-11T18:03:56.284 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.284 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.284 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "8080", 2023-12-11T18:03:56.284 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.284 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.284 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.284 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.284 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.284 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.284 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.284 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.284 INFO:teuthology.orchestra.run.smithi059.stdout: "ssl": { 2023-12-11T18:03:56.285 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ssl", 2023-12-11T18:03:56.285 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.285 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.285 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.285 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2023-12-11T18:03:56.285 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.285 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.285 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.285 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.285 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.285 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.285 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.285 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.285 INFO:teuthology.orchestra.run.smithi059.stdout: "ssl_server_port": { 2023-12-11T18:03:56.286 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ssl_server_port", 2023-12-11T18:03:56.286 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2023-12-11T18:03:56.286 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.286 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.286 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "8443", 2023-12-11T18:03:56.286 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.286 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.286 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.286 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.286 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.286 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.286 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.287 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.287 INFO:teuthology.orchestra.run.smithi059.stdout: "standby_behaviour": { 2023-12-11T18:03:56.287 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "standby_behaviour", 2023-12-11T18:03:56.287 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.287 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.287 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.287 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "redirect", 2023-12-11T18:03:56.287 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.287 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.287 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:56.287 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:56.287 INFO:teuthology.orchestra.run.smithi059.stdout: "redirect" 2023-12-11T18:03:56.287 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:56.287 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.288 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.288 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.288 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.288 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.288 INFO:teuthology.orchestra.run.smithi059.stdout: "standby_error_status_code": { 2023-12-11T18:03:56.288 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "standby_error_status_code", 2023-12-11T18:03:56.288 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2023-12-11T18:03:56.288 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.288 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.288 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "500", 2023-12-11T18:03:56.288 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "400", 2023-12-11T18:03:56.288 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "599", 2023-12-11T18:03:56.288 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.288 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.288 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.289 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.289 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.289 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.289 INFO:teuthology.orchestra.run.smithi059.stdout: "url_prefix": { 2023-12-11T18:03:56.289 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "url_prefix", 2023-12-11T18:03:56.289 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.289 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.289 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.289 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:56.289 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.289 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.289 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.289 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.289 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.289 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.290 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.290 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:56.290 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:56.290 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.290 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-12-11T18:03:56.290 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "devicehealth", 2023-12-11T18:03:56.290 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2023-12-11T18:03:56.290 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2023-12-11T18:03:56.290 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2023-12-11T18:03:56.290 INFO:teuthology.orchestra.run.smithi059.stdout: "enable_monitoring": { 2023-12-11T18:03:56.290 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "enable_monitoring", 2023-12-11T18:03:56.290 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.290 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.290 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.291 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2023-12-11T18:03:56.291 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.291 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.291 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.291 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "monitor device health metrics", 2023-12-11T18:03:56.291 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.291 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.291 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.291 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.291 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2023-12-11T18:03:56.291 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2023-12-11T18:03:56.291 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.291 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.291 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.291 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:56.292 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.292 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.292 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:56.292 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:56.292 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:56.292 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:56.292 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:56.292 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:56.292 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:56.292 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:56.292 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.292 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.292 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.292 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.292 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.293 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2023-12-11T18:03:56.293 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2023-12-11T18:03:56.293 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.293 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.293 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.293 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:56.293 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.293 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.293 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.293 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.293 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.293 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.293 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.293 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.293 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2023-12-11T18:03:56.294 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2023-12-11T18:03:56.294 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.294 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.294 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.294 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2023-12-11T18:03:56.294 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.294 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.294 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:56.294 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:56.294 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:56.294 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:56.294 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:56.294 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:56.294 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:56.295 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:56.295 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.295 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.295 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.295 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.295 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.295 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2023-12-11T18:03:56.295 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2023-12-11T18:03:56.295 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.295 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.295 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.295 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:56.295 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.295 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.295 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.296 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.296 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.296 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.296 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.296 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.296 INFO:teuthology.orchestra.run.smithi059.stdout: "mark_out_threshold": { 2023-12-11T18:03:56.296 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "mark_out_threshold", 2023-12-11T18:03:56.296 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2023-12-11T18:03:56.296 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.296 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.296 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "2419200", 2023-12-11T18:03:56.296 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.296 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.296 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.296 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-12-11T18:03:56.297 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.297 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.297 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.297 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.297 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_name": { 2023-12-11T18:03:56.297 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "pool_name", 2023-12-11T18:03:56.297 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.297 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.297 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.297 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "device_health_metrics", 2023-12-11T18:03:56.297 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.297 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.298 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.298 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "name of pool in which to store device health metrics", 2023-12-11T18:03:56.298 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.298 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.298 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.298 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.298 INFO:teuthology.orchestra.run.smithi059.stdout: "retention_period": { 2023-12-11T18:03:56.298 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "retention_period", 2023-12-11T18:03:56.298 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2023-12-11T18:03:56.298 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.298 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.298 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "15552000", 2023-12-11T18:03:56.298 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.298 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.298 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.299 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "how long to retain device health metrics", 2023-12-11T18:03:56.299 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.299 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.299 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.299 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.299 INFO:teuthology.orchestra.run.smithi059.stdout: "scrape_frequency": { 2023-12-11T18:03:56.299 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "scrape_frequency", 2023-12-11T18:03:56.299 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2023-12-11T18:03:56.299 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.299 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.299 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "86400", 2023-12-11T18:03:56.299 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.299 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.299 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.299 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "how frequently to scrape device health metrics", 2023-12-11T18:03:56.300 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.300 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.300 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.300 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.300 INFO:teuthology.orchestra.run.smithi059.stdout: "self_heal": { 2023-12-11T18:03:56.300 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "self_heal", 2023-12-11T18:03:56.300 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.300 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.300 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.300 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2023-12-11T18:03:56.300 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.300 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.300 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.300 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-12-11T18:03:56.300 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.301 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.301 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.301 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.301 INFO:teuthology.orchestra.run.smithi059.stdout: "sleep_interval": { 2023-12-11T18:03:56.301 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sleep_interval", 2023-12-11T18:03:56.301 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2023-12-11T18:03:56.301 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.301 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.301 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "600", 2023-12-11T18:03:56.301 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.301 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.301 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.302 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "how frequently to wake up and check device health", 2023-12-11T18:03:56.302 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.302 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.302 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.302 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.302 INFO:teuthology.orchestra.run.smithi059.stdout: "warn_threshold": { 2023-12-11T18:03:56.302 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "warn_threshold", 2023-12-11T18:03:56.302 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2023-12-11T18:03:56.302 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.302 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.302 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "7257600", 2023-12-11T18:03:56.302 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.302 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.302 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.302 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-12-11T18:03:56.303 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.303 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.303 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.303 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:56.303 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:56.303 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.303 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-12-11T18:03:56.303 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "diskprediction_local", 2023-12-11T18:03:56.303 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2023-12-11T18:03:56.303 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2023-12-11T18:03:56.303 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2023-12-11T18:03:56.303 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2023-12-11T18:03:56.303 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2023-12-11T18:03:56.303 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.303 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.304 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.304 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:56.305 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.305 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.305 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:56.305 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:56.305 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:56.305 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:56.305 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:56.305 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:56.305 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:56.305 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:56.306 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.306 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.306 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.306 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.306 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.306 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2023-12-11T18:03:56.306 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2023-12-11T18:03:56.306 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.306 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.306 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.306 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:56.306 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.306 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.306 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.307 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.307 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.307 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.307 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.307 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.307 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2023-12-11T18:03:56.307 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2023-12-11T18:03:56.307 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.307 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.307 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.307 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2023-12-11T18:03:56.307 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.308 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.308 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:56.308 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:56.308 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:56.308 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:56.308 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:56.308 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:56.308 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:56.308 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:56.308 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.308 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.309 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.309 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.309 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.309 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2023-12-11T18:03:56.309 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2023-12-11T18:03:56.309 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.309 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.309 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.309 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:56.309 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.309 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.310 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.310 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.310 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.310 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.310 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.310 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.310 INFO:teuthology.orchestra.run.smithi059.stdout: "predict_interval": { 2023-12-11T18:03:56.310 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "predict_interval", 2023-12-11T18:03:56.310 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.310 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.310 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.311 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "86400", 2023-12-11T18:03:56.311 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.311 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.311 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.311 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.311 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.312 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.312 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.312 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.312 INFO:teuthology.orchestra.run.smithi059.stdout: "predictor_model": { 2023-12-11T18:03:56.312 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "predictor_model", 2023-12-11T18:03:56.312 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.312 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.312 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.312 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "prophetstor", 2023-12-11T18:03:56.312 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.312 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.312 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.313 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.313 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.313 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.313 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.313 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.313 INFO:teuthology.orchestra.run.smithi059.stdout: "sleep_interval": { 2023-12-11T18:03:56.313 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sleep_interval", 2023-12-11T18:03:56.313 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.313 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.313 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.313 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "600", 2023-12-11T18:03:56.313 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.313 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.313 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.314 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.314 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.314 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.314 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.314 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:56.314 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:56.314 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.314 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-12-11T18:03:56.314 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "influx", 2023-12-11T18:03:56.314 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": false, 2023-12-11T18:03:56.314 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "influxdb python module not found", 2023-12-11T18:03:56.314 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2023-12-11T18:03:56.314 INFO:teuthology.orchestra.run.smithi059.stdout: "batch_size": { 2023-12-11T18:03:56.314 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "batch_size", 2023-12-11T18:03:56.315 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2023-12-11T18:03:56.315 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.315 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.315 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "5000", 2023-12-11T18:03:56.315 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.315 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.315 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.315 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-12-11T18:03:56.315 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.315 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.315 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.315 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.315 INFO:teuthology.orchestra.run.smithi059.stdout: "database": { 2023-12-11T18:03:56.315 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "database", 2023-12-11T18:03:56.315 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.316 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.316 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.316 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "ceph", 2023-12-11T18:03:56.316 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.316 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.316 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.316 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-12-11T18:03:56.316 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.316 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.317 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.317 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.317 INFO:teuthology.orchestra.run.smithi059.stdout: "hostname": { 2023-12-11T18:03:56.317 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "hostname", 2023-12-11T18:03:56.317 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.317 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.317 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.317 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:56.318 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.318 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.318 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.318 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "InfluxDB server hostname", 2023-12-11T18:03:56.318 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.318 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.318 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.319 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.319 INFO:teuthology.orchestra.run.smithi059.stdout: "interval": { 2023-12-11T18:03:56.319 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "interval", 2023-12-11T18:03:56.319 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2023-12-11T18:03:56.319 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.319 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.319 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "30", 2023-12-11T18:03:56.320 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "5", 2023-12-11T18:03:56.320 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.320 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.320 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-12-11T18:03:56.320 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.320 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.321 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.321 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.321 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2023-12-11T18:03:56.321 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2023-12-11T18:03:56.321 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.321 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.322 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.322 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:56.322 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.322 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.322 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:56.322 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:56.322 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:56.322 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:56.323 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:56.323 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:56.323 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:56.323 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:56.323 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.323 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.323 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.324 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.324 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.324 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2023-12-11T18:03:56.324 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2023-12-11T18:03:56.324 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.324 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.324 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.325 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:56.325 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.325 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.325 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.325 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.325 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.326 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.326 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.326 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.326 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2023-12-11T18:03:56.326 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2023-12-11T18:03:56.326 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.326 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.326 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.326 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2023-12-11T18:03:56.326 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.327 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.327 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:56.327 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:56.327 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:56.327 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:56.327 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:56.327 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:56.327 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:56.327 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:56.327 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.327 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.327 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.327 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.327 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.328 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2023-12-11T18:03:56.328 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2023-12-11T18:03:56.328 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.328 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.328 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.328 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:56.328 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.328 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.328 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.328 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.328 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.328 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.329 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.329 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.329 INFO:teuthology.orchestra.run.smithi059.stdout: "password": { 2023-12-11T18:03:56.329 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "password", 2023-12-11T18:03:56.329 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.329 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.329 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.329 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:56.329 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.330 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.330 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.330 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "password of InfluxDB server user", 2023-12-11T18:03:56.330 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.330 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.330 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.330 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.330 INFO:teuthology.orchestra.run.smithi059.stdout: "port": { 2023-12-11T18:03:56.330 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "port", 2023-12-11T18:03:56.330 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2023-12-11T18:03:56.330 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.330 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.330 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "8086", 2023-12-11T18:03:56.330 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.331 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.331 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.331 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "InfluxDB server port", 2023-12-11T18:03:56.331 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.331 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.331 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.331 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.331 INFO:teuthology.orchestra.run.smithi059.stdout: "ssl": { 2023-12-11T18:03:56.331 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ssl", 2023-12-11T18:03:56.331 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.332 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.332 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.332 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "false", 2023-12-11T18:03:56.332 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.332 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.332 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.332 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-12-11T18:03:56.333 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.333 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.333 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.333 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.333 INFO:teuthology.orchestra.run.smithi059.stdout: "threads": { 2023-12-11T18:03:56.333 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "threads", 2023-12-11T18:03:56.333 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2023-12-11T18:03:56.334 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.334 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.334 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "5", 2023-12-11T18:03:56.334 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "1", 2023-12-11T18:03:56.334 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "32", 2023-12-11T18:03:56.334 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.335 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-12-11T18:03:56.335 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.335 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.335 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.335 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.335 INFO:teuthology.orchestra.run.smithi059.stdout: "username": { 2023-12-11T18:03:56.335 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "username", 2023-12-11T18:03:56.335 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.335 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.336 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.336 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:56.336 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.336 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.336 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.336 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "username of InfluxDB server user", 2023-12-11T18:03:56.336 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.336 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.337 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.337 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.337 INFO:teuthology.orchestra.run.smithi059.stdout: "verify_ssl": { 2023-12-11T18:03:56.337 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "verify_ssl", 2023-12-11T18:03:56.337 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.337 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.337 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.337 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "true", 2023-12-11T18:03:56.337 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.338 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.338 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.338 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-12-11T18:03:56.338 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.338 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.338 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.338 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:56.338 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:56.339 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.339 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-12-11T18:03:56.339 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "insights", 2023-12-11T18:03:56.339 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2023-12-11T18:03:56.339 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2023-12-11T18:03:56.339 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2023-12-11T18:03:56.339 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2023-12-11T18:03:56.339 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2023-12-11T18:03:56.339 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.340 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.340 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.340 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:56.340 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.340 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.340 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:56.340 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:56.340 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:56.341 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:56.341 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:56.341 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:56.341 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:56.341 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:56.341 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.341 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.341 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.342 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.342 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.342 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2023-12-11T18:03:56.342 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2023-12-11T18:03:56.342 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.342 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.342 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.342 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:56.343 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.343 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.343 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.343 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.343 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.343 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.343 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.343 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.343 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2023-12-11T18:03:56.343 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2023-12-11T18:03:56.343 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.343 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.343 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.344 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2023-12-11T18:03:56.344 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.344 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.344 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:56.344 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:56.344 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:56.344 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:56.344 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:56.345 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:56.345 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:56.345 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:56.345 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.345 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.345 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.345 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.345 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.345 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2023-12-11T18:03:56.345 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2023-12-11T18:03:56.345 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.345 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.346 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.346 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:56.346 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.346 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.346 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.346 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.346 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.346 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.346 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.346 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:56.346 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:56.346 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.347 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-12-11T18:03:56.347 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "iostat", 2023-12-11T18:03:56.347 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2023-12-11T18:03:56.347 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2023-12-11T18:03:56.347 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2023-12-11T18:03:56.347 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2023-12-11T18:03:56.347 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2023-12-11T18:03:56.347 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.347 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.347 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.347 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:56.347 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.347 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.347 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:56.347 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:56.348 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:56.348 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:56.348 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:56.348 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:56.348 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:56.348 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:56.348 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.348 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.348 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.348 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.348 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.348 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2023-12-11T18:03:56.349 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2023-12-11T18:03:56.349 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.349 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.349 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.349 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:56.349 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.349 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.349 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.349 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.349 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.349 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.349 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.349 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.349 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2023-12-11T18:03:56.350 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2023-12-11T18:03:56.350 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.350 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.350 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.350 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2023-12-11T18:03:56.350 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.350 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.350 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:56.350 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:56.350 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:56.350 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:56.350 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:56.350 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:56.350 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:56.351 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:56.351 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.351 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.351 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.351 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.351 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.351 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2023-12-11T18:03:56.351 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2023-12-11T18:03:56.351 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.351 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.351 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.351 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:56.351 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.351 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.352 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.352 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.352 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.352 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.352 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.352 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:56.352 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:56.352 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.352 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-12-11T18:03:56.352 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "k8sevents", 2023-12-11T18:03:56.352 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2023-12-11T18:03:56.352 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2023-12-11T18:03:56.352 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2023-12-11T18:03:56.353 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_event_retention_days": { 2023-12-11T18:03:56.353 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ceph_event_retention_days", 2023-12-11T18:03:56.353 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2023-12-11T18:03:56.353 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.353 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.353 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "7", 2023-12-11T18:03:56.353 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.353 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.353 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.353 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Days to hold ceph event information within local cache", 2023-12-11T18:03:56.353 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.353 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.353 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.353 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.353 INFO:teuthology.orchestra.run.smithi059.stdout: "config_check_secs": { 2023-12-11T18:03:56.354 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "config_check_secs", 2023-12-11T18:03:56.354 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2023-12-11T18:03:56.354 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.354 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.354 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "10", 2023-12-11T18:03:56.354 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "10", 2023-12-11T18:03:56.354 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.354 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.354 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-12-11T18:03:56.354 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.354 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.354 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.355 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.355 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2023-12-11T18:03:56.355 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2023-12-11T18:03:56.355 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.355 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.355 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.355 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:56.355 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.355 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.355 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:56.355 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:56.355 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:56.355 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:56.355 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:56.355 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:56.356 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:56.356 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:56.356 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.356 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.356 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.356 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.356 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.356 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2023-12-11T18:03:56.356 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2023-12-11T18:03:56.356 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.356 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.356 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.356 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:56.356 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.357 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.357 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.357 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.357 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.357 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.357 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.357 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.357 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2023-12-11T18:03:56.357 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2023-12-11T18:03:56.357 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.357 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.357 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.357 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2023-12-11T18:03:56.357 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.358 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.358 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:56.358 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:56.358 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:56.358 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:56.358 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:56.358 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:56.358 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:56.358 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:56.358 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.358 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.358 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.359 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.359 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.359 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2023-12-11T18:03:56.359 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2023-12-11T18:03:56.359 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.359 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.359 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.359 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:56.359 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.359 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.359 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.359 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.359 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.359 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.360 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.360 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:56.360 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:56.360 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.360 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-12-11T18:03:56.360 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "localpool", 2023-12-11T18:03:56.360 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2023-12-11T18:03:56.360 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2023-12-11T18:03:56.360 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2023-12-11T18:03:56.360 INFO:teuthology.orchestra.run.smithi059.stdout: "failure_domain": { 2023-12-11T18:03:56.360 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "failure_domain", 2023-12-11T18:03:56.360 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.360 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.360 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.361 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "host", 2023-12-11T18:03:56.361 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.361 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.361 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.361 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "failure domain for any created local pool", 2023-12-11T18:03:56.361 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-12-11T18:03:56.361 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.361 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.361 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.361 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2023-12-11T18:03:56.361 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2023-12-11T18:03:56.361 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.361 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.361 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.362 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:56.362 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.362 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.362 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:56.362 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:56.362 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:56.362 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:56.362 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:56.362 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:56.362 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:56.362 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:56.362 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.362 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.363 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.363 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.363 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.363 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2023-12-11T18:03:56.363 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2023-12-11T18:03:56.363 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.363 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.363 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.363 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:56.363 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.363 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.363 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.363 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.363 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.364 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.364 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.364 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.364 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2023-12-11T18:03:56.364 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2023-12-11T18:03:56.364 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.364 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.364 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.364 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2023-12-11T18:03:56.364 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.364 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.364 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:56.364 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:56.364 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:56.364 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:56.365 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:56.365 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:56.365 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:56.365 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:56.365 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.365 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.365 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.365 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.365 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.365 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2023-12-11T18:03:56.365 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2023-12-11T18:03:56.365 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.365 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.365 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.365 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:56.366 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.366 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.366 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.366 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.366 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.366 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.366 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.366 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.366 INFO:teuthology.orchestra.run.smithi059.stdout: "min_size": { 2023-12-11T18:03:56.366 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "min_size", 2023-12-11T18:03:56.366 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2023-12-11T18:03:56.366 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.366 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.367 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:56.367 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.367 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.367 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.367 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "default min_size for any created local pool", 2023-12-11T18:03:56.367 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-12-11T18:03:56.367 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.367 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.367 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.367 INFO:teuthology.orchestra.run.smithi059.stdout: "num_rep": { 2023-12-11T18:03:56.367 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "num_rep", 2023-12-11T18:03:56.367 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2023-12-11T18:03:56.368 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.368 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.368 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "3", 2023-12-11T18:03:56.368 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.368 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.368 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.368 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "default replica count for any created local pool", 2023-12-11T18:03:56.368 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.368 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.368 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.368 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.368 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num": { 2023-12-11T18:03:56.368 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "pg_num", 2023-12-11T18:03:56.368 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2023-12-11T18:03:56.368 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.369 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.369 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "128", 2023-12-11T18:03:56.369 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.369 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.369 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.369 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "default pg_num for any created local pool", 2023-12-11T18:03:56.369 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.369 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.369 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.369 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.369 INFO:teuthology.orchestra.run.smithi059.stdout: "prefix": { 2023-12-11T18:03:56.369 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "prefix", 2023-12-11T18:03:56.369 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.369 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.370 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.370 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:56.370 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.370 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.370 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.370 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "name prefix for any created local pool", 2023-12-11T18:03:56.370 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.370 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.370 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.370 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.370 INFO:teuthology.orchestra.run.smithi059.stdout: "subtree": { 2023-12-11T18:03:56.370 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "subtree", 2023-12-11T18:03:56.370 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.370 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.371 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.371 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "rack", 2023-12-11T18:03:56.371 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.371 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.371 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.371 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "CRUSH level for which to create a local pool", 2023-12-11T18:03:56.371 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-12-11T18:03:56.371 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.371 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.371 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:56.371 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:56.371 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.371 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-12-11T18:03:56.371 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "mds_autoscaler", 2023-12-11T18:03:56.371 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2023-12-11T18:03:56.372 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2023-12-11T18:03:56.372 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2023-12-11T18:03:56.372 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2023-12-11T18:03:56.372 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2023-12-11T18:03:56.372 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.372 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.372 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.372 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:56.372 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.372 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.372 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:56.372 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:56.373 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:56.373 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:56.373 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:56.373 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:56.373 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:56.373 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:56.373 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.373 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.374 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.374 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.374 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.374 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2023-12-11T18:03:56.374 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2023-12-11T18:03:56.374 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.375 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.375 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.375 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:56.375 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.375 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.375 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.376 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.376 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.376 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.376 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.376 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.376 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2023-12-11T18:03:56.376 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2023-12-11T18:03:56.376 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.377 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.377 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.377 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2023-12-11T18:03:56.377 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.377 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.377 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:56.377 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:56.377 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:56.378 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:56.378 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:56.378 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:56.378 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:56.378 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:56.378 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.378 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.379 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.379 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.379 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.379 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2023-12-11T18:03:56.379 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2023-12-11T18:03:56.379 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.379 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.379 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.379 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:56.379 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.379 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.379 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.379 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.379 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.380 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.380 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.380 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:56.380 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:56.380 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.380 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-12-11T18:03:56.380 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "mirroring", 2023-12-11T18:03:56.380 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2023-12-11T18:03:56.380 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2023-12-11T18:03:56.380 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2023-12-11T18:03:56.380 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2023-12-11T18:03:56.380 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2023-12-11T18:03:56.380 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.380 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.380 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.381 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:56.381 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.381 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.381 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:56.381 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:56.381 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:56.381 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:56.381 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:56.381 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:56.381 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:56.381 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:56.381 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.381 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.381 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.382 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.382 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.382 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2023-12-11T18:03:56.382 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2023-12-11T18:03:56.382 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.382 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.382 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.382 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:56.382 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.382 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.382 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.382 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.382 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.382 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.383 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.383 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.383 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2023-12-11T18:03:56.383 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2023-12-11T18:03:56.383 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.383 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.383 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.383 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2023-12-11T18:03:56.383 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.383 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.383 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:56.383 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:56.383 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:56.383 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:56.383 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:56.383 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:56.384 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:56.384 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:56.384 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.384 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.384 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.384 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.384 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.384 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2023-12-11T18:03:56.384 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2023-12-11T18:03:56.384 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.384 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.384 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.384 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:56.385 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.385 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.385 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.385 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.385 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.385 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.385 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.385 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:56.385 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:56.385 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.385 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-12-11T18:03:56.385 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "nfs", 2023-12-11T18:03:56.385 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2023-12-11T18:03:56.385 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2023-12-11T18:03:56.385 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2023-12-11T18:03:56.386 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2023-12-11T18:03:56.386 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2023-12-11T18:03:56.386 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.386 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.386 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.386 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:56.386 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.386 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.386 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:56.386 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:56.386 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:56.386 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:56.386 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:56.387 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:56.387 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:56.387 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:56.387 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.387 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.387 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.387 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.387 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.387 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2023-12-11T18:03:56.387 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2023-12-11T18:03:56.387 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.387 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.388 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.388 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:56.388 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.388 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.388 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.388 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.388 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.388 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.388 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.388 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.388 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2023-12-11T18:03:56.388 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2023-12-11T18:03:56.388 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.388 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.388 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.389 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2023-12-11T18:03:56.389 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.389 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.389 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:56.389 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:56.389 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:56.389 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:56.389 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:56.389 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:56.389 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:56.389 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:56.389 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.389 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.390 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.390 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.390 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.390 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2023-12-11T18:03:56.390 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2023-12-11T18:03:56.390 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.390 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.390 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.390 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:56.390 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.390 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.390 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.390 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.390 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.391 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.391 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.391 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:56.391 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:56.391 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.391 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-12-11T18:03:56.391 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "orchestrator", 2023-12-11T18:03:56.391 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2023-12-11T18:03:56.391 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2023-12-11T18:03:56.391 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2023-12-11T18:03:56.391 INFO:teuthology.orchestra.run.smithi059.stdout: "fail_fs": { 2023-12-11T18:03:56.391 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "fail_fs", 2023-12-11T18:03:56.392 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.392 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.392 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.392 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:56.392 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.392 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.392 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.392 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-12-11T18:03:56.392 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.392 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.392 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.392 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.392 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2023-12-11T18:03:56.392 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2023-12-11T18:03:56.393 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.393 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.393 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.393 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:56.393 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.393 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.393 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:56.393 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:56.393 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:56.393 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:56.393 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:56.393 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:56.393 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:56.393 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:56.394 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.394 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.394 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.394 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.394 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.394 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2023-12-11T18:03:56.394 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2023-12-11T18:03:56.394 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.394 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.394 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.394 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:56.394 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.394 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.395 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.395 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.395 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.395 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.395 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.395 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.395 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2023-12-11T18:03:56.395 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2023-12-11T18:03:56.395 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.395 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.395 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.395 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2023-12-11T18:03:56.395 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.395 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.396 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:56.396 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:56.396 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:56.396 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:56.396 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:56.396 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:56.396 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:56.396 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:56.396 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.396 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.396 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.396 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.396 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.396 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2023-12-11T18:03:56.396 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2023-12-11T18:03:56.397 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.397 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.397 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.397 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:56.397 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.397 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.397 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.397 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.397 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.397 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.397 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.397 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.397 INFO:teuthology.orchestra.run.smithi059.stdout: "orchestrator": { 2023-12-11T18:03:56.397 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "orchestrator", 2023-12-11T18:03:56.397 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.398 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.398 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.398 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:56.398 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.398 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.398 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:56.398 INFO:teuthology.orchestra.run.smithi059.stdout: "cephadm", 2023-12-11T18:03:56.398 INFO:teuthology.orchestra.run.smithi059.stdout: "rook", 2023-12-11T18:03:56.398 INFO:teuthology.orchestra.run.smithi059.stdout: "test_orchestrator" 2023-12-11T18:03:56.398 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:56.398 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Orchestrator backend", 2023-12-11T18:03:56.398 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.398 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.399 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.399 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:56.399 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:56.399 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.399 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-12-11T18:03:56.399 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "osd_perf_query", 2023-12-11T18:03:56.399 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2023-12-11T18:03:56.399 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2023-12-11T18:03:56.399 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2023-12-11T18:03:56.399 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2023-12-11T18:03:56.399 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2023-12-11T18:03:56.399 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.399 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.399 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.399 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:56.400 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.400 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.400 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:56.400 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:56.400 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:56.400 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:56.400 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:56.400 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:56.400 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:56.400 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:56.400 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.400 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.400 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.400 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.400 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.401 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2023-12-11T18:03:56.401 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2023-12-11T18:03:56.401 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.401 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.401 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.401 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:56.401 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.401 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.401 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.401 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.401 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.401 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.401 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.401 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.402 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2023-12-11T18:03:56.402 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2023-12-11T18:03:56.402 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.402 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.402 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.402 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2023-12-11T18:03:56.402 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.402 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.402 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:56.402 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:56.402 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:56.402 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:56.402 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:56.402 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:56.403 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:56.403 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:56.403 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.403 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.403 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.403 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.403 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.403 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2023-12-11T18:03:56.403 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2023-12-11T18:03:56.403 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.403 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.403 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.403 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:56.403 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.403 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.404 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.404 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.404 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.404 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.404 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.404 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:56.404 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:56.404 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.404 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-12-11T18:03:56.404 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "osd_support", 2023-12-11T18:03:56.404 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2023-12-11T18:03:56.404 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2023-12-11T18:03:56.404 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2023-12-11T18:03:56.404 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2023-12-11T18:03:56.404 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2023-12-11T18:03:56.405 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.405 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.405 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.405 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:56.405 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.405 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.405 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:56.405 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:56.405 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:56.405 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:56.405 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:56.405 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:56.405 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:56.405 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:56.405 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.405 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.406 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.406 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.406 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.406 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2023-12-11T18:03:56.406 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2023-12-11T18:03:56.406 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.406 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.406 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.406 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:56.406 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.406 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.407 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.407 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.407 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.407 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.407 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.407 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.407 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2023-12-11T18:03:56.407 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2023-12-11T18:03:56.407 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.407 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.407 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.407 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2023-12-11T18:03:56.407 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.408 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.408 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:56.408 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:56.408 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:56.408 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:56.408 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:56.408 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:56.408 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:56.408 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:56.408 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.408 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.408 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.409 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.409 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.409 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2023-12-11T18:03:56.409 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2023-12-11T18:03:56.409 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.409 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.409 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.409 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:56.409 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.409 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.409 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.409 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.410 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.410 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.410 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.410 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:56.410 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:56.410 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.410 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-12-11T18:03:56.410 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "pg_autoscaler", 2023-12-11T18:03:56.410 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2023-12-11T18:03:56.410 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2023-12-11T18:03:56.410 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2023-12-11T18:03:56.410 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2023-12-11T18:03:56.410 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2023-12-11T18:03:56.410 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.411 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.411 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.411 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:56.411 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.411 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.411 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:56.411 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:56.411 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:56.411 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:56.411 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:56.411 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:56.411 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:56.411 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:56.411 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.411 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.412 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.412 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.412 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.412 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2023-12-11T18:03:56.412 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2023-12-11T18:03:56.412 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.412 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.412 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.412 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:56.412 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.413 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.413 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.413 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.413 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.413 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.413 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.413 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.413 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2023-12-11T18:03:56.413 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2023-12-11T18:03:56.413 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.413 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.413 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.414 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2023-12-11T18:03:56.414 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.414 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.414 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:56.414 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:56.414 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:56.414 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:56.414 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:56.414 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:56.414 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:56.414 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:56.414 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.414 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.414 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.415 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.415 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.415 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2023-12-11T18:03:56.415 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2023-12-11T18:03:56.415 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.415 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.415 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.415 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:56.415 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.415 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.415 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.415 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.415 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.415 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.415 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.416 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.416 INFO:teuthology.orchestra.run.smithi059.stdout: "sleep_interval": { 2023-12-11T18:03:56.416 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sleep_interval", 2023-12-11T18:03:56.416 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2023-12-11T18:03:56.416 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.416 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.416 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "60", 2023-12-11T18:03:56.416 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.416 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.416 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.416 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.416 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.416 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.416 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.417 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.417 INFO:teuthology.orchestra.run.smithi059.stdout: "threshold": { 2023-12-11T18:03:56.417 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "threshold", 2023-12-11T18:03:56.417 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "float", 2023-12-11T18:03:56.417 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.417 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.417 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "3.0", 2023-12-11T18:03:56.417 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "1.0", 2023-12-11T18:03:56.417 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.417 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.417 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "scaling threshold", 2023-12-11T18:03:56.417 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-12-11T18:03:56.417 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.417 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.418 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:56.418 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:56.418 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.418 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-12-11T18:03:56.418 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "progress", 2023-12-11T18:03:56.418 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2023-12-11T18:03:56.418 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2023-12-11T18:03:56.418 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2023-12-11T18:03:56.418 INFO:teuthology.orchestra.run.smithi059.stdout: "allow_pg_recovery_event": { 2023-12-11T18:03:56.418 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "allow_pg_recovery_event", 2023-12-11T18:03:56.418 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.418 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.418 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.419 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:56.419 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.419 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.419 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.419 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "allow the module to show pg recovery progress", 2023-12-11T18:03:56.419 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.419 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.419 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.419 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.419 INFO:teuthology.orchestra.run.smithi059.stdout: "enabled": { 2023-12-11T18:03:56.419 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "enabled", 2023-12-11T18:03:56.419 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.419 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.419 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.419 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2023-12-11T18:03:56.420 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.420 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.420 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.420 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.420 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.420 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.420 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.420 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.420 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2023-12-11T18:03:56.420 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2023-12-11T18:03:56.420 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.421 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.421 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.421 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:56.421 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.421 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.421 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:56.421 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:56.421 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:56.421 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:56.421 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:56.421 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:56.421 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:56.421 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:56.421 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.422 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.422 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.422 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.422 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.422 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2023-12-11T18:03:56.422 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2023-12-11T18:03:56.422 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.422 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.422 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.422 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:56.422 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.422 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.422 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.423 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.423 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.423 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.423 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.423 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.423 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2023-12-11T18:03:56.423 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2023-12-11T18:03:56.423 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.423 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.423 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.423 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2023-12-11T18:03:56.423 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.423 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.423 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:56.423 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:56.424 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:56.424 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:56.424 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:56.424 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:56.424 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:56.424 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:56.424 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.424 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.424 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.424 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.424 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.424 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2023-12-11T18:03:56.424 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2023-12-11T18:03:56.424 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.425 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.425 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.425 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:56.425 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.425 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.425 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.425 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.425 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.425 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.425 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.425 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.425 INFO:teuthology.orchestra.run.smithi059.stdout: "max_completed_events": { 2023-12-11T18:03:56.425 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "max_completed_events", 2023-12-11T18:03:56.425 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2023-12-11T18:03:56.426 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.426 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.426 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "50", 2023-12-11T18:03:56.426 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.426 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.426 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.426 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "number of past completed events to remember", 2023-12-11T18:03:56.426 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.426 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.426 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.426 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.426 INFO:teuthology.orchestra.run.smithi059.stdout: "sleep_interval": { 2023-12-11T18:03:56.426 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sleep_interval", 2023-12-11T18:03:56.427 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2023-12-11T18:03:56.427 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.427 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.427 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "5", 2023-12-11T18:03:56.427 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.427 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.427 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.427 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "how long the module is going to sleep", 2023-12-11T18:03:56.427 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.427 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.427 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.427 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:56.427 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:56.427 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.427 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-12-11T18:03:56.428 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "prometheus", 2023-12-11T18:03:56.428 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2023-12-11T18:03:56.428 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2023-12-11T18:03:56.428 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2023-12-11T18:03:56.428 INFO:teuthology.orchestra.run.smithi059.stdout: "cache": { 2023-12-11T18:03:56.428 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "cache", 2023-12-11T18:03:56.428 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.428 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.428 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.428 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2023-12-11T18:03:56.428 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.428 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.428 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.428 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.428 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.429 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.429 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.429 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.429 INFO:teuthology.orchestra.run.smithi059.stdout: "exclude_perf_counters": { 2023-12-11T18:03:56.429 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "exclude_perf_counters", 2023-12-11T18:03:56.429 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.429 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.429 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.429 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2023-12-11T18:03:56.429 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.429 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.429 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.429 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-12-11T18:03:56.429 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-12-11T18:03:56.430 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.430 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.430 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.430 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2023-12-11T18:03:56.430 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2023-12-11T18:03:56.430 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.430 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.430 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.430 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:56.430 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.430 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.430 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:56.430 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:56.431 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:56.431 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:56.431 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:56.431 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:56.431 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:56.431 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:56.431 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.431 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.431 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.431 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.431 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.431 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2023-12-11T18:03:56.431 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2023-12-11T18:03:56.431 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.431 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.431 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.432 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:56.432 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.432 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.432 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.432 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.432 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.432 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.432 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.432 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.432 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2023-12-11T18:03:56.432 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2023-12-11T18:03:56.432 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.432 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.432 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.433 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2023-12-11T18:03:56.433 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.433 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.433 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:56.433 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:56.433 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:56.433 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:56.433 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:56.433 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:56.433 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:56.433 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:56.433 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.433 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.433 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.433 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.434 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.434 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2023-12-11T18:03:56.434 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2023-12-11T18:03:56.434 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.434 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.434 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.434 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:56.434 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.434 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.434 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.434 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.434 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.434 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.434 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.435 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.435 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_stats_pools": { 2023-12-11T18:03:56.435 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rbd_stats_pools", 2023-12-11T18:03:56.435 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.435 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.435 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.435 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:56.435 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.435 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.435 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.435 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.435 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.435 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.435 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.435 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.436 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_stats_pools_refresh_interval": { 2023-12-11T18:03:56.436 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-12-11T18:03:56.436 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2023-12-11T18:03:56.436 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.436 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.436 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "300", 2023-12-11T18:03:56.436 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.436 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.436 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.436 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.436 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.436 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.436 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.436 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.436 INFO:teuthology.orchestra.run.smithi059.stdout: "scrape_interval": { 2023-12-11T18:03:56.437 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "scrape_interval", 2023-12-11T18:03:56.437 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "float", 2023-12-11T18:03:56.437 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.437 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.437 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "15.0", 2023-12-11T18:03:56.437 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.437 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.437 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.437 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.437 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.437 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.437 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.437 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.437 INFO:teuthology.orchestra.run.smithi059.stdout: "server_addr": { 2023-12-11T18:03:56.437 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "server_addr", 2023-12-11T18:03:56.438 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.438 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.438 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.438 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "::", 2023-12-11T18:03:56.438 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.438 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.438 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.438 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-12-11T18:03:56.438 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.438 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.438 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.439 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.439 INFO:teuthology.orchestra.run.smithi059.stdout: "server_port": { 2023-12-11T18:03:56.439 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "server_port", 2023-12-11T18:03:56.439 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2023-12-11T18:03:56.439 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.439 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.439 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "9283", 2023-12-11T18:03:56.439 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.439 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.439 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.439 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-12-11T18:03:56.439 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.439 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.439 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.439 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.440 INFO:teuthology.orchestra.run.smithi059.stdout: "stale_cache_strategy": { 2023-12-11T18:03:56.440 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "stale_cache_strategy", 2023-12-11T18:03:56.440 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.440 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.440 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.440 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "log", 2023-12-11T18:03:56.440 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.440 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.440 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.440 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.440 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.440 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.440 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.440 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.441 INFO:teuthology.orchestra.run.smithi059.stdout: "standby_behaviour": { 2023-12-11T18:03:56.441 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "standby_behaviour", 2023-12-11T18:03:56.441 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.441 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.441 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.441 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "default", 2023-12-11T18:03:56.441 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.441 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.441 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:56.441 INFO:teuthology.orchestra.run.smithi059.stdout: "default", 2023-12-11T18:03:56.441 INFO:teuthology.orchestra.run.smithi059.stdout: "error" 2023-12-11T18:03:56.441 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:56.441 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.441 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.441 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.442 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.442 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.442 INFO:teuthology.orchestra.run.smithi059.stdout: "standby_error_status_code": { 2023-12-11T18:03:56.442 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "standby_error_status_code", 2023-12-11T18:03:56.442 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2023-12-11T18:03:56.442 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.442 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.442 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "500", 2023-12-11T18:03:56.442 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "400", 2023-12-11T18:03:56.442 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "599", 2023-12-11T18:03:56.442 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.442 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.442 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.442 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.443 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.443 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:56.443 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:56.443 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.443 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-12-11T18:03:56.443 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rbd_support", 2023-12-11T18:03:56.443 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2023-12-11T18:03:56.443 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2023-12-11T18:03:56.443 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2023-12-11T18:03:56.443 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2023-12-11T18:03:56.443 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2023-12-11T18:03:56.443 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.443 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.443 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.443 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:56.443 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.444 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.444 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:56.444 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:56.444 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:56.444 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:56.444 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:56.444 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:56.444 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:56.444 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:56.444 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.444 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.444 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.444 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.444 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.444 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2023-12-11T18:03:56.445 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2023-12-11T18:03:56.445 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.445 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.445 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.445 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:56.445 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.445 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.445 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.445 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.445 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.445 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.445 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.445 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.445 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2023-12-11T18:03:56.445 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2023-12-11T18:03:56.446 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.446 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.446 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.446 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2023-12-11T18:03:56.446 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.446 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.446 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:56.446 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:56.446 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:56.446 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:56.446 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:56.447 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:56.447 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:56.447 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:56.447 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.447 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.447 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.447 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.447 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.447 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2023-12-11T18:03:56.447 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2023-12-11T18:03:56.447 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.447 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.447 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.447 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:56.448 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.448 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.448 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.448 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.448 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.448 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.448 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.448 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.448 INFO:teuthology.orchestra.run.smithi059.stdout: "max_concurrent_snap_create": { 2023-12-11T18:03:56.448 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "max_concurrent_snap_create", 2023-12-11T18:03:56.448 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2023-12-11T18:03:56.448 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.448 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.448 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "10", 2023-12-11T18:03:56.448 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.449 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.449 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.449 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.449 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.449 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.449 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.449 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.449 INFO:teuthology.orchestra.run.smithi059.stdout: "mirror_snapshot_schedule": { 2023-12-11T18:03:56.449 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "mirror_snapshot_schedule", 2023-12-11T18:03:56.449 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.449 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.450 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.450 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:56.450 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.450 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.450 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.450 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.450 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.450 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.450 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.450 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.450 INFO:teuthology.orchestra.run.smithi059.stdout: "trash_purge_schedule": { 2023-12-11T18:03:56.450 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "trash_purge_schedule", 2023-12-11T18:03:56.450 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.450 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.451 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.451 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:56.451 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.451 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.451 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.451 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.451 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.451 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.451 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.451 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:56.451 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:56.451 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.451 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-12-11T18:03:56.451 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "restful", 2023-12-11T18:03:56.452 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2023-12-11T18:03:56.452 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2023-12-11T18:03:56.452 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2023-12-11T18:03:56.452 INFO:teuthology.orchestra.run.smithi059.stdout: "enable_auth": { 2023-12-11T18:03:56.452 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "enable_auth", 2023-12-11T18:03:56.452 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.452 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.452 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.452 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2023-12-11T18:03:56.452 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.452 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.452 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.452 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.452 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.452 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.453 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.453 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.453 INFO:teuthology.orchestra.run.smithi059.stdout: "key_file": { 2023-12-11T18:03:56.453 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "key_file", 2023-12-11T18:03:56.453 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.453 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.453 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.453 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:56.453 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.453 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.453 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.453 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.453 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.453 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.453 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.454 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.454 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2023-12-11T18:03:56.454 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2023-12-11T18:03:56.454 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.454 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.454 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.454 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:56.454 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.454 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.454 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:56.454 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:56.454 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:56.454 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:56.454 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:56.455 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:56.455 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:56.455 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:56.455 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.455 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.455 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.455 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.455 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.455 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2023-12-11T18:03:56.455 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2023-12-11T18:03:56.455 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.455 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.455 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.455 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:56.455 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.456 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.456 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.456 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.456 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.456 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.456 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.456 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.456 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2023-12-11T18:03:56.456 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2023-12-11T18:03:56.456 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.456 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.456 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.456 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2023-12-11T18:03:56.456 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.457 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.457 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:56.457 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:56.457 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:56.457 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:56.457 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:56.457 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:56.457 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:56.457 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:56.457 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.457 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.457 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.457 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.457 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.457 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2023-12-11T18:03:56.457 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2023-12-11T18:03:56.458 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.458 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.458 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.458 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:56.458 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.458 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.458 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.458 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.458 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.458 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.458 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.458 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.458 INFO:teuthology.orchestra.run.smithi059.stdout: "server_addr": { 2023-12-11T18:03:56.458 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "server_addr", 2023-12-11T18:03:56.459 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.459 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.459 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.459 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:56.459 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.459 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.459 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.460 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.460 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.460 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.460 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.460 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.460 INFO:teuthology.orchestra.run.smithi059.stdout: "server_port": { 2023-12-11T18:03:56.460 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "server_port", 2023-12-11T18:03:56.460 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.461 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.461 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.461 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:56.461 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.461 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.461 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.461 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.461 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.461 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.461 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.461 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:56.461 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:56.461 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.461 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-12-11T18:03:56.462 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rgw", 2023-12-11T18:03:56.462 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2023-12-11T18:03:56.462 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2023-12-11T18:03:56.462 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2023-12-11T18:03:56.462 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2023-12-11T18:03:56.462 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2023-12-11T18:03:56.462 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.462 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.462 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.462 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:56.462 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.462 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.463 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:56.463 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:56.463 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:56.463 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:56.463 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:56.463 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:56.463 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:56.463 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:56.463 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.463 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.463 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.463 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.463 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.464 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2023-12-11T18:03:56.464 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2023-12-11T18:03:56.464 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.464 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.464 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.464 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:56.464 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.464 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.464 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.464 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.464 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.464 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.464 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.464 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.465 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2023-12-11T18:03:56.465 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2023-12-11T18:03:56.465 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.465 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.465 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.465 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2023-12-11T18:03:56.465 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.465 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.465 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:56.465 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:56.465 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:56.465 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:56.465 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:56.465 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:56.465 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:56.465 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:56.466 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.466 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.466 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.466 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.466 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.466 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2023-12-11T18:03:56.466 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2023-12-11T18:03:56.466 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.466 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.466 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.466 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:56.466 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.466 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.467 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.467 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.467 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.467 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.467 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.467 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:56.467 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:56.467 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.467 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-12-11T18:03:56.467 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rook", 2023-12-11T18:03:56.467 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2023-12-11T18:03:56.467 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2023-12-11T18:03:56.467 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2023-12-11T18:03:56.467 INFO:teuthology.orchestra.run.smithi059.stdout: "drive_group_interval": { 2023-12-11T18:03:56.467 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "drive_group_interval", 2023-12-11T18:03:56.468 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "float", 2023-12-11T18:03:56.468 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.468 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.468 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "300.0", 2023-12-11T18:03:56.468 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.468 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.468 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.468 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-12-11T18:03:56.468 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.468 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.469 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.469 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.469 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2023-12-11T18:03:56.469 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2023-12-11T18:03:56.469 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.469 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.469 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.469 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:56.469 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.470 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.470 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:56.470 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:56.470 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:56.470 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:56.470 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:56.470 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:56.470 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:56.470 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:56.470 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.470 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.470 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.470 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.470 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.471 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2023-12-11T18:03:56.471 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2023-12-11T18:03:56.471 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.471 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.471 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.471 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:56.471 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.471 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.471 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.471 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.471 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.471 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.471 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.471 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.472 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2023-12-11T18:03:56.472 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2023-12-11T18:03:56.472 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.472 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.472 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.472 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2023-12-11T18:03:56.472 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.472 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.472 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:56.472 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:56.472 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:56.472 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:56.472 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:56.472 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:56.473 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:56.473 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:56.473 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.473 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.473 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.473 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.473 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.473 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2023-12-11T18:03:56.473 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2023-12-11T18:03:56.473 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.473 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.473 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.473 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:56.473 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.473 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.474 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.474 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.474 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.474 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.474 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.474 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.474 INFO:teuthology.orchestra.run.smithi059.stdout: "storage_class": { 2023-12-11T18:03:56.474 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "storage_class", 2023-12-11T18:03:56.474 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.474 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.474 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.474 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "local", 2023-12-11T18:03:56.474 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.474 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.475 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.475 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-12-11T18:03:56.475 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.475 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.475 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.475 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:56.475 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:56.475 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.475 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-12-11T18:03:56.475 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "selftest", 2023-12-11T18:03:56.475 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2023-12-11T18:03:56.475 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2023-12-11T18:03:56.475 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2023-12-11T18:03:56.475 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2023-12-11T18:03:56.476 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2023-12-11T18:03:56.476 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.476 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.476 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.476 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:56.476 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.476 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.476 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:56.476 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:56.477 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:56.477 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:56.477 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:56.477 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:56.477 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:56.477 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:56.477 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.477 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.478 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.478 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.478 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.478 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2023-12-11T18:03:56.478 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2023-12-11T18:03:56.478 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.478 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.478 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.478 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:56.478 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.478 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.478 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.478 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.478 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.479 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.479 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.479 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.479 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2023-12-11T18:03:56.479 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2023-12-11T18:03:56.479 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.479 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.479 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.479 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2023-12-11T18:03:56.479 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.479 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.479 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:56.479 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:56.479 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:56.479 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:56.480 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:56.480 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:56.480 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:56.480 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:56.480 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.480 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.480 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.480 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.480 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.480 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2023-12-11T18:03:56.480 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2023-12-11T18:03:56.480 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.480 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.480 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.481 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:56.481 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.481 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.481 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.481 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.481 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.481 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.481 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.481 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.481 INFO:teuthology.orchestra.run.smithi059.stdout: "roption1": { 2023-12-11T18:03:56.481 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "roption1", 2023-12-11T18:03:56.481 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.481 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.481 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.481 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:56.482 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.482 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.482 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.482 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.482 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.482 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.482 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.482 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.482 INFO:teuthology.orchestra.run.smithi059.stdout: "roption2": { 2023-12-11T18:03:56.482 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "roption2", 2023-12-11T18:03:56.482 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.482 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.482 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.482 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "xyz", 2023-12-11T18:03:56.483 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.483 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.483 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.483 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.483 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.483 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.483 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.483 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.483 INFO:teuthology.orchestra.run.smithi059.stdout: "rwoption1": { 2023-12-11T18:03:56.483 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rwoption1", 2023-12-11T18:03:56.483 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.483 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.483 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.483 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:56.483 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.484 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.484 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.484 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.484 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.484 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.484 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.484 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.484 INFO:teuthology.orchestra.run.smithi059.stdout: "rwoption2": { 2023-12-11T18:03:56.484 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rwoption2", 2023-12-11T18:03:56.484 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2023-12-11T18:03:56.484 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.484 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.484 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:56.485 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.485 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.485 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.485 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.485 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.485 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.485 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.485 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.485 INFO:teuthology.orchestra.run.smithi059.stdout: "rwoption3": { 2023-12-11T18:03:56.486 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rwoption3", 2023-12-11T18:03:56.486 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "float", 2023-12-11T18:03:56.486 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.486 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.486 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:56.486 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.486 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.486 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.486 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.486 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.486 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.486 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.486 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.486 INFO:teuthology.orchestra.run.smithi059.stdout: "rwoption4": { 2023-12-11T18:03:56.487 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rwoption4", 2023-12-11T18:03:56.487 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.487 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.487 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.487 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:56.487 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.487 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.487 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.487 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.487 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.487 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.487 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.487 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.488 INFO:teuthology.orchestra.run.smithi059.stdout: "rwoption5": { 2023-12-11T18:03:56.488 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rwoption5", 2023-12-11T18:03:56.488 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.488 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.488 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.488 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:56.488 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.488 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.488 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.488 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.488 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.488 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.488 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.489 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.489 INFO:teuthology.orchestra.run.smithi059.stdout: "rwoption6": { 2023-12-11T18:03:56.489 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rwoption6", 2023-12-11T18:03:56.489 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.489 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.489 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.489 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2023-12-11T18:03:56.489 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.489 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.489 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.489 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.489 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.489 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.489 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.489 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.490 INFO:teuthology.orchestra.run.smithi059.stdout: "rwoption7": { 2023-12-11T18:03:56.490 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rwoption7", 2023-12-11T18:03:56.490 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2023-12-11T18:03:56.490 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.490 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.490 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:56.490 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "1", 2023-12-11T18:03:56.490 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "42", 2023-12-11T18:03:56.490 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.490 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.490 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.490 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.490 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.490 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.490 INFO:teuthology.orchestra.run.smithi059.stdout: "testkey": { 2023-12-11T18:03:56.491 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "testkey", 2023-12-11T18:03:56.491 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.491 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.491 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.491 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:56.491 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.491 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.491 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.491 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.491 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.491 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.491 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.491 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.491 INFO:teuthology.orchestra.run.smithi059.stdout: "testlkey": { 2023-12-11T18:03:56.491 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "testlkey", 2023-12-11T18:03:56.492 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.492 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.492 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.492 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:56.492 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.492 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.492 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.492 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.492 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.492 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.492 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.492 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.492 INFO:teuthology.orchestra.run.smithi059.stdout: "testnewline": { 2023-12-11T18:03:56.492 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "testnewline", 2023-12-11T18:03:56.492 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.493 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.493 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.493 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:56.493 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.493 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.493 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.493 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.493 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.493 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.493 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.493 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:56.493 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:56.493 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.493 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-12-11T18:03:56.493 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "snap_schedule", 2023-12-11T18:03:56.494 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2023-12-11T18:03:56.494 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2023-12-11T18:03:56.494 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2023-12-11T18:03:56.494 INFO:teuthology.orchestra.run.smithi059.stdout: "allow_m_granularity": { 2023-12-11T18:03:56.494 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "allow_m_granularity", 2023-12-11T18:03:56.494 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.494 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.494 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.494 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:56.494 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.494 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.494 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.494 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "allow minute scheduled snapshots", 2023-12-11T18:03:56.494 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.495 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.495 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.495 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.495 INFO:teuthology.orchestra.run.smithi059.stdout: "dump_on_update": { 2023-12-11T18:03:56.495 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "dump_on_update", 2023-12-11T18:03:56.495 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.495 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.495 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.495 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:56.495 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.495 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.495 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.495 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "dump database to debug log on update", 2023-12-11T18:03:56.495 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.495 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.495 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.496 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.496 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2023-12-11T18:03:56.496 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2023-12-11T18:03:56.496 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.496 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.496 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.496 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:56.496 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.496 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.496 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:56.496 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:56.496 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:56.496 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:56.496 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:56.496 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:56.497 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:56.497 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:56.497 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.497 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.497 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.497 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.497 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.497 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2023-12-11T18:03:56.497 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2023-12-11T18:03:56.497 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.497 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.497 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.497 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:56.497 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.498 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.498 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.498 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.498 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.498 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.498 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.498 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.498 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2023-12-11T18:03:56.498 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2023-12-11T18:03:56.498 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.498 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.498 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.498 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2023-12-11T18:03:56.498 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.498 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.499 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:56.499 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:56.499 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:56.499 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:56.499 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:56.499 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:56.499 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:56.499 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:56.499 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.499 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.499 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.499 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.499 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.499 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2023-12-11T18:03:56.499 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2023-12-11T18:03:56.500 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.500 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.500 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.500 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:56.500 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.500 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.500 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.500 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.500 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.500 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.500 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.500 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:56.500 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:56.500 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.501 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-12-11T18:03:56.501 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "stats", 2023-12-11T18:03:56.501 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2023-12-11T18:03:56.501 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2023-12-11T18:03:56.501 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2023-12-11T18:03:56.501 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2023-12-11T18:03:56.501 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2023-12-11T18:03:56.501 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.501 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.501 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.501 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:56.501 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.501 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.501 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:56.502 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:56.502 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:56.502 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:56.502 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:56.502 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:56.502 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:56.502 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:56.502 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.502 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.502 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.502 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.502 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.502 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2023-12-11T18:03:56.502 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2023-12-11T18:03:56.503 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.503 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.503 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.503 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:56.503 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.503 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.503 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.503 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.503 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.503 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.503 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.503 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.503 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2023-12-11T18:03:56.503 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2023-12-11T18:03:56.504 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.504 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.504 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.504 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2023-12-11T18:03:56.504 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.504 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.504 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:56.504 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:56.504 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:56.504 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:56.504 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:56.504 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:56.504 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:56.504 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:56.504 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.505 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.505 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.505 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.505 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.505 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2023-12-11T18:03:56.505 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2023-12-11T18:03:56.505 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.505 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.505 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.505 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:56.505 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.505 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.505 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.505 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.505 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.506 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.506 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.506 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:56.506 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:56.506 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.506 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-12-11T18:03:56.506 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "status", 2023-12-11T18:03:56.506 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2023-12-11T18:03:56.506 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2023-12-11T18:03:56.506 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2023-12-11T18:03:56.506 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2023-12-11T18:03:56.506 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2023-12-11T18:03:56.506 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.506 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.507 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.507 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:56.507 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.507 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.507 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:56.507 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:56.507 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:56.507 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:56.507 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:56.507 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:56.507 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:56.507 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:56.507 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.507 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.507 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.508 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.508 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.508 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2023-12-11T18:03:56.508 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2023-12-11T18:03:56.508 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.508 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.508 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.508 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:56.508 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.508 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.508 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.508 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.508 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.508 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.508 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.509 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.509 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2023-12-11T18:03:56.509 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2023-12-11T18:03:56.509 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.509 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.509 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.509 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2023-12-11T18:03:56.509 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.509 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.509 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:56.509 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:56.509 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:56.509 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:56.509 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:56.509 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:56.510 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:56.510 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:56.510 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.510 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.510 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.510 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.510 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.510 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2023-12-11T18:03:56.510 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2023-12-11T18:03:56.510 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.513 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.513 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.514 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:56.514 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.514 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.514 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.514 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.514 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.514 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.514 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.514 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:56.514 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:56.514 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.514 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-12-11T18:03:56.514 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "telegraf", 2023-12-11T18:03:56.514 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2023-12-11T18:03:56.515 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2023-12-11T18:03:56.515 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2023-12-11T18:03:56.515 INFO:teuthology.orchestra.run.smithi059.stdout: "address": { 2023-12-11T18:03:56.515 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "address", 2023-12-11T18:03:56.515 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.515 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.515 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.515 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-12-11T18:03:56.515 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.515 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.515 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.515 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.515 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.515 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.515 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.516 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.516 INFO:teuthology.orchestra.run.smithi059.stdout: "interval": { 2023-12-11T18:03:56.516 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "interval", 2023-12-11T18:03:56.516 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2023-12-11T18:03:56.516 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.516 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.516 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "15", 2023-12-11T18:03:56.516 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.516 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.516 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.516 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.516 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.516 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.516 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.516 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.517 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2023-12-11T18:03:56.517 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2023-12-11T18:03:56.517 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.517 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.517 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.517 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:56.517 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.517 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.517 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:56.517 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:56.517 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:56.517 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:56.517 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:56.517 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:56.518 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:56.518 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:56.518 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.518 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.518 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.518 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.518 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.518 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2023-12-11T18:03:56.518 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2023-12-11T18:03:56.518 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.518 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.518 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.518 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:56.518 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.519 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.519 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.519 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.519 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.519 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.519 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.519 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.519 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2023-12-11T18:03:56.519 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2023-12-11T18:03:56.519 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.519 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.519 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.519 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2023-12-11T18:03:56.519 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.519 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.520 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:56.520 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:56.520 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:56.520 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:56.520 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:56.520 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:56.520 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:56.520 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:56.520 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.520 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.520 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.520 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.520 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.520 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2023-12-11T18:03:56.520 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2023-12-11T18:03:56.521 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.521 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.521 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.521 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:56.521 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.521 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.521 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.521 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.521 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.521 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.521 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.521 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:56.521 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:56.521 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.521 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-12-11T18:03:56.522 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "telemetry", 2023-12-11T18:03:56.522 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2023-12-11T18:03:56.522 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2023-12-11T18:03:56.522 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2023-12-11T18:03:56.522 INFO:teuthology.orchestra.run.smithi059.stdout: "channel_basic": { 2023-12-11T18:03:56.522 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "channel_basic", 2023-12-11T18:03:56.522 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.522 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.522 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.522 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2023-12-11T18:03:56.522 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.522 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.522 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.522 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Share basic cluster information (size, version)", 2023-12-11T18:03:56.523 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.523 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.523 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.523 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.523 INFO:teuthology.orchestra.run.smithi059.stdout: "channel_crash": { 2023-12-11T18:03:56.523 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "channel_crash", 2023-12-11T18:03:56.523 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.523 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.523 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.523 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2023-12-11T18:03:56.523 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.523 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.523 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.523 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-12-11T18:03:56.523 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.524 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.524 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.524 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.524 INFO:teuthology.orchestra.run.smithi059.stdout: "channel_device": { 2023-12-11T18:03:56.524 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "channel_device", 2023-12-11T18:03:56.524 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.524 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.524 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.524 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2023-12-11T18:03:56.524 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.524 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.524 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.524 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-12-11T18:03:56.524 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.524 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.525 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.525 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.525 INFO:teuthology.orchestra.run.smithi059.stdout: "channel_ident": { 2023-12-11T18:03:56.525 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "channel_ident", 2023-12-11T18:03:56.525 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.525 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.525 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.525 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:56.525 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.525 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.525 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.525 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-12-11T18:03:56.525 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.525 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.526 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.526 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.526 INFO:teuthology.orchestra.run.smithi059.stdout: "channel_perf": { 2023-12-11T18:03:56.526 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "channel_perf", 2023-12-11T18:03:56.526 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.526 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.526 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.526 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:56.526 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.526 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.526 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.526 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Share various performance metrics of a cluster", 2023-12-11T18:03:56.526 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.526 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.526 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.527 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.527 INFO:teuthology.orchestra.run.smithi059.stdout: "contact": { 2023-12-11T18:03:56.527 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "contact", 2023-12-11T18:03:56.527 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.527 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.527 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.527 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:56.527 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.527 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.527 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.527 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.527 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.527 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.527 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.527 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.528 INFO:teuthology.orchestra.run.smithi059.stdout: "description": { 2023-12-11T18:03:56.528 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "description", 2023-12-11T18:03:56.528 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.528 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.528 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.528 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:56.528 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.528 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.528 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.528 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.528 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.528 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.528 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.528 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.528 INFO:teuthology.orchestra.run.smithi059.stdout: "device_url": { 2023-12-11T18:03:56.529 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "device_url", 2023-12-11T18:03:56.529 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.529 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.529 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.529 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-12-11T18:03:56.529 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.529 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.529 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.529 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.529 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.529 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.529 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.529 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.529 INFO:teuthology.orchestra.run.smithi059.stdout: "enabled": { 2023-12-11T18:03:56.530 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "enabled", 2023-12-11T18:03:56.530 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.530 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.530 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.530 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:56.530 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.530 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.530 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.530 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.530 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.530 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.530 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.530 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.530 INFO:teuthology.orchestra.run.smithi059.stdout: "interval": { 2023-12-11T18:03:56.531 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "interval", 2023-12-11T18:03:56.531 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2023-12-11T18:03:56.531 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.531 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.531 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "24", 2023-12-11T18:03:56.531 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "8", 2023-12-11T18:03:56.531 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.531 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.531 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.531 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.531 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.531 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.531 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.531 INFO:teuthology.orchestra.run.smithi059.stdout: "last_opt_revision": { 2023-12-11T18:03:56.531 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "last_opt_revision", 2023-12-11T18:03:56.532 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2023-12-11T18:03:56.532 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.532 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.532 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "1", 2023-12-11T18:03:56.532 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.532 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.532 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.532 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.532 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.532 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.532 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.532 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.532 INFO:teuthology.orchestra.run.smithi059.stdout: "leaderboard": { 2023-12-11T18:03:56.532 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "leaderboard", 2023-12-11T18:03:56.533 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.533 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.533 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.533 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:56.533 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.533 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.533 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.533 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.533 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.533 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.533 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.533 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.533 INFO:teuthology.orchestra.run.smithi059.stdout: "leaderboard_description": { 2023-12-11T18:03:56.533 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "leaderboard_description", 2023-12-11T18:03:56.534 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.534 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.534 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.534 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:56.534 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.534 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.534 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.534 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.534 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.534 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.534 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.534 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.534 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2023-12-11T18:03:56.534 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2023-12-11T18:03:56.535 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.535 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.535 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.535 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:56.535 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.535 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.535 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:56.535 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:56.535 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:56.535 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:56.535 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:56.535 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:56.535 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:56.535 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:56.535 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.536 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.536 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.536 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.536 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.536 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2023-12-11T18:03:56.536 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2023-12-11T18:03:56.536 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.536 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.536 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.536 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:56.536 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.536 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.536 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.536 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.537 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.537 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.537 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.537 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.537 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2023-12-11T18:03:56.537 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2023-12-11T18:03:56.537 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.537 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.537 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.537 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2023-12-11T18:03:56.537 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.537 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.537 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:56.537 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:56.537 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:56.538 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:56.538 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:56.538 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:56.538 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:56.538 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:56.538 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.538 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.538 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.538 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.538 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.538 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2023-12-11T18:03:56.538 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2023-12-11T18:03:56.538 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.538 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.539 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.539 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:56.539 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.539 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.539 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.539 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.539 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.539 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.539 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.539 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.539 INFO:teuthology.orchestra.run.smithi059.stdout: "organization": { 2023-12-11T18:03:56.539 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "organization", 2023-12-11T18:03:56.539 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.539 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.539 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.540 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:56.540 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.540 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.540 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.540 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.540 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.540 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.540 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.540 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.540 INFO:teuthology.orchestra.run.smithi059.stdout: "proxy": { 2023-12-11T18:03:56.540 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "proxy", 2023-12-11T18:03:56.540 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.540 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.540 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.540 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:56.541 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.541 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.541 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.541 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.541 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.541 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.541 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.541 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.541 INFO:teuthology.orchestra.run.smithi059.stdout: "url": { 2023-12-11T18:03:56.541 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "url", 2023-12-11T18:03:56.541 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.541 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.541 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.541 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-12-11T18:03:56.541 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.542 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.542 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.542 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.542 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.542 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.542 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.542 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:56.542 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:56.542 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.542 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-12-11T18:03:56.542 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "test_orchestrator", 2023-12-11T18:03:56.542 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2023-12-11T18:03:56.542 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2023-12-11T18:03:56.542 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2023-12-11T18:03:56.543 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2023-12-11T18:03:56.543 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2023-12-11T18:03:56.543 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.543 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.543 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.543 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:56.543 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.543 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.543 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:56.543 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:56.543 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:56.543 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:56.543 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:56.543 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:56.543 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:56.543 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:56.544 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.544 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.544 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.544 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.544 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.544 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2023-12-11T18:03:56.544 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2023-12-11T18:03:56.544 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.544 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.544 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.544 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:56.544 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.544 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.544 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.544 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.545 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.545 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.545 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.545 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.545 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2023-12-11T18:03:56.545 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2023-12-11T18:03:56.545 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.545 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.545 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.545 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2023-12-11T18:03:56.545 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.545 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.545 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:56.545 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:56.545 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:56.546 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:56.546 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:56.546 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:56.546 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:56.546 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:56.546 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.546 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.546 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.546 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.546 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.546 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2023-12-11T18:03:56.546 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2023-12-11T18:03:56.546 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.546 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.547 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.547 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:56.547 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.547 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.547 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.547 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.547 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.547 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.547 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.547 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:56.547 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:56.547 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.547 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-12-11T18:03:56.547 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "volumes", 2023-12-11T18:03:56.548 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2023-12-11T18:03:56.548 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2023-12-11T18:03:56.548 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2023-12-11T18:03:56.548 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2023-12-11T18:03:56.548 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2023-12-11T18:03:56.548 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.548 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.548 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.548 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:56.548 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.548 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.548 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:56.548 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:56.548 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:56.549 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:56.549 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:56.549 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:56.549 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:56.549 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:56.549 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.549 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.549 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.549 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.549 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.549 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2023-12-11T18:03:56.549 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2023-12-11T18:03:56.549 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.549 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.550 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.550 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:56.550 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.550 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.550 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.550 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.550 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.550 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.550 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.550 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.550 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2023-12-11T18:03:56.550 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2023-12-11T18:03:56.550 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.550 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.551 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.551 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2023-12-11T18:03:56.551 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.551 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.551 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:56.551 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:56.551 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:56.551 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:56.551 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:56.551 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:56.551 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:56.551 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:56.551 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.552 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.552 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.552 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.552 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.552 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2023-12-11T18:03:56.552 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2023-12-11T18:03:56.552 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.552 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.552 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.552 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:56.552 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.552 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.552 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.552 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.552 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.553 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.553 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.553 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.553 INFO:teuthology.orchestra.run.smithi059.stdout: "max_concurrent_clones": { 2023-12-11T18:03:56.553 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "max_concurrent_clones", 2023-12-11T18:03:56.553 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2023-12-11T18:03:56.553 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.553 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.553 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "4", 2023-12-11T18:03:56.553 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.553 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.553 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.553 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Number of asynchronous cloner threads", 2023-12-11T18:03:56.553 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.553 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.554 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.554 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.554 INFO:teuthology.orchestra.run.smithi059.stdout: "periodic_async_work": { 2023-12-11T18:03:56.554 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "periodic_async_work", 2023-12-11T18:03:56.554 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.554 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.554 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.554 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:56.554 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.554 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.554 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.554 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Periodically check for async work", 2023-12-11T18:03:56.554 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.554 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.555 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.555 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.555 INFO:teuthology.orchestra.run.smithi059.stdout: "snapshot_clone_delay": { 2023-12-11T18:03:56.555 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "snapshot_clone_delay", 2023-12-11T18:03:56.555 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2023-12-11T18:03:56.555 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.555 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.555 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2023-12-11T18:03:56.555 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.555 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.555 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.555 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-12-11T18:03:56.555 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.555 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.556 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.556 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:56.556 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:56.556 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.556 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-12-11T18:03:56.556 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "zabbix", 2023-12-11T18:03:56.556 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2023-12-11T18:03:56.556 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2023-12-11T18:03:56.556 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2023-12-11T18:03:56.556 INFO:teuthology.orchestra.run.smithi059.stdout: "discovery_interval": { 2023-12-11T18:03:56.556 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "discovery_interval", 2023-12-11T18:03:56.556 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "uint", 2023-12-11T18:03:56.556 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.556 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.556 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "100", 2023-12-11T18:03:56.557 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.557 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.557 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.557 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.557 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.557 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.557 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.557 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.557 INFO:teuthology.orchestra.run.smithi059.stdout: "identifier": { 2023-12-11T18:03:56.557 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "identifier", 2023-12-11T18:03:56.557 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.557 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.557 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.557 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:56.558 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.558 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.558 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.558 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.558 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.558 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.558 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.558 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.558 INFO:teuthology.orchestra.run.smithi059.stdout: "interval": { 2023-12-11T18:03:56.558 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "interval", 2023-12-11T18:03:56.558 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2023-12-11T18:03:56.558 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.558 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.559 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "60", 2023-12-11T18:03:56.559 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.559 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.559 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.559 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.559 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.559 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.559 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.559 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.559 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2023-12-11T18:03:56.559 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2023-12-11T18:03:56.559 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.559 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.559 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.559 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:56.560 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.560 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.560 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:56.560 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:56.560 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:56.560 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:56.560 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:56.560 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:56.560 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:56.560 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:56.560 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.560 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.560 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.560 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.560 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.561 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2023-12-11T18:03:56.561 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2023-12-11T18:03:56.561 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.561 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.561 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.561 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:56.561 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.561 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.561 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.561 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.561 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.561 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.561 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.561 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.561 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2023-12-11T18:03:56.562 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2023-12-11T18:03:56.562 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.562 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.562 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.562 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2023-12-11T18:03:56.562 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.562 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.562 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2023-12-11T18:03:56.562 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2023-12-11T18:03:56.562 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2023-12-11T18:03:56.562 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2023-12-11T18:03:56.562 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2023-12-11T18:03:56.562 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2023-12-11T18:03:56.563 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2023-12-11T18:03:56.563 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:56.563 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.563 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.563 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.563 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.563 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.563 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2023-12-11T18:03:56.563 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2023-12-11T18:03:56.563 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2023-12-11T18:03:56.563 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.563 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2023-12-11T18:03:56.563 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2023-12-11T18:03:56.563 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.563 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.564 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.564 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.564 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.564 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.564 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.564 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.564 INFO:teuthology.orchestra.run.smithi059.stdout: "zabbix_host": { 2023-12-11T18:03:56.564 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "zabbix_host", 2023-12-11T18:03:56.564 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.564 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.564 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.564 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2023-12-11T18:03:56.564 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.564 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.564 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.564 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.565 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.565 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.565 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.565 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.565 INFO:teuthology.orchestra.run.smithi059.stdout: "zabbix_port": { 2023-12-11T18:03:56.565 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "zabbix_port", 2023-12-11T18:03:56.565 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2023-12-11T18:03:56.565 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.565 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.565 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "10051", 2023-12-11T18:03:56.565 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.565 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.565 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.565 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.565 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.565 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.566 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.566 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.566 INFO:teuthology.orchestra.run.smithi059.stdout: "zabbix_sender": { 2023-12-11T18:03:56.566 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "zabbix_sender", 2023-12-11T18:03:56.566 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2023-12-11T18:03:56.566 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2023-12-11T18:03:56.566 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2023-12-11T18:03:56.566 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-12-11T18:03:56.566 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2023-12-11T18:03:56.566 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2023-12-11T18:03:56.566 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2023-12-11T18:03:56.566 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2023-12-11T18:03:56.566 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2023-12-11T18:03:56.566 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2023-12-11T18:03:56.567 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2023-12-11T18:03:56.567 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:56.567 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:56.567 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:56.567 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:56.567 INFO:teuthology.orchestra.run.smithi059.stdout: "services": { 2023-12-11T18:03:56.567 INFO:teuthology.orchestra.run.smithi059.stdout: "dashboard": "https://172.21.15.59:8443/" 2023-12-11T18:03:56.567 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.567 INFO:teuthology.orchestra.run.smithi059.stdout: "always_on_modules": { 2023-12-11T18:03:56.567 INFO:teuthology.orchestra.run.smithi059.stdout: "octopus": [ 2023-12-11T18:03:56.567 INFO:teuthology.orchestra.run.smithi059.stdout: "balancer", 2023-12-11T18:03:56.567 INFO:teuthology.orchestra.run.smithi059.stdout: "crash", 2023-12-11T18:03:56.567 INFO:teuthology.orchestra.run.smithi059.stdout: "devicehealth", 2023-12-11T18:03:56.567 INFO:teuthology.orchestra.run.smithi059.stdout: "orchestrator", 2023-12-11T18:03:56.568 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_autoscaler", 2023-12-11T18:03:56.568 INFO:teuthology.orchestra.run.smithi059.stdout: "progress", 2023-12-11T18:03:56.568 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_support", 2023-12-11T18:03:56.568 INFO:teuthology.orchestra.run.smithi059.stdout: "status", 2023-12-11T18:03:56.568 INFO:teuthology.orchestra.run.smithi059.stdout: "telemetry", 2023-12-11T18:03:56.568 INFO:teuthology.orchestra.run.smithi059.stdout: "volumes" 2023-12-11T18:03:56.568 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:56.568 INFO:teuthology.orchestra.run.smithi059.stdout: "pacific": [ 2023-12-11T18:03:56.568 INFO:teuthology.orchestra.run.smithi059.stdout: "balancer", 2023-12-11T18:03:56.568 INFO:teuthology.orchestra.run.smithi059.stdout: "crash", 2023-12-11T18:03:56.568 INFO:teuthology.orchestra.run.smithi059.stdout: "devicehealth", 2023-12-11T18:03:56.568 INFO:teuthology.orchestra.run.smithi059.stdout: "orchestrator", 2023-12-11T18:03:56.568 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_autoscaler", 2023-12-11T18:03:56.568 INFO:teuthology.orchestra.run.smithi059.stdout: "progress", 2023-12-11T18:03:56.569 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_support", 2023-12-11T18:03:56.569 INFO:teuthology.orchestra.run.smithi059.stdout: "status", 2023-12-11T18:03:56.569 INFO:teuthology.orchestra.run.smithi059.stdout: "telemetry", 2023-12-11T18:03:56.569 INFO:teuthology.orchestra.run.smithi059.stdout: "volumes" 2023-12-11T18:03:56.569 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:56.569 INFO:teuthology.orchestra.run.smithi059.stdout: "quincy": [ 2023-12-11T18:03:56.569 INFO:teuthology.orchestra.run.smithi059.stdout: "balancer", 2023-12-11T18:03:56.569 INFO:teuthology.orchestra.run.smithi059.stdout: "crash", 2023-12-11T18:03:56.569 INFO:teuthology.orchestra.run.smithi059.stdout: "devicehealth", 2023-12-11T18:03:56.569 INFO:teuthology.orchestra.run.smithi059.stdout: "orchestrator", 2023-12-11T18:03:56.569 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_autoscaler", 2023-12-11T18:03:56.569 INFO:teuthology.orchestra.run.smithi059.stdout: "progress", 2023-12-11T18:03:56.569 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_support", 2023-12-11T18:03:56.569 INFO:teuthology.orchestra.run.smithi059.stdout: "status", 2023-12-11T18:03:56.569 INFO:teuthology.orchestra.run.smithi059.stdout: "telemetry", 2023-12-11T18:03:56.570 INFO:teuthology.orchestra.run.smithi059.stdout: "volumes" 2023-12-11T18:03:56.570 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-12-11T18:03:56.570 INFO:teuthology.orchestra.run.smithi059.stdout: "reef": [ 2023-12-11T18:03:56.570 INFO:teuthology.orchestra.run.smithi059.stdout: "balancer", 2023-12-11T18:03:56.570 INFO:teuthology.orchestra.run.smithi059.stdout: "crash", 2023-12-11T18:03:56.570 INFO:teuthology.orchestra.run.smithi059.stdout: "devicehealth", 2023-12-11T18:03:56.570 INFO:teuthology.orchestra.run.smithi059.stdout: "orchestrator", 2023-12-11T18:03:56.570 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_autoscaler", 2023-12-11T18:03:56.570 INFO:teuthology.orchestra.run.smithi059.stdout: "progress", 2023-12-11T18:03:56.570 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_support", 2023-12-11T18:03:56.570 INFO:teuthology.orchestra.run.smithi059.stdout: "status", 2023-12-11T18:03:56.570 INFO:teuthology.orchestra.run.smithi059.stdout: "telemetry", 2023-12-11T18:03:56.570 INFO:teuthology.orchestra.run.smithi059.stdout: "volumes" 2023-12-11T18:03:56.570 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2023-12-11T18:03:56.571 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.571 INFO:teuthology.orchestra.run.smithi059.stdout: "last_failure_osd_epoch": 89, 2023-12-11T18:03:56.571 INFO:teuthology.orchestra.run.smithi059.stdout: "active_clients": [ 2023-12-11T18:03:56.571 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-12-11T18:03:56.571 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "devicehealth", 2023-12-11T18:03:56.571 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2023-12-11T18:03:56.571 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-12-11T18:03:56.571 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2023-12-11T18:03:56.571 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:0", 2023-12-11T18:03:56.571 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4020254809 2023-12-11T18:03:56.571 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:56.571 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2023-12-11T18:03:56.571 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.571 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-12-11T18:03:56.572 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "libcephsqlite", 2023-12-11T18:03:56.572 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2023-12-11T18:03:56.572 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-12-11T18:03:56.572 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2023-12-11T18:03:56.572 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:0", 2023-12-11T18:03:56.572 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 272293616 2023-12-11T18:03:56.572 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:56.572 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2023-12-11T18:03:56.572 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.572 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-12-11T18:03:56.572 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rbd_support", 2023-12-11T18:03:56.572 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2023-12-11T18:03:56.572 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-12-11T18:03:56.572 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2023-12-11T18:03:56.572 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:0", 2023-12-11T18:03:56.573 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 3977195652 2023-12-11T18:03:56.573 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:56.573 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2023-12-11T18:03:56.573 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.573 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-12-11T18:03:56.573 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "snap_schedule", 2023-12-11T18:03:56.573 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2023-12-11T18:03:56.573 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-12-11T18:03:56.573 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2023-12-11T18:03:56.573 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:0", 2023-12-11T18:03:56.573 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 4195543184 2023-12-11T18:03:56.573 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:56.573 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2023-12-11T18:03:56.573 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.574 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-12-11T18:03:56.574 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "snap_schedule", 2023-12-11T18:03:56.574 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2023-12-11T18:03:56.574 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-12-11T18:03:56.574 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2023-12-11T18:03:56.574 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:0", 2023-12-11T18:03:56.574 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 86880727 2023-12-11T18:03:56.574 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:56.574 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2023-12-11T18:03:56.574 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:56.574 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-12-11T18:03:56.574 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "volumes", 2023-12-11T18:03:56.574 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2023-12-11T18:03:56.574 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-12-11T18:03:56.575 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2023-12-11T18:03:56.575 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:0", 2023-12-11T18:03:56.575 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 3643139831 2023-12-11T18:03:56.575 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:56.575 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2023-12-11T18:03:56.575 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:56.575 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2023-12-11T18:03:56.575 INFO:teuthology.orchestra.run.smithi059.stdout:} 2023-12-11T18:03:56.576 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:03:55 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/2396342680' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-11T18:03:56.576 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:03:55 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/2003391227' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-12-11T18:03:56.577 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:55.228+0000 7faa11739700 1 -- 172.21.15.59:0/2003391227 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7fa9f40a1620 msgr2=0x7fa9f40a3ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T18:03:56.577 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:55.228+0000 7faa11739700 1 --2- 172.21.15.59:0/2003391227 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7fa9f40a1620 0x7fa9f40a3ae0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7faa04009610 tx=0x7faa0400b700 comp rx=0 tx=0).stop 2023-12-11T18:03:56.577 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:55.228+0000 7faa11739700 1 -- 172.21.15.59:0/2003391227 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa0c1085d0 msgr2=0x7faa0c11dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T18:03:56.577 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:55.228+0000 7faa11739700 1 --2- 172.21.15.59:0/2003391227 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa0c1085d0 0x7faa0c11dd10 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fa9fc006fd0 tx=0x7fa9fc00b730 comp rx=0 tx=0).stop 2023-12-11T18:03:56.577 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:55.238+0000 7faa11739700 1 -- 172.21.15.59:0/2003391227 shutdown_connections 2023-12-11T18:03:56.577 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:55.238+0000 7faa11739700 1 --2- 172.21.15.59:0/2003391227 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7fa9f40a1620 0x7fa9f40a3ae0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:03:56.577 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:55.238+0000 7faa11739700 1 --2- 172.21.15.59:0/2003391227 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa0c10ceb0 0x7faa0c117f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:03:56.578 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:55.238+0000 7faa11739700 1 --2- 172.21.15.59:0/2003391227 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa0c1085d0 0x7faa0c11dd10 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:03:56.578 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:55.238+0000 7faa11739700 1 --2- 172.21.15.59:0/2003391227 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faa0c108f90 0x7faa0c11e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:03:56.578 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:55.238+0000 7faa11739700 1 -- 172.21.15.59:0/2003391227 >> 172.21.15.59:0/2003391227 conn(0x7faa0c06c990 msgr2=0x7faa0c10a930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T18:03:56.578 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:55.238+0000 7faa11739700 1 -- 172.21.15.59:0/2003391227 shutdown_connections 2023-12-11T18:03:56.578 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:03:55.238+0000 7faa11739700 1 -- 172.21.15.59:0/2003391227 wait complete. 2023-12-11T18:03:56.584 DEBUG:tasks.check_counter:Getting stats from a 2023-12-11T18:03:56.584 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell --fsid d0a07eaa-984b-11ee-95a3-87774f69a715 -- ceph daemon mds.a perf dump 2023-12-11T18:03:56.585 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:03:55 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/2396342680' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-11T18:03:56.585 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:03:55 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/2003391227' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-12-11T18:03:56.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:03:56 smithi059 ceph-mon[98847]: pgmap v561: 129 pgs: 41 active+clean+snaptrim_wait, 13 active+clean+snaptrim, 75 active+clean; 1.8 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 578 KiB/s rd, 569 KiB/s wr, 82 op/s 2023-12-11T18:03:56.782 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:03:56 smithi115 ceph-mon[102748]: pgmap v561: 129 pgs: 41 active+clean+snaptrim_wait, 13 active+clean+snaptrim, 75 active+clean; 1.8 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 578 KiB/s rd, 569 KiB/s wr, 82 op/s 2023-12-11T18:03:57.408 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/mon.a/config 2023-12-11T18:03:58.782 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:03:58 smithi115 ceph-mon[102748]: pgmap v562: 129 pgs: 41 active+clean+snaptrim_wait, 13 active+clean+snaptrim, 75 active+clean; 1.8 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 315 KiB/s wr, 62 op/s 2023-12-11T18:03:58.874 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:03:58 smithi059 ceph-mon[98847]: pgmap v562: 129 pgs: 41 active+clean+snaptrim_wait, 13 active+clean+snaptrim, 75 active+clean; 1.8 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 315 KiB/s wr, 62 op/s 2023-12-11T18:03:59.617 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2023-12-11T18:03:59.617 INFO:teuthology.orchestra.run.smithi059.stdout: "AsyncMessenger::Worker-0": { 2023-12-11T18:03:59.617 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_messages": 329, 2023-12-11T18:03:59.617 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages": 316, 2023-12-11T18:03:59.618 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_bytes": 1818208, 2023-12-11T18:03:59.618 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_bytes": 190336, 2023-12-11T18:03:59.618 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_created_connections": 4, 2023-12-11T18:03:59.618 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_active_connections": 0, 2023-12-11T18:03:59.618 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_total_time": 0.061502061, 2023-12-11T18:03:59.618 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_send_time": 0.019592655, 2023-12-11T18:03:59.618 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_recv_time": 0.041807068, 2023-12-11T18:03:59.618 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_fast_dispatch_time": 0.008585614, 2023-12-11T18:03:59.618 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages_queue_lat": { 2023-12-11T18:03:59.618 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 315, 2023-12-11T18:03:59.618 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.100574298, 2023-12-11T18:03:59.618 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000319283 2023-12-11T18:03:59.618 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:59.619 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_handle_ack_lat": { 2023-12-11T18:03:59.619 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2023-12-11T18:03:59.619 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2023-12-11T18:03:59.619 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2023-12-11T18:03:59.619 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:59.619 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_encrypted_bytes": 1818208, 2023-12-11T18:03:59.619 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_encrypted_bytes": 190336 2023-12-11T18:03:59.619 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:59.619 INFO:teuthology.orchestra.run.smithi059.stdout: "AsyncMessenger::Worker-1": { 2023-12-11T18:03:59.619 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_messages": 3, 2023-12-11T18:03:59.619 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages": 243, 2023-12-11T18:03:59.619 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_bytes": 480, 2023-12-11T18:03:59.619 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_bytes": 273600, 2023-12-11T18:03:59.619 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_created_connections": 6, 2023-12-11T18:03:59.620 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_active_connections": 3, 2023-12-11T18:03:59.620 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_total_time": 0.018433998, 2023-12-11T18:03:59.620 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_send_time": 0.013119558, 2023-12-11T18:03:59.620 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_recv_time": 0.002054397, 2023-12-11T18:03:59.620 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-12-11T18:03:59.620 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages_queue_lat": { 2023-12-11T18:03:59.620 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 240, 2023-12-11T18:03:59.620 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.095855044, 2023-12-11T18:03:59.620 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000399396 2023-12-11T18:03:59.620 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:59.620 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_handle_ack_lat": { 2023-12-11T18:03:59.620 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2023-12-11T18:03:59.620 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2023-12-11T18:03:59.620 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2023-12-11T18:03:59.620 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:59.621 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_encrypted_bytes": 480, 2023-12-11T18:03:59.621 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_encrypted_bytes": 273600 2023-12-11T18:03:59.621 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:59.621 INFO:teuthology.orchestra.run.smithi059.stdout: "AsyncMessenger::Worker-2": { 2023-12-11T18:03:59.621 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_messages": 4, 2023-12-11T18:03:59.621 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages": 4, 2023-12-11T18:03:59.621 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_bytes": 2944, 2023-12-11T18:03:59.621 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_bytes": 448, 2023-12-11T18:03:59.621 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_created_connections": 2, 2023-12-11T18:03:59.621 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_active_connections": 1, 2023-12-11T18:03:59.621 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_total_time": 0.000793931, 2023-12-11T18:03:59.621 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_send_time": 0.000036674, 2023-12-11T18:03:59.621 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_recv_time": 0.000725482, 2023-12-11T18:03:59.621 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-12-11T18:03:59.622 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages_queue_lat": { 2023-12-11T18:03:59.622 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 3, 2023-12-11T18:03:59.622 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000790365, 2023-12-11T18:03:59.622 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000263455 2023-12-11T18:03:59.622 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:59.622 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_handle_ack_lat": { 2023-12-11T18:03:59.622 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2023-12-11T18:03:59.622 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2023-12-11T18:03:59.622 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2023-12-11T18:03:59.622 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:59.622 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_encrypted_bytes": 2944, 2023-12-11T18:03:59.622 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_encrypted_bytes": 448 2023-12-11T18:03:59.622 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:59.623 INFO:teuthology.orchestra.run.smithi059.stdout: "cct": { 2023-12-11T18:03:59.623 INFO:teuthology.orchestra.run.smithi059.stdout: "total_workers": 0, 2023-12-11T18:03:59.623 INFO:teuthology.orchestra.run.smithi059.stdout: "unhealthy_workers": 0 2023-12-11T18:03:59.623 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:59.623 INFO:teuthology.orchestra.run.smithi059.stdout: "mempool": { 2023-12-11T18:03:59.623 INFO:teuthology.orchestra.run.smithi059.stdout: "bloom_filter_bytes": 0, 2023-12-11T18:03:59.623 INFO:teuthology.orchestra.run.smithi059.stdout: "bloom_filter_items": 0, 2023-12-11T18:03:59.623 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_alloc_bytes": 0, 2023-12-11T18:03:59.623 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_alloc_items": 0, 2023-12-11T18:03:59.623 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_data_bytes": 0, 2023-12-11T18:03:59.623 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_data_items": 0, 2023-12-11T18:03:59.623 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_onode_bytes": 0, 2023-12-11T18:03:59.623 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_onode_items": 0, 2023-12-11T18:03:59.623 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_meta_bytes": 0, 2023-12-11T18:03:59.624 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_meta_items": 0, 2023-12-11T18:03:59.624 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_other_bytes": 0, 2023-12-11T18:03:59.624 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_other_items": 0, 2023-12-11T18:03:59.624 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_buffer_bytes": 0, 2023-12-11T18:03:59.624 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_buffer_items": 0, 2023-12-11T18:03:59.624 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_extent_bytes": 0, 2023-12-11T18:03:59.624 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_extent_items": 0, 2023-12-11T18:03:59.624 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_blob_bytes": 0, 2023-12-11T18:03:59.624 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_blob_items": 0, 2023-12-11T18:03:59.624 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_shared_blob_bytes": 0, 2023-12-11T18:03:59.624 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_shared_blob_items": 0, 2023-12-11T18:03:59.624 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_inline_bl_bytes": 0, 2023-12-11T18:03:59.624 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_inline_bl_items": 0, 2023-12-11T18:03:59.624 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_fsck_bytes": 0, 2023-12-11T18:03:59.624 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_fsck_items": 0, 2023-12-11T18:03:59.625 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_txc_bytes": 0, 2023-12-11T18:03:59.625 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_txc_items": 0, 2023-12-11T18:03:59.625 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_writing_deferred_bytes": 0, 2023-12-11T18:03:59.625 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_writing_deferred_items": 0, 2023-12-11T18:03:59.625 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_writing_bytes": 0, 2023-12-11T18:03:59.625 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_writing_items": 0, 2023-12-11T18:03:59.625 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_bytes": 0, 2023-12-11T18:03:59.625 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_items": 0, 2023-12-11T18:03:59.625 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_file_reader_bytes": 0, 2023-12-11T18:03:59.625 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_file_reader_items": 0, 2023-12-11T18:03:59.625 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_file_writer_bytes": 0, 2023-12-11T18:03:59.625 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_file_writer_items": 0, 2023-12-11T18:03:59.625 INFO:teuthology.orchestra.run.smithi059.stdout: "buffer_anon_bytes": 148704, 2023-12-11T18:03:59.625 INFO:teuthology.orchestra.run.smithi059.stdout: "buffer_anon_items": 29, 2023-12-11T18:03:59.626 INFO:teuthology.orchestra.run.smithi059.stdout: "buffer_meta_bytes": 0, 2023-12-11T18:03:59.626 INFO:teuthology.orchestra.run.smithi059.stdout: "buffer_meta_items": 0, 2023-12-11T18:03:59.626 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_bytes": 0, 2023-12-11T18:03:59.626 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_items": 0, 2023-12-11T18:03:59.626 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_mapbl_bytes": 0, 2023-12-11T18:03:59.626 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_mapbl_items": 0, 2023-12-11T18:03:59.626 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_pglog_bytes": 0, 2023-12-11T18:03:59.626 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_pglog_items": 0, 2023-12-11T18:03:59.626 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_bytes": 0, 2023-12-11T18:03:59.626 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_items": 0, 2023-12-11T18:03:59.626 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_mapping_bytes": 0, 2023-12-11T18:03:59.626 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_mapping_items": 0, 2023-12-11T18:03:59.626 INFO:teuthology.orchestra.run.smithi059.stdout: "pgmap_bytes": 0, 2023-12-11T18:03:59.626 INFO:teuthology.orchestra.run.smithi059.stdout: "pgmap_items": 0, 2023-12-11T18:03:59.627 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_co_bytes": 928, 2023-12-11T18:03:59.627 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_co_items": 2, 2023-12-11T18:03:59.627 INFO:teuthology.orchestra.run.smithi059.stdout: "unittest_1_bytes": 0, 2023-12-11T18:03:59.627 INFO:teuthology.orchestra.run.smithi059.stdout: "unittest_1_items": 0, 2023-12-11T18:03:59.627 INFO:teuthology.orchestra.run.smithi059.stdout: "unittest_2_bytes": 0, 2023-12-11T18:03:59.627 INFO:teuthology.orchestra.run.smithi059.stdout: "unittest_2_items": 0 2023-12-11T18:03:59.627 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:03:59.627 INFO:teuthology.orchestra.run.smithi059.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-12-11T18:03:59.627 INFO:teuthology.orchestra.run.smithi059.stdout: "val": 0, 2023-12-11T18:03:59.627 INFO:teuthology.orchestra.run.smithi059.stdout: "max": 104857600, 2023-12-11T18:03:59.627 INFO:teuthology.orchestra.run.smithi059.stdout: "get_started": 0, 2023-12-11T18:03:59.627 INFO:teuthology.orchestra.run.smithi059.stdout: "get": 332, 2023-12-11T18:03:59.627 INFO:teuthology.orchestra.run.smithi059.stdout: "get_sum": 1771802, 2023-12-11T18:03:59.627 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_fail": 0, 2023-12-11T18:03:59.627 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_success": 332, 2023-12-11T18:03:59.628 INFO:teuthology.orchestra.run.smithi059.stdout: "take": 0, 2023-12-11T18:03:59.628 INFO:teuthology.orchestra.run.smithi059.stdout: "take_sum": 0, 2023-12-11T18:03:59.628 INFO:teuthology.orchestra.run.smithi059.stdout: "put": 332, 2023-12-11T18:03:59.628 INFO:teuthology.orchestra.run.smithi059.stdout: "put_sum": 1771802, 2023-12-11T18:03:59.628 INFO:teuthology.orchestra.run.smithi059.stdout: "wait": { 2023-12-11T18:03:59.628 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2023-12-11T18:03:59.628 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2023-12-11T18:03:59.628 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2023-12-11T18:03:59.628 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:59.628 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:03:59.628 INFO:teuthology.orchestra.run.smithi059.stdout:} 2023-12-11T18:04:00.101 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2023-12-11T18:04:00.101 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2023-12-11T18:04:00.101 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.a 2023-12-11T18:04:00.101 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.a 2023-12-11T18:04:00.101 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.a 2023-12-11T18:04:00.102 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.a 2023-12-11T18:04:00.102 DEBUG:tasks.check_counter:Getting stats from d 2023-12-11T18:04:00.102 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell --fsid d0a07eaa-984b-11ee-95a3-87774f69a715 -- ceph daemon mds.d perf dump 2023-12-11T18:04:00.782 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:04:00 smithi115 ceph-mon[102748]: pgmap v563: 129 pgs: 41 active+clean+snaptrim_wait, 13 active+clean+snaptrim, 75 active+clean; 1.8 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 307 KiB/s wr, 47 op/s 2023-12-11T18:04:00.907 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:04:00 smithi059 ceph-mon[98847]: pgmap v563: 129 pgs: 41 active+clean+snaptrim_wait, 13 active+clean+snaptrim, 75 active+clean; 1.8 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 307 KiB/s wr, 47 op/s 2023-12-11T18:04:00.955 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/mon.a/config 2023-12-11T18:04:01.509 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:04:01 smithi059 ceph-mon[98847]: pgmap v564: 129 pgs: 41 active+clean+snaptrim_wait, 13 active+clean+snaptrim, 75 active+clean; 1.8 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 17 KiB/s wr, 31 op/s 2023-12-11T18:04:01.782 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:04:01 smithi115 ceph-mon[102748]: pgmap v564: 129 pgs: 41 active+clean+snaptrim_wait, 13 active+clean+snaptrim, 75 active+clean; 1.8 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 17 KiB/s wr, 31 op/s 2023-12-11T18:04:02.792 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2023-12-11T18:04:02.792 INFO:teuthology.orchestra.run.smithi059.stdout: "AsyncMessenger::Worker-0": { 2023-12-11T18:04:02.792 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_messages": 0, 2023-12-11T18:04:02.793 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages": 2, 2023-12-11T18:04:02.793 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_bytes": 0, 2023-12-11T18:04:02.793 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_bytes": 0, 2023-12-11T18:04:02.793 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_created_connections": 4, 2023-12-11T18:04:02.793 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_active_connections": 0, 2023-12-11T18:04:02.793 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_total_time": 0.001774203, 2023-12-11T18:04:02.793 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_send_time": 0.000000000, 2023-12-11T18:04:02.793 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_recv_time": 0.000477849, 2023-12-11T18:04:02.793 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-12-11T18:04:02.793 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages_queue_lat": { 2023-12-11T18:04:02.793 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2023-12-11T18:04:02.793 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2023-12-11T18:04:02.794 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2023-12-11T18:04:02.794 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:04:02.794 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_handle_ack_lat": { 2023-12-11T18:04:02.794 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2023-12-11T18:04:02.794 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2023-12-11T18:04:02.794 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2023-12-11T18:04:02.794 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:04:02.794 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_encrypted_bytes": 0, 2023-12-11T18:04:02.794 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_encrypted_bytes": 0 2023-12-11T18:04:02.794 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:04:02.794 INFO:teuthology.orchestra.run.smithi059.stdout: "AsyncMessenger::Worker-1": { 2023-12-11T18:04:02.794 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_messages": 338, 2023-12-11T18:04:02.794 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages": 323, 2023-12-11T18:04:02.795 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_bytes": 1822512, 2023-12-11T18:04:02.795 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_bytes": 193904, 2023-12-11T18:04:02.795 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_created_connections": 2, 2023-12-11T18:04:02.795 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_active_connections": 1, 2023-12-11T18:04:02.795 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_total_time": 0.062881182, 2023-12-11T18:04:02.795 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_send_time": 0.020582929, 2023-12-11T18:04:02.795 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_recv_time": 0.043363008, 2023-12-11T18:04:02.795 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_fast_dispatch_time": 0.009039362, 2023-12-11T18:04:02.795 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages_queue_lat": { 2023-12-11T18:04:02.795 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 323, 2023-12-11T18:04:02.795 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.076730051, 2023-12-11T18:04:02.795 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000237554 2023-12-11T18:04:02.795 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:04:02.796 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_handle_ack_lat": { 2023-12-11T18:04:02.796 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2023-12-11T18:04:02.796 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2023-12-11T18:04:02.796 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2023-12-11T18:04:02.796 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:04:02.796 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_encrypted_bytes": 1822512, 2023-12-11T18:04:02.796 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_encrypted_bytes": 193904 2023-12-11T18:04:02.796 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:04:02.796 INFO:teuthology.orchestra.run.smithi059.stdout: "AsyncMessenger::Worker-2": { 2023-12-11T18:04:02.796 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_messages": 3, 2023-12-11T18:04:02.796 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages": 246, 2023-12-11T18:04:02.796 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_bytes": 480, 2023-12-11T18:04:02.797 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_bytes": 274128, 2023-12-11T18:04:02.797 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_created_connections": 6, 2023-12-11T18:04:02.797 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_active_connections": 3, 2023-12-11T18:04:02.797 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_total_time": 0.018949480, 2023-12-11T18:04:02.797 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_send_time": 0.013306634, 2023-12-11T18:04:02.797 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_recv_time": 0.002327859, 2023-12-11T18:04:02.797 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-12-11T18:04:02.797 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages_queue_lat": { 2023-12-11T18:04:02.797 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 243, 2023-12-11T18:04:02.797 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.069903011, 2023-12-11T18:04:02.798 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000287666 2023-12-11T18:04:02.798 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:04:02.798 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_handle_ack_lat": { 2023-12-11T18:04:02.798 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2023-12-11T18:04:02.798 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2023-12-11T18:04:02.798 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2023-12-11T18:04:02.799 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:04:02.799 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_encrypted_bytes": 480, 2023-12-11T18:04:02.799 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_encrypted_bytes": 274128 2023-12-11T18:04:02.799 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:04:02.799 INFO:teuthology.orchestra.run.smithi059.stdout: "cct": { 2023-12-11T18:04:02.799 INFO:teuthology.orchestra.run.smithi059.stdout: "total_workers": 0, 2023-12-11T18:04:02.799 INFO:teuthology.orchestra.run.smithi059.stdout: "unhealthy_workers": 0 2023-12-11T18:04:02.800 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:04:02.800 INFO:teuthology.orchestra.run.smithi059.stdout: "mempool": { 2023-12-11T18:04:02.800 INFO:teuthology.orchestra.run.smithi059.stdout: "bloom_filter_bytes": 0, 2023-12-11T18:04:02.800 INFO:teuthology.orchestra.run.smithi059.stdout: "bloom_filter_items": 0, 2023-12-11T18:04:02.800 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_alloc_bytes": 0, 2023-12-11T18:04:02.800 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_alloc_items": 0, 2023-12-11T18:04:02.801 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_data_bytes": 0, 2023-12-11T18:04:02.801 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_data_items": 0, 2023-12-11T18:04:02.801 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_onode_bytes": 0, 2023-12-11T18:04:02.801 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_onode_items": 0, 2023-12-11T18:04:02.801 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_meta_bytes": 0, 2023-12-11T18:04:02.801 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_meta_items": 0, 2023-12-11T18:04:02.802 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_other_bytes": 0, 2023-12-11T18:04:02.802 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_other_items": 0, 2023-12-11T18:04:02.802 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_buffer_bytes": 0, 2023-12-11T18:04:02.802 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_buffer_items": 0, 2023-12-11T18:04:02.802 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_extent_bytes": 0, 2023-12-11T18:04:02.802 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_extent_items": 0, 2023-12-11T18:04:02.803 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_blob_bytes": 0, 2023-12-11T18:04:02.803 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_blob_items": 0, 2023-12-11T18:04:02.803 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_shared_blob_bytes": 0, 2023-12-11T18:04:02.803 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_shared_blob_items": 0, 2023-12-11T18:04:02.803 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_inline_bl_bytes": 0, 2023-12-11T18:04:02.803 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_inline_bl_items": 0, 2023-12-11T18:04:02.804 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_fsck_bytes": 0, 2023-12-11T18:04:02.804 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_fsck_items": 0, 2023-12-11T18:04:02.804 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_txc_bytes": 0, 2023-12-11T18:04:02.804 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_txc_items": 0, 2023-12-11T18:04:02.804 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_writing_deferred_bytes": 0, 2023-12-11T18:04:02.804 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_writing_deferred_items": 0, 2023-12-11T18:04:02.804 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_writing_bytes": 0, 2023-12-11T18:04:02.805 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_writing_items": 0, 2023-12-11T18:04:02.805 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_bytes": 0, 2023-12-11T18:04:02.805 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_items": 0, 2023-12-11T18:04:02.805 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_file_reader_bytes": 0, 2023-12-11T18:04:02.805 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_file_reader_items": 0, 2023-12-11T18:04:02.805 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_file_writer_bytes": 0, 2023-12-11T18:04:02.806 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_file_writer_items": 0, 2023-12-11T18:04:02.806 INFO:teuthology.orchestra.run.smithi059.stdout: "buffer_anon_bytes": 148744, 2023-12-11T18:04:02.806 INFO:teuthology.orchestra.run.smithi059.stdout: "buffer_anon_items": 30, 2023-12-11T18:04:02.806 INFO:teuthology.orchestra.run.smithi059.stdout: "buffer_meta_bytes": 0, 2023-12-11T18:04:02.806 INFO:teuthology.orchestra.run.smithi059.stdout: "buffer_meta_items": 0, 2023-12-11T18:04:02.807 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_bytes": 0, 2023-12-11T18:04:02.807 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_items": 0, 2023-12-11T18:04:02.807 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_mapbl_bytes": 0, 2023-12-11T18:04:02.807 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_mapbl_items": 0, 2023-12-11T18:04:02.807 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_pglog_bytes": 0, 2023-12-11T18:04:02.807 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_pglog_items": 0, 2023-12-11T18:04:02.808 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_bytes": 0, 2023-12-11T18:04:02.808 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_items": 0, 2023-12-11T18:04:02.808 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_mapping_bytes": 0, 2023-12-11T18:04:02.808 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_mapping_items": 0, 2023-12-11T18:04:02.808 INFO:teuthology.orchestra.run.smithi059.stdout: "pgmap_bytes": 0, 2023-12-11T18:04:02.808 INFO:teuthology.orchestra.run.smithi059.stdout: "pgmap_items": 0, 2023-12-11T18:04:02.809 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_co_bytes": 928, 2023-12-11T18:04:02.809 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_co_items": 2, 2023-12-11T18:04:02.809 INFO:teuthology.orchestra.run.smithi059.stdout: "unittest_1_bytes": 0, 2023-12-11T18:04:02.809 INFO:teuthology.orchestra.run.smithi059.stdout: "unittest_1_items": 0, 2023-12-11T18:04:02.809 INFO:teuthology.orchestra.run.smithi059.stdout: "unittest_2_bytes": 0, 2023-12-11T18:04:02.809 INFO:teuthology.orchestra.run.smithi059.stdout: "unittest_2_items": 0 2023-12-11T18:04:02.810 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:04:02.810 INFO:teuthology.orchestra.run.smithi059.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-12-11T18:04:02.810 INFO:teuthology.orchestra.run.smithi059.stdout: "val": 0, 2023-12-11T18:04:02.810 INFO:teuthology.orchestra.run.smithi059.stdout: "max": 104857600, 2023-12-11T18:04:02.810 INFO:teuthology.orchestra.run.smithi059.stdout: "get_started": 0, 2023-12-11T18:04:02.811 INFO:teuthology.orchestra.run.smithi059.stdout: "get": 337, 2023-12-11T18:04:02.811 INFO:teuthology.orchestra.run.smithi059.stdout: "get_sum": 1772452, 2023-12-11T18:04:02.811 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_fail": 0, 2023-12-11T18:04:02.811 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_success": 337, 2023-12-11T18:04:02.811 INFO:teuthology.orchestra.run.smithi059.stdout: "take": 0, 2023-12-11T18:04:02.811 INFO:teuthology.orchestra.run.smithi059.stdout: "take_sum": 0, 2023-12-11T18:04:02.811 INFO:teuthology.orchestra.run.smithi059.stdout: "put": 337, 2023-12-11T18:04:02.811 INFO:teuthology.orchestra.run.smithi059.stdout: "put_sum": 1772452, 2023-12-11T18:04:02.812 INFO:teuthology.orchestra.run.smithi059.stdout: "wait": { 2023-12-11T18:04:02.812 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2023-12-11T18:04:02.812 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2023-12-11T18:04:02.812 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2023-12-11T18:04:02.812 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:04:02.812 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:04:02.812 INFO:teuthology.orchestra.run.smithi059.stdout:} 2023-12-11T18:04:03.356 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2023-12-11T18:04:03.356 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2023-12-11T18:04:03.356 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2023-12-11T18:04:03.356 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2023-12-11T18:04:03.356 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2023-12-11T18:04:03.357 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.d 2023-12-11T18:04:03.357 DEBUG:tasks.check_counter:Getting stats from g 2023-12-11T18:04:03.357 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell --fsid d0a07eaa-984b-11ee-95a3-87774f69a715 -- ceph daemon mds.g perf dump 2023-12-11T18:04:03.638 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:04:03 smithi059 ceph-mon[98847]: pgmap v565: 129 pgs: 41 active+clean+snaptrim_wait, 12 active+clean+snaptrim, 76 active+clean; 1.8 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 36 KiB/s wr, 39 op/s 2023-12-11T18:04:04.032 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:04:03 smithi115 ceph-mon[102748]: pgmap v565: 129 pgs: 41 active+clean+snaptrim_wait, 12 active+clean+snaptrim, 76 active+clean; 1.8 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 36 KiB/s wr, 39 op/s 2023-12-11T18:04:04.362 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/mon.a/config 2023-12-11T18:04:06.405 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:04:06 smithi059 ceph-mon[98847]: pgmap v566: 129 pgs: 41 active+clean+snaptrim_wait, 12 active+clean+snaptrim, 76 active+clean; 1.8 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 59 KiB/s wr, 28 op/s 2023-12-11T18:04:06.532 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:04:06 smithi115 ceph-mon[102748]: pgmap v566: 129 pgs: 41 active+clean+snaptrim_wait, 12 active+clean+snaptrim, 76 active+clean; 1.8 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 59 KiB/s wr, 28 op/s 2023-12-11T18:04:07.050 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2023-12-11T18:04:07.050 INFO:teuthology.orchestra.run.smithi059.stdout: "AsyncMessenger::Worker-0": { 2023-12-11T18:04:07.050 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_messages": 332, 2023-12-11T18:04:07.050 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages": 319, 2023-12-11T18:04:07.051 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_bytes": 1819024, 2023-12-11T18:04:07.051 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_bytes": 192208, 2023-12-11T18:04:07.051 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_created_connections": 4, 2023-12-11T18:04:07.051 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_active_connections": 0, 2023-12-11T18:04:07.051 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_total_time": 0.063315134, 2023-12-11T18:04:07.051 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_send_time": 0.020235094, 2023-12-11T18:04:07.051 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_recv_time": 0.043098685, 2023-12-11T18:04:07.051 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_fast_dispatch_time": 0.008934186, 2023-12-11T18:04:07.051 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages_queue_lat": { 2023-12-11T18:04:07.051 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 318, 2023-12-11T18:04:07.051 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.089033730, 2023-12-11T18:04:07.051 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000279980 2023-12-11T18:04:07.052 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:04:07.052 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_handle_ack_lat": { 2023-12-11T18:04:07.052 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2023-12-11T18:04:07.052 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2023-12-11T18:04:07.052 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2023-12-11T18:04:07.052 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:04:07.052 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_encrypted_bytes": 1819024, 2023-12-11T18:04:07.052 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_encrypted_bytes": 192208 2023-12-11T18:04:07.052 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:04:07.052 INFO:teuthology.orchestra.run.smithi059.stdout: "AsyncMessenger::Worker-1": { 2023-12-11T18:04:07.052 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_messages": 3, 2023-12-11T18:04:07.052 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages": 244, 2023-12-11T18:04:07.052 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_bytes": 480, 2023-12-11T18:04:07.053 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_bytes": 273776, 2023-12-11T18:04:07.053 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_created_connections": 6, 2023-12-11T18:04:07.053 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_active_connections": 3, 2023-12-11T18:04:07.053 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_total_time": 0.018776372, 2023-12-11T18:04:07.053 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_send_time": 0.013570020, 2023-12-11T18:04:07.053 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_recv_time": 0.002147041, 2023-12-11T18:04:07.053 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-12-11T18:04:07.053 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages_queue_lat": { 2023-12-11T18:04:07.053 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 241, 2023-12-11T18:04:07.053 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.055165515, 2023-12-11T18:04:07.053 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000228902 2023-12-11T18:04:07.053 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:04:07.053 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_handle_ack_lat": { 2023-12-11T18:04:07.054 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2023-12-11T18:04:07.054 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2023-12-11T18:04:07.054 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2023-12-11T18:04:07.054 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:04:07.054 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_encrypted_bytes": 480, 2023-12-11T18:04:07.054 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_encrypted_bytes": 273776 2023-12-11T18:04:07.054 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:04:07.054 INFO:teuthology.orchestra.run.smithi059.stdout: "AsyncMessenger::Worker-2": { 2023-12-11T18:04:07.055 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_messages": 4, 2023-12-11T18:04:07.055 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages": 4, 2023-12-11T18:04:07.055 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_bytes": 2944, 2023-12-11T18:04:07.055 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_bytes": 448, 2023-12-11T18:04:07.055 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_created_connections": 2, 2023-12-11T18:04:07.055 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_active_connections": 1, 2023-12-11T18:04:07.055 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_total_time": 0.000788260, 2023-12-11T18:04:07.055 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_send_time": 0.000034727, 2023-12-11T18:04:07.055 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_recv_time": 0.000716971, 2023-12-11T18:04:07.055 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-12-11T18:04:07.055 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages_queue_lat": { 2023-12-11T18:04:07.055 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 3, 2023-12-11T18:04:07.055 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000693306, 2023-12-11T18:04:07.055 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000231102 2023-12-11T18:04:07.056 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:04:07.056 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_handle_ack_lat": { 2023-12-11T18:04:07.056 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2023-12-11T18:04:07.056 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2023-12-11T18:04:07.056 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2023-12-11T18:04:07.056 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:04:07.056 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_encrypted_bytes": 2944, 2023-12-11T18:04:07.056 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_encrypted_bytes": 448 2023-12-11T18:04:07.056 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:04:07.056 INFO:teuthology.orchestra.run.smithi059.stdout: "cct": { 2023-12-11T18:04:07.056 INFO:teuthology.orchestra.run.smithi059.stdout: "total_workers": 0, 2023-12-11T18:04:07.056 INFO:teuthology.orchestra.run.smithi059.stdout: "unhealthy_workers": 0 2023-12-11T18:04:07.056 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:04:07.056 INFO:teuthology.orchestra.run.smithi059.stdout: "mempool": { 2023-12-11T18:04:07.057 INFO:teuthology.orchestra.run.smithi059.stdout: "bloom_filter_bytes": 0, 2023-12-11T18:04:07.057 INFO:teuthology.orchestra.run.smithi059.stdout: "bloom_filter_items": 0, 2023-12-11T18:04:07.057 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_alloc_bytes": 0, 2023-12-11T18:04:07.057 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_alloc_items": 0, 2023-12-11T18:04:07.057 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_data_bytes": 0, 2023-12-11T18:04:07.057 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_data_items": 0, 2023-12-11T18:04:07.057 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_onode_bytes": 0, 2023-12-11T18:04:07.057 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_onode_items": 0, 2023-12-11T18:04:07.057 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_meta_bytes": 0, 2023-12-11T18:04:07.057 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_meta_items": 0, 2023-12-11T18:04:07.057 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_other_bytes": 0, 2023-12-11T18:04:07.057 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_other_items": 0, 2023-12-11T18:04:07.057 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_buffer_bytes": 0, 2023-12-11T18:04:07.057 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_buffer_items": 0, 2023-12-11T18:04:07.058 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_extent_bytes": 0, 2023-12-11T18:04:07.058 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_extent_items": 0, 2023-12-11T18:04:07.058 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_blob_bytes": 0, 2023-12-11T18:04:07.058 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_blob_items": 0, 2023-12-11T18:04:07.058 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_shared_blob_bytes": 0, 2023-12-11T18:04:07.058 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_shared_blob_items": 0, 2023-12-11T18:04:07.058 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_inline_bl_bytes": 0, 2023-12-11T18:04:07.058 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_inline_bl_items": 0, 2023-12-11T18:04:07.058 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_fsck_bytes": 0, 2023-12-11T18:04:07.058 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_fsck_items": 0, 2023-12-11T18:04:07.058 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_txc_bytes": 0, 2023-12-11T18:04:07.058 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_txc_items": 0, 2023-12-11T18:04:07.058 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_writing_deferred_bytes": 0, 2023-12-11T18:04:07.059 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_writing_deferred_items": 0, 2023-12-11T18:04:07.059 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_writing_bytes": 0, 2023-12-11T18:04:07.059 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_writing_items": 0, 2023-12-11T18:04:07.059 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_bytes": 0, 2023-12-11T18:04:07.059 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_items": 0, 2023-12-11T18:04:07.059 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_file_reader_bytes": 0, 2023-12-11T18:04:07.059 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_file_reader_items": 0, 2023-12-11T18:04:07.059 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_file_writer_bytes": 0, 2023-12-11T18:04:07.059 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_file_writer_items": 0, 2023-12-11T18:04:07.059 INFO:teuthology.orchestra.run.smithi059.stdout: "buffer_anon_bytes": 148744, 2023-12-11T18:04:07.059 INFO:teuthology.orchestra.run.smithi059.stdout: "buffer_anon_items": 30, 2023-12-11T18:04:07.059 INFO:teuthology.orchestra.run.smithi059.stdout: "buffer_meta_bytes": 0, 2023-12-11T18:04:07.059 INFO:teuthology.orchestra.run.smithi059.stdout: "buffer_meta_items": 0, 2023-12-11T18:04:07.059 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_bytes": 0, 2023-12-11T18:04:07.059 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_items": 0, 2023-12-11T18:04:07.060 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_mapbl_bytes": 0, 2023-12-11T18:04:07.060 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_mapbl_items": 0, 2023-12-11T18:04:07.060 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_pglog_bytes": 0, 2023-12-11T18:04:07.060 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_pglog_items": 0, 2023-12-11T18:04:07.060 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_bytes": 0, 2023-12-11T18:04:07.060 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_items": 0, 2023-12-11T18:04:07.060 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_mapping_bytes": 0, 2023-12-11T18:04:07.060 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_mapping_items": 0, 2023-12-11T18:04:07.060 INFO:teuthology.orchestra.run.smithi059.stdout: "pgmap_bytes": 0, 2023-12-11T18:04:07.060 INFO:teuthology.orchestra.run.smithi059.stdout: "pgmap_items": 0, 2023-12-11T18:04:07.060 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_co_bytes": 928, 2023-12-11T18:04:07.060 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_co_items": 2, 2023-12-11T18:04:07.060 INFO:teuthology.orchestra.run.smithi059.stdout: "unittest_1_bytes": 0, 2023-12-11T18:04:07.060 INFO:teuthology.orchestra.run.smithi059.stdout: "unittest_1_items": 0, 2023-12-11T18:04:07.060 INFO:teuthology.orchestra.run.smithi059.stdout: "unittest_2_bytes": 0, 2023-12-11T18:04:07.061 INFO:teuthology.orchestra.run.smithi059.stdout: "unittest_2_items": 0 2023-12-11T18:04:07.061 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:04:07.061 INFO:teuthology.orchestra.run.smithi059.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-12-11T18:04:07.061 INFO:teuthology.orchestra.run.smithi059.stdout: "val": 0, 2023-12-11T18:04:07.061 INFO:teuthology.orchestra.run.smithi059.stdout: "max": 104857600, 2023-12-11T18:04:07.061 INFO:teuthology.orchestra.run.smithi059.stdout: "get_started": 0, 2023-12-11T18:04:07.061 INFO:teuthology.orchestra.run.smithi059.stdout: "get": 335, 2023-12-11T18:04:07.061 INFO:teuthology.orchestra.run.smithi059.stdout: "get_sum": 1772192, 2023-12-11T18:04:07.061 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_fail": 0, 2023-12-11T18:04:07.062 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_success": 335, 2023-12-11T18:04:07.062 INFO:teuthology.orchestra.run.smithi059.stdout: "take": 0, 2023-12-11T18:04:07.062 INFO:teuthology.orchestra.run.smithi059.stdout: "take_sum": 0, 2023-12-11T18:04:07.062 INFO:teuthology.orchestra.run.smithi059.stdout: "put": 335, 2023-12-11T18:04:07.062 INFO:teuthology.orchestra.run.smithi059.stdout: "put_sum": 1772192, 2023-12-11T18:04:07.062 INFO:teuthology.orchestra.run.smithi059.stdout: "wait": { 2023-12-11T18:04:07.062 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2023-12-11T18:04:07.063 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2023-12-11T18:04:07.063 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2023-12-11T18:04:07.063 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:04:07.063 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:04:07.063 INFO:teuthology.orchestra.run.smithi059.stdout:} 2023-12-11T18:04:07.551 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2023-12-11T18:04:07.551 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2023-12-11T18:04:07.551 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.g 2023-12-11T18:04:07.551 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.g 2023-12-11T18:04:07.551 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.g 2023-12-11T18:04:07.551 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.g 2023-12-11T18:04:07.551 DEBUG:tasks.check_counter:Getting stats from j 2023-12-11T18:04:07.552 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell --fsid d0a07eaa-984b-11ee-95a3-87774f69a715 -- ceph daemon mds.j perf dump 2023-12-11T18:04:08.504 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/mon.a/config 2023-12-11T18:04:08.532 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:04:08 smithi115 ceph-mon[102748]: pgmap v567: 129 pgs: 41 active+clean+snaptrim_wait, 12 active+clean+snaptrim, 76 active+clean; 1.8 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 55 KiB/s wr, 23 op/s 2023-12-11T18:04:08.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:04:08 smithi059 ceph-mon[98847]: pgmap v567: 129 pgs: 41 active+clean+snaptrim_wait, 12 active+clean+snaptrim, 76 active+clean; 1.8 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 55 KiB/s wr, 23 op/s 2023-12-11T18:04:10.300 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:04:10 smithi059 ceph-mon[98847]: pgmap v568: 129 pgs: 41 active+clean+snaptrim_wait, 12 active+clean+snaptrim, 76 active+clean; 1.8 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 53 KiB/s wr, 21 op/s 2023-12-11T18:04:10.412 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2023-12-11T18:04:10.412 INFO:teuthology.orchestra.run.smithi059.stdout: "AsyncMessenger::Worker-0": { 2023-12-11T18:04:10.412 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_messages": 336, 2023-12-11T18:04:10.412 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages": 323, 2023-12-11T18:04:10.413 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_bytes": 1820112, 2023-12-11T18:04:10.413 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_bytes": 194704, 2023-12-11T18:04:10.413 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_created_connections": 4, 2023-12-11T18:04:10.413 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_active_connections": 0, 2023-12-11T18:04:10.413 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_total_time": 0.064749002, 2023-12-11T18:04:10.413 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_send_time": 0.021146432, 2023-12-11T18:04:10.413 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_recv_time": 0.043190674, 2023-12-11T18:04:10.413 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_fast_dispatch_time": 0.008972252, 2023-12-11T18:04:10.413 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages_queue_lat": { 2023-12-11T18:04:10.413 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 322, 2023-12-11T18:04:10.413 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.083832117, 2023-12-11T18:04:10.413 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000260348 2023-12-11T18:04:10.413 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:04:10.413 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_handle_ack_lat": { 2023-12-11T18:04:10.413 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2023-12-11T18:04:10.414 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2023-12-11T18:04:10.414 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2023-12-11T18:04:10.414 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:04:10.414 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_encrypted_bytes": 1820112, 2023-12-11T18:04:10.414 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_encrypted_bytes": 194704 2023-12-11T18:04:10.414 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:04:10.414 INFO:teuthology.orchestra.run.smithi059.stdout: "AsyncMessenger::Worker-1": { 2023-12-11T18:04:10.414 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_messages": 3, 2023-12-11T18:04:10.414 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages": 248, 2023-12-11T18:04:10.414 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_bytes": 480, 2023-12-11T18:04:10.414 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_bytes": 274480, 2023-12-11T18:04:10.414 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_created_connections": 6, 2023-12-11T18:04:10.414 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_active_connections": 3, 2023-12-11T18:04:10.415 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_total_time": 0.018717136, 2023-12-11T18:04:10.415 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_send_time": 0.013395025, 2023-12-11T18:04:10.415 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_recv_time": 0.002231007, 2023-12-11T18:04:10.415 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-12-11T18:04:10.415 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages_queue_lat": { 2023-12-11T18:04:10.415 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 245, 2023-12-11T18:04:10.415 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.073062320, 2023-12-11T18:04:10.415 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000298213 2023-12-11T18:04:10.415 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:04:10.415 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_handle_ack_lat": { 2023-12-11T18:04:10.415 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2023-12-11T18:04:10.415 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2023-12-11T18:04:10.415 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2023-12-11T18:04:10.415 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:04:10.415 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_encrypted_bytes": 480, 2023-12-11T18:04:10.416 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_encrypted_bytes": 274480 2023-12-11T18:04:10.416 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:04:10.416 INFO:teuthology.orchestra.run.smithi059.stdout: "AsyncMessenger::Worker-2": { 2023-12-11T18:04:10.416 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_messages": 4, 2023-12-11T18:04:10.416 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages": 4, 2023-12-11T18:04:10.416 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_bytes": 2944, 2023-12-11T18:04:10.416 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_bytes": 448, 2023-12-11T18:04:10.416 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_created_connections": 2, 2023-12-11T18:04:10.416 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_active_connections": 1, 2023-12-11T18:04:10.416 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_total_time": 0.000993785, 2023-12-11T18:04:10.416 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_send_time": 0.000047138, 2023-12-11T18:04:10.416 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_recv_time": 0.000918120, 2023-12-11T18:04:10.416 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-12-11T18:04:10.416 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages_queue_lat": { 2023-12-11T18:04:10.416 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 3, 2023-12-11T18:04:10.416 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000915188, 2023-12-11T18:04:10.417 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000305062 2023-12-11T18:04:10.417 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:04:10.417 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_handle_ack_lat": { 2023-12-11T18:04:10.417 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2023-12-11T18:04:10.417 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2023-12-11T18:04:10.417 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2023-12-11T18:04:10.417 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:04:10.417 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_encrypted_bytes": 2944, 2023-12-11T18:04:10.417 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_encrypted_bytes": 448 2023-12-11T18:04:10.417 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:04:10.417 INFO:teuthology.orchestra.run.smithi059.stdout: "cct": { 2023-12-11T18:04:10.417 INFO:teuthology.orchestra.run.smithi059.stdout: "total_workers": 0, 2023-12-11T18:04:10.417 INFO:teuthology.orchestra.run.smithi059.stdout: "unhealthy_workers": 0 2023-12-11T18:04:10.418 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:04:10.418 INFO:teuthology.orchestra.run.smithi059.stdout: "mempool": { 2023-12-11T18:04:10.418 INFO:teuthology.orchestra.run.smithi059.stdout: "bloom_filter_bytes": 0, 2023-12-11T18:04:10.418 INFO:teuthology.orchestra.run.smithi059.stdout: "bloom_filter_items": 0, 2023-12-11T18:04:10.418 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_alloc_bytes": 0, 2023-12-11T18:04:10.418 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_alloc_items": 0, 2023-12-11T18:04:10.418 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_data_bytes": 0, 2023-12-11T18:04:10.418 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_data_items": 0, 2023-12-11T18:04:10.418 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_onode_bytes": 0, 2023-12-11T18:04:10.418 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_onode_items": 0, 2023-12-11T18:04:10.418 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_meta_bytes": 0, 2023-12-11T18:04:10.418 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_meta_items": 0, 2023-12-11T18:04:10.418 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_other_bytes": 0, 2023-12-11T18:04:10.418 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_other_items": 0, 2023-12-11T18:04:10.419 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_buffer_bytes": 0, 2023-12-11T18:04:10.419 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_buffer_items": 0, 2023-12-11T18:04:10.419 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_extent_bytes": 0, 2023-12-11T18:04:10.419 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_extent_items": 0, 2023-12-11T18:04:10.419 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_blob_bytes": 0, 2023-12-11T18:04:10.419 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_blob_items": 0, 2023-12-11T18:04:10.419 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_shared_blob_bytes": 0, 2023-12-11T18:04:10.419 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_shared_blob_items": 0, 2023-12-11T18:04:10.419 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_inline_bl_bytes": 0, 2023-12-11T18:04:10.419 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_inline_bl_items": 0, 2023-12-11T18:04:10.419 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_fsck_bytes": 0, 2023-12-11T18:04:10.419 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_fsck_items": 0, 2023-12-11T18:04:10.419 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_txc_bytes": 0, 2023-12-11T18:04:10.419 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_txc_items": 0, 2023-12-11T18:04:10.420 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_writing_deferred_bytes": 0, 2023-12-11T18:04:10.420 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_writing_deferred_items": 0, 2023-12-11T18:04:10.420 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_writing_bytes": 0, 2023-12-11T18:04:10.420 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_writing_items": 0, 2023-12-11T18:04:10.420 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_bytes": 0, 2023-12-11T18:04:10.420 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_items": 0, 2023-12-11T18:04:10.420 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_file_reader_bytes": 0, 2023-12-11T18:04:10.420 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_file_reader_items": 0, 2023-12-11T18:04:10.420 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_file_writer_bytes": 0, 2023-12-11T18:04:10.420 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_file_writer_items": 0, 2023-12-11T18:04:10.420 INFO:teuthology.orchestra.run.smithi059.stdout: "buffer_anon_bytes": 148721, 2023-12-11T18:04:10.420 INFO:teuthology.orchestra.run.smithi059.stdout: "buffer_anon_items": 30, 2023-12-11T18:04:10.420 INFO:teuthology.orchestra.run.smithi059.stdout: "buffer_meta_bytes": 0, 2023-12-11T18:04:10.420 INFO:teuthology.orchestra.run.smithi059.stdout: "buffer_meta_items": 0, 2023-12-11T18:04:10.420 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_bytes": 0, 2023-12-11T18:04:10.421 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_items": 0, 2023-12-11T18:04:10.421 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_mapbl_bytes": 0, 2023-12-11T18:04:10.421 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_mapbl_items": 0, 2023-12-11T18:04:10.421 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_pglog_bytes": 0, 2023-12-11T18:04:10.421 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_pglog_items": 0, 2023-12-11T18:04:10.421 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_bytes": 0, 2023-12-11T18:04:10.421 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_items": 0, 2023-12-11T18:04:10.421 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_mapping_bytes": 0, 2023-12-11T18:04:10.421 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_mapping_items": 0, 2023-12-11T18:04:10.421 INFO:teuthology.orchestra.run.smithi059.stdout: "pgmap_bytes": 0, 2023-12-11T18:04:10.421 INFO:teuthology.orchestra.run.smithi059.stdout: "pgmap_items": 0, 2023-12-11T18:04:10.421 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_co_bytes": 928, 2023-12-11T18:04:10.421 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_co_items": 2, 2023-12-11T18:04:10.421 INFO:teuthology.orchestra.run.smithi059.stdout: "unittest_1_bytes": 0, 2023-12-11T18:04:10.421 INFO:teuthology.orchestra.run.smithi059.stdout: "unittest_1_items": 0, 2023-12-11T18:04:10.422 INFO:teuthology.orchestra.run.smithi059.stdout: "unittest_2_bytes": 0, 2023-12-11T18:04:10.422 INFO:teuthology.orchestra.run.smithi059.stdout: "unittest_2_items": 0 2023-12-11T18:04:10.422 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-12-11T18:04:10.422 INFO:teuthology.orchestra.run.smithi059.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-12-11T18:04:10.422 INFO:teuthology.orchestra.run.smithi059.stdout: "val": 0, 2023-12-11T18:04:10.422 INFO:teuthology.orchestra.run.smithi059.stdout: "max": 104857600, 2023-12-11T18:04:10.422 INFO:teuthology.orchestra.run.smithi059.stdout: "get_started": 0, 2023-12-11T18:04:10.422 INFO:teuthology.orchestra.run.smithi059.stdout: "get": 339, 2023-12-11T18:04:10.422 INFO:teuthology.orchestra.run.smithi059.stdout: "get_sum": 1772712, 2023-12-11T18:04:10.422 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_fail": 0, 2023-12-11T18:04:10.422 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_success": 339, 2023-12-11T18:04:10.422 INFO:teuthology.orchestra.run.smithi059.stdout: "take": 0, 2023-12-11T18:04:10.422 INFO:teuthology.orchestra.run.smithi059.stdout: "take_sum": 0, 2023-12-11T18:04:10.422 INFO:teuthology.orchestra.run.smithi059.stdout: "put": 339, 2023-12-11T18:04:10.423 INFO:teuthology.orchestra.run.smithi059.stdout: "put_sum": 1772712, 2023-12-11T18:04:10.423 INFO:teuthology.orchestra.run.smithi059.stdout: "wait": { 2023-12-11T18:04:10.423 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2023-12-11T18:04:10.423 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2023-12-11T18:04:10.423 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2023-12-11T18:04:10.423 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:04:10.423 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-12-11T18:04:10.423 INFO:teuthology.orchestra.run.smithi059.stdout:} 2023-12-11T18:04:10.782 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:04:10 smithi115 ceph-mon[102748]: pgmap v568: 129 pgs: 41 active+clean+snaptrim_wait, 12 active+clean+snaptrim, 76 active+clean; 1.8 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 53 KiB/s wr, 21 op/s 2023-12-11T18:04:10.957 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2023-12-11T18:04:10.957 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2023-12-11T18:04:10.957 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2023-12-11T18:04:10.957 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2023-12-11T18:04:10.957 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2023-12-11T18:04:10.957 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.j 2023-12-11T18:04:10.957 DEBUG:tasks.check_counter:Getting stats from b 2023-12-11T18:04:10.958 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell --fsid d0a07eaa-984b-11ee-95a3-87774f69a715 -- ceph daemon mds.b perf dump 2023-12-11T18:04:11.846 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/mon.b/config 2023-12-11T18:04:12.536 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:04:12 smithi115 ceph-mon[102748]: pgmap v569: 129 pgs: 41 active+clean+snaptrim_wait, 12 active+clean+snaptrim, 76 active+clean; 1.8 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 53 KiB/s wr, 22 op/s 2023-12-11T18:04:12.536 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:04:12 smithi115 ceph-mon[102748]: osdmap e119: 12 total, 12 up, 12 in 2023-12-11T18:04:12.662 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:04:12 smithi059 ceph-mon[98847]: pgmap v569: 129 pgs: 41 active+clean+snaptrim_wait, 12 active+clean+snaptrim, 76 active+clean; 1.8 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 53 KiB/s wr, 22 op/s 2023-12-11T18:04:12.662 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:04:12 smithi059 ceph-mon[98847]: osdmap e119: 12 total, 12 up, 12 in 2023-12-11T18:04:14.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:04:14 smithi059 ceph-mon[98847]: pgmap v571: 129 pgs: 44 active+clean+snaptrim_wait, 14 active+clean+snaptrim, 71 active+clean; 1.8 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 30 KiB/s wr, 9 op/s 2023-12-11T18:04:14.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:04:14 smithi115 ceph-mon[102748]: pgmap v571: 129 pgs: 44 active+clean+snaptrim_wait, 14 active+clean+snaptrim, 71 active+clean; 1.8 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 30 KiB/s wr, 9 op/s 2023-12-11T18:04:16.533 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:04:16 smithi115 ceph-mon[102748]: pgmap v572: 129 pgs: 51 active+clean+snaptrim_wait, 17 active+clean+snaptrim, 61 active+clean; 1.7 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2023-12-11T18:04:16.657 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:04:16 smithi059 ceph-mon[98847]: pgmap v572: 129 pgs: 51 active+clean+snaptrim_wait, 17 active+clean+snaptrim, 61 active+clean; 1.7 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2023-12-11T18:04:18.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:04:18 smithi059 ceph-mon[98847]: pgmap v573: 129 pgs: 59 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 50 active+clean; 1.7 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-12-11T18:04:18.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:04:18 smithi059 ceph-mon[98847]: fsmap cephfs:3 {0=b=up:active,1=k=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-12-11T18:04:18.687 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:04:18 smithi115 ceph-mon[102748]: pgmap v573: 129 pgs: 59 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 50 active+clean; 1.7 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-12-11T18:04:18.687 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:04:18 smithi115 ceph-mon[102748]: fsmap cephfs:3 {0=b=up:active,1=k=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-12-11T18:04:19.155 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 18:04:18 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T18:04:18.676+0000 7f4642beb700 -1 calc_pg_upmaps abort due to max <= 0 2023-12-11T18:04:19.404 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2023-12-11T18:04:19.404 INFO:teuthology.orchestra.run.smithi115.stdout: "AsyncMessenger::Worker-0": { 2023-12-11T18:04:19.404 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_messages": 34106, 2023-12-11T18:04:19.404 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages": 42460, 2023-12-11T18:04:19.404 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_bytes": 32323625, 2023-12-11T18:04:19.404 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_bytes": 147744236, 2023-12-11T18:04:19.404 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_created_connections": 14, 2023-12-11T18:04:19.404 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_active_connections": 5, 2023-12-11T18:04:19.404 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_total_time": 2.675923313, 2023-12-11T18:04:19.404 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_send_time": 1.183201453, 2023-12-11T18:04:19.404 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_recv_time": 1.174433802, 2023-12-11T18:04:19.404 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_fast_dispatch_time": 0.469937902, 2023-12-11T18:04:19.405 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages_queue_lat": { 2023-12-11T18:04:19.405 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 42456, 2023-12-11T18:04:19.405 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 4.817891262, 2023-12-11T18:04:19.405 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000113479 2023-12-11T18:04:19.405 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:19.405 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_handle_ack_lat": { 2023-12-11T18:04:19.405 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 55915, 2023-12-11T18:04:19.405 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.008790919, 2023-12-11T18:04:19.405 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000157 2023-12-11T18:04:19.405 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:19.405 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_encrypted_bytes": 2305008, 2023-12-11T18:04:19.405 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_encrypted_bytes": 445936 2023-12-11T18:04:19.405 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:19.405 INFO:teuthology.orchestra.run.smithi115.stdout: "AsyncMessenger::Worker-1": { 2023-12-11T18:04:19.406 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_messages": 17007, 2023-12-11T18:04:19.406 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages": 16869, 2023-12-11T18:04:19.406 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_bytes": 17197016, 2023-12-11T18:04:19.406 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_bytes": 144255658, 2023-12-11T18:04:19.406 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_created_connections": 16, 2023-12-11T18:04:19.406 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_active_connections": 5, 2023-12-11T18:04:19.406 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_total_time": 1.757647396, 2023-12-11T18:04:19.406 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_send_time": 0.565402046, 2023-12-11T18:04:19.406 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_recv_time": 0.361314797, 2023-12-11T18:04:19.406 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_fast_dispatch_time": 0.787489305, 2023-12-11T18:04:19.406 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages_queue_lat": { 2023-12-11T18:04:19.406 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 16869, 2023-12-11T18:04:19.406 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 5.853330766, 2023-12-11T18:04:19.406 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000346987 2023-12-11T18:04:19.407 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:19.407 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_handle_ack_lat": { 2023-12-11T18:04:19.407 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 2377, 2023-12-11T18:04:19.407 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000486269, 2023-12-11T18:04:19.407 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000204 2023-12-11T18:04:19.407 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:19.407 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_encrypted_bytes": 101376, 2023-12-11T18:04:19.407 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_encrypted_bytes": 568960 2023-12-11T18:04:19.407 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:19.407 INFO:teuthology.orchestra.run.smithi115.stdout: "AsyncMessenger::Worker-2": { 2023-12-11T18:04:19.407 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_messages": 241346, 2023-12-11T18:04:19.407 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages": 352685, 2023-12-11T18:04:19.407 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_bytes": 102686814, 2023-12-11T18:04:19.407 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_bytes": 303329890, 2023-12-11T18:04:19.407 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_created_connections": 17, 2023-12-11T18:04:19.407 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_active_connections": 6, 2023-12-11T18:04:19.408 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_total_time": 11.870087207, 2023-12-11T18:04:19.408 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_send_time": 5.458160723, 2023-12-11T18:04:19.408 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_recv_time": 148.962294546, 2023-12-11T18:04:19.408 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_fast_dispatch_time": 0.795782937, 2023-12-11T18:04:19.408 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages_queue_lat": { 2023-12-11T18:04:19.408 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 352683, 2023-12-11T18:04:19.408 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 67.531883992, 2023-12-11T18:04:19.408 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000191480 2023-12-11T18:04:19.408 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:19.408 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_handle_ack_lat": { 2023-12-11T18:04:19.408 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 395309, 2023-12-11T18:04:19.408 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.063628409, 2023-12-11T18:04:19.408 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000160 2023-12-11T18:04:19.408 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:19.409 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_encrypted_bytes": 77003168, 2023-12-11T18:04:19.409 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_encrypted_bytes": 170568288 2023-12-11T18:04:19.409 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:19.409 INFO:teuthology.orchestra.run.smithi115.stdout: "cct": { 2023-12-11T18:04:19.409 INFO:teuthology.orchestra.run.smithi115.stdout: "total_workers": 1, 2023-12-11T18:04:19.409 INFO:teuthology.orchestra.run.smithi115.stdout: "unhealthy_workers": 0 2023-12-11T18:04:19.409 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:19.409 INFO:teuthology.orchestra.run.smithi115.stdout: "finisher-MDSRank": { 2023-12-11T18:04:19.409 INFO:teuthology.orchestra.run.smithi115.stdout: "queue_len": 0, 2023-12-11T18:04:19.409 INFO:teuthology.orchestra.run.smithi115.stdout: "complete_latency": { 2023-12-11T18:04:19.409 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 7052, 2023-12-11T18:04:19.409 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 71.909490057, 2023-12-11T18:04:19.409 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.010197034 2023-12-11T18:04:19.409 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-11T18:04:19.410 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:19.410 INFO:teuthology.orchestra.run.smithi115.stdout: "finisher-PurgeQueue": { 2023-12-11T18:04:19.410 INFO:teuthology.orchestra.run.smithi115.stdout: "queue_len": 0, 2023-12-11T18:04:19.410 INFO:teuthology.orchestra.run.smithi115.stdout: "complete_latency": { 2023-12-11T18:04:19.410 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 26704, 2023-12-11T18:04:19.410 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 6.753503990, 2023-12-11T18:04:19.410 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000252902 2023-12-11T18:04:19.410 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-11T18:04:19.410 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:19.410 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": { 2023-12-11T18:04:19.410 INFO:teuthology.orchestra.run.smithi115.stdout: "request": 104178, 2023-12-11T18:04:19.410 INFO:teuthology.orchestra.run.smithi115.stdout: "reply": 98581, 2023-12-11T18:04:19.410 INFO:teuthology.orchestra.run.smithi115.stdout: "reply_latency": { 2023-12-11T18:04:19.410 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 98581, 2023-12-11T18:04:19.411 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 1690.078146259, 2023-12-11T18:04:19.411 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.017144055 2023-12-11T18:04:19.411 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:19.411 INFO:teuthology.orchestra.run.smithi115.stdout: "slow_reply": 0, 2023-12-11T18:04:19.411 INFO:teuthology.orchestra.run.smithi115.stdout: "forward": 5596, 2023-12-11T18:04:19.411 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_fetch_complete": 147, 2023-12-11T18:04:19.411 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_fetch_keys": 0, 2023-12-11T18:04:19.411 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_commit": 876, 2023-12-11T18:04:19.411 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_split": 95, 2023-12-11T18:04:19.411 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_merge": 0, 2023-12-11T18:04:19.411 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes": 33695, 2023-12-11T18:04:19.411 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_top": 0, 2023-12-11T18:04:19.411 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_bottom": 5, 2023-12-11T18:04:19.411 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_pin_tail": 33690, 2023-12-11T18:04:19.411 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_pinned": 33695, 2023-12-11T18:04:19.412 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_expired": 82005, 2023-12-11T18:04:19.412 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_with_caps": 3, 2023-12-11T18:04:19.412 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": 3, 2023-12-11T18:04:19.412 INFO:teuthology.orchestra.run.smithi115.stdout: "subtrees": 3, 2023-12-11T18:04:19.412 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse": 171329, 2023-12-11T18:04:19.412 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_hit": 147643, 2023-12-11T18:04:19.412 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_forward": 5573, 2023-12-11T18:04:19.412 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_discover": 744, 2023-12-11T18:04:19.412 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_dir_fetch": 62, 2023-12-11T18:04:19.412 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_remote_ino": 0, 2023-12-11T18:04:19.412 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_lock": 883, 2023-12-11T18:04:19.412 INFO:teuthology.orchestra.run.smithi115.stdout: "load_cent": 105, 2023-12-11T18:04:19.412 INFO:teuthology.orchestra.run.smithi115.stdout: "q": 0, 2023-12-11T18:04:19.412 INFO:teuthology.orchestra.run.smithi115.stdout: "exported": 301, 2023-12-11T18:04:19.412 INFO:teuthology.orchestra.run.smithi115.stdout: "exported_inodes": 67500, 2023-12-11T18:04:19.412 INFO:teuthology.orchestra.run.smithi115.stdout: "imported": 297, 2023-12-11T18:04:19.413 INFO:teuthology.orchestra.run.smithi115.stdout: "imported_inodes": 62769, 2023-12-11T18:04:19.413 INFO:teuthology.orchestra.run.smithi115.stdout: "openino_dir_fetch": 0, 2023-12-11T18:04:19.413 INFO:teuthology.orchestra.run.smithi115.stdout: "openino_backtrace_fetch": 0, 2023-12-11T18:04:19.413 INFO:teuthology.orchestra.run.smithi115.stdout: "openino_peer_discover": 0, 2023-12-11T18:04:19.413 INFO:teuthology.orchestra.run.smithi115.stdout: "root_rfiles": 4, 2023-12-11T18:04:19.413 INFO:teuthology.orchestra.run.smithi115.stdout: "root_rbytes": 246, 2023-12-11T18:04:19.413 INFO:teuthology.orchestra.run.smithi115.stdout: "root_rsnaps": 6, 2023-12-11T18:04:19.413 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_backtrace_fetch": 0, 2023-12-11T18:04:19.413 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_set_tag": 0, 2023-12-11T18:04:19.413 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_backtrace_repaired": 0, 2023-12-11T18:04:19.413 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_inotable_repaired": 0, 2023-12-11T18:04:19.413 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_dir_inodes": 0, 2023-12-11T18:04:19.413 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_dir_base_inodes": 0, 2023-12-11T18:04:19.413 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_dirfrag_rstats": 0, 2023-12-11T18:04:19.413 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_file_inodes": 0, 2023-12-11T18:04:19.413 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_inode_file_caps": 212, 2023-12-11T18:04:19.414 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_revoke": 16354, 2023-12-11T18:04:19.414 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_grant": 36874, 2023-12-11T18:04:19.414 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_trunc": 5, 2023-12-11T18:04:19.414 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_flushsnap_ack": 6, 2023-12-11T18:04:19.414 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_flush_ack": 0, 2023-12-11T18:04:19.414 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_caps": 47165, 2023-12-11T18:04:19.414 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_caps_dirty": 21955, 2023-12-11T18:04:19.414 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_cap_release": 788, 2023-12-11T18:04:19.414 INFO:teuthology.orchestra.run.smithi115.stdout: "process_request_cap_release": 36465 2023-12-11T18:04:19.414 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:19.414 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_cache": { 2023-12-11T18:04:19.414 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_update": 1, 2023-12-11T18:04:19.414 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_update_receipt": 0, 2023-12-11T18:04:19.415 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_try_discover": 0, 2023-12-11T18:04:19.415 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_send_discover": 812, 2023-12-11T18:04:19.415 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_handle_discover": 997, 2023-12-11T18:04:19.415 INFO:teuthology.orchestra.run.smithi115.stdout: "num_strays": 90, 2023-12-11T18:04:19.415 INFO:teuthology.orchestra.run.smithi115.stdout: "num_strays_delayed": 0, 2023-12-11T18:04:19.415 INFO:teuthology.orchestra.run.smithi115.stdout: "num_strays_enqueuing": 0, 2023-12-11T18:04:19.415 INFO:teuthology.orchestra.run.smithi115.stdout: "strays_created": 15133, 2023-12-11T18:04:19.415 INFO:teuthology.orchestra.run.smithi115.stdout: "strays_enqueued": 15043, 2023-12-11T18:04:19.415 INFO:teuthology.orchestra.run.smithi115.stdout: "strays_reintegrated": 0, 2023-12-11T18:04:19.415 INFO:teuthology.orchestra.run.smithi115.stdout: "strays_migrated": 0, 2023-12-11T18:04:19.415 INFO:teuthology.orchestra.run.smithi115.stdout: "num_recovering_processing": 0, 2023-12-11T18:04:19.415 INFO:teuthology.orchestra.run.smithi115.stdout: "num_recovering_enqueued": 0, 2023-12-11T18:04:19.415 INFO:teuthology.orchestra.run.smithi115.stdout: "num_recovering_prioritized": 0, 2023-12-11T18:04:19.415 INFO:teuthology.orchestra.run.smithi115.stdout: "recovery_started": 0, 2023-12-11T18:04:19.416 INFO:teuthology.orchestra.run.smithi115.stdout: "recovery_completed": 0, 2023-12-11T18:04:19.416 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_enqueue_scrub": 0, 2023-12-11T18:04:19.416 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_exportdir": 301, 2023-12-11T18:04:19.416 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_flush": 0, 2023-12-11T18:04:19.416 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_fragmentdir": 95, 2023-12-11T18:04:19.416 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_fragstats": 0, 2023-12-11T18:04:19.416 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_inodestats": 0 2023-12-11T18:04:19.416 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:19.416 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_log": { 2023-12-11T18:04:19.416 INFO:teuthology.orchestra.run.smithi115.stdout: "evlrg": 5, 2023-12-11T18:04:19.416 INFO:teuthology.orchestra.run.smithi115.stdout: "evadd": 105203, 2023-12-11T18:04:19.417 INFO:teuthology.orchestra.run.smithi115.stdout: "evex": 22308, 2023-12-11T18:04:19.417 INFO:teuthology.orchestra.run.smithi115.stdout: "evtrm": 12291, 2023-12-11T18:04:19.417 INFO:teuthology.orchestra.run.smithi115.stdout: "ev": 92912, 2023-12-11T18:04:19.417 INFO:teuthology.orchestra.run.smithi115.stdout: "evexg": 0, 2023-12-11T18:04:19.417 INFO:teuthology.orchestra.run.smithi115.stdout: "evexd": 10017, 2023-12-11T18:04:19.417 INFO:teuthology.orchestra.run.smithi115.stdout: "segadd": 159, 2023-12-11T18:04:19.417 INFO:teuthology.orchestra.run.smithi115.stdout: "segex": 31, 2023-12-11T18:04:19.417 INFO:teuthology.orchestra.run.smithi115.stdout: "segtrm": 17, 2023-12-11T18:04:19.417 INFO:teuthology.orchestra.run.smithi115.stdout: "seg": 142, 2023-12-11T18:04:19.417 INFO:teuthology.orchestra.run.smithi115.stdout: "segmjr": 8, 2023-12-11T18:04:19.417 INFO:teuthology.orchestra.run.smithi115.stdout: "segexg": 0, 2023-12-11T18:04:19.417 INFO:teuthology.orchestra.run.smithi115.stdout: "segexd": 14, 2023-12-11T18:04:19.417 INFO:teuthology.orchestra.run.smithi115.stdout: "expos": 34452427, 2023-12-11T18:04:19.417 INFO:teuthology.orchestra.run.smithi115.stdout: "wrpos": 401555912, 2023-12-11T18:04:19.418 INFO:teuthology.orchestra.run.smithi115.stdout: "rdpos": 0, 2023-12-11T18:04:19.418 INFO:teuthology.orchestra.run.smithi115.stdout: "jlat": { 2023-12-11T18:04:19.418 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 3576, 2023-12-11T18:04:19.418 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 254.890981328, 2023-12-11T18:04:19.418 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.071278238 2023-12-11T18:04:19.418 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:19.418 INFO:teuthology.orchestra.run.smithi115.stdout: "replayed": 0 2023-12-11T18:04:19.418 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:19.418 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_mem": { 2023-12-11T18:04:19.418 INFO:teuthology.orchestra.run.smithi115.stdout: "ino": 15534, 2023-12-11T18:04:19.418 INFO:teuthology.orchestra.run.smithi115.stdout: "ino+": 79237, 2023-12-11T18:04:19.418 INFO:teuthology.orchestra.run.smithi115.stdout: "ino-": 63703, 2023-12-11T18:04:19.418 INFO:teuthology.orchestra.run.smithi115.stdout: "dir": 605, 2023-12-11T18:04:19.419 INFO:teuthology.orchestra.run.smithi115.stdout: "dir+": 2194, 2023-12-11T18:04:19.419 INFO:teuthology.orchestra.run.smithi115.stdout: "dir-": 1589, 2023-12-11T18:04:19.419 INFO:teuthology.orchestra.run.smithi115.stdout: "dn": 33695, 2023-12-11T18:04:19.419 INFO:teuthology.orchestra.run.smithi115.stdout: "dn+": 132078, 2023-12-11T18:04:19.419 INFO:teuthology.orchestra.run.smithi115.stdout: "dn-": 98383, 2023-12-11T18:04:19.419 INFO:teuthology.orchestra.run.smithi115.stdout: "cap": 3, 2023-12-11T18:04:19.419 INFO:teuthology.orchestra.run.smithi115.stdout: "cap+": 45864, 2023-12-11T18:04:19.419 INFO:teuthology.orchestra.run.smithi115.stdout: "cap-": 45861, 2023-12-11T18:04:19.419 INFO:teuthology.orchestra.run.smithi115.stdout: "rss": 191748, 2023-12-11T18:04:19.419 INFO:teuthology.orchestra.run.smithi115.stdout: "heap": 207228 2023-12-11T18:04:19.419 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:19.419 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_server": { 2023-12-11T18:04:19.419 INFO:teuthology.orchestra.run.smithi115.stdout: "dispatch_client_request": 114814, 2023-12-11T18:04:19.419 INFO:teuthology.orchestra.run.smithi115.stdout: "dispatch_server_request": 3591, 2023-12-11T18:04:19.419 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_request": 104178, 2023-12-11T18:04:19.420 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_session": 493, 2023-12-11T18:04:19.420 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_peer_request": 6091, 2023-12-11T18:04:19.420 INFO:teuthology.orchestra.run.smithi115.stdout: "req_create_latency": { 2023-12-11T18:04:19.420 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 17726, 2023-12-11T18:04:19.420 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 67.382968753, 2023-12-11T18:04:19.420 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.003801363 2023-12-11T18:04:19.420 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:19.420 INFO:teuthology.orchestra.run.smithi115.stdout: "req_getattr_latency": { 2023-12-11T18:04:19.420 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 4596, 2023-12-11T18:04:19.420 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 56.506611612, 2023-12-11T18:04:19.420 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.012294737 2023-12-11T18:04:19.420 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:19.420 INFO:teuthology.orchestra.run.smithi115.stdout: "req_getfilelock_latency": { 2023-12-11T18:04:19.420 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-12-11T18:04:19.420 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-12-11T18:04:19.421 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-12-11T18:04:19.421 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:19.421 INFO:teuthology.orchestra.run.smithi115.stdout: "req_link_latency": { 2023-12-11T18:04:19.421 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-12-11T18:04:19.421 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-12-11T18:04:19.421 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-12-11T18:04:19.421 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:19.421 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookup_latency": { 2023-12-11T18:04:19.421 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 40302, 2023-12-11T18:04:19.421 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 1023.857923661, 2023-12-11T18:04:19.421 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.025404643 2023-12-11T18:04:19.422 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:19.422 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookuphash_latency": { 2023-12-11T18:04:19.422 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-12-11T18:04:19.422 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-12-11T18:04:19.422 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-12-11T18:04:19.422 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:19.422 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookupino_latency": { 2023-12-11T18:04:19.422 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 1, 2023-12-11T18:04:19.422 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000184156, 2023-12-11T18:04:19.422 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000184156 2023-12-11T18:04:19.422 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:19.422 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookupname_latency": { 2023-12-11T18:04:19.422 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-12-11T18:04:19.422 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-12-11T18:04:19.423 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-12-11T18:04:19.423 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:19.423 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookupparent_latency": { 2023-12-11T18:04:19.423 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-12-11T18:04:19.423 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-12-11T18:04:19.423 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-12-11T18:04:19.423 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:19.423 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookupsnap_latency": { 2023-12-11T18:04:19.423 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-12-11T18:04:19.423 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-12-11T18:04:19.423 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-12-11T18:04:19.423 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:19.423 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lssnap_latency": { 2023-12-11T18:04:19.423 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 16, 2023-12-11T18:04:19.423 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.005956873, 2023-12-11T18:04:19.424 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000372304 2023-12-11T18:04:19.424 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:19.424 INFO:teuthology.orchestra.run.smithi115.stdout: "req_mkdir_latency": { 2023-12-11T18:04:19.424 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 113, 2023-12-11T18:04:19.424 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 15.875409644, 2023-12-11T18:04:19.424 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.140490350 2023-12-11T18:04:19.424 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:19.424 INFO:teuthology.orchestra.run.smithi115.stdout: "req_mknod_latency": { 2023-12-11T18:04:19.424 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-12-11T18:04:19.424 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-12-11T18:04:19.424 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-12-11T18:04:19.424 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:19.424 INFO:teuthology.orchestra.run.smithi115.stdout: "req_mksnap_latency": { 2023-12-11T18:04:19.424 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 16, 2023-12-11T18:04:19.425 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 149.700631846, 2023-12-11T18:04:19.425 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 9.356289490 2023-12-11T18:04:19.425 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:19.425 INFO:teuthology.orchestra.run.smithi115.stdout: "req_open_latency": { 2023-12-11T18:04:19.425 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 5395, 2023-12-11T18:04:19.425 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 22.302945761, 2023-12-11T18:04:19.425 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.004134002 2023-12-11T18:04:19.425 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:19.425 INFO:teuthology.orchestra.run.smithi115.stdout: "req_readdir_latency": { 2023-12-11T18:04:19.425 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 650, 2023-12-11T18:04:19.425 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 3.691009131, 2023-12-11T18:04:19.425 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.005678475 2023-12-11T18:04:19.425 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:19.425 INFO:teuthology.orchestra.run.smithi115.stdout: "req_rename_latency": { 2023-12-11T18:04:19.426 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 17314, 2023-12-11T18:04:19.426 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 294.217108627, 2023-12-11T18:04:19.426 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.016993017 2023-12-11T18:04:19.426 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:19.426 INFO:teuthology.orchestra.run.smithi115.stdout: "req_renamesnap_latency": { 2023-12-11T18:04:19.426 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-12-11T18:04:19.426 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-12-11T18:04:19.426 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-12-11T18:04:19.426 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:19.426 INFO:teuthology.orchestra.run.smithi115.stdout: "req_snapdiff_latency": { 2023-12-11T18:04:19.426 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-12-11T18:04:19.426 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-12-11T18:04:19.426 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-12-11T18:04:19.427 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:19.427 INFO:teuthology.orchestra.run.smithi115.stdout: "req_rmdir_latency": { 2023-12-11T18:04:19.427 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 94, 2023-12-11T18:04:19.427 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 19.230632563, 2023-12-11T18:04:19.427 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.204581197 2023-12-11T18:04:19.427 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:19.427 INFO:teuthology.orchestra.run.smithi115.stdout: "req_rmsnap_latency": { 2023-12-11T18:04:19.427 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 10, 2023-12-11T18:04:19.427 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 14.961335239, 2023-12-11T18:04:19.427 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 1.496133523 2023-12-11T18:04:19.427 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:19.427 INFO:teuthology.orchestra.run.smithi115.stdout: "req_rmxattr_latency": { 2023-12-11T18:04:19.427 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-12-11T18:04:19.428 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-12-11T18:04:19.428 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-12-11T18:04:19.428 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:19.428 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setattr_latency": { 2023-12-11T18:04:19.428 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 7, 2023-12-11T18:04:19.428 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.040561860, 2023-12-11T18:04:19.428 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.005794551 2023-12-11T18:04:19.428 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:19.428 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setdirlayout_latency": { 2023-12-11T18:04:19.428 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-12-11T18:04:19.428 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-12-11T18:04:19.428 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-12-11T18:04:19.428 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:19.428 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setfilelock_latency": { 2023-12-11T18:04:19.428 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-12-11T18:04:19.429 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-12-11T18:04:19.429 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-12-11T18:04:19.429 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:19.429 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setlayout_latency": { 2023-12-11T18:04:19.429 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-12-11T18:04:19.429 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-12-11T18:04:19.429 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-12-11T18:04:19.429 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:19.429 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setxattr_latency": { 2023-12-11T18:04:19.429 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 7, 2023-12-11T18:04:19.429 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.018981514, 2023-12-11T18:04:19.429 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.002711644 2023-12-11T18:04:19.429 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:19.430 INFO:teuthology.orchestra.run.smithi115.stdout: "req_symlink_latency": { 2023-12-11T18:04:19.430 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 3, 2023-12-11T18:04:19.430 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.001968998, 2023-12-11T18:04:19.430 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000656332 2023-12-11T18:04:19.430 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:19.430 INFO:teuthology.orchestra.run.smithi115.stdout: "req_unlink_latency": { 2023-12-11T18:04:19.430 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 12331, 2023-12-11T18:04:19.430 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 22.283916021, 2023-12-11T18:04:19.430 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.001807145 2023-12-11T18:04:19.430 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:19.430 INFO:teuthology.orchestra.run.smithi115.stdout: "cap_revoke_eviction": 0, 2023-12-11T18:04:19.430 INFO:teuthology.orchestra.run.smithi115.stdout: "cap_acquisition_throttle": 0, 2023-12-11T18:04:19.430 INFO:teuthology.orchestra.run.smithi115.stdout: "req_getvxattr_latency": { 2023-12-11T18:04:19.430 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-12-11T18:04:19.431 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-12-11T18:04:19.431 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-12-11T18:04:19.431 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-11T18:04:19.431 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:19.431 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_sessions": { 2023-12-11T18:04:19.431 INFO:teuthology.orchestra.run.smithi115.stdout: "session_count": 2, 2023-12-11T18:04:19.431 INFO:teuthology.orchestra.run.smithi115.stdout: "session_add": 7, 2023-12-11T18:04:19.431 INFO:teuthology.orchestra.run.smithi115.stdout: "session_remove": 5, 2023-12-11T18:04:19.431 INFO:teuthology.orchestra.run.smithi115.stdout: "sessions_open": 2, 2023-12-11T18:04:19.431 INFO:teuthology.orchestra.run.smithi115.stdout: "sessions_stale": 0, 2023-12-11T18:04:19.431 INFO:teuthology.orchestra.run.smithi115.stdout: "total_load": 7035, 2023-12-11T18:04:19.431 INFO:teuthology.orchestra.run.smithi115.stdout: "average_load": 3517, 2023-12-11T18:04:19.431 INFO:teuthology.orchestra.run.smithi115.stdout: "avg_session_uptime": 779, 2023-12-11T18:04:19.432 INFO:teuthology.orchestra.run.smithi115.stdout: "mdthresh_evicted": 0 2023-12-11T18:04:19.432 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:19.432 INFO:teuthology.orchestra.run.smithi115.stdout: "mempool": { 2023-12-11T18:04:19.432 INFO:teuthology.orchestra.run.smithi115.stdout: "bloom_filter_bytes": 360, 2023-12-11T18:04:19.432 INFO:teuthology.orchestra.run.smithi115.stdout: "bloom_filter_items": 360, 2023-12-11T18:04:19.432 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_alloc_bytes": 0, 2023-12-11T18:04:19.432 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_alloc_items": 0, 2023-12-11T18:04:19.432 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_data_bytes": 0, 2023-12-11T18:04:19.432 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_data_items": 0, 2023-12-11T18:04:19.432 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_onode_bytes": 0, 2023-12-11T18:04:19.432 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_onode_items": 0, 2023-12-11T18:04:19.432 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_meta_bytes": 0, 2023-12-11T18:04:19.432 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_meta_items": 0, 2023-12-11T18:04:19.432 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_other_bytes": 0, 2023-12-11T18:04:19.432 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_other_items": 0, 2023-12-11T18:04:19.432 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_buffer_bytes": 0, 2023-12-11T18:04:19.433 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_buffer_items": 0, 2023-12-11T18:04:19.433 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_extent_bytes": 0, 2023-12-11T18:04:19.433 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_extent_items": 0, 2023-12-11T18:04:19.433 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_blob_bytes": 0, 2023-12-11T18:04:19.433 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_blob_items": 0, 2023-12-11T18:04:19.433 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_shared_blob_bytes": 0, 2023-12-11T18:04:19.433 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_shared_blob_items": 0, 2023-12-11T18:04:19.433 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_inline_bl_bytes": 0, 2023-12-11T18:04:19.433 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_inline_bl_items": 0, 2023-12-11T18:04:19.433 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_fsck_bytes": 0, 2023-12-11T18:04:19.433 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_fsck_items": 0, 2023-12-11T18:04:19.433 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_txc_bytes": 0, 2023-12-11T18:04:19.433 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_txc_items": 0, 2023-12-11T18:04:19.433 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_writing_deferred_bytes": 0, 2023-12-11T18:04:19.433 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_writing_deferred_items": 0, 2023-12-11T18:04:19.434 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_writing_bytes": 0, 2023-12-11T18:04:19.434 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_writing_items": 0, 2023-12-11T18:04:19.434 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_bytes": 0, 2023-12-11T18:04:19.434 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_items": 0, 2023-12-11T18:04:19.434 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_file_reader_bytes": 0, 2023-12-11T18:04:19.434 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_file_reader_items": 0, 2023-12-11T18:04:19.434 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_file_writer_bytes": 0, 2023-12-11T18:04:19.434 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_file_writer_items": 0, 2023-12-11T18:04:19.434 INFO:teuthology.orchestra.run.smithi115.stdout: "buffer_anon_bytes": 1623649, 2023-12-11T18:04:19.434 INFO:teuthology.orchestra.run.smithi115.stdout: "buffer_anon_items": 15979, 2023-12-11T18:04:19.434 INFO:teuthology.orchestra.run.smithi115.stdout: "buffer_meta_bytes": 176, 2023-12-11T18:04:19.434 INFO:teuthology.orchestra.run.smithi115.stdout: "buffer_meta_items": 2, 2023-12-11T18:04:19.434 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_bytes": 0, 2023-12-11T18:04:19.435 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_items": 0, 2023-12-11T18:04:19.435 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_mapbl_bytes": 0, 2023-12-11T18:04:19.435 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_mapbl_items": 0, 2023-12-11T18:04:19.435 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_pglog_bytes": 0, 2023-12-11T18:04:19.435 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_pglog_items": 0, 2023-12-11T18:04:19.435 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_bytes": 10464, 2023-12-11T18:04:19.435 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_items": 226, 2023-12-11T18:04:19.435 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_mapping_bytes": 0, 2023-12-11T18:04:19.435 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_mapping_items": 0, 2023-12-11T18:04:19.435 INFO:teuthology.orchestra.run.smithi115.stdout: "pgmap_bytes": 0, 2023-12-11T18:04:19.435 INFO:teuthology.orchestra.run.smithi115.stdout: "pgmap_items": 0, 2023-12-11T18:04:19.435 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_co_bytes": 60809624, 2023-12-11T18:04:19.435 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_co_items": 714183, 2023-12-11T18:04:19.435 INFO:teuthology.orchestra.run.smithi115.stdout: "unittest_1_bytes": 0, 2023-12-11T18:04:19.435 INFO:teuthology.orchestra.run.smithi115.stdout: "unittest_1_items": 0, 2023-12-11T18:04:19.435 INFO:teuthology.orchestra.run.smithi115.stdout: "unittest_2_bytes": 0, 2023-12-11T18:04:19.436 INFO:teuthology.orchestra.run.smithi115.stdout: "unittest_2_items": 0 2023-12-11T18:04:19.436 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:19.436 INFO:teuthology.orchestra.run.smithi115.stdout: "objecter": { 2023-12-11T18:04:19.436 INFO:teuthology.orchestra.run.smithi115.stdout: "op_active": 0, 2023-12-11T18:04:19.436 INFO:teuthology.orchestra.run.smithi115.stdout: "op_laggy": 0, 2023-12-11T18:04:19.436 INFO:teuthology.orchestra.run.smithi115.stdout: "op_send": 37059, 2023-12-11T18:04:19.436 INFO:teuthology.orchestra.run.smithi115.stdout: "op_send_bytes": 417398659, 2023-12-11T18:04:19.436 INFO:teuthology.orchestra.run.smithi115.stdout: "op_resend": 54, 2023-12-11T18:04:19.436 INFO:teuthology.orchestra.run.smithi115.stdout: "op_reply": 37040, 2023-12-11T18:04:19.436 INFO:teuthology.orchestra.run.smithi115.stdout: "op_latency": { 2023-12-11T18:04:19.436 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 37040, 2023-12-11T18:04:19.436 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 1364.137113168, 2023-12-11T18:04:19.436 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.036828755 2023-12-11T18:04:19.436 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:19.436 INFO:teuthology.orchestra.run.smithi115.stdout: "op_inflight": 0, 2023-12-11T18:04:19.437 INFO:teuthology.orchestra.run.smithi115.stdout: "oplen_avg": { 2023-12-11T18:04:19.437 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 37040, 2023-12-11T18:04:19.437 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 38539 2023-12-11T18:04:19.437 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:19.437 INFO:teuthology.orchestra.run.smithi115.stdout: "op": 37040, 2023-12-11T18:04:19.437 INFO:teuthology.orchestra.run.smithi115.stdout: "op_r": 853, 2023-12-11T18:04:19.437 INFO:teuthology.orchestra.run.smithi115.stdout: "op_w": 36187, 2023-12-11T18:04:19.437 INFO:teuthology.orchestra.run.smithi115.stdout: "op_rmw": 0, 2023-12-11T18:04:19.437 INFO:teuthology.orchestra.run.smithi115.stdout: "op_pg": 0, 2023-12-11T18:04:19.437 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_stat": 0, 2023-12-11T18:04:19.437 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_create": 94, 2023-12-11T18:04:19.437 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_read": 447, 2023-12-11T18:04:19.437 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_write": 4026, 2023-12-11T18:04:19.437 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_writefull": 266, 2023-12-11T18:04:19.438 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_writesame": 0, 2023-12-11T18:04:19.438 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_append": 0, 2023-12-11T18:04:19.438 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_zero": 0, 2023-12-11T18:04:19.438 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_truncate": 93, 2023-12-11T18:04:19.438 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_delete": 30694, 2023-12-11T18:04:19.438 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_mapext": 0, 2023-12-11T18:04:19.438 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_sparse_read": 0, 2023-12-11T18:04:19.438 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_clonerange": 0, 2023-12-11T18:04:19.438 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_getxattr": 4, 2023-12-11T18:04:19.438 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_setxattr": 158, 2023-12-11T18:04:19.438 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_cmpxattr": 0, 2023-12-11T18:04:19.438 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_rmxattr": 0, 2023-12-11T18:04:19.438 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_resetxattrs": 0, 2023-12-11T18:04:19.438 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_call": 0, 2023-12-11T18:04:19.439 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_watch": 0, 2023-12-11T18:04:19.439 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_notify": 0, 2023-12-11T18:04:19.439 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_src_cmpxattr": 0, 2023-12-11T18:04:19.439 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_pgls": 0, 2023-12-11T18:04:19.439 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_pgls_filter": 0, 2023-12-11T18:04:19.439 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_other": 5, 2023-12-11T18:04:19.439 INFO:teuthology.orchestra.run.smithi115.stdout: "linger_active": 0, 2023-12-11T18:04:19.439 INFO:teuthology.orchestra.run.smithi115.stdout: "linger_send": 0, 2023-12-11T18:04:19.439 INFO:teuthology.orchestra.run.smithi115.stdout: "linger_resend": 0, 2023-12-11T18:04:19.439 INFO:teuthology.orchestra.run.smithi115.stdout: "linger_ping": 0, 2023-12-11T18:04:19.439 INFO:teuthology.orchestra.run.smithi115.stdout: "poolop_active": 0, 2023-12-11T18:04:19.439 INFO:teuthology.orchestra.run.smithi115.stdout: "poolop_send": 0, 2023-12-11T18:04:19.439 INFO:teuthology.orchestra.run.smithi115.stdout: "poolop_resend": 0, 2023-12-11T18:04:19.439 INFO:teuthology.orchestra.run.smithi115.stdout: "poolstat_active": 0, 2023-12-11T18:04:19.439 INFO:teuthology.orchestra.run.smithi115.stdout: "poolstat_send": 0, 2023-12-11T18:04:19.440 INFO:teuthology.orchestra.run.smithi115.stdout: "poolstat_resend": 0, 2023-12-11T18:04:19.440 INFO:teuthology.orchestra.run.smithi115.stdout: "statfs_active": 0, 2023-12-11T18:04:19.440 INFO:teuthology.orchestra.run.smithi115.stdout: "statfs_send": 0, 2023-12-11T18:04:19.440 INFO:teuthology.orchestra.run.smithi115.stdout: "statfs_resend": 0, 2023-12-11T18:04:19.440 INFO:teuthology.orchestra.run.smithi115.stdout: "command_active": 0, 2023-12-11T18:04:19.440 INFO:teuthology.orchestra.run.smithi115.stdout: "command_send": 0, 2023-12-11T18:04:19.440 INFO:teuthology.orchestra.run.smithi115.stdout: "command_resend": 0, 2023-12-11T18:04:19.440 INFO:teuthology.orchestra.run.smithi115.stdout: "map_epoch": 119, 2023-12-11T18:04:19.440 INFO:teuthology.orchestra.run.smithi115.stdout: "map_full": 0, 2023-12-11T18:04:19.440 INFO:teuthology.orchestra.run.smithi115.stdout: "map_inc": 38, 2023-12-11T18:04:19.440 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_sessions": 12, 2023-12-11T18:04:19.440 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_session_open": 16, 2023-12-11T18:04:19.440 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_session_close": 4, 2023-12-11T18:04:19.440 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_laggy": 0, 2023-12-11T18:04:19.440 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_wr": 1993, 2023-12-11T18:04:19.441 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_rd": 553, 2023-12-11T18:04:19.441 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_del": 206 2023-12-11T18:04:19.441 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:19.441 INFO:teuthology.orchestra.run.smithi115.stdout: "oft": { 2023-12-11T18:04:19.441 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_total_objs": 1, 2023-12-11T18:04:19.441 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_total_kv_pairs": 0, 2023-12-11T18:04:19.441 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_total_updates": 0, 2023-12-11T18:04:19.441 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_total_removes": 0 2023-12-11T18:04:19.441 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:19.441 INFO:teuthology.orchestra.run.smithi115.stdout: "osd-slow-ops": { 2023-12-11T18:04:19.441 INFO:teuthology.orchestra.run.smithi115.stdout: "slow_ops_count": 9 2023-12-11T18:04:19.441 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:19.441 INFO:teuthology.orchestra.run.smithi115.stdout: "purge_queue": { 2023-12-11T18:04:19.441 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executing_ops": 0, 2023-12-11T18:04:19.442 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executing_ops_high_water": 17, 2023-12-11T18:04:19.442 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executing": 0, 2023-12-11T18:04:19.442 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executing_high_water": 10, 2023-12-11T18:04:19.442 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executed_ops": 30079, 2023-12-11T18:04:19.442 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executed": 15043, 2023-12-11T18:04:19.442 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_item_in_journal": 0 2023-12-11T18:04:19.442 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:19.442 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-12-11T18:04:19.442 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2023-12-11T18:04:19.442 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 104857600, 2023-12-11T18:04:19.442 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2023-12-11T18:04:19.442 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 292455, 2023-12-11T18:04:19.442 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 116356620, 2023-12-11T18:04:19.443 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2023-12-11T18:04:19.443 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 292455, 2023-12-11T18:04:19.443 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 0, 2023-12-11T18:04:19.443 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 0, 2023-12-11T18:04:19.443 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 292455, 2023-12-11T18:04:19.443 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 116356620, 2023-12-11T18:04:19.443 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2023-12-11T18:04:19.443 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-12-11T18:04:19.443 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-12-11T18:04:19.443 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-12-11T18:04:19.443 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-11T18:04:19.443 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:19.443 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-objecter_bytes": { 2023-12-11T18:04:19.443 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2023-12-11T18:04:19.443 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 104857600, 2023-12-11T18:04:19.444 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2023-12-11T18:04:19.444 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 0, 2023-12-11T18:04:19.444 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 0, 2023-12-11T18:04:19.444 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2023-12-11T18:04:19.444 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 0, 2023-12-11T18:04:19.444 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 37040, 2023-12-11T18:04:19.444 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 419571210, 2023-12-11T18:04:19.444 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 37040, 2023-12-11T18:04:19.444 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 419571210, 2023-12-11T18:04:19.444 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2023-12-11T18:04:19.444 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-12-11T18:04:19.444 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-12-11T18:04:19.444 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-12-11T18:04:19.444 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-11T18:04:19.444 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:19.444 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-objecter_ops": { 2023-12-11T18:04:19.445 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2023-12-11T18:04:19.445 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 1024, 2023-12-11T18:04:19.445 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2023-12-11T18:04:19.445 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 0, 2023-12-11T18:04:19.445 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 0, 2023-12-11T18:04:19.445 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2023-12-11T18:04:19.445 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 0, 2023-12-11T18:04:19.445 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 37040, 2023-12-11T18:04:19.445 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 37040, 2023-12-11T18:04:19.445 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 37040, 2023-12-11T18:04:19.445 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 37040, 2023-12-11T18:04:19.445 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2023-12-11T18:04:19.445 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-12-11T18:04:19.445 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-12-11T18:04:19.445 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-12-11T18:04:19.446 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-11T18:04:19.446 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:19.446 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-write_buf_throttle": { 2023-12-11T18:04:19.446 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2023-12-11T18:04:19.446 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 3758096384, 2023-12-11T18:04:19.446 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2023-12-11T18:04:19.446 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 15043, 2023-12-11T18:04:19.446 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 2185527, 2023-12-11T18:04:19.446 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2023-12-11T18:04:19.446 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 15043, 2023-12-11T18:04:19.446 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 0, 2023-12-11T18:04:19.446 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 0, 2023-12-11T18:04:19.446 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 450, 2023-12-11T18:04:19.447 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 2185527, 2023-12-11T18:04:19.447 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2023-12-11T18:04:19.447 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-12-11T18:04:19.447 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-12-11T18:04:19.447 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-12-11T18:04:19.447 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-11T18:04:19.447 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:19.447 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-write_buf_throttle-0x55dd6951c6e0": { 2023-12-11T18:04:19.447 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2023-12-11T18:04:19.447 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 3758096384, 2023-12-11T18:04:19.447 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2023-12-11T18:04:19.447 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 105203, 2023-12-11T18:04:19.448 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 397361608, 2023-12-11T18:04:19.448 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2023-12-11T18:04:19.448 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 105203, 2023-12-11T18:04:19.448 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 0, 2023-12-11T18:04:19.448 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 0, 2023-12-11T18:04:19.448 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 3576, 2023-12-11T18:04:19.448 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 397361608, 2023-12-11T18:04:19.448 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2023-12-11T18:04:19.448 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-12-11T18:04:19.448 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-12-11T18:04:19.448 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-12-11T18:04:19.448 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-11T18:04:19.448 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-11T18:04:19.448 INFO:teuthology.orchestra.run.smithi115.stdout:} 2023-12-11T18:04:20.480 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:04:20 smithi115 ceph-mon[102748]: pgmap v574: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.7 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 204 B/s wr, 5 op/s 2023-12-11T18:04:20.513 INFO:tasks.check_counter:Daemon mds.b mds.exported=301 2023-12-11T18:04:20.513 INFO:tasks.check_counter:Daemon mds.b mds.imported=297 2023-12-11T18:04:20.514 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2023-12-11T18:04:20.514 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=16 2023-12-11T18:04:20.514 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=10 2023-12-11T18:04:20.514 INFO:tasks.check_counter:Daemon mds.b mds.dir_split=95 2023-12-11T18:04:20.514 DEBUG:tasks.check_counter:Getting stats from e 2023-12-11T18:04:20.514 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell --fsid d0a07eaa-984b-11ee-95a3-87774f69a715 -- ceph daemon mds.e perf dump 2023-12-11T18:04:20.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:04:20 smithi059 ceph-mon[98847]: pgmap v574: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.7 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 204 B/s wr, 5 op/s 2023-12-11T18:04:21.677 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/mon.b/config 2023-12-11T18:04:22.532 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:04:22 smithi115 ceph-mon[102748]: pgmap v575: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.7 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-12-11T18:04:22.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:04:22 smithi059 ceph-mon[98847]: pgmap v575: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.7 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-12-11T18:04:24.647 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2023-12-11T18:04:24.647 INFO:teuthology.orchestra.run.smithi115.stdout: "AsyncMessenger::Worker-0": { 2023-12-11T18:04:24.648 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_messages": 1573, 2023-12-11T18:04:24.648 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages": 1571, 2023-12-11T18:04:24.648 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_bytes": 176462386, 2023-12-11T18:04:24.648 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_bytes": 487615, 2023-12-11T18:04:24.648 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_created_connections": 9, 2023-12-11T18:04:24.648 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_active_connections": 3, 2023-12-11T18:04:24.648 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_total_time": 0.206504088, 2023-12-11T18:04:24.648 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_send_time": 0.038154567, 2023-12-11T18:04:24.648 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_recv_time": 0.114117589, 2023-12-11T18:04:24.648 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_fast_dispatch_time": 0.043021228, 2023-12-11T18:04:24.648 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages_queue_lat": { 2023-12-11T18:04:24.648 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 1570, 2023-12-11T18:04:24.649 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.150860648, 2023-12-11T18:04:24.649 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000096089 2023-12-11T18:04:24.649 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:24.649 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_handle_ack_lat": { 2023-12-11T18:04:24.649 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-12-11T18:04:24.649 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-12-11T18:04:24.649 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-12-11T18:04:24.649 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:24.649 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_encrypted_bytes": 2944, 2023-12-11T18:04:24.649 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_encrypted_bytes": 448 2023-12-11T18:04:24.649 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:24.649 INFO:teuthology.orchestra.run.smithi115.stdout: "AsyncMessenger::Worker-1": { 2023-12-11T18:04:24.649 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_messages": 855, 2023-12-11T18:04:24.649 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages": 1102, 2023-12-11T18:04:24.649 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_bytes": 96698411, 2023-12-11T18:04:24.650 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_bytes": 937681, 2023-12-11T18:04:24.650 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_created_connections": 11, 2023-12-11T18:04:24.650 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_active_connections": 5, 2023-12-11T18:04:24.650 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_total_time": 0.142573685, 2023-12-11T18:04:24.650 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_send_time": 0.026702524, 2023-12-11T18:04:24.650 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_recv_time": 0.078106174, 2023-12-11T18:04:24.650 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_fast_dispatch_time": 0.028013574, 2023-12-11T18:04:24.650 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages_queue_lat": { 2023-12-11T18:04:24.650 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 1099, 2023-12-11T18:04:24.650 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.109604443, 2023-12-11T18:04:24.650 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000099731 2023-12-11T18:04:24.650 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:24.650 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_handle_ack_lat": { 2023-12-11T18:04:24.651 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-12-11T18:04:24.651 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-12-11T18:04:24.651 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-12-11T18:04:24.651 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:24.651 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_encrypted_bytes": 480, 2023-12-11T18:04:24.651 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_encrypted_bytes": 672704 2023-12-11T18:04:24.651 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:24.651 INFO:teuthology.orchestra.run.smithi115.stdout: "AsyncMessenger::Worker-2": { 2023-12-11T18:04:24.651 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_messages": 2100, 2023-12-11T18:04:24.651 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages": 2057, 2023-12-11T18:04:24.651 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_bytes": 119963874, 2023-12-11T18:04:24.651 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_bytes": 730039, 2023-12-11T18:04:24.651 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_created_connections": 6, 2023-12-11T18:04:24.651 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_active_connections": 4, 2023-12-11T18:04:24.652 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_total_time": 0.285550075, 2023-12-11T18:04:24.652 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_send_time": 0.060990020, 2023-12-11T18:04:24.652 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_recv_time": 0.151590519, 2023-12-11T18:04:24.652 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_fast_dispatch_time": 0.069522113, 2023-12-11T18:04:24.652 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages_queue_lat": { 2023-12-11T18:04:24.652 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 2056, 2023-12-11T18:04:24.652 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.271359199, 2023-12-11T18:04:24.652 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000131984 2023-12-11T18:04:24.652 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:24.652 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_handle_ack_lat": { 2023-12-11T18:04:24.652 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-12-11T18:04:24.652 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-12-11T18:04:24.652 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-12-11T18:04:24.652 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:24.653 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_encrypted_bytes": 1994464, 2023-12-11T18:04:24.653 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_encrypted_bytes": 207248 2023-12-11T18:04:24.653 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:24.653 INFO:teuthology.orchestra.run.smithi115.stdout: "cct": { 2023-12-11T18:04:24.653 INFO:teuthology.orchestra.run.smithi115.stdout: "total_workers": 1, 2023-12-11T18:04:24.653 INFO:teuthology.orchestra.run.smithi115.stdout: "unhealthy_workers": 0 2023-12-11T18:04:24.653 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:24.653 INFO:teuthology.orchestra.run.smithi115.stdout: "finisher-MDSRank": { 2023-12-11T18:04:24.653 INFO:teuthology.orchestra.run.smithi115.stdout: "queue_len": 0, 2023-12-11T18:04:24.653 INFO:teuthology.orchestra.run.smithi115.stdout: "complete_latency": { 2023-12-11T18:04:24.653 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 6572, 2023-12-11T18:04:24.653 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.193905918, 2023-12-11T18:04:24.653 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000029504 2023-12-11T18:04:24.653 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-11T18:04:24.653 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:24.654 INFO:teuthology.orchestra.run.smithi115.stdout: "finisher-PurgeQueue": { 2023-12-11T18:04:24.654 INFO:teuthology.orchestra.run.smithi115.stdout: "queue_len": 0, 2023-12-11T18:04:24.654 INFO:teuthology.orchestra.run.smithi115.stdout: "complete_latency": { 2023-12-11T18:04:24.654 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-12-11T18:04:24.654 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-12-11T18:04:24.654 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-12-11T18:04:24.654 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-11T18:04:24.654 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:24.654 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": { 2023-12-11T18:04:24.654 INFO:teuthology.orchestra.run.smithi115.stdout: "request": 0, 2023-12-11T18:04:24.654 INFO:teuthology.orchestra.run.smithi115.stdout: "reply": 0, 2023-12-11T18:04:24.654 INFO:teuthology.orchestra.run.smithi115.stdout: "reply_latency": { 2023-12-11T18:04:24.654 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-12-11T18:04:24.655 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-12-11T18:04:24.655 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-12-11T18:04:24.655 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:24.655 INFO:teuthology.orchestra.run.smithi115.stdout: "slow_reply": 0, 2023-12-11T18:04:24.655 INFO:teuthology.orchestra.run.smithi115.stdout: "forward": 0, 2023-12-11T18:04:24.655 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_fetch_complete": 0, 2023-12-11T18:04:24.655 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_fetch_keys": 0, 2023-12-11T18:04:24.655 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_commit": 0, 2023-12-11T18:04:24.655 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_split": 0, 2023-12-11T18:04:24.655 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_merge": 0, 2023-12-11T18:04:24.655 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes": 59905, 2023-12-11T18:04:24.655 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_top": 3238, 2023-12-11T18:04:24.655 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_bottom": 56667, 2023-12-11T18:04:24.656 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_pin_tail": 0, 2023-12-11T18:04:24.656 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_pinned": 55278, 2023-12-11T18:04:24.656 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_expired": 1, 2023-12-11T18:04:24.656 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_with_caps": 0, 2023-12-11T18:04:24.656 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": 0, 2023-12-11T18:04:24.656 INFO:teuthology.orchestra.run.smithi115.stdout: "subtrees": 1, 2023-12-11T18:04:24.656 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse": 0, 2023-12-11T18:04:24.656 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_hit": 0, 2023-12-11T18:04:24.656 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_forward": 0, 2023-12-11T18:04:24.656 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_discover": 0, 2023-12-11T18:04:24.656 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_dir_fetch": 0, 2023-12-11T18:04:24.656 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_remote_ino": 0, 2023-12-11T18:04:24.656 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_lock": 0, 2023-12-11T18:04:24.656 INFO:teuthology.orchestra.run.smithi115.stdout: "load_cent": 0, 2023-12-11T18:04:24.656 INFO:teuthology.orchestra.run.smithi115.stdout: "q": 0, 2023-12-11T18:04:24.657 INFO:teuthology.orchestra.run.smithi115.stdout: "exported": 0, 2023-12-11T18:04:24.657 INFO:teuthology.orchestra.run.smithi115.stdout: "exported_inodes": 0, 2023-12-11T18:04:24.657 INFO:teuthology.orchestra.run.smithi115.stdout: "imported": 0, 2023-12-11T18:04:24.657 INFO:teuthology.orchestra.run.smithi115.stdout: "imported_inodes": 0, 2023-12-11T18:04:24.657 INFO:teuthology.orchestra.run.smithi115.stdout: "openino_dir_fetch": 0, 2023-12-11T18:04:24.657 INFO:teuthology.orchestra.run.smithi115.stdout: "openino_backtrace_fetch": 0, 2023-12-11T18:04:24.657 INFO:teuthology.orchestra.run.smithi115.stdout: "openino_peer_discover": 0, 2023-12-11T18:04:24.657 INFO:teuthology.orchestra.run.smithi115.stdout: "root_rfiles": 0, 2023-12-11T18:04:24.657 INFO:teuthology.orchestra.run.smithi115.stdout: "root_rbytes": 0, 2023-12-11T18:04:24.657 INFO:teuthology.orchestra.run.smithi115.stdout: "root_rsnaps": 0, 2023-12-11T18:04:24.657 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_backtrace_fetch": 0, 2023-12-11T18:04:24.657 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_set_tag": 0, 2023-12-11T18:04:24.657 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_backtrace_repaired": 0, 2023-12-11T18:04:24.657 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_inotable_repaired": 0, 2023-12-11T18:04:24.658 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_dir_inodes": 0, 2023-12-11T18:04:24.658 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_dir_base_inodes": 0, 2023-12-11T18:04:24.658 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_dirfrag_rstats": 0, 2023-12-11T18:04:24.658 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_file_inodes": 0, 2023-12-11T18:04:24.658 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_inode_file_caps": 0, 2023-12-11T18:04:24.658 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_revoke": 0, 2023-12-11T18:04:24.658 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_grant": 0, 2023-12-11T18:04:24.658 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_trunc": 0, 2023-12-11T18:04:24.658 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-12-11T18:04:24.658 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_flush_ack": 0, 2023-12-11T18:04:24.658 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_caps": 0, 2023-12-11T18:04:24.658 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_caps_dirty": 0, 2023-12-11T18:04:24.659 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_cap_release": 0, 2023-12-11T18:04:24.659 INFO:teuthology.orchestra.run.smithi115.stdout: "process_request_cap_release": 0 2023-12-11T18:04:24.659 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:24.659 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_cache": { 2023-12-11T18:04:24.659 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_update": 0, 2023-12-11T18:04:24.659 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_update_receipt": 0, 2023-12-11T18:04:24.659 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_try_discover": 0, 2023-12-11T18:04:24.659 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_send_discover": 0, 2023-12-11T18:04:24.659 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_handle_discover": 0, 2023-12-11T18:04:24.659 INFO:teuthology.orchestra.run.smithi115.stdout: "num_strays": 0, 2023-12-11T18:04:24.659 INFO:teuthology.orchestra.run.smithi115.stdout: "num_strays_delayed": 0, 2023-12-11T18:04:24.659 INFO:teuthology.orchestra.run.smithi115.stdout: "num_strays_enqueuing": 0, 2023-12-11T18:04:24.659 INFO:teuthology.orchestra.run.smithi115.stdout: "strays_created": 0, 2023-12-11T18:04:24.659 INFO:teuthology.orchestra.run.smithi115.stdout: "strays_enqueued": 0, 2023-12-11T18:04:24.659 INFO:teuthology.orchestra.run.smithi115.stdout: "strays_reintegrated": 0, 2023-12-11T18:04:24.660 INFO:teuthology.orchestra.run.smithi115.stdout: "strays_migrated": 0, 2023-12-11T18:04:24.660 INFO:teuthology.orchestra.run.smithi115.stdout: "num_recovering_processing": 0, 2023-12-11T18:04:24.660 INFO:teuthology.orchestra.run.smithi115.stdout: "num_recovering_enqueued": 0, 2023-12-11T18:04:24.660 INFO:teuthology.orchestra.run.smithi115.stdout: "num_recovering_prioritized": 0, 2023-12-11T18:04:24.660 INFO:teuthology.orchestra.run.smithi115.stdout: "recovery_started": 0, 2023-12-11T18:04:24.660 INFO:teuthology.orchestra.run.smithi115.stdout: "recovery_completed": 0, 2023-12-11T18:04:24.660 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_enqueue_scrub": 0, 2023-12-11T18:04:24.660 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_exportdir": 0, 2023-12-11T18:04:24.660 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_flush": 0, 2023-12-11T18:04:24.660 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_fragmentdir": 0, 2023-12-11T18:04:24.660 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_fragstats": 0, 2023-12-11T18:04:24.660 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_inodestats": 0 2023-12-11T18:04:24.660 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:24.660 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_log": { 2023-12-11T18:04:24.660 INFO:teuthology.orchestra.run.smithi115.stdout: "evlrg": 0, 2023-12-11T18:04:24.661 INFO:teuthology.orchestra.run.smithi115.stdout: "evadd": 0, 2023-12-11T18:04:24.661 INFO:teuthology.orchestra.run.smithi115.stdout: "evex": 0, 2023-12-11T18:04:24.661 INFO:teuthology.orchestra.run.smithi115.stdout: "evtrm": 0, 2023-12-11T18:04:24.661 INFO:teuthology.orchestra.run.smithi115.stdout: "ev": 103828, 2023-12-11T18:04:24.661 INFO:teuthology.orchestra.run.smithi115.stdout: "evexg": 0, 2023-12-11T18:04:24.661 INFO:teuthology.orchestra.run.smithi115.stdout: "evexd": 0, 2023-12-11T18:04:24.661 INFO:teuthology.orchestra.run.smithi115.stdout: "segadd": 0, 2023-12-11T18:04:24.661 INFO:teuthology.orchestra.run.smithi115.stdout: "segex": 0, 2023-12-11T18:04:24.661 INFO:teuthology.orchestra.run.smithi115.stdout: "segtrm": 0, 2023-12-11T18:04:24.661 INFO:teuthology.orchestra.run.smithi115.stdout: "seg": 132, 2023-12-11T18:04:24.661 INFO:teuthology.orchestra.run.smithi115.stdout: "segmjr": 10, 2023-12-11T18:04:24.661 INFO:teuthology.orchestra.run.smithi115.stdout: "segexg": 0, 2023-12-11T18:04:24.661 INFO:teuthology.orchestra.run.smithi115.stdout: "segexd": 0, 2023-12-11T18:04:24.661 INFO:teuthology.orchestra.run.smithi115.stdout: "expos": 35472749, 2023-12-11T18:04:24.661 INFO:teuthology.orchestra.run.smithi115.stdout: "wrpos": 394116278, 2023-12-11T18:04:24.662 INFO:teuthology.orchestra.run.smithi115.stdout: "rdpos": 394108956, 2023-12-11T18:04:24.662 INFO:teuthology.orchestra.run.smithi115.stdout: "jlat": { 2023-12-11T18:04:24.662 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-12-11T18:04:24.662 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-12-11T18:04:24.662 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-12-11T18:04:24.662 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:24.662 INFO:teuthology.orchestra.run.smithi115.stdout: "replayed": 103828 2023-12-11T18:04:24.662 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:24.662 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_mem": { 2023-12-11T18:04:24.662 INFO:teuthology.orchestra.run.smithi115.stdout: "ino": 20246, 2023-12-11T18:04:24.662 INFO:teuthology.orchestra.run.smithi115.stdout: "ino+": 71115, 2023-12-11T18:04:24.662 INFO:teuthology.orchestra.run.smithi115.stdout: "ino-": 50869, 2023-12-11T18:04:24.662 INFO:teuthology.orchestra.run.smithi115.stdout: "dir": 745, 2023-12-11T18:04:24.662 INFO:teuthology.orchestra.run.smithi115.stdout: "dir+": 1965, 2023-12-11T18:04:24.663 INFO:teuthology.orchestra.run.smithi115.stdout: "dir-": 1220, 2023-12-11T18:04:24.663 INFO:teuthology.orchestra.run.smithi115.stdout: "dn": 59905, 2023-12-11T18:04:24.663 INFO:teuthology.orchestra.run.smithi115.stdout: "dn+": 110571, 2023-12-11T18:04:24.663 INFO:teuthology.orchestra.run.smithi115.stdout: "dn-": 50666, 2023-12-11T18:04:24.663 INFO:teuthology.orchestra.run.smithi115.stdout: "cap": 0, 2023-12-11T18:04:24.663 INFO:teuthology.orchestra.run.smithi115.stdout: "cap+": 0, 2023-12-11T18:04:24.663 INFO:teuthology.orchestra.run.smithi115.stdout: "cap-": 0, 2023-12-11T18:04:24.663 INFO:teuthology.orchestra.run.smithi115.stdout: "rss": 180704, 2023-12-11T18:04:24.663 INFO:teuthology.orchestra.run.smithi115.stdout: "heap": 207228 2023-12-11T18:04:24.663 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:24.663 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_server": { 2023-12-11T18:04:24.663 INFO:teuthology.orchestra.run.smithi115.stdout: "dispatch_client_request": 0, 2023-12-11T18:04:24.663 INFO:teuthology.orchestra.run.smithi115.stdout: "dispatch_server_request": 0, 2023-12-11T18:04:24.663 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_request": 0, 2023-12-11T18:04:24.664 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_session": 0, 2023-12-11T18:04:24.664 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_peer_request": 0, 2023-12-11T18:04:24.664 INFO:teuthology.orchestra.run.smithi115.stdout: "req_create_latency": { 2023-12-11T18:04:24.664 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-12-11T18:04:24.664 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-12-11T18:04:24.664 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-12-11T18:04:24.664 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:24.664 INFO:teuthology.orchestra.run.smithi115.stdout: "req_getattr_latency": { 2023-12-11T18:04:24.664 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-12-11T18:04:24.664 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-12-11T18:04:24.664 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-12-11T18:04:24.664 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:24.664 INFO:teuthology.orchestra.run.smithi115.stdout: "req_getfilelock_latency": { 2023-12-11T18:04:24.664 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-12-11T18:04:24.665 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-12-11T18:04:24.665 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-12-11T18:04:24.665 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:24.665 INFO:teuthology.orchestra.run.smithi115.stdout: "req_link_latency": { 2023-12-11T18:04:24.665 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-12-11T18:04:24.665 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-12-11T18:04:24.665 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-12-11T18:04:24.665 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:24.665 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookup_latency": { 2023-12-11T18:04:24.665 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-12-11T18:04:24.665 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-12-11T18:04:24.665 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-12-11T18:04:24.665 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:24.665 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookuphash_latency": { 2023-12-11T18:04:24.665 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-12-11T18:04:24.666 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-12-11T18:04:24.666 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-12-11T18:04:24.666 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:24.666 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookupino_latency": { 2023-12-11T18:04:24.666 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-12-11T18:04:24.666 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-12-11T18:04:24.666 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-12-11T18:04:24.666 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:24.666 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookupname_latency": { 2023-12-11T18:04:24.666 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-12-11T18:04:24.666 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-12-11T18:04:24.666 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-12-11T18:04:24.666 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:24.666 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookupparent_latency": { 2023-12-11T18:04:24.667 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-12-11T18:04:24.667 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-12-11T18:04:24.667 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-12-11T18:04:24.667 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:24.667 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookupsnap_latency": { 2023-12-11T18:04:24.667 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-12-11T18:04:24.667 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-12-11T18:04:24.667 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-12-11T18:04:24.667 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:24.667 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lssnap_latency": { 2023-12-11T18:04:24.667 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-12-11T18:04:24.667 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-12-11T18:04:24.667 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-12-11T18:04:24.667 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:24.668 INFO:teuthology.orchestra.run.smithi115.stdout: "req_mkdir_latency": { 2023-12-11T18:04:24.668 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-12-11T18:04:24.668 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-12-11T18:04:24.668 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-12-11T18:04:24.668 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:24.668 INFO:teuthology.orchestra.run.smithi115.stdout: "req_mknod_latency": { 2023-12-11T18:04:24.668 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-12-11T18:04:24.668 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-12-11T18:04:24.668 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-12-11T18:04:24.668 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:24.668 INFO:teuthology.orchestra.run.smithi115.stdout: "req_mksnap_latency": { 2023-12-11T18:04:24.668 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-12-11T18:04:24.668 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-12-11T18:04:24.668 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-12-11T18:04:24.669 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:24.669 INFO:teuthology.orchestra.run.smithi115.stdout: "req_open_latency": { 2023-12-11T18:04:24.669 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-12-11T18:04:24.669 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-12-11T18:04:24.669 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-12-11T18:04:24.669 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:24.669 INFO:teuthology.orchestra.run.smithi115.stdout: "req_readdir_latency": { 2023-12-11T18:04:24.669 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-12-11T18:04:24.669 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-12-11T18:04:24.669 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-12-11T18:04:24.669 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:24.669 INFO:teuthology.orchestra.run.smithi115.stdout: "req_rename_latency": { 2023-12-11T18:04:24.669 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-12-11T18:04:24.669 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-12-11T18:04:24.669 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-12-11T18:04:24.670 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:24.670 INFO:teuthology.orchestra.run.smithi115.stdout: "req_renamesnap_latency": { 2023-12-11T18:04:24.670 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-12-11T18:04:24.670 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-12-11T18:04:24.670 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-12-11T18:04:24.670 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:24.670 INFO:teuthology.orchestra.run.smithi115.stdout: "req_snapdiff_latency": { 2023-12-11T18:04:24.670 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-12-11T18:04:24.670 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-12-11T18:04:24.670 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-12-11T18:04:24.670 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:24.670 INFO:teuthology.orchestra.run.smithi115.stdout: "req_rmdir_latency": { 2023-12-11T18:04:24.670 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-12-11T18:04:24.670 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-12-11T18:04:24.671 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-12-11T18:04:24.671 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:24.671 INFO:teuthology.orchestra.run.smithi115.stdout: "req_rmsnap_latency": { 2023-12-11T18:04:24.671 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-12-11T18:04:24.671 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-12-11T18:04:24.671 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-12-11T18:04:24.671 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:24.671 INFO:teuthology.orchestra.run.smithi115.stdout: "req_rmxattr_latency": { 2023-12-11T18:04:24.671 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-12-11T18:04:24.671 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-12-11T18:04:24.671 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-12-11T18:04:24.671 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:24.671 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setattr_latency": { 2023-12-11T18:04:24.671 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-12-11T18:04:24.672 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-12-11T18:04:24.672 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-12-11T18:04:24.672 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:24.672 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setdirlayout_latency": { 2023-12-11T18:04:24.672 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-12-11T18:04:24.672 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-12-11T18:04:24.672 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-12-11T18:04:24.672 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:24.672 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setfilelock_latency": { 2023-12-11T18:04:24.672 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-12-11T18:04:24.672 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-12-11T18:04:24.672 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-12-11T18:04:24.672 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:24.672 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setlayout_latency": { 2023-12-11T18:04:24.673 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-12-11T18:04:24.673 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-12-11T18:04:24.673 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-12-11T18:04:24.673 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:24.673 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setxattr_latency": { 2023-12-11T18:04:24.673 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-12-11T18:04:24.673 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-12-11T18:04:24.673 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-12-11T18:04:24.673 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:24.673 INFO:teuthology.orchestra.run.smithi115.stdout: "req_symlink_latency": { 2023-12-11T18:04:24.673 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-12-11T18:04:24.673 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-12-11T18:04:24.673 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-12-11T18:04:24.673 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:24.674 INFO:teuthology.orchestra.run.smithi115.stdout: "req_unlink_latency": { 2023-12-11T18:04:24.674 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-12-11T18:04:24.674 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-12-11T18:04:24.674 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-12-11T18:04:24.674 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:24.674 INFO:teuthology.orchestra.run.smithi115.stdout: "cap_revoke_eviction": 0, 2023-12-11T18:04:24.674 INFO:teuthology.orchestra.run.smithi115.stdout: "cap_acquisition_throttle": 0, 2023-12-11T18:04:24.674 INFO:teuthology.orchestra.run.smithi115.stdout: "req_getvxattr_latency": { 2023-12-11T18:04:24.674 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-12-11T18:04:24.674 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-12-11T18:04:24.674 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-12-11T18:04:24.674 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-11T18:04:24.674 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:24.674 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_sessions": { 2023-12-11T18:04:24.675 INFO:teuthology.orchestra.run.smithi115.stdout: "session_count": 2, 2023-12-11T18:04:24.675 INFO:teuthology.orchestra.run.smithi115.stdout: "session_add": 5, 2023-12-11T18:04:24.675 INFO:teuthology.orchestra.run.smithi115.stdout: "session_remove": 3, 2023-12-11T18:04:24.675 INFO:teuthology.orchestra.run.smithi115.stdout: "sessions_open": 2, 2023-12-11T18:04:24.675 INFO:teuthology.orchestra.run.smithi115.stdout: "sessions_stale": 0, 2023-12-11T18:04:24.675 INFO:teuthology.orchestra.run.smithi115.stdout: "total_load": 0, 2023-12-11T18:04:24.675 INFO:teuthology.orchestra.run.smithi115.stdout: "average_load": 0, 2023-12-11T18:04:24.675 INFO:teuthology.orchestra.run.smithi115.stdout: "avg_session_uptime": 5385, 2023-12-11T18:04:24.675 INFO:teuthology.orchestra.run.smithi115.stdout: "mdthresh_evicted": 0 2023-12-11T18:04:24.675 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:24.675 INFO:teuthology.orchestra.run.smithi115.stdout: "mempool": { 2023-12-11T18:04:24.675 INFO:teuthology.orchestra.run.smithi115.stdout: "bloom_filter_bytes": 0, 2023-12-11T18:04:24.675 INFO:teuthology.orchestra.run.smithi115.stdout: "bloom_filter_items": 0, 2023-12-11T18:04:24.675 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_alloc_bytes": 0, 2023-12-11T18:04:24.675 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_alloc_items": 0, 2023-12-11T18:04:24.676 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_data_bytes": 0, 2023-12-11T18:04:24.676 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_data_items": 0, 2023-12-11T18:04:24.676 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_onode_bytes": 0, 2023-12-11T18:04:24.676 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_onode_items": 0, 2023-12-11T18:04:24.676 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_meta_bytes": 0, 2023-12-11T18:04:24.676 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_meta_items": 0, 2023-12-11T18:04:24.676 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_other_bytes": 0, 2023-12-11T18:04:24.676 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_other_items": 0, 2023-12-11T18:04:24.676 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_buffer_bytes": 0, 2023-12-11T18:04:24.676 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_buffer_items": 0, 2023-12-11T18:04:24.676 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_extent_bytes": 0, 2023-12-11T18:04:24.676 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_extent_items": 0, 2023-12-11T18:04:24.676 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_blob_bytes": 0, 2023-12-11T18:04:24.676 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_blob_items": 0, 2023-12-11T18:04:24.677 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_shared_blob_bytes": 0, 2023-12-11T18:04:24.677 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_shared_blob_items": 0, 2023-12-11T18:04:24.677 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_inline_bl_bytes": 0, 2023-12-11T18:04:24.677 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_inline_bl_items": 0, 2023-12-11T18:04:24.677 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_fsck_bytes": 0, 2023-12-11T18:04:24.677 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_fsck_items": 0, 2023-12-11T18:04:24.677 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_txc_bytes": 0, 2023-12-11T18:04:24.677 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_txc_items": 0, 2023-12-11T18:04:24.677 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_writing_deferred_bytes": 0, 2023-12-11T18:04:24.677 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_writing_deferred_items": 0, 2023-12-11T18:04:24.677 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_writing_bytes": 0, 2023-12-11T18:04:24.677 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_writing_items": 0, 2023-12-11T18:04:24.677 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_bytes": 0, 2023-12-11T18:04:24.677 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_items": 0, 2023-12-11T18:04:24.678 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_file_reader_bytes": 0, 2023-12-11T18:04:24.678 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_file_reader_items": 0, 2023-12-11T18:04:24.678 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_file_writer_bytes": 0, 2023-12-11T18:04:24.678 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_file_writer_items": 0, 2023-12-11T18:04:24.678 INFO:teuthology.orchestra.run.smithi115.stdout: "buffer_anon_bytes": 932469, 2023-12-11T18:04:24.678 INFO:teuthology.orchestra.run.smithi115.stdout: "buffer_anon_items": 20674, 2023-12-11T18:04:24.678 INFO:teuthology.orchestra.run.smithi115.stdout: "buffer_meta_bytes": 0, 2023-12-11T18:04:24.678 INFO:teuthology.orchestra.run.smithi115.stdout: "buffer_meta_items": 0, 2023-12-11T18:04:24.678 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_bytes": 0, 2023-12-11T18:04:24.678 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_items": 0, 2023-12-11T18:04:24.678 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_mapbl_bytes": 0, 2023-12-11T18:04:24.678 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_mapbl_items": 0, 2023-12-11T18:04:24.678 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_pglog_bytes": 0, 2023-12-11T18:04:24.678 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_pglog_items": 0, 2023-12-11T18:04:24.679 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_bytes": 10464, 2023-12-11T18:04:24.679 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_items": 226, 2023-12-11T18:04:24.679 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_mapping_bytes": 0, 2023-12-11T18:04:24.679 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_mapping_items": 0, 2023-12-11T18:04:24.679 INFO:teuthology.orchestra.run.smithi115.stdout: "pgmap_bytes": 0, 2023-12-11T18:04:24.679 INFO:teuthology.orchestra.run.smithi115.stdout: "pgmap_items": 0, 2023-12-11T18:04:24.679 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_co_bytes": 89521058, 2023-12-11T18:04:24.679 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_co_items": 884003, 2023-12-11T18:04:24.679 INFO:teuthology.orchestra.run.smithi115.stdout: "unittest_1_bytes": 0, 2023-12-11T18:04:24.679 INFO:teuthology.orchestra.run.smithi115.stdout: "unittest_1_items": 0, 2023-12-11T18:04:24.679 INFO:teuthology.orchestra.run.smithi115.stdout: "unittest_2_bytes": 0, 2023-12-11T18:04:24.679 INFO:teuthology.orchestra.run.smithi115.stdout: "unittest_2_items": 0 2023-12-11T18:04:24.679 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:24.679 INFO:teuthology.orchestra.run.smithi115.stdout: "objecter": { 2023-12-11T18:04:24.680 INFO:teuthology.orchestra.run.smithi115.stdout: "op_active": 0, 2023-12-11T18:04:24.680 INFO:teuthology.orchestra.run.smithi115.stdout: "op_laggy": 0, 2023-12-11T18:04:24.680 INFO:teuthology.orchestra.run.smithi115.stdout: "op_send": 4099, 2023-12-11T18:04:24.680 INFO:teuthology.orchestra.run.smithi115.stdout: "op_send_bytes": 21, 2023-12-11T18:04:24.680 INFO:teuthology.orchestra.run.smithi115.stdout: "op_resend": 3, 2023-12-11T18:04:24.680 INFO:teuthology.orchestra.run.smithi115.stdout: "op_reply": 4099, 2023-12-11T18:04:24.680 INFO:teuthology.orchestra.run.smithi115.stdout: "op_latency": { 2023-12-11T18:04:24.680 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 4099, 2023-12-11T18:04:24.680 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 40.889210763, 2023-12-11T18:04:24.680 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.009975411 2023-12-11T18:04:24.680 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:24.680 INFO:teuthology.orchestra.run.smithi115.stdout: "op_inflight": 0, 2023-12-11T18:04:24.680 INFO:teuthology.orchestra.run.smithi115.stdout: "oplen_avg": { 2023-12-11T18:04:24.680 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 4099, 2023-12-11T18:04:24.680 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 4100 2023-12-11T18:04:24.681 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:24.681 INFO:teuthology.orchestra.run.smithi115.stdout: "op": 4099, 2023-12-11T18:04:24.681 INFO:teuthology.orchestra.run.smithi115.stdout: "op_r": 4099, 2023-12-11T18:04:24.681 INFO:teuthology.orchestra.run.smithi115.stdout: "op_w": 0, 2023-12-11T18:04:24.681 INFO:teuthology.orchestra.run.smithi115.stdout: "op_rmw": 0, 2023-12-11T18:04:24.681 INFO:teuthology.orchestra.run.smithi115.stdout: "op_pg": 0, 2023-12-11T18:04:24.681 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_stat": 2524, 2023-12-11T18:04:24.681 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_create": 0, 2023-12-11T18:04:24.681 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_read": 1573, 2023-12-11T18:04:24.681 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_write": 0, 2023-12-11T18:04:24.681 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_writefull": 0, 2023-12-11T18:04:24.681 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_writesame": 0, 2023-12-11T18:04:24.681 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_append": 0, 2023-12-11T18:04:24.681 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_zero": 0, 2023-12-11T18:04:24.681 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_truncate": 0, 2023-12-11T18:04:24.682 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_delete": 0, 2023-12-11T18:04:24.682 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_mapext": 0, 2023-12-11T18:04:24.682 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_sparse_read": 0, 2023-12-11T18:04:24.682 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_clonerange": 0, 2023-12-11T18:04:24.682 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_getxattr": 1, 2023-12-11T18:04:24.682 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_setxattr": 0, 2023-12-11T18:04:24.682 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_cmpxattr": 0, 2023-12-11T18:04:24.682 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_rmxattr": 0, 2023-12-11T18:04:24.682 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_resetxattrs": 0, 2023-12-11T18:04:24.682 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_call": 0, 2023-12-11T18:04:24.682 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_watch": 0, 2023-12-11T18:04:24.682 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_notify": 0, 2023-12-11T18:04:24.682 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_src_cmpxattr": 0, 2023-12-11T18:04:24.682 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_pgls": 0, 2023-12-11T18:04:24.683 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_pgls_filter": 0, 2023-12-11T18:04:24.683 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_other": 0, 2023-12-11T18:04:24.683 INFO:teuthology.orchestra.run.smithi115.stdout: "linger_active": 0, 2023-12-11T18:04:24.683 INFO:teuthology.orchestra.run.smithi115.stdout: "linger_send": 0, 2023-12-11T18:04:24.683 INFO:teuthology.orchestra.run.smithi115.stdout: "linger_resend": 0, 2023-12-11T18:04:24.683 INFO:teuthology.orchestra.run.smithi115.stdout: "linger_ping": 0, 2023-12-11T18:04:24.683 INFO:teuthology.orchestra.run.smithi115.stdout: "poolop_active": 0, 2023-12-11T18:04:24.683 INFO:teuthology.orchestra.run.smithi115.stdout: "poolop_send": 0, 2023-12-11T18:04:24.683 INFO:teuthology.orchestra.run.smithi115.stdout: "poolop_resend": 0, 2023-12-11T18:04:24.683 INFO:teuthology.orchestra.run.smithi115.stdout: "poolstat_active": 0, 2023-12-11T18:04:24.683 INFO:teuthology.orchestra.run.smithi115.stdout: "poolstat_send": 0, 2023-12-11T18:04:24.683 INFO:teuthology.orchestra.run.smithi115.stdout: "poolstat_resend": 0, 2023-12-11T18:04:24.683 INFO:teuthology.orchestra.run.smithi115.stdout: "statfs_active": 0, 2023-12-11T18:04:24.683 INFO:teuthology.orchestra.run.smithi115.stdout: "statfs_send": 0, 2023-12-11T18:04:24.683 INFO:teuthology.orchestra.run.smithi115.stdout: "statfs_resend": 0, 2023-12-11T18:04:24.684 INFO:teuthology.orchestra.run.smithi115.stdout: "command_active": 0, 2023-12-11T18:04:24.684 INFO:teuthology.orchestra.run.smithi115.stdout: "command_send": 0, 2023-12-11T18:04:24.684 INFO:teuthology.orchestra.run.smithi115.stdout: "command_resend": 0, 2023-12-11T18:04:24.684 INFO:teuthology.orchestra.run.smithi115.stdout: "map_epoch": 119, 2023-12-11T18:04:24.684 INFO:teuthology.orchestra.run.smithi115.stdout: "map_full": 0, 2023-12-11T18:04:24.684 INFO:teuthology.orchestra.run.smithi115.stdout: "map_inc": 33, 2023-12-11T18:04:24.684 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_sessions": 10, 2023-12-11T18:04:24.684 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_session_open": 14, 2023-12-11T18:04:24.684 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_session_close": 4, 2023-12-11T18:04:24.684 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_laggy": 0, 2023-12-11T18:04:24.684 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_wr": 0, 2023-12-11T18:04:24.684 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_rd": 2, 2023-12-11T18:04:24.684 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_del": 0 2023-12-11T18:04:24.685 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:24.685 INFO:teuthology.orchestra.run.smithi115.stdout: "oft": { 2023-12-11T18:04:24.685 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_total_objs": 0, 2023-12-11T18:04:24.685 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_total_kv_pairs": 0, 2023-12-11T18:04:24.685 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_total_updates": 0, 2023-12-11T18:04:24.685 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_total_removes": 0 2023-12-11T18:04:24.685 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:24.685 INFO:teuthology.orchestra.run.smithi115.stdout: "osd-slow-ops": { 2023-12-11T18:04:24.685 INFO:teuthology.orchestra.run.smithi115.stdout: "slow_ops_count": 0 2023-12-11T18:04:24.685 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:24.685 INFO:teuthology.orchestra.run.smithi115.stdout: "purge_queue": { 2023-12-11T18:04:24.685 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executing_ops": 0, 2023-12-11T18:04:24.685 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executing_ops_high_water": 0, 2023-12-11T18:04:24.685 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executing": 0, 2023-12-11T18:04:24.685 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executing_high_water": 0, 2023-12-11T18:04:24.686 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executed_ops": 0, 2023-12-11T18:04:24.686 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executed": 0, 2023-12-11T18:04:24.686 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_item_in_journal": 0 2023-12-11T18:04:24.686 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:24.686 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-12-11T18:04:24.686 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2023-12-11T18:04:24.686 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 104857600, 2023-12-11T18:04:24.686 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2023-12-11T18:04:24.686 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 4524, 2023-12-11T18:04:24.686 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 392693490, 2023-12-11T18:04:24.686 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2023-12-11T18:04:24.686 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 4524, 2023-12-11T18:04:24.686 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 0, 2023-12-11T18:04:24.686 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 0, 2023-12-11T18:04:24.687 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 4524, 2023-12-11T18:04:24.687 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 392693490, 2023-12-11T18:04:24.687 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2023-12-11T18:04:24.687 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-12-11T18:04:24.687 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-12-11T18:04:24.687 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-12-11T18:04:24.687 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-11T18:04:24.687 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:24.687 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-objecter_bytes": { 2023-12-11T18:04:24.687 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2023-12-11T18:04:24.687 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 104857600, 2023-12-11T18:04:24.687 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2023-12-11T18:04:24.687 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 0, 2023-12-11T18:04:24.687 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 0, 2023-12-11T18:04:24.688 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2023-12-11T18:04:24.688 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 0, 2023-12-11T18:04:24.688 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 4099, 2023-12-11T18:04:24.688 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 389983703, 2023-12-11T18:04:24.688 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 4099, 2023-12-11T18:04:24.688 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 389983703, 2023-12-11T18:04:24.688 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2023-12-11T18:04:24.688 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-12-11T18:04:24.688 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-12-11T18:04:24.688 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-12-11T18:04:24.688 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-11T18:04:24.688 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:24.688 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-objecter_ops": { 2023-12-11T18:04:24.688 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2023-12-11T18:04:24.689 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 1024, 2023-12-11T18:04:24.689 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2023-12-11T18:04:24.689 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 0, 2023-12-11T18:04:24.689 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 0, 2023-12-11T18:04:24.689 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2023-12-11T18:04:24.689 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 0, 2023-12-11T18:04:24.689 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 4099, 2023-12-11T18:04:24.689 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 4099, 2023-12-11T18:04:24.689 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 4099, 2023-12-11T18:04:24.689 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 4099, 2023-12-11T18:04:24.689 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2023-12-11T18:04:24.689 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-12-11T18:04:24.689 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-12-11T18:04:24.689 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-12-11T18:04:24.689 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-11T18:04:24.690 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:24.690 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-write_buf_throttle": { 2023-12-11T18:04:24.690 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2023-12-11T18:04:24.690 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 3758096384, 2023-12-11T18:04:24.690 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2023-12-11T18:04:24.690 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 0, 2023-12-11T18:04:24.690 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 0, 2023-12-11T18:04:24.690 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2023-12-11T18:04:24.690 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 0, 2023-12-11T18:04:24.690 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 0, 2023-12-11T18:04:24.690 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 0, 2023-12-11T18:04:24.690 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 0, 2023-12-11T18:04:24.690 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 0, 2023-12-11T18:04:24.690 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2023-12-11T18:04:24.691 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-12-11T18:04:24.691 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-12-11T18:04:24.691 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-12-11T18:04:24.691 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-11T18:04:24.691 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:24.691 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-write_buf_throttle-0x55a2bfa42960": { 2023-12-11T18:04:24.691 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2023-12-11T18:04:24.691 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 3758096384, 2023-12-11T18:04:24.691 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2023-12-11T18:04:24.691 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 0, 2023-12-11T18:04:24.691 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 0, 2023-12-11T18:04:24.691 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2023-12-11T18:04:24.691 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 0, 2023-12-11T18:04:24.691 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 0, 2023-12-11T18:04:24.692 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 0, 2023-12-11T18:04:24.692 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 0, 2023-12-11T18:04:24.692 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 0, 2023-12-11T18:04:24.692 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2023-12-11T18:04:24.692 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-12-11T18:04:24.692 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-12-11T18:04:24.692 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-12-11T18:04:24.692 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-11T18:04:24.692 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-11T18:04:24.692 INFO:teuthology.orchestra.run.smithi115.stdout:} 2023-12-11T18:04:24.696 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:04:24 smithi115 ceph-mon[102748]: pgmap v576: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.7 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 180 B/s wr, 5 op/s 2023-12-11T18:04:24.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:04:24 smithi059 ceph-mon[98847]: pgmap v576: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.7 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 180 B/s wr, 5 op/s 2023-12-11T18:04:25.698 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2023-12-11T18:04:25.698 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2023-12-11T18:04:25.698 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=0 2023-12-11T18:04:25.698 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2023-12-11T18:04:25.698 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2023-12-11T18:04:25.698 INFO:tasks.check_counter:Daemon mds.e mds.dir_split=0 2023-12-11T18:04:25.698 DEBUG:tasks.check_counter:Getting stats from h 2023-12-11T18:04:25.699 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell --fsid d0a07eaa-984b-11ee-95a3-87774f69a715 -- ceph daemon mds.h perf dump 2023-12-11T18:04:26.643 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/mon.b/config 2023-12-11T18:04:26.786 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:04:26 smithi115 ceph-mon[102748]: pgmap v577: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.7 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 6 op/s 2023-12-11T18:04:26.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:04:26 smithi059 ceph-mon[98847]: pgmap v577: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.7 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 6 op/s 2023-12-11T18:04:28.470 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:04:28 smithi115 ceph-mon[102748]: pgmap v578: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.7 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2023-12-11T18:04:28.896 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2023-12-11T18:04:28.896 INFO:teuthology.orchestra.run.smithi115.stdout: "AsyncMessenger::Worker-0": { 2023-12-11T18:04:28.896 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_messages": 49099, 2023-12-11T18:04:28.896 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages": 40268, 2023-12-11T18:04:28.896 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_bytes": 21355065, 2023-12-11T18:04:28.896 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_bytes": 151105388, 2023-12-11T18:04:28.896 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_created_connections": 11, 2023-12-11T18:04:28.896 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_active_connections": 5, 2023-12-11T18:04:28.896 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_total_time": 3.038039105, 2023-12-11T18:04:28.896 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_send_time": 1.218247889, 2023-12-11T18:04:28.896 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_recv_time": 1.349903068, 2023-12-11T18:04:28.896 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_fast_dispatch_time": 0.695693863, 2023-12-11T18:04:28.897 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages_queue_lat": { 2023-12-11T18:04:28.897 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 40269, 2023-12-11T18:04:28.897 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 5.093439195, 2023-12-11T18:04:28.897 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000126485 2023-12-11T18:04:28.897 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:28.897 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_handle_ack_lat": { 2023-12-11T18:04:28.897 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 55134, 2023-12-11T18:04:28.897 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.007756031, 2023-12-11T18:04:28.897 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000140 2023-12-11T18:04:28.897 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:28.897 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_encrypted_bytes": 6592, 2023-12-11T18:04:28.897 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_encrypted_bytes": 10384 2023-12-11T18:04:28.897 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:28.897 INFO:teuthology.orchestra.run.smithi115.stdout: "AsyncMessenger::Worker-1": { 2023-12-11T18:04:28.897 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_messages": 16296, 2023-12-11T18:04:28.897 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages": 15955, 2023-12-11T18:04:28.898 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_bytes": 25931195, 2023-12-11T18:04:28.898 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_bytes": 171150125, 2023-12-11T18:04:28.898 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_created_connections": 19, 2023-12-11T18:04:28.898 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_active_connections": 7, 2023-12-11T18:04:28.898 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_total_time": 1.708821306, 2023-12-11T18:04:28.898 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_send_time": 0.614263905, 2023-12-11T18:04:28.898 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_recv_time": 0.384641877, 2023-12-11T18:04:28.898 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_fast_dispatch_time": 0.706898700, 2023-12-11T18:04:28.898 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages_queue_lat": { 2023-12-11T18:04:28.898 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 15952, 2023-12-11T18:04:28.898 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 5.655003449, 2023-12-11T18:04:28.898 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000354501 2023-12-11T18:04:28.898 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:28.899 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_handle_ack_lat": { 2023-12-11T18:04:28.899 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 2362, 2023-12-11T18:04:28.899 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000396058, 2023-12-11T18:04:28.899 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000167 2023-12-11T18:04:28.899 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:28.899 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_encrypted_bytes": 4576, 2023-12-11T18:04:28.899 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_encrypted_bytes": 601808 2023-12-11T18:04:28.899 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:28.899 INFO:teuthology.orchestra.run.smithi115.stdout: "AsyncMessenger::Worker-2": { 2023-12-11T18:04:28.899 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_messages": 241455, 2023-12-11T18:04:28.899 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages": 325755, 2023-12-11T18:04:28.899 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_bytes": 104302908, 2023-12-11T18:04:28.899 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_bytes": 260174708, 2023-12-11T18:04:28.899 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_created_connections": 12, 2023-12-11T18:04:28.899 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_active_connections": 7, 2023-12-11T18:04:28.899 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_total_time": 10.861261783, 2023-12-11T18:04:28.900 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_send_time": 5.140630716, 2023-12-11T18:04:28.900 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_recv_time": 508.171385662, 2023-12-11T18:04:28.900 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_fast_dispatch_time": 0.434872627, 2023-12-11T18:04:28.900 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages_queue_lat": { 2023-12-11T18:04:28.900 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 325752, 2023-12-11T18:04:28.900 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 52.937936970, 2023-12-11T18:04:28.900 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000162509 2023-12-11T18:04:28.900 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:28.900 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_handle_ack_lat": { 2023-12-11T18:04:28.900 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 373737, 2023-12-11T18:04:28.900 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.057917850, 2023-12-11T18:04:28.900 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000154 2023-12-11T18:04:28.900 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:28.900 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_encrypted_bytes": 76299920, 2023-12-11T18:04:28.900 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_encrypted_bytes": 158694304 2023-12-11T18:04:28.900 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:28.901 INFO:teuthology.orchestra.run.smithi115.stdout: "cct": { 2023-12-11T18:04:28.901 INFO:teuthology.orchestra.run.smithi115.stdout: "total_workers": 1, 2023-12-11T18:04:28.901 INFO:teuthology.orchestra.run.smithi115.stdout: "unhealthy_workers": 0 2023-12-11T18:04:28.901 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:28.901 INFO:teuthology.orchestra.run.smithi115.stdout: "finisher-MDSRank": { 2023-12-11T18:04:28.901 INFO:teuthology.orchestra.run.smithi115.stdout: "queue_len": 0, 2023-12-11T18:04:28.901 INFO:teuthology.orchestra.run.smithi115.stdout: "complete_latency": { 2023-12-11T18:04:28.901 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 7656, 2023-12-11T18:04:28.901 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 67.936248196, 2023-12-11T18:04:28.901 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.008873595 2023-12-11T18:04:28.901 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-11T18:04:28.901 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:28.901 INFO:teuthology.orchestra.run.smithi115.stdout: "finisher-PurgeQueue": { 2023-12-11T18:04:28.901 INFO:teuthology.orchestra.run.smithi115.stdout: "queue_len": 0, 2023-12-11T18:04:28.901 INFO:teuthology.orchestra.run.smithi115.stdout: "complete_latency": { 2023-12-11T18:04:28.901 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 29716, 2023-12-11T18:04:28.902 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 7.680498476, 2023-12-11T18:04:28.902 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000258463 2023-12-11T18:04:28.902 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-11T18:04:28.902 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:28.902 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": { 2023-12-11T18:04:28.902 INFO:teuthology.orchestra.run.smithi115.stdout: "request": 99417, 2023-12-11T18:04:28.902 INFO:teuthology.orchestra.run.smithi115.stdout: "reply": 97274, 2023-12-11T18:04:28.902 INFO:teuthology.orchestra.run.smithi115.stdout: "reply_latency": { 2023-12-11T18:04:28.902 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 97274, 2023-12-11T18:04:28.902 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 3417.805758643, 2023-12-11T18:04:28.902 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.035135861 2023-12-11T18:04:28.902 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:28.902 INFO:teuthology.orchestra.run.smithi115.stdout: "slow_reply": 0, 2023-12-11T18:04:28.902 INFO:teuthology.orchestra.run.smithi115.stdout: "forward": 2143, 2023-12-11T18:04:28.902 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_fetch_complete": 237, 2023-12-11T18:04:28.903 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_fetch_keys": 0, 2023-12-11T18:04:28.903 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_commit": 840, 2023-12-11T18:04:28.903 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_split": 79, 2023-12-11T18:04:28.903 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_merge": 1, 2023-12-11T18:04:28.903 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes": 35694, 2023-12-11T18:04:28.903 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_top": 4526, 2023-12-11T18:04:28.903 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_bottom": 12858, 2023-12-11T18:04:28.903 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_pin_tail": 18310, 2023-12-11T18:04:28.903 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_pinned": 29225, 2023-12-11T18:04:28.903 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_expired": 60143, 2023-12-11T18:04:28.903 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_with_caps": 1, 2023-12-11T18:04:28.903 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": 2, 2023-12-11T18:04:28.903 INFO:teuthology.orchestra.run.smithi115.stdout: "subtrees": 4, 2023-12-11T18:04:28.903 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse": 173337, 2023-12-11T18:04:28.903 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_hit": 147533, 2023-12-11T18:04:28.903 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_forward": 2008, 2023-12-11T18:04:28.904 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_discover": 1164, 2023-12-11T18:04:28.904 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_dir_fetch": 98, 2023-12-11T18:04:28.904 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_remote_ino": 0, 2023-12-11T18:04:28.904 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_lock": 1472, 2023-12-11T18:04:28.904 INFO:teuthology.orchestra.run.smithi115.stdout: "load_cent": 0, 2023-12-11T18:04:28.904 INFO:teuthology.orchestra.run.smithi115.stdout: "q": 0, 2023-12-11T18:04:28.904 INFO:teuthology.orchestra.run.smithi115.stdout: "exported": 234, 2023-12-11T18:04:28.904 INFO:teuthology.orchestra.run.smithi115.stdout: "exported_inodes": 47592, 2023-12-11T18:04:28.904 INFO:teuthology.orchestra.run.smithi115.stdout: "imported": 264, 2023-12-11T18:04:28.904 INFO:teuthology.orchestra.run.smithi115.stdout: "imported_inodes": 50726, 2023-12-11T18:04:28.904 INFO:teuthology.orchestra.run.smithi115.stdout: "openino_dir_fetch": 0, 2023-12-11T18:04:28.904 INFO:teuthology.orchestra.run.smithi115.stdout: "openino_backtrace_fetch": 0, 2023-12-11T18:04:28.904 INFO:teuthology.orchestra.run.smithi115.stdout: "openino_peer_discover": 0, 2023-12-11T18:04:28.904 INFO:teuthology.orchestra.run.smithi115.stdout: "root_rfiles": 4, 2023-12-11T18:04:28.904 INFO:teuthology.orchestra.run.smithi115.stdout: "root_rbytes": 246, 2023-12-11T18:04:28.904 INFO:teuthology.orchestra.run.smithi115.stdout: "root_rsnaps": 0, 2023-12-11T18:04:28.905 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_backtrace_fetch": 0, 2023-12-11T18:04:28.905 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_set_tag": 0, 2023-12-11T18:04:28.905 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_backtrace_repaired": 0, 2023-12-11T18:04:28.905 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_inotable_repaired": 0, 2023-12-11T18:04:28.905 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_dir_inodes": 0, 2023-12-11T18:04:28.905 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_dir_base_inodes": 0, 2023-12-11T18:04:28.905 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_dirfrag_rstats": 0, 2023-12-11T18:04:28.905 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_file_inodes": 0, 2023-12-11T18:04:28.905 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_inode_file_caps": 319, 2023-12-11T18:04:28.905 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_revoke": 16018, 2023-12-11T18:04:28.905 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_grant": 37751, 2023-12-11T18:04:28.905 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_trunc": 0, 2023-12-11T18:04:28.905 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_flushsnap_ack": 1, 2023-12-11T18:04:28.905 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_flush_ack": 0, 2023-12-11T18:04:28.905 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_caps": 49399, 2023-12-11T18:04:28.906 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_caps_dirty": 19325, 2023-12-11T18:04:28.906 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_cap_release": 618, 2023-12-11T18:04:28.906 INFO:teuthology.orchestra.run.smithi115.stdout: "process_request_cap_release": 37159 2023-12-11T18:04:28.906 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:28.906 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_cache": { 2023-12-11T18:04:28.906 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_update": 0, 2023-12-11T18:04:28.906 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_update_receipt": 9, 2023-12-11T18:04:28.906 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_try_discover": 6, 2023-12-11T18:04:28.906 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_send_discover": 1245, 2023-12-11T18:04:28.906 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_handle_discover": 1204, 2023-12-11T18:04:28.906 INFO:teuthology.orchestra.run.smithi115.stdout: "num_strays": 97, 2023-12-11T18:04:28.906 INFO:teuthology.orchestra.run.smithi115.stdout: "num_strays_delayed": 0, 2023-12-11T18:04:28.906 INFO:teuthology.orchestra.run.smithi115.stdout: "num_strays_enqueuing": 0, 2023-12-11T18:04:28.906 INFO:teuthology.orchestra.run.smithi115.stdout: "strays_created": 17316, 2023-12-11T18:04:28.907 INFO:teuthology.orchestra.run.smithi115.stdout: "strays_enqueued": 17219, 2023-12-11T18:04:28.907 INFO:teuthology.orchestra.run.smithi115.stdout: "strays_reintegrated": 0, 2023-12-11T18:04:28.907 INFO:teuthology.orchestra.run.smithi115.stdout: "strays_migrated": 0, 2023-12-11T18:04:28.907 INFO:teuthology.orchestra.run.smithi115.stdout: "num_recovering_processing": 0, 2023-12-11T18:04:28.907 INFO:teuthology.orchestra.run.smithi115.stdout: "num_recovering_enqueued": 0, 2023-12-11T18:04:28.907 INFO:teuthology.orchestra.run.smithi115.stdout: "num_recovering_prioritized": 0, 2023-12-11T18:04:28.907 INFO:teuthology.orchestra.run.smithi115.stdout: "recovery_started": 0, 2023-12-11T18:04:28.907 INFO:teuthology.orchestra.run.smithi115.stdout: "recovery_completed": 0, 2023-12-11T18:04:28.907 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_enqueue_scrub": 0, 2023-12-11T18:04:28.907 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_exportdir": 238, 2023-12-11T18:04:28.907 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_flush": 0, 2023-12-11T18:04:28.907 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_fragmentdir": 80, 2023-12-11T18:04:28.907 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_fragstats": 0, 2023-12-11T18:04:28.907 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_inodestats": 0 2023-12-11T18:04:28.907 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:28.908 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_log": { 2023-12-11T18:04:28.908 INFO:teuthology.orchestra.run.smithi115.stdout: "evlrg": 5, 2023-12-11T18:04:28.908 INFO:teuthology.orchestra.run.smithi115.stdout: "evadd": 103828, 2023-12-11T18:04:28.908 INFO:teuthology.orchestra.run.smithi115.stdout: "evex": 15551, 2023-12-11T18:04:28.908 INFO:teuthology.orchestra.run.smithi115.stdout: "evtrm": 12291, 2023-12-11T18:04:28.908 INFO:teuthology.orchestra.run.smithi115.stdout: "ev": 91537, 2023-12-11T18:04:28.908 INFO:teuthology.orchestra.run.smithi115.stdout: "evexg": 0, 2023-12-11T18:04:28.908 INFO:teuthology.orchestra.run.smithi115.stdout: "evexd": 3260, 2023-12-11T18:04:28.908 INFO:teuthology.orchestra.run.smithi115.stdout: "segadd": 148, 2023-12-11T18:04:28.908 INFO:teuthology.orchestra.run.smithi115.stdout: "segex": 20, 2023-12-11T18:04:28.908 INFO:teuthology.orchestra.run.smithi115.stdout: "segtrm": 16, 2023-12-11T18:04:28.908 INFO:teuthology.orchestra.run.smithi115.stdout: "seg": 132, 2023-12-11T18:04:28.908 INFO:teuthology.orchestra.run.smithi115.stdout: "segmjr": 8, 2023-12-11T18:04:28.908 INFO:teuthology.orchestra.run.smithi115.stdout: "segexg": 0, 2023-12-11T18:04:28.908 INFO:teuthology.orchestra.run.smithi115.stdout: "segexd": 4, 2023-12-11T18:04:28.908 INFO:teuthology.orchestra.run.smithi115.stdout: "expos": 35472749, 2023-12-11T18:04:28.909 INFO:teuthology.orchestra.run.smithi115.stdout: "wrpos": 394116278, 2023-12-11T18:04:28.909 INFO:teuthology.orchestra.run.smithi115.stdout: "rdpos": 0, 2023-12-11T18:04:28.909 INFO:teuthology.orchestra.run.smithi115.stdout: "jlat": { 2023-12-11T18:04:28.909 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 3583, 2023-12-11T18:04:28.909 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 175.540645988, 2023-12-11T18:04:28.909 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.048992644 2023-12-11T18:04:28.909 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:28.909 INFO:teuthology.orchestra.run.smithi115.stdout: "replayed": 0 2023-12-11T18:04:28.909 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:28.909 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_mem": { 2023-12-11T18:04:28.909 INFO:teuthology.orchestra.run.smithi115.stdout: "ino": 18752, 2023-12-11T18:04:28.909 INFO:teuthology.orchestra.run.smithi115.stdout: "ino+": 72404, 2023-12-11T18:04:28.909 INFO:teuthology.orchestra.run.smithi115.stdout: "ino-": 53652, 2023-12-11T18:04:28.909 INFO:teuthology.orchestra.run.smithi115.stdout: "dir": 756, 2023-12-11T18:04:28.909 INFO:teuthology.orchestra.run.smithi115.stdout: "dir+": 2020, 2023-12-11T18:04:28.909 INFO:teuthology.orchestra.run.smithi115.stdout: "dir-": 1264, 2023-12-11T18:04:28.910 INFO:teuthology.orchestra.run.smithi115.stdout: "dn": 35694, 2023-12-11T18:04:28.910 INFO:teuthology.orchestra.run.smithi115.stdout: "dn+": 114786, 2023-12-11T18:04:28.910 INFO:teuthology.orchestra.run.smithi115.stdout: "dn-": 79092, 2023-12-11T18:04:28.910 INFO:teuthology.orchestra.run.smithi115.stdout: "cap": 2, 2023-12-11T18:04:28.910 INFO:teuthology.orchestra.run.smithi115.stdout: "cap+": 34796, 2023-12-11T18:04:28.910 INFO:teuthology.orchestra.run.smithi115.stdout: "cap-": 34794, 2023-12-11T18:04:28.910 INFO:teuthology.orchestra.run.smithi115.stdout: "rss": 232976, 2023-12-11T18:04:28.910 INFO:teuthology.orchestra.run.smithi115.stdout: "heap": 199036 2023-12-11T18:04:28.910 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:28.910 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_server": { 2023-12-11T18:04:28.910 INFO:teuthology.orchestra.run.smithi115.stdout: "dispatch_client_request": 107977, 2023-12-11T18:04:28.910 INFO:teuthology.orchestra.run.smithi115.stdout: "dispatch_server_request": 6856, 2023-12-11T18:04:28.910 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_request": 99417, 2023-12-11T18:04:28.910 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_session": 541, 2023-12-11T18:04:28.910 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_peer_request": 10661, 2023-12-11T18:04:28.911 INFO:teuthology.orchestra.run.smithi115.stdout: "req_create_latency": { 2023-12-11T18:04:28.911 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 14414, 2023-12-11T18:04:28.911 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 92.674722194, 2023-12-11T18:04:28.911 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.006429493 2023-12-11T18:04:28.911 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:28.911 INFO:teuthology.orchestra.run.smithi115.stdout: "req_getattr_latency": { 2023-12-11T18:04:28.911 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 329, 2023-12-11T18:04:28.911 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 76.199589170, 2023-12-11T18:04:28.911 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.231609693 2023-12-11T18:04:28.911 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:28.911 INFO:teuthology.orchestra.run.smithi115.stdout: "req_getfilelock_latency": { 2023-12-11T18:04:28.911 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-12-11T18:04:28.911 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-12-11T18:04:28.911 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-12-11T18:04:28.912 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:28.912 INFO:teuthology.orchestra.run.smithi115.stdout: "req_link_latency": { 2023-12-11T18:04:28.912 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-12-11T18:04:28.912 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-12-11T18:04:28.912 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-12-11T18:04:28.912 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:28.912 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookup_latency": { 2023-12-11T18:04:28.912 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 49535, 2023-12-11T18:04:28.912 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 2767.264864554, 2023-12-11T18:04:28.912 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.055864840 2023-12-11T18:04:28.912 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:28.912 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookuphash_latency": { 2023-12-11T18:04:28.912 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-12-11T18:04:28.912 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-12-11T18:04:28.912 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-12-11T18:04:28.912 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:28.913 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookupino_latency": { 2023-12-11T18:04:28.913 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 1, 2023-12-11T18:04:28.913 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000137948, 2023-12-11T18:04:28.913 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000137948 2023-12-11T18:04:28.913 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:28.913 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookupname_latency": { 2023-12-11T18:04:28.913 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-12-11T18:04:28.913 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-12-11T18:04:28.913 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-12-11T18:04:28.913 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:28.913 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookupparent_latency": { 2023-12-11T18:04:28.913 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-12-11T18:04:28.913 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-12-11T18:04:28.913 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-12-11T18:04:28.913 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:28.914 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookupsnap_latency": { 2023-12-11T18:04:28.914 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-12-11T18:04:28.914 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-12-11T18:04:28.914 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-12-11T18:04:28.914 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:28.914 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lssnap_latency": { 2023-12-11T18:04:28.914 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-12-11T18:04:28.914 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-12-11T18:04:28.914 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-12-11T18:04:28.914 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:28.914 INFO:teuthology.orchestra.run.smithi115.stdout: "req_mkdir_latency": { 2023-12-11T18:04:28.914 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 63, 2023-12-11T18:04:28.914 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.886608056, 2023-12-11T18:04:28.914 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.014073143 2023-12-11T18:04:28.914 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:28.915 INFO:teuthology.orchestra.run.smithi115.stdout: "req_mknod_latency": { 2023-12-11T18:04:28.915 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-12-11T18:04:28.915 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-12-11T18:04:28.915 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-12-11T18:04:28.915 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:28.915 INFO:teuthology.orchestra.run.smithi115.stdout: "req_mksnap_latency": { 2023-12-11T18:04:28.915 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-12-11T18:04:28.915 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-12-11T18:04:28.915 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-12-11T18:04:28.915 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:28.915 INFO:teuthology.orchestra.run.smithi115.stdout: "req_open_latency": { 2023-12-11T18:04:28.915 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 3908, 2023-12-11T18:04:28.915 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 22.383052205, 2023-12-11T18:04:28.915 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.005727495 2023-12-11T18:04:28.916 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:28.916 INFO:teuthology.orchestra.run.smithi115.stdout: "req_readdir_latency": { 2023-12-11T18:04:28.916 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 737, 2023-12-11T18:04:28.916 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 2.660973492, 2023-12-11T18:04:28.916 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.003610547 2023-12-11T18:04:28.916 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:28.916 INFO:teuthology.orchestra.run.smithi115.stdout: "req_rename_latency": { 2023-12-11T18:04:28.916 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 14446, 2023-12-11T18:04:28.916 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 401.653207494, 2023-12-11T18:04:28.916 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.027803766 2023-12-11T18:04:28.916 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:28.916 INFO:teuthology.orchestra.run.smithi115.stdout: "req_renamesnap_latency": { 2023-12-11T18:04:28.916 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-12-11T18:04:28.916 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-12-11T18:04:28.916 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-12-11T18:04:28.916 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:28.917 INFO:teuthology.orchestra.run.smithi115.stdout: "req_snapdiff_latency": { 2023-12-11T18:04:28.917 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-12-11T18:04:28.917 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-12-11T18:04:28.917 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-12-11T18:04:28.917 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:28.917 INFO:teuthology.orchestra.run.smithi115.stdout: "req_rmdir_latency": { 2023-12-11T18:04:28.917 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 97, 2023-12-11T18:04:28.917 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 29.688971379, 2023-12-11T18:04:28.917 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.306071869 2023-12-11T18:04:28.917 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:28.917 INFO:teuthology.orchestra.run.smithi115.stdout: "req_rmsnap_latency": { 2023-12-11T18:04:28.917 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-12-11T18:04:28.917 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-12-11T18:04:28.917 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-12-11T18:04:28.917 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:28.918 INFO:teuthology.orchestra.run.smithi115.stdout: "req_rmxattr_latency": { 2023-12-11T18:04:28.918 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-12-11T18:04:28.918 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-12-11T18:04:28.918 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-12-11T18:04:28.918 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:28.918 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setattr_latency": { 2023-12-11T18:04:28.918 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-12-11T18:04:28.918 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-12-11T18:04:28.918 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-12-11T18:04:28.918 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:28.918 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setdirlayout_latency": { 2023-12-11T18:04:28.918 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-12-11T18:04:28.918 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-12-11T18:04:28.918 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-12-11T18:04:28.918 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:28.919 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setfilelock_latency": { 2023-12-11T18:04:28.919 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-12-11T18:04:28.919 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-12-11T18:04:28.919 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-12-11T18:04:28.919 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:28.919 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setlayout_latency": { 2023-12-11T18:04:28.919 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-12-11T18:04:28.919 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-12-11T18:04:28.919 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-12-11T18:04:28.919 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:28.919 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setxattr_latency": { 2023-12-11T18:04:28.919 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-12-11T18:04:28.919 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-12-11T18:04:28.919 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-12-11T18:04:28.919 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:28.920 INFO:teuthology.orchestra.run.smithi115.stdout: "req_symlink_latency": { 2023-12-11T18:04:28.920 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-12-11T18:04:28.920 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-12-11T18:04:28.920 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-12-11T18:04:28.920 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:28.920 INFO:teuthology.orchestra.run.smithi115.stdout: "req_unlink_latency": { 2023-12-11T18:04:28.920 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 13744, 2023-12-11T18:04:28.920 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 24.393632151, 2023-12-11T18:04:28.920 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.001774856 2023-12-11T18:04:28.920 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:28.920 INFO:teuthology.orchestra.run.smithi115.stdout: "cap_revoke_eviction": 0, 2023-12-11T18:04:28.920 INFO:teuthology.orchestra.run.smithi115.stdout: "cap_acquisition_throttle": 0, 2023-12-11T18:04:28.920 INFO:teuthology.orchestra.run.smithi115.stdout: "req_getvxattr_latency": { 2023-12-11T18:04:28.920 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-12-11T18:04:28.920 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-12-11T18:04:28.920 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-12-11T18:04:28.921 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-11T18:04:28.921 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:28.921 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_sessions": { 2023-12-11T18:04:28.921 INFO:teuthology.orchestra.run.smithi115.stdout: "session_count": 2, 2023-12-11T18:04:28.921 INFO:teuthology.orchestra.run.smithi115.stdout: "session_add": 5, 2023-12-11T18:04:28.921 INFO:teuthology.orchestra.run.smithi115.stdout: "session_remove": 3, 2023-12-11T18:04:28.921 INFO:teuthology.orchestra.run.smithi115.stdout: "sessions_open": 2, 2023-12-11T18:04:28.921 INFO:teuthology.orchestra.run.smithi115.stdout: "sessions_stale": 0, 2023-12-11T18:04:28.921 INFO:teuthology.orchestra.run.smithi115.stdout: "total_load": 2883, 2023-12-11T18:04:28.921 INFO:teuthology.orchestra.run.smithi115.stdout: "average_load": 1441, 2023-12-11T18:04:28.921 INFO:teuthology.orchestra.run.smithi115.stdout: "avg_session_uptime": 784, 2023-12-11T18:04:28.921 INFO:teuthology.orchestra.run.smithi115.stdout: "mdthresh_evicted": 0 2023-12-11T18:04:28.921 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:28.921 INFO:teuthology.orchestra.run.smithi115.stdout: "mempool": { 2023-12-11T18:04:28.921 INFO:teuthology.orchestra.run.smithi115.stdout: "bloom_filter_bytes": 0, 2023-12-11T18:04:28.921 INFO:teuthology.orchestra.run.smithi115.stdout: "bloom_filter_items": 0, 2023-12-11T18:04:28.922 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_alloc_bytes": 0, 2023-12-11T18:04:28.922 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_alloc_items": 0, 2023-12-11T18:04:28.922 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_data_bytes": 0, 2023-12-11T18:04:28.922 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_data_items": 0, 2023-12-11T18:04:28.922 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_onode_bytes": 0, 2023-12-11T18:04:28.922 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_onode_items": 0, 2023-12-11T18:04:28.922 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_meta_bytes": 0, 2023-12-11T18:04:28.922 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_meta_items": 0, 2023-12-11T18:04:28.922 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_other_bytes": 0, 2023-12-11T18:04:28.922 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_other_items": 0, 2023-12-11T18:04:28.922 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_buffer_bytes": 0, 2023-12-11T18:04:28.922 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_buffer_items": 0, 2023-12-11T18:04:28.922 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_extent_bytes": 0, 2023-12-11T18:04:28.922 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_extent_items": 0, 2023-12-11T18:04:28.922 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_blob_bytes": 0, 2023-12-11T18:04:28.922 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_blob_items": 0, 2023-12-11T18:04:28.923 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_shared_blob_bytes": 0, 2023-12-11T18:04:28.923 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_shared_blob_items": 0, 2023-12-11T18:04:28.923 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_inline_bl_bytes": 0, 2023-12-11T18:04:28.923 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_inline_bl_items": 0, 2023-12-11T18:04:28.923 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_fsck_bytes": 0, 2023-12-11T18:04:28.923 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_fsck_items": 0, 2023-12-11T18:04:28.923 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_txc_bytes": 0, 2023-12-11T18:04:28.923 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_txc_items": 0, 2023-12-11T18:04:28.923 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_writing_deferred_bytes": 0, 2023-12-11T18:04:28.923 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_writing_deferred_items": 0, 2023-12-11T18:04:28.923 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_writing_bytes": 0, 2023-12-11T18:04:28.923 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_writing_items": 0, 2023-12-11T18:04:28.923 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_bytes": 0, 2023-12-11T18:04:28.923 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_items": 0, 2023-12-11T18:04:28.923 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_file_reader_bytes": 0, 2023-12-11T18:04:28.924 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_file_reader_items": 0, 2023-12-11T18:04:28.924 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_file_writer_bytes": 0, 2023-12-11T18:04:28.924 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_file_writer_items": 0, 2023-12-11T18:04:28.924 INFO:teuthology.orchestra.run.smithi115.stdout: "buffer_anon_bytes": 1735071, 2023-12-11T18:04:28.924 INFO:teuthology.orchestra.run.smithi115.stdout: "buffer_anon_items": 18238, 2023-12-11T18:04:28.924 INFO:teuthology.orchestra.run.smithi115.stdout: "buffer_meta_bytes": 0, 2023-12-11T18:04:28.924 INFO:teuthology.orchestra.run.smithi115.stdout: "buffer_meta_items": 0, 2023-12-11T18:04:28.924 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_bytes": 0, 2023-12-11T18:04:28.924 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_items": 0, 2023-12-11T18:04:28.924 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_mapbl_bytes": 0, 2023-12-11T18:04:28.924 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_mapbl_items": 0, 2023-12-11T18:04:28.924 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_pglog_bytes": 0, 2023-12-11T18:04:28.924 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_pglog_items": 0, 2023-12-11T18:04:28.924 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_bytes": 10464, 2023-12-11T18:04:28.924 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_items": 226, 2023-12-11T18:04:28.924 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_mapping_bytes": 0, 2023-12-11T18:04:28.924 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_mapping_items": 0, 2023-12-11T18:04:28.925 INFO:teuthology.orchestra.run.smithi115.stdout: "pgmap_bytes": 0, 2023-12-11T18:04:28.925 INFO:teuthology.orchestra.run.smithi115.stdout: "pgmap_items": 0, 2023-12-11T18:04:28.925 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_co_bytes": 69716847, 2023-12-11T18:04:28.925 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_co_items": 762638, 2023-12-11T18:04:28.925 INFO:teuthology.orchestra.run.smithi115.stdout: "unittest_1_bytes": 0, 2023-12-11T18:04:28.925 INFO:teuthology.orchestra.run.smithi115.stdout: "unittest_1_items": 0, 2023-12-11T18:04:28.925 INFO:teuthology.orchestra.run.smithi115.stdout: "unittest_2_bytes": 0, 2023-12-11T18:04:28.925 INFO:teuthology.orchestra.run.smithi115.stdout: "unittest_2_items": 0 2023-12-11T18:04:28.925 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:28.925 INFO:teuthology.orchestra.run.smithi115.stdout: "objecter": { 2023-12-11T18:04:28.925 INFO:teuthology.orchestra.run.smithi115.stdout: "op_active": 0, 2023-12-11T18:04:28.925 INFO:teuthology.orchestra.run.smithi115.stdout: "op_laggy": 0, 2023-12-11T18:04:28.925 INFO:teuthology.orchestra.run.smithi115.stdout: "op_send": 40954, 2023-12-11T18:04:28.925 INFO:teuthology.orchestra.run.smithi115.stdout: "op_send_bytes": 410289553, 2023-12-11T18:04:28.925 INFO:teuthology.orchestra.run.smithi115.stdout: "op_resend": 28, 2023-12-11T18:04:28.925 INFO:teuthology.orchestra.run.smithi115.stdout: "op_reply": 40954, 2023-12-11T18:04:28.926 INFO:teuthology.orchestra.run.smithi115.stdout: "op_latency": { 2023-12-11T18:04:28.926 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 40954, 2023-12-11T18:04:28.926 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 1056.957659021, 2023-12-11T18:04:28.926 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.025808410 2023-12-11T18:04:28.926 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:28.926 INFO:teuthology.orchestra.run.smithi115.stdout: "op_inflight": 0, 2023-12-11T18:04:28.926 INFO:teuthology.orchestra.run.smithi115.stdout: "oplen_avg": { 2023-12-11T18:04:28.926 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 40954, 2023-12-11T18:04:28.926 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 42585 2023-12-11T18:04:28.926 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:28.926 INFO:teuthology.orchestra.run.smithi115.stdout: "op": 40954, 2023-12-11T18:04:28.926 INFO:teuthology.orchestra.run.smithi115.stdout: "op_r": 989, 2023-12-11T18:04:28.926 INFO:teuthology.orchestra.run.smithi115.stdout: "op_w": 39965, 2023-12-11T18:04:28.926 INFO:teuthology.orchestra.run.smithi115.stdout: "op_rmw": 0, 2023-12-11T18:04:28.926 INFO:teuthology.orchestra.run.smithi115.stdout: "op_pg": 0, 2023-12-11T18:04:28.926 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_stat": 117, 2023-12-11T18:04:28.927 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_create": 21, 2023-12-11T18:04:28.927 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_read": 488, 2023-12-11T18:04:28.927 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_write": 4073, 2023-12-11T18:04:28.927 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_writefull": 268, 2023-12-11T18:04:28.927 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_writesame": 0, 2023-12-11T18:04:28.927 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_append": 0, 2023-12-11T18:04:28.927 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_zero": 0, 2023-12-11T18:04:28.927 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_truncate": 78, 2023-12-11T18:04:28.927 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_delete": 34558, 2023-12-11T18:04:28.927 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_mapext": 0, 2023-12-11T18:04:28.927 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_sparse_read": 0, 2023-12-11T18:04:28.927 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_clonerange": 0, 2023-12-11T18:04:28.927 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_getxattr": 0, 2023-12-11T18:04:28.927 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_setxattr": 42, 2023-12-11T18:04:28.927 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_cmpxattr": 0, 2023-12-11T18:04:28.927 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_rmxattr": 0, 2023-12-11T18:04:28.928 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_resetxattrs": 0, 2023-12-11T18:04:28.928 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_call": 0, 2023-12-11T18:04:28.928 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_watch": 0, 2023-12-11T18:04:28.928 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_notify": 0, 2023-12-11T18:04:28.928 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_src_cmpxattr": 0, 2023-12-11T18:04:28.928 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_pgls": 0, 2023-12-11T18:04:28.928 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_pgls_filter": 0, 2023-12-11T18:04:28.928 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_other": 0, 2023-12-11T18:04:28.928 INFO:teuthology.orchestra.run.smithi115.stdout: "linger_active": 0, 2023-12-11T18:04:28.928 INFO:teuthology.orchestra.run.smithi115.stdout: "linger_send": 0, 2023-12-11T18:04:28.928 INFO:teuthology.orchestra.run.smithi115.stdout: "linger_resend": 0, 2023-12-11T18:04:28.928 INFO:teuthology.orchestra.run.smithi115.stdout: "linger_ping": 0, 2023-12-11T18:04:28.928 INFO:teuthology.orchestra.run.smithi115.stdout: "poolop_active": 0, 2023-12-11T18:04:28.928 INFO:teuthology.orchestra.run.smithi115.stdout: "poolop_send": 0, 2023-12-11T18:04:28.928 INFO:teuthology.orchestra.run.smithi115.stdout: "poolop_resend": 0, 2023-12-11T18:04:28.928 INFO:teuthology.orchestra.run.smithi115.stdout: "poolstat_active": 0, 2023-12-11T18:04:28.929 INFO:teuthology.orchestra.run.smithi115.stdout: "poolstat_send": 0, 2023-12-11T18:04:28.929 INFO:teuthology.orchestra.run.smithi115.stdout: "poolstat_resend": 0, 2023-12-11T18:04:28.929 INFO:teuthology.orchestra.run.smithi115.stdout: "statfs_active": 0, 2023-12-11T18:04:28.929 INFO:teuthology.orchestra.run.smithi115.stdout: "statfs_send": 0, 2023-12-11T18:04:28.929 INFO:teuthology.orchestra.run.smithi115.stdout: "statfs_resend": 0, 2023-12-11T18:04:28.929 INFO:teuthology.orchestra.run.smithi115.stdout: "command_active": 0, 2023-12-11T18:04:28.929 INFO:teuthology.orchestra.run.smithi115.stdout: "command_send": 0, 2023-12-11T18:04:28.929 INFO:teuthology.orchestra.run.smithi115.stdout: "command_resend": 0, 2023-12-11T18:04:28.929 INFO:teuthology.orchestra.run.smithi115.stdout: "map_epoch": 119, 2023-12-11T18:04:28.929 INFO:teuthology.orchestra.run.smithi115.stdout: "map_full": 0, 2023-12-11T18:04:28.929 INFO:teuthology.orchestra.run.smithi115.stdout: "map_inc": 33, 2023-12-11T18:04:28.929 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_sessions": 12, 2023-12-11T18:04:28.929 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_session_open": 16, 2023-12-11T18:04:28.929 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_session_close": 4, 2023-12-11T18:04:28.929 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_laggy": 0, 2023-12-11T18:04:28.930 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_wr": 1903, 2023-12-11T18:04:28.930 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_rd": 738, 2023-12-11T18:04:28.930 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_del": 299 2023-12-11T18:04:28.930 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:28.930 INFO:teuthology.orchestra.run.smithi115.stdout: "oft": { 2023-12-11T18:04:28.930 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_total_objs": 1, 2023-12-11T18:04:28.930 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_total_kv_pairs": 0, 2023-12-11T18:04:28.930 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_total_updates": 0, 2023-12-11T18:04:28.930 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_total_removes": 0 2023-12-11T18:04:28.930 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:28.930 INFO:teuthology.orchestra.run.smithi115.stdout: "osd-slow-ops": { 2023-12-11T18:04:28.930 INFO:teuthology.orchestra.run.smithi115.stdout: "slow_ops_count": 35 2023-12-11T18:04:28.930 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:28.930 INFO:teuthology.orchestra.run.smithi115.stdout: "purge_queue": { 2023-12-11T18:04:28.930 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executing_ops": 0, 2023-12-11T18:04:28.931 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executing_ops_high_water": 16, 2023-12-11T18:04:28.931 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executing": 0, 2023-12-11T18:04:28.931 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executing_high_water": 8, 2023-12-11T18:04:28.931 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executed_ops": 34438, 2023-12-11T18:04:28.931 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executed": 17219, 2023-12-11T18:04:28.931 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_item_in_journal": 0 2023-12-11T18:04:28.931 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:28.931 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-12-11T18:04:28.931 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2023-12-11T18:04:28.931 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 104857600, 2023-12-11T18:04:28.931 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2023-12-11T18:04:28.931 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 306846, 2023-12-11T18:04:28.931 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 114809227, 2023-12-11T18:04:28.931 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2023-12-11T18:04:28.931 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 306846, 2023-12-11T18:04:28.932 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 0, 2023-12-11T18:04:28.932 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 0, 2023-12-11T18:04:28.932 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 306846, 2023-12-11T18:04:28.932 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 114809227, 2023-12-11T18:04:28.932 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2023-12-11T18:04:28.932 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-12-11T18:04:28.932 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-12-11T18:04:28.932 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-12-11T18:04:28.932 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-11T18:04:28.932 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:28.932 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-objecter_bytes": { 2023-12-11T18:04:28.932 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2023-12-11T18:04:28.932 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 104857600, 2023-12-11T18:04:28.932 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2023-12-11T18:04:28.932 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 0, 2023-12-11T18:04:28.933 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 0, 2023-12-11T18:04:28.933 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2023-12-11T18:04:28.933 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 0, 2023-12-11T18:04:28.933 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 40954, 2023-12-11T18:04:28.933 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 412762458, 2023-12-11T18:04:28.933 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 40954, 2023-12-11T18:04:28.933 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 412762458, 2023-12-11T18:04:28.933 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2023-12-11T18:04:28.933 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-12-11T18:04:28.933 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-12-11T18:04:28.933 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-12-11T18:04:28.933 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-11T18:04:28.933 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:28.933 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-objecter_ops": { 2023-12-11T18:04:28.933 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2023-12-11T18:04:28.934 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 1024, 2023-12-11T18:04:28.934 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2023-12-11T18:04:28.934 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 0, 2023-12-11T18:04:28.934 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 0, 2023-12-11T18:04:28.934 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2023-12-11T18:04:28.934 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 0, 2023-12-11T18:04:28.934 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 40954, 2023-12-11T18:04:28.934 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 40954, 2023-12-11T18:04:28.934 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 40954, 2023-12-11T18:04:28.934 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 40954, 2023-12-11T18:04:28.934 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2023-12-11T18:04:28.934 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-12-11T18:04:28.934 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-12-11T18:04:28.934 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-12-11T18:04:28.934 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-11T18:04:28.935 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:28.935 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-write_buf_throttle": { 2023-12-11T18:04:28.935 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2023-12-11T18:04:28.935 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 3758096384, 2023-12-11T18:04:28.935 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2023-12-11T18:04:28.935 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 17219, 2023-12-11T18:04:28.935 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 2485711, 2023-12-11T18:04:28.935 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2023-12-11T18:04:28.935 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 17219, 2023-12-11T18:04:28.935 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 0, 2023-12-11T18:04:28.935 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 0, 2023-12-11T18:04:28.935 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 489, 2023-12-11T18:04:28.935 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 2485711, 2023-12-11T18:04:28.935 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2023-12-11T18:04:28.935 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-12-11T18:04:28.935 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-12-11T18:04:28.935 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-12-11T18:04:28.936 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-11T18:04:28.936 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:28.936 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-write_buf_throttle-0x55dcb91486e0": { 2023-12-11T18:04:28.936 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2023-12-11T18:04:28.936 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 3758096384, 2023-12-11T18:04:28.936 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2023-12-11T18:04:28.936 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 103828, 2023-12-11T18:04:28.936 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 389921974, 2023-12-11T18:04:28.936 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2023-12-11T18:04:28.936 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 103828, 2023-12-11T18:04:28.936 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 0, 2023-12-11T18:04:28.936 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 0, 2023-12-11T18:04:28.936 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 3583, 2023-12-11T18:04:28.936 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 389921974, 2023-12-11T18:04:28.936 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2023-12-11T18:04:28.936 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-12-11T18:04:28.937 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-12-11T18:04:28.937 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-12-11T18:04:28.937 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-11T18:04:28.937 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-11T18:04:28.937 INFO:teuthology.orchestra.run.smithi115.stdout:} 2023-12-11T18:04:28.937 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:04:28 smithi059 ceph-mon[98847]: pgmap v578: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.7 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2023-12-11T18:04:30.413 INFO:tasks.check_counter:Daemon mds.h mds.exported=234 2023-12-11T18:04:30.413 INFO:tasks.check_counter:Daemon mds.h mds.imported=264 2023-12-11T18:04:30.414 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=0 2023-12-11T18:04:30.414 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2023-12-11T18:04:30.414 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2023-12-11T18:04:30.414 INFO:tasks.check_counter:Daemon mds.h mds.dir_split=79 2023-12-11T18:04:30.414 DEBUG:tasks.check_counter:Getting stats from k 2023-12-11T18:04:30.414 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell --fsid d0a07eaa-984b-11ee-95a3-87774f69a715 -- ceph daemon mds.k perf dump 2023-12-11T18:04:30.785 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:04:30 smithi115 ceph-mon[102748]: pgmap v579: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.7 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2023-12-11T18:04:30.912 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:04:30 smithi059 ceph-mon[98847]: pgmap v579: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.7 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2023-12-11T18:04:31.386 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/mon.b/config 2023-12-11T18:04:31.785 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:04:31 smithi115 ceph-mon[102748]: pgmap v580: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.7 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2023-12-11T18:04:31.907 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:04:31 smithi059 ceph-mon[98847]: pgmap v580: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.7 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2023-12-11T18:04:33.576 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2023-12-11T18:04:33.576 INFO:teuthology.orchestra.run.smithi115.stdout: "AsyncMessenger::Worker-0": { 2023-12-11T18:04:33.576 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_messages": 19589, 2023-12-11T18:04:33.576 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages": 20203, 2023-12-11T18:04:33.576 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_bytes": 121539355, 2023-12-11T18:04:33.576 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_bytes": 66434960, 2023-12-11T18:04:33.576 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_created_connections": 9, 2023-12-11T18:04:33.576 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_active_connections": 5, 2023-12-11T18:04:33.576 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_total_time": 1.697587994, 2023-12-11T18:04:33.576 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_send_time": 0.462682302, 2023-12-11T18:04:33.576 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_recv_time": 0.494719736, 2023-12-11T18:04:33.577 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_fast_dispatch_time": 0.688234576, 2023-12-11T18:04:33.577 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages_queue_lat": { 2023-12-11T18:04:33.577 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 20202, 2023-12-11T18:04:33.577 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 8.173554177, 2023-12-11T18:04:33.577 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000404591 2023-12-11T18:04:33.577 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:33.577 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_handle_ack_lat": { 2023-12-11T18:04:33.577 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 2195, 2023-12-11T18:04:33.577 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.003574519, 2023-12-11T18:04:33.577 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000001628 2023-12-11T18:04:33.577 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:33.577 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_encrypted_bytes": 1908320, 2023-12-11T18:04:33.577 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_encrypted_bytes": 205520 2023-12-11T18:04:33.577 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:33.577 INFO:teuthology.orchestra.run.smithi115.stdout: "AsyncMessenger::Worker-1": { 2023-12-11T18:04:33.577 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_messages": 19256, 2023-12-11T18:04:33.578 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages": 19504, 2023-12-11T18:04:33.578 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_bytes": 135675837, 2023-12-11T18:04:33.578 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_bytes": 120042513, 2023-12-11T18:04:33.578 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_created_connections": 12, 2023-12-11T18:04:33.578 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_active_connections": 5, 2023-12-11T18:04:33.578 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_total_time": 1.764392237, 2023-12-11T18:04:33.578 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_send_time": 0.543829343, 2023-12-11T18:04:33.578 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_recv_time": 0.395104804, 2023-12-11T18:04:33.578 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_fast_dispatch_time": 0.740030103, 2023-12-11T18:04:33.578 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages_queue_lat": { 2023-12-11T18:04:33.578 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 19501, 2023-12-11T18:04:33.578 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 8.659792780, 2023-12-11T18:04:33.578 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000444069 2023-12-11T18:04:33.579 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:33.579 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_handle_ack_lat": { 2023-12-11T18:04:33.579 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 2, 2023-12-11T18:04:33.579 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000627, 2023-12-11T18:04:33.579 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000313 2023-12-11T18:04:33.579 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:33.579 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_encrypted_bytes": 624, 2023-12-11T18:04:33.579 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_encrypted_bytes": 431472 2023-12-11T18:04:33.579 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:33.579 INFO:teuthology.orchestra.run.smithi115.stdout: "AsyncMessenger::Worker-2": { 2023-12-11T18:04:33.579 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_messages": 63919, 2023-12-11T18:04:33.579 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages": 111937, 2023-12-11T18:04:33.579 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_bytes": 101321827, 2023-12-11T18:04:33.579 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_bytes": 113413046, 2023-12-11T18:04:33.580 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_created_connections": 9, 2023-12-11T18:04:33.580 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_active_connections": 5, 2023-12-11T18:04:33.580 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_total_time": 5.328456858, 2023-12-11T18:04:33.580 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_send_time": 2.255167474, 2023-12-11T18:04:33.580 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_recv_time": 2.980039242, 2023-12-11T18:04:33.580 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_running_fast_dispatch_time": 0.514957884, 2023-12-11T18:04:33.580 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_messages_queue_lat": { 2023-12-11T18:04:33.580 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 111936, 2023-12-11T18:04:33.580 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 45.452287236, 2023-12-11T18:04:33.580 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000406056 2023-12-11T18:04:33.580 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:33.580 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_handle_ack_lat": { 2023-12-11T18:04:33.580 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 115120, 2023-12-11T18:04:33.580 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.023467303, 2023-12-11T18:04:33.581 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000203 2023-12-11T18:04:33.581 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:33.581 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_recv_encrypted_bytes": 16109840, 2023-12-11T18:04:33.581 INFO:teuthology.orchestra.run.smithi115.stdout: "msgr_send_encrypted_bytes": 67865216 2023-12-11T18:04:33.581 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:33.581 INFO:teuthology.orchestra.run.smithi115.stdout: "cct": { 2023-12-11T18:04:33.581 INFO:teuthology.orchestra.run.smithi115.stdout: "total_workers": 1, 2023-12-11T18:04:33.581 INFO:teuthology.orchestra.run.smithi115.stdout: "unhealthy_workers": 0 2023-12-11T18:04:33.581 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:33.581 INFO:teuthology.orchestra.run.smithi115.stdout: "finisher-MDSRank": { 2023-12-11T18:04:33.581 INFO:teuthology.orchestra.run.smithi115.stdout: "queue_len": 0, 2023-12-11T18:04:33.581 INFO:teuthology.orchestra.run.smithi115.stdout: "complete_latency": { 2023-12-11T18:04:33.581 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 1819, 2023-12-11T18:04:33.581 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 26.943402225, 2023-12-11T18:04:33.582 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.014812205 2023-12-11T18:04:33.582 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-11T18:04:33.582 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:33.582 INFO:teuthology.orchestra.run.smithi115.stdout: "finisher-PurgeQueue": { 2023-12-11T18:04:33.582 INFO:teuthology.orchestra.run.smithi115.stdout: "queue_len": 0, 2023-12-11T18:04:33.582 INFO:teuthology.orchestra.run.smithi115.stdout: "complete_latency": { 2023-12-11T18:04:33.582 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 41673, 2023-12-11T18:04:33.582 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 9.168049650, 2023-12-11T18:04:33.582 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000219999 2023-12-11T18:04:33.582 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-11T18:04:33.582 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:33.582 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": { 2023-12-11T18:04:33.582 INFO:teuthology.orchestra.run.smithi115.stdout: "request": 48622, 2023-12-11T18:04:33.583 INFO:teuthology.orchestra.run.smithi115.stdout: "reply": 48519, 2023-12-11T18:04:33.583 INFO:teuthology.orchestra.run.smithi115.stdout: "reply_latency": { 2023-12-11T18:04:33.583 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 48519, 2023-12-11T18:04:33.583 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 88.966077910, 2023-12-11T18:04:33.583 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.001833633 2023-12-11T18:04:33.583 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:33.583 INFO:teuthology.orchestra.run.smithi115.stdout: "slow_reply": 0, 2023-12-11T18:04:33.583 INFO:teuthology.orchestra.run.smithi115.stdout: "forward": 44, 2023-12-11T18:04:33.583 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_fetch_complete": 206, 2023-12-11T18:04:33.583 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_fetch_keys": 1, 2023-12-11T18:04:33.583 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_commit": 291, 2023-12-11T18:04:33.583 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_split": 0, 2023-12-11T18:04:33.583 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_merge": 0, 2023-12-11T18:04:33.583 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes": 67701, 2023-12-11T18:04:33.583 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_top": 7936, 2023-12-11T18:04:33.584 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_bottom": 41548, 2023-12-11T18:04:33.584 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_pin_tail": 18217, 2023-12-11T18:04:33.584 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_pinned": 56360, 2023-12-11T18:04:33.584 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_expired": 51597, 2023-12-11T18:04:33.584 INFO:teuthology.orchestra.run.smithi115.stdout: "inodes_with_caps": 0, 2023-12-11T18:04:33.584 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": 0, 2023-12-11T18:04:33.584 INFO:teuthology.orchestra.run.smithi115.stdout: "subtrees": 4, 2023-12-11T18:04:33.584 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse": 72331, 2023-12-11T18:04:33.584 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_hit": 72163, 2023-12-11T18:04:33.584 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_forward": 43, 2023-12-11T18:04:33.584 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_discover": 9, 2023-12-11T18:04:33.584 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_dir_fetch": 1, 2023-12-11T18:04:33.584 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_remote_ino": 0, 2023-12-11T18:04:33.584 INFO:teuthology.orchestra.run.smithi115.stdout: "traverse_lock": 0, 2023-12-11T18:04:33.584 INFO:teuthology.orchestra.run.smithi115.stdout: "load_cent": 0, 2023-12-11T18:04:33.585 INFO:teuthology.orchestra.run.smithi115.stdout: "q": 0, 2023-12-11T18:04:33.585 INFO:teuthology.orchestra.run.smithi115.stdout: "exported": 29, 2023-12-11T18:04:33.585 INFO:teuthology.orchestra.run.smithi115.stdout: "exported_inodes": 42367, 2023-12-11T18:04:33.585 INFO:teuthology.orchestra.run.smithi115.stdout: "imported": 29, 2023-12-11T18:04:33.585 INFO:teuthology.orchestra.run.smithi115.stdout: "imported_inodes": 15356, 2023-12-11T18:04:33.585 INFO:teuthology.orchestra.run.smithi115.stdout: "openino_dir_fetch": 1, 2023-12-11T18:04:33.585 INFO:teuthology.orchestra.run.smithi115.stdout: "openino_backtrace_fetch": 0, 2023-12-11T18:04:33.585 INFO:teuthology.orchestra.run.smithi115.stdout: "openino_peer_discover": 0, 2023-12-11T18:04:33.585 INFO:teuthology.orchestra.run.smithi115.stdout: "root_rfiles": 38748, 2023-12-11T18:04:33.585 INFO:teuthology.orchestra.run.smithi115.stdout: "root_rbytes": 803088135, 2023-12-11T18:04:33.585 INFO:teuthology.orchestra.run.smithi115.stdout: "root_rsnaps": 6, 2023-12-11T18:04:33.585 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_backtrace_fetch": 0, 2023-12-11T18:04:33.585 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_set_tag": 0, 2023-12-11T18:04:33.585 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_backtrace_repaired": 0, 2023-12-11T18:04:33.585 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_inotable_repaired": 0, 2023-12-11T18:04:33.585 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_dir_inodes": 0, 2023-12-11T18:04:33.585 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_dir_base_inodes": 0, 2023-12-11T18:04:33.586 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_dirfrag_rstats": 0, 2023-12-11T18:04:33.586 INFO:teuthology.orchestra.run.smithi115.stdout: "scrub_file_inodes": 0, 2023-12-11T18:04:33.586 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_inode_file_caps": 40, 2023-12-11T18:04:33.586 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_revoke": 314, 2023-12-11T18:04:33.586 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_grant": 24024, 2023-12-11T18:04:33.586 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_trunc": 0, 2023-12-11T18:04:33.586 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-12-11T18:04:33.586 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_cap_op_flush_ack": 0, 2023-12-11T18:04:33.586 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_caps": 327, 2023-12-11T18:04:33.586 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_caps_dirty": 0, 2023-12-11T18:04:33.586 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_cap_release": 207, 2023-12-11T18:04:33.586 INFO:teuthology.orchestra.run.smithi115.stdout: "process_request_cap_release": 27808 2023-12-11T18:04:33.586 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:33.586 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_cache": { 2023-12-11T18:04:33.587 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_update": 0, 2023-12-11T18:04:33.587 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_update_receipt": 0, 2023-12-11T18:04:33.587 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_try_discover": 0, 2023-12-11T18:04:33.587 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_send_discover": 9, 2023-12-11T18:04:33.587 INFO:teuthology.orchestra.run.smithi115.stdout: "dir_handle_discover": 11, 2023-12-11T18:04:33.587 INFO:teuthology.orchestra.run.smithi115.stdout: "num_strays": 162, 2023-12-11T18:04:33.587 INFO:teuthology.orchestra.run.smithi115.stdout: "num_strays_delayed": 0, 2023-12-11T18:04:33.587 INFO:teuthology.orchestra.run.smithi115.stdout: "num_strays_enqueuing": 0, 2023-12-11T18:04:33.587 INFO:teuthology.orchestra.run.smithi115.stdout: "strays_created": 23813, 2023-12-11T18:04:33.587 INFO:teuthology.orchestra.run.smithi115.stdout: "strays_enqueued": 23846, 2023-12-11T18:04:33.587 INFO:teuthology.orchestra.run.smithi115.stdout: "strays_reintegrated": 0, 2023-12-11T18:04:33.587 INFO:teuthology.orchestra.run.smithi115.stdout: "strays_migrated": 0, 2023-12-11T18:04:33.587 INFO:teuthology.orchestra.run.smithi115.stdout: "num_recovering_processing": 0, 2023-12-11T18:04:33.587 INFO:teuthology.orchestra.run.smithi115.stdout: "num_recovering_enqueued": 0, 2023-12-11T18:04:33.587 INFO:teuthology.orchestra.run.smithi115.stdout: "num_recovering_prioritized": 0, 2023-12-11T18:04:33.588 INFO:teuthology.orchestra.run.smithi115.stdout: "recovery_started": 0, 2023-12-11T18:04:33.588 INFO:teuthology.orchestra.run.smithi115.stdout: "recovery_completed": 0, 2023-12-11T18:04:33.588 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_enqueue_scrub": 0, 2023-12-11T18:04:33.588 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_exportdir": 34, 2023-12-11T18:04:33.588 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_flush": 0, 2023-12-11T18:04:33.588 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_fragmentdir": 0, 2023-12-11T18:04:33.588 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_fragstats": 0, 2023-12-11T18:04:33.588 INFO:teuthology.orchestra.run.smithi115.stdout: "ireq_inodestats": 0 2023-12-11T18:04:33.588 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:33.588 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_log": { 2023-12-11T18:04:33.588 INFO:teuthology.orchestra.run.smithi115.stdout: "evlrg": 1, 2023-12-11T18:04:33.588 INFO:teuthology.orchestra.run.smithi115.stdout: "evadd": 48168, 2023-12-11T18:04:33.588 INFO:teuthology.orchestra.run.smithi115.stdout: "evex": 61450, 2023-12-11T18:04:33.588 INFO:teuthology.orchestra.run.smithi115.stdout: "evtrm": 61450, 2023-12-11T18:04:33.588 INFO:teuthology.orchestra.run.smithi115.stdout: "ev": 79059, 2023-12-11T18:04:33.588 INFO:teuthology.orchestra.run.smithi115.stdout: "evexg": 0, 2023-12-11T18:04:33.589 INFO:teuthology.orchestra.run.smithi115.stdout: "evexd": 12290, 2023-12-11T18:04:33.589 INFO:teuthology.orchestra.run.smithi115.stdout: "segadd": 75, 2023-12-11T18:04:33.589 INFO:teuthology.orchestra.run.smithi115.stdout: "segex": 84, 2023-12-11T18:04:33.589 INFO:teuthology.orchestra.run.smithi115.stdout: "segtrm": 84, 2023-12-11T18:04:33.589 INFO:teuthology.orchestra.run.smithi115.stdout: "seg": 128, 2023-12-11T18:04:33.589 INFO:teuthology.orchestra.run.smithi115.stdout: "segmjr": 7, 2023-12-11T18:04:33.589 INFO:teuthology.orchestra.run.smithi115.stdout: "segexg": 0, 2023-12-11T18:04:33.589 INFO:teuthology.orchestra.run.smithi115.stdout: "segexd": 18, 2023-12-11T18:04:33.589 INFO:teuthology.orchestra.run.smithi115.stdout: "expos": 244592624, 2023-12-11T18:04:33.589 INFO:teuthology.orchestra.run.smithi115.stdout: "wrpos": 582709489, 2023-12-11T18:04:33.589 INFO:teuthology.orchestra.run.smithi115.stdout: "rdpos": 383402836, 2023-12-11T18:04:33.589 INFO:teuthology.orchestra.run.smithi115.stdout: "jlat": { 2023-12-11T18:04:33.589 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 496, 2023-12-11T18:04:33.589 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 24.226649266, 2023-12-11T18:04:33.589 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.048844050 2023-12-11T18:04:33.589 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:33.590 INFO:teuthology.orchestra.run.smithi115.stdout: "replayed": 92341 2023-12-11T18:04:33.590 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:33.590 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_mem": { 2023-12-11T18:04:33.590 INFO:teuthology.orchestra.run.smithi115.stdout: "ino": 34119, 2023-12-11T18:04:33.590 INFO:teuthology.orchestra.run.smithi115.stdout: "ino+": 111322, 2023-12-11T18:04:33.590 INFO:teuthology.orchestra.run.smithi115.stdout: "ino-": 77203, 2023-12-11T18:04:33.590 INFO:teuthology.orchestra.run.smithi115.stdout: "dir": 1131, 2023-12-11T18:04:33.590 INFO:teuthology.orchestra.run.smithi115.stdout: "dir+": 2978, 2023-12-11T18:04:33.590 INFO:teuthology.orchestra.run.smithi115.stdout: "dir-": 1847, 2023-12-11T18:04:33.590 INFO:teuthology.orchestra.run.smithi115.stdout: "dn": 67701, 2023-12-11T18:04:33.590 INFO:teuthology.orchestra.run.smithi115.stdout: "dn+": 178315, 2023-12-11T18:04:33.590 INFO:teuthology.orchestra.run.smithi115.stdout: "dn-": 110614, 2023-12-11T18:04:33.590 INFO:teuthology.orchestra.run.smithi115.stdout: "cap": 0, 2023-12-11T18:04:33.590 INFO:teuthology.orchestra.run.smithi115.stdout: "cap+": 23928, 2023-12-11T18:04:33.591 INFO:teuthology.orchestra.run.smithi115.stdout: "cap-": 23928, 2023-12-11T18:04:33.591 INFO:teuthology.orchestra.run.smithi115.stdout: "rss": 239424, 2023-12-11T18:04:33.591 INFO:teuthology.orchestra.run.smithi115.stdout: "heap": 215420 2023-12-11T18:04:33.591 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:33.591 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_server": { 2023-12-11T18:04:33.591 INFO:teuthology.orchestra.run.smithi115.stdout: "dispatch_client_request": 49720, 2023-12-11T18:04:33.591 INFO:teuthology.orchestra.run.smithi115.stdout: "dispatch_server_request": 18, 2023-12-11T18:04:33.591 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_request": 48622, 2023-12-11T18:04:33.591 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_client_session": 10, 2023-12-11T18:04:33.591 INFO:teuthology.orchestra.run.smithi115.stdout: "handle_peer_request": 44, 2023-12-11T18:04:33.591 INFO:teuthology.orchestra.run.smithi115.stdout: "req_create_latency": { 2023-12-11T18:04:33.591 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-12-11T18:04:33.591 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-12-11T18:04:33.591 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-12-11T18:04:33.591 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:33.591 INFO:teuthology.orchestra.run.smithi115.stdout: "req_getattr_latency": { 2023-12-11T18:04:33.592 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-12-11T18:04:33.592 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-12-11T18:04:33.592 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-12-11T18:04:33.592 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:33.592 INFO:teuthology.orchestra.run.smithi115.stdout: "req_getfilelock_latency": { 2023-12-11T18:04:33.592 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-12-11T18:04:33.592 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-12-11T18:04:33.592 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-12-11T18:04:33.592 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:33.592 INFO:teuthology.orchestra.run.smithi115.stdout: "req_link_latency": { 2023-12-11T18:04:33.592 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-12-11T18:04:33.592 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-12-11T18:04:33.592 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-12-11T18:04:33.592 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:33.592 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookup_latency": { 2023-12-11T18:04:33.593 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 23583, 2023-12-11T18:04:33.593 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 22.186954317, 2023-12-11T18:04:33.593 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000940802 2023-12-11T18:04:33.593 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:33.593 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookuphash_latency": { 2023-12-11T18:04:33.593 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-12-11T18:04:33.593 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-12-11T18:04:33.593 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-12-11T18:04:33.593 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:33.593 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookupino_latency": { 2023-12-11T18:04:33.593 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-12-11T18:04:33.593 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-12-11T18:04:33.593 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-12-11T18:04:33.593 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:33.593 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookupname_latency": { 2023-12-11T18:04:33.593 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-12-11T18:04:33.594 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-12-11T18:04:33.594 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-12-11T18:04:33.594 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:33.594 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookupparent_latency": { 2023-12-11T18:04:33.594 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-12-11T18:04:33.594 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-12-11T18:04:33.594 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-12-11T18:04:33.594 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:33.594 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lookupsnap_latency": { 2023-12-11T18:04:33.594 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-12-11T18:04:33.594 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-12-11T18:04:33.594 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-12-11T18:04:33.594 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:33.594 INFO:teuthology.orchestra.run.smithi115.stdout: "req_lssnap_latency": { 2023-12-11T18:04:33.594 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-12-11T18:04:33.595 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-12-11T18:04:33.595 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-12-11T18:04:33.595 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:33.595 INFO:teuthology.orchestra.run.smithi115.stdout: "req_mkdir_latency": { 2023-12-11T18:04:33.595 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-12-11T18:04:33.595 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-12-11T18:04:33.595 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-12-11T18:04:33.595 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:33.595 INFO:teuthology.orchestra.run.smithi115.stdout: "req_mknod_latency": { 2023-12-11T18:04:33.595 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-12-11T18:04:33.595 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-12-11T18:04:33.595 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-12-11T18:04:33.595 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:33.595 INFO:teuthology.orchestra.run.smithi115.stdout: "req_mksnap_latency": { 2023-12-11T18:04:33.595 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-12-11T18:04:33.595 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-12-11T18:04:33.595 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-12-11T18:04:33.596 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:33.596 INFO:teuthology.orchestra.run.smithi115.stdout: "req_open_latency": { 2023-12-11T18:04:33.596 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-12-11T18:04:33.596 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-12-11T18:04:33.596 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-12-11T18:04:33.596 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:33.596 INFO:teuthology.orchestra.run.smithi115.stdout: "req_readdir_latency": { 2023-12-11T18:04:33.596 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 1181, 2023-12-11T18:04:33.596 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 3.621530814, 2023-12-11T18:04:33.596 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.003066495 2023-12-11T18:04:33.596 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:33.596 INFO:teuthology.orchestra.run.smithi115.stdout: "req_rename_latency": { 2023-12-11T18:04:33.596 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-12-11T18:04:33.596 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-12-11T18:04:33.596 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-12-11T18:04:33.596 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:33.597 INFO:teuthology.orchestra.run.smithi115.stdout: "req_renamesnap_latency": { 2023-12-11T18:04:33.597 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-12-11T18:04:33.597 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-12-11T18:04:33.597 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-12-11T18:04:33.597 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:33.597 INFO:teuthology.orchestra.run.smithi115.stdout: "req_snapdiff_latency": { 2023-12-11T18:04:33.597 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-12-11T18:04:33.597 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-12-11T18:04:33.597 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-12-11T18:04:33.597 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:33.597 INFO:teuthology.orchestra.run.smithi115.stdout: "req_rmdir_latency": { 2023-12-11T18:04:33.597 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 160, 2023-12-11T18:04:33.597 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 21.030451602, 2023-12-11T18:04:33.597 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.131440322 2023-12-11T18:04:33.597 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:33.597 INFO:teuthology.orchestra.run.smithi115.stdout: "req_rmsnap_latency": { 2023-12-11T18:04:33.598 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-12-11T18:04:33.598 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-12-11T18:04:33.598 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-12-11T18:04:33.598 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:33.598 INFO:teuthology.orchestra.run.smithi115.stdout: "req_rmxattr_latency": { 2023-12-11T18:04:33.598 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-12-11T18:04:33.598 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-12-11T18:04:33.598 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-12-11T18:04:33.598 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:33.598 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setattr_latency": { 2023-12-11T18:04:33.598 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-12-11T18:04:33.598 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-12-11T18:04:33.598 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-12-11T18:04:33.598 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:33.599 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setdirlayout_latency": { 2023-12-11T18:04:33.599 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-12-11T18:04:33.599 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-12-11T18:04:33.599 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-12-11T18:04:33.599 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:33.599 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setfilelock_latency": { 2023-12-11T18:04:33.599 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-12-11T18:04:33.599 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-12-11T18:04:33.599 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-12-11T18:04:33.599 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:33.599 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setlayout_latency": { 2023-12-11T18:04:33.599 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-12-11T18:04:33.599 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-12-11T18:04:33.599 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-12-11T18:04:33.599 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:33.599 INFO:teuthology.orchestra.run.smithi115.stdout: "req_setxattr_latency": { 2023-12-11T18:04:33.600 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-12-11T18:04:33.600 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-12-11T18:04:33.600 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-12-11T18:04:33.600 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:33.600 INFO:teuthology.orchestra.run.smithi115.stdout: "req_symlink_latency": { 2023-12-11T18:04:33.600 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-12-11T18:04:33.600 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-12-11T18:04:33.600 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-12-11T18:04:33.600 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:33.600 INFO:teuthology.orchestra.run.smithi115.stdout: "req_unlink_latency": { 2023-12-11T18:04:33.600 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 23595, 2023-12-11T18:04:33.600 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 42.127141177, 2023-12-11T18:04:33.600 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.001785426 2023-12-11T18:04:33.601 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:33.601 INFO:teuthology.orchestra.run.smithi115.stdout: "cap_revoke_eviction": 0, 2023-12-11T18:04:33.601 INFO:teuthology.orchestra.run.smithi115.stdout: "cap_acquisition_throttle": 0, 2023-12-11T18:04:33.601 INFO:teuthology.orchestra.run.smithi115.stdout: "req_getvxattr_latency": { 2023-12-11T18:04:33.601 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-12-11T18:04:33.601 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-12-11T18:04:33.601 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-12-11T18:04:33.601 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-11T18:04:33.601 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:33.601 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_sessions": { 2023-12-11T18:04:33.601 INFO:teuthology.orchestra.run.smithi115.stdout: "session_count": 1, 2023-12-11T18:04:33.601 INFO:teuthology.orchestra.run.smithi115.stdout: "session_add": 1, 2023-12-11T18:04:33.601 INFO:teuthology.orchestra.run.smithi115.stdout: "session_remove": 0, 2023-12-11T18:04:33.601 INFO:teuthology.orchestra.run.smithi115.stdout: "sessions_open": 0, 2023-12-11T18:04:33.601 INFO:teuthology.orchestra.run.smithi115.stdout: "sessions_stale": 0, 2023-12-11T18:04:33.602 INFO:teuthology.orchestra.run.smithi115.stdout: "total_load": 5610, 2023-12-11T18:04:33.602 INFO:teuthology.orchestra.run.smithi115.stdout: "average_load": 5610, 2023-12-11T18:04:33.602 INFO:teuthology.orchestra.run.smithi115.stdout: "avg_session_uptime": 2513, 2023-12-11T18:04:33.602 INFO:teuthology.orchestra.run.smithi115.stdout: "mdthresh_evicted": 0 2023-12-11T18:04:33.602 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:33.602 INFO:teuthology.orchestra.run.smithi115.stdout: "mempool": { 2023-12-11T18:04:33.602 INFO:teuthology.orchestra.run.smithi115.stdout: "bloom_filter_bytes": 0, 2023-12-11T18:04:33.602 INFO:teuthology.orchestra.run.smithi115.stdout: "bloom_filter_items": 0, 2023-12-11T18:04:33.602 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_alloc_bytes": 0, 2023-12-11T18:04:33.602 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_alloc_items": 0, 2023-12-11T18:04:33.602 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_data_bytes": 0, 2023-12-11T18:04:33.602 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_data_items": 0, 2023-12-11T18:04:33.602 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_onode_bytes": 0, 2023-12-11T18:04:33.602 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_onode_items": 0, 2023-12-11T18:04:33.603 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_meta_bytes": 0, 2023-12-11T18:04:33.603 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_meta_items": 0, 2023-12-11T18:04:33.603 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_other_bytes": 0, 2023-12-11T18:04:33.603 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_other_items": 0, 2023-12-11T18:04:33.603 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_buffer_bytes": 0, 2023-12-11T18:04:33.603 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_buffer_items": 0, 2023-12-11T18:04:33.603 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_extent_bytes": 0, 2023-12-11T18:04:33.603 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_extent_items": 0, 2023-12-11T18:04:33.603 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_blob_bytes": 0, 2023-12-11T18:04:33.603 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_blob_items": 0, 2023-12-11T18:04:33.603 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_shared_blob_bytes": 0, 2023-12-11T18:04:33.603 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_shared_blob_items": 0, 2023-12-11T18:04:33.603 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_inline_bl_bytes": 0, 2023-12-11T18:04:33.603 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_inline_bl_items": 0, 2023-12-11T18:04:33.603 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_fsck_bytes": 0, 2023-12-11T18:04:33.603 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_fsck_items": 0, 2023-12-11T18:04:33.604 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_txc_bytes": 0, 2023-12-11T18:04:33.604 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_txc_items": 0, 2023-12-11T18:04:33.604 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_writing_deferred_bytes": 0, 2023-12-11T18:04:33.604 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_writing_deferred_items": 0, 2023-12-11T18:04:33.604 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_writing_bytes": 0, 2023-12-11T18:04:33.604 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_writing_items": 0, 2023-12-11T18:04:33.604 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_bytes": 0, 2023-12-11T18:04:33.604 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_items": 0, 2023-12-11T18:04:33.604 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_file_reader_bytes": 0, 2023-12-11T18:04:33.604 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_file_reader_items": 0, 2023-12-11T18:04:33.604 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_file_writer_bytes": 0, 2023-12-11T18:04:33.604 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_file_writer_items": 0, 2023-12-11T18:04:33.604 INFO:teuthology.orchestra.run.smithi115.stdout: "buffer_anon_bytes": 2804858, 2023-12-11T18:04:33.604 INFO:teuthology.orchestra.run.smithi115.stdout: "buffer_anon_items": 34797, 2023-12-11T18:04:33.605 INFO:teuthology.orchestra.run.smithi115.stdout: "buffer_meta_bytes": 0, 2023-12-11T18:04:33.605 INFO:teuthology.orchestra.run.smithi115.stdout: "buffer_meta_items": 0, 2023-12-11T18:04:33.605 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_bytes": 0, 2023-12-11T18:04:33.605 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_items": 0, 2023-12-11T18:04:33.605 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_mapbl_bytes": 0, 2023-12-11T18:04:33.605 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_mapbl_items": 0, 2023-12-11T18:04:33.605 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_pglog_bytes": 0, 2023-12-11T18:04:33.605 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_pglog_items": 0, 2023-12-11T18:04:33.605 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_bytes": 10464, 2023-12-11T18:04:33.605 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_items": 226, 2023-12-11T18:04:33.605 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_mapping_bytes": 0, 2023-12-11T18:04:33.605 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_mapping_items": 0, 2023-12-11T18:04:33.605 INFO:teuthology.orchestra.run.smithi115.stdout: "pgmap_bytes": 0, 2023-12-11T18:04:33.606 INFO:teuthology.orchestra.run.smithi115.stdout: "pgmap_items": 0, 2023-12-11T18:04:33.606 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_co_bytes": 128240588, 2023-12-11T18:04:33.606 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_co_items": 1431037, 2023-12-11T18:04:33.606 INFO:teuthology.orchestra.run.smithi115.stdout: "unittest_1_bytes": 0, 2023-12-11T18:04:33.606 INFO:teuthology.orchestra.run.smithi115.stdout: "unittest_1_items": 0, 2023-12-11T18:04:33.606 INFO:teuthology.orchestra.run.smithi115.stdout: "unittest_2_bytes": 0, 2023-12-11T18:04:33.606 INFO:teuthology.orchestra.run.smithi115.stdout: "unittest_2_items": 0 2023-12-11T18:04:33.606 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:33.606 INFO:teuthology.orchestra.run.smithi115.stdout: "objecter": { 2023-12-11T18:04:33.606 INFO:teuthology.orchestra.run.smithi115.stdout: "op_active": 0, 2023-12-11T18:04:33.606 INFO:teuthology.orchestra.run.smithi115.stdout: "op_laggy": 0, 2023-12-11T18:04:33.606 INFO:teuthology.orchestra.run.smithi115.stdout: "op_send": 51311, 2023-12-11T18:04:33.606 INFO:teuthology.orchestra.run.smithi115.stdout: "op_send_bytes": 210015327, 2023-12-11T18:04:33.607 INFO:teuthology.orchestra.run.smithi115.stdout: "op_resend": 1, 2023-12-11T18:04:33.607 INFO:teuthology.orchestra.run.smithi115.stdout: "op_reply": 51309, 2023-12-11T18:04:33.607 INFO:teuthology.orchestra.run.smithi115.stdout: "op_latency": { 2023-12-11T18:04:33.607 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 51309, 2023-12-11T18:04:33.607 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 1440.537701700, 2023-12-11T18:04:33.607 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.028075731 2023-12-11T18:04:33.607 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:33.607 INFO:teuthology.orchestra.run.smithi115.stdout: "op_inflight": 0, 2023-12-11T18:04:33.607 INFO:teuthology.orchestra.run.smithi115.stdout: "oplen_avg": { 2023-12-11T18:04:33.607 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 51309, 2023-12-11T18:04:33.607 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 54505 2023-12-11T18:04:33.607 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:33.607 INFO:teuthology.orchestra.run.smithi115.stdout: "op": 51309, 2023-12-11T18:04:33.607 INFO:teuthology.orchestra.run.smithi115.stdout: "op_r": 949, 2023-12-11T18:04:33.607 INFO:teuthology.orchestra.run.smithi115.stdout: "op_w": 50360, 2023-12-11T18:04:33.608 INFO:teuthology.orchestra.run.smithi115.stdout: "op_rmw": 0, 2023-12-11T18:04:33.608 INFO:teuthology.orchestra.run.smithi115.stdout: "op_pg": 0, 2023-12-11T18:04:33.608 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_stat": 69, 2023-12-11T18:04:33.608 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_create": 1325, 2023-12-11T18:04:33.608 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_read": 369, 2023-12-11T18:04:33.608 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_write": 783, 2023-12-11T18:04:33.608 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_writefull": 124, 2023-12-11T18:04:33.608 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_writesame": 0, 2023-12-11T18:04:33.608 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_append": 0, 2023-12-11T18:04:33.608 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_zero": 2, 2023-12-11T18:04:33.608 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_truncate": 0, 2023-12-11T18:04:33.608 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_delete": 47794, 2023-12-11T18:04:33.608 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_mapext": 0, 2023-12-11T18:04:33.608 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_sparse_read": 0, 2023-12-11T18:04:33.609 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_clonerange": 0, 2023-12-11T18:04:33.609 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_getxattr": 13, 2023-12-11T18:04:33.609 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_setxattr": 2264, 2023-12-11T18:04:33.609 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_cmpxattr": 0, 2023-12-11T18:04:33.609 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_rmxattr": 0, 2023-12-11T18:04:33.609 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_resetxattrs": 0, 2023-12-11T18:04:33.609 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_call": 0, 2023-12-11T18:04:33.609 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_watch": 0, 2023-12-11T18:04:33.609 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_notify": 0, 2023-12-11T18:04:33.609 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_src_cmpxattr": 0, 2023-12-11T18:04:33.609 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_pgls": 0, 2023-12-11T18:04:33.609 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_pgls_filter": 0, 2023-12-11T18:04:33.609 INFO:teuthology.orchestra.run.smithi115.stdout: "osdop_other": 0, 2023-12-11T18:04:33.609 INFO:teuthology.orchestra.run.smithi115.stdout: "linger_active": 0, 2023-12-11T18:04:33.609 INFO:teuthology.orchestra.run.smithi115.stdout: "linger_send": 0, 2023-12-11T18:04:33.610 INFO:teuthology.orchestra.run.smithi115.stdout: "linger_resend": 0, 2023-12-11T18:04:33.610 INFO:teuthology.orchestra.run.smithi115.stdout: "linger_ping": 0, 2023-12-11T18:04:33.610 INFO:teuthology.orchestra.run.smithi115.stdout: "poolop_active": 0, 2023-12-11T18:04:33.610 INFO:teuthology.orchestra.run.smithi115.stdout: "poolop_send": 0, 2023-12-11T18:04:33.610 INFO:teuthology.orchestra.run.smithi115.stdout: "poolop_resend": 0, 2023-12-11T18:04:33.610 INFO:teuthology.orchestra.run.smithi115.stdout: "poolstat_active": 0, 2023-12-11T18:04:33.610 INFO:teuthology.orchestra.run.smithi115.stdout: "poolstat_send": 0, 2023-12-11T18:04:33.610 INFO:teuthology.orchestra.run.smithi115.stdout: "poolstat_resend": 0, 2023-12-11T18:04:33.610 INFO:teuthology.orchestra.run.smithi115.stdout: "statfs_active": 0, 2023-12-11T18:04:33.610 INFO:teuthology.orchestra.run.smithi115.stdout: "statfs_send": 0, 2023-12-11T18:04:33.610 INFO:teuthology.orchestra.run.smithi115.stdout: "statfs_resend": 0, 2023-12-11T18:04:33.610 INFO:teuthology.orchestra.run.smithi115.stdout: "command_active": 0, 2023-12-11T18:04:33.610 INFO:teuthology.orchestra.run.smithi115.stdout: "command_send": 0, 2023-12-11T18:04:33.610 INFO:teuthology.orchestra.run.smithi115.stdout: "command_resend": 0, 2023-12-11T18:04:33.610 INFO:teuthology.orchestra.run.smithi115.stdout: "map_epoch": 119, 2023-12-11T18:04:33.611 INFO:teuthology.orchestra.run.smithi115.stdout: "map_full": 0, 2023-12-11T18:04:33.611 INFO:teuthology.orchestra.run.smithi115.stdout: "map_inc": 19, 2023-12-11T18:04:33.611 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_sessions": 12, 2023-12-11T18:04:33.611 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_session_open": 12, 2023-12-11T18:04:33.611 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_session_close": 0, 2023-12-11T18:04:33.611 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_laggy": 0, 2023-12-11T18:04:33.611 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_wr": 655, 2023-12-11T18:04:33.611 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_rd": 784, 2023-12-11T18:04:33.611 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_del": 323 2023-12-11T18:04:33.611 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:33.611 INFO:teuthology.orchestra.run.smithi115.stdout: "oft": { 2023-12-11T18:04:33.611 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_total_objs": 0, 2023-12-11T18:04:33.611 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_total_kv_pairs": 0, 2023-12-11T18:04:33.611 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_total_updates": 0, 2023-12-11T18:04:33.612 INFO:teuthology.orchestra.run.smithi115.stdout: "omap_total_removes": 0 2023-12-11T18:04:33.612 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:33.612 INFO:teuthology.orchestra.run.smithi115.stdout: "osd-slow-ops": { 2023-12-11T18:04:33.612 INFO:teuthology.orchestra.run.smithi115.stdout: "slow_ops_count": 0 2023-12-11T18:04:33.612 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:33.612 INFO:teuthology.orchestra.run.smithi115.stdout: "purge_queue": { 2023-12-11T18:04:33.612 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executing_ops": 0, 2023-12-11T18:04:33.612 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executing_ops_high_water": 16, 2023-12-11T18:04:33.612 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executing": 0, 2023-12-11T18:04:33.612 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executing_high_water": 8, 2023-12-11T18:04:33.612 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executed_ops": 47694, 2023-12-11T18:04:33.612 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_executed": 23847, 2023-12-11T18:04:33.612 INFO:teuthology.orchestra.run.smithi115.stdout: "pq_item_in_journal": 0 2023-12-11T18:04:33.612 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:33.613 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-12-11T18:04:33.613 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2023-12-11T18:04:33.613 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 104857600, 2023-12-11T18:04:33.613 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2023-12-11T18:04:33.613 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 102760, 2023-12-11T18:04:33.613 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 347173065, 2023-12-11T18:04:33.613 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2023-12-11T18:04:33.613 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 102760, 2023-12-11T18:04:33.613 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 0, 2023-12-11T18:04:33.613 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 0, 2023-12-11T18:04:33.613 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 102760, 2023-12-11T18:04:33.613 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 347173065, 2023-12-11T18:04:33.613 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2023-12-11T18:04:33.613 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-12-11T18:04:33.613 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-12-11T18:04:33.614 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-12-11T18:04:33.614 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-11T18:04:33.614 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:33.614 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-objecter_bytes": { 2023-12-11T18:04:33.614 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2023-12-11T18:04:33.614 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 104857600, 2023-12-11T18:04:33.614 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2023-12-11T18:04:33.614 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 0, 2023-12-11T18:04:33.614 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 0, 2023-12-11T18:04:33.614 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2023-12-11T18:04:33.614 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 0, 2023-12-11T18:04:33.614 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 51309, 2023-12-11T18:04:33.614 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 522937476, 2023-12-11T18:04:33.614 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 51309, 2023-12-11T18:04:33.614 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 522937476, 2023-12-11T18:04:33.615 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2023-12-11T18:04:33.615 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-12-11T18:04:33.615 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-12-11T18:04:33.615 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-12-11T18:04:33.615 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-11T18:04:33.615 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:33.615 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-objecter_ops": { 2023-12-11T18:04:33.615 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2023-12-11T18:04:33.615 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 1024, 2023-12-11T18:04:33.615 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2023-12-11T18:04:33.615 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 0, 2023-12-11T18:04:33.615 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 0, 2023-12-11T18:04:33.615 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2023-12-11T18:04:33.615 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 0, 2023-12-11T18:04:33.615 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 51309, 2023-12-11T18:04:33.615 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 51309, 2023-12-11T18:04:33.616 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 51309, 2023-12-11T18:04:33.616 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 51309, 2023-12-11T18:04:33.616 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2023-12-11T18:04:33.616 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-12-11T18:04:33.616 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-12-11T18:04:33.616 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-12-11T18:04:33.616 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-11T18:04:33.616 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:33.616 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-write_buf_throttle": { 2023-12-11T18:04:33.616 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2023-12-11T18:04:33.616 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 3758096384, 2023-12-11T18:04:33.616 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2023-12-11T18:04:33.616 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 23846, 2023-12-11T18:04:33.616 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 3716278, 2023-12-11T18:04:33.616 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2023-12-11T18:04:33.617 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 23846, 2023-12-11T18:04:33.617 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 0, 2023-12-11T18:04:33.617 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 0, 2023-12-11T18:04:33.617 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 287, 2023-12-11T18:04:33.617 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 3716278, 2023-12-11T18:04:33.617 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2023-12-11T18:04:33.617 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-12-11T18:04:33.617 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-12-11T18:04:33.617 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-12-11T18:04:33.617 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-11T18:04:33.617 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-11T18:04:33.617 INFO:teuthology.orchestra.run.smithi115.stdout: "throttle-write_buf_throttle-0x556978e08960": { 2023-12-11T18:04:33.617 INFO:teuthology.orchestra.run.smithi115.stdout: "val": 0, 2023-12-11T18:04:33.617 INFO:teuthology.orchestra.run.smithi115.stdout: "max": 3758096384, 2023-12-11T18:04:33.617 INFO:teuthology.orchestra.run.smithi115.stdout: "get_started": 0, 2023-12-11T18:04:33.617 INFO:teuthology.orchestra.run.smithi115.stdout: "get": 48168, 2023-12-11T18:04:33.617 INFO:teuthology.orchestra.run.smithi115.stdout: "get_sum": 199297612, 2023-12-11T18:04:33.618 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_fail": 0, 2023-12-11T18:04:33.618 INFO:teuthology.orchestra.run.smithi115.stdout: "get_or_fail_success": 48168, 2023-12-11T18:04:33.618 INFO:teuthology.orchestra.run.smithi115.stdout: "take": 0, 2023-12-11T18:04:33.618 INFO:teuthology.orchestra.run.smithi115.stdout: "take_sum": 0, 2023-12-11T18:04:33.618 INFO:teuthology.orchestra.run.smithi115.stdout: "put": 496, 2023-12-11T18:04:33.618 INFO:teuthology.orchestra.run.smithi115.stdout: "put_sum": 199297612, 2023-12-11T18:04:33.618 INFO:teuthology.orchestra.run.smithi115.stdout: "wait": { 2023-12-11T18:04:33.618 INFO:teuthology.orchestra.run.smithi115.stdout: "avgcount": 0, 2023-12-11T18:04:33.618 INFO:teuthology.orchestra.run.smithi115.stdout: "sum": 0.000000000, 2023-12-11T18:04:33.618 INFO:teuthology.orchestra.run.smithi115.stdout: "avgtime": 0.000000000 2023-12-11T18:04:33.618 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-11T18:04:33.618 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-11T18:04:33.618 INFO:teuthology.orchestra.run.smithi115.stdout:} 2023-12-11T18:04:34.036 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:04:33 smithi115 ceph-mon[102748]: pgmap v581: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.7 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-11T18:04:34.110 INFO:tasks.check_counter:Daemon mds.k mds.exported=29 2023-12-11T18:04:34.111 INFO:tasks.check_counter:Daemon mds.k mds.imported=29 2023-12-11T18:04:34.111 INFO:tasks.check_counter:Daemon mds.k mds.root_rsnaps=6 2023-12-11T18:04:34.111 INFO:tasks.check_counter:Daemon mds.k mds_server.req_mksnap_latency.avgcount=0 2023-12-11T18:04:34.111 INFO:tasks.check_counter:Daemon mds.k mds_server.req_rmsnap_latency.avgcount=0 2023-12-11T18:04:34.111 INFO:tasks.check_counter:Daemon mds.k mds.dir_split=0 2023-12-11T18:04:34.111 DEBUG:tasks.check_counter:Getting stats from c 2023-12-11T18:04:34.111 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a62fda73efba5520d409bb5a21bd8373289dda5d shell --fsid d0a07eaa-984b-11ee-95a3-87774f69a715 -- ceph daemon mds.c perf dump 2023-12-11T18:04:34.112 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2023-12-11T18:04:34.114 ERROR:teuthology.run_tasks:Manager failed: check-counter Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/task/__init__.py", line 132, in __exit__ self.end() File "/home/teuthworker/src/git.ceph.com_ceph-c_a62fda73efba5520d409bb5a21bd8373289dda5d/qa/tasks/check_counter.py", line 84, in end proc = manager.admin_socket(daemon_type, daemon_id, ["perf", "dump"]) File "/home/teuthworker/src/git.ceph.com_ceph-c_a62fda73efba5520d409bb5a21bd8373289dda5d/qa/tasks/ceph_manager.py", line 1848, in admin_socket return shell( File "/home/teuthworker/src/git.ceph.com_ceph-c_a62fda73efba5520d409bb5a21bd8373289dda5d/qa/tasks/ceph_manager.py", line 43, in shell return remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/orchestra/run.py", line 450, in run r.execute() File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/orchestra/run.py", line 100, in execute self.client.exec_command(self.command) File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/virtualenv/lib/python3.8/site-packages/paramiko/client.py", line 525, in exec_command chan = self._transport.open_session(timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/virtualenv/lib/python3.8/site-packages/paramiko/transport.py", line 920, in open_session return self.open_channel( File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/virtualenv/lib/python3.8/site-packages/paramiko/transport.py", line 1051, in open_channel raise e File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/virtualenv/lib/python3.8/site-packages/paramiko/transport.py", line 2110, in run ptype, m = self.packetizer.read_message() File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/virtualenv/lib/python3.8/site-packages/paramiko/packet.py", line 459, in read_message header = self.read_all(self.__block_size_in, check_rekey=True) File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/virtualenv/lib/python3.8/site-packages/paramiko/packet.py", line 301, in read_all x = self.__socket.recv(n) File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/virtualenv/lib/python3.8/site-packages/gevent/_socketcommon.py", line 657, in recv return self._sock.recv(*args) ConnectionResetError: [Errno 104] Connection reset by peer 2023-12-11T18:04:34.117 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2023-12-11T18:04:34.127 INFO:tasks.kclient:Unmounting kernel clients... 2023-12-11T18:04:34.127 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2023-12-11T18:04:34.128 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-11T18:04:34.156 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:04:33 smithi059 ceph-mon[98847]: pgmap v581: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.7 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-11T18:04:34.190 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2023-12-11T18:04:34.191 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-11T18:04:34.294 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-11T18:04:34.592 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:34.590+0000 7feefd142700 1 -- 172.21.15.59:0/1622866565 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feef8109f70 msgr2=0x7feef810a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T18:04:34.592 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:34.590+0000 7feefd142700 1 --2- 172.21.15.59:0/1622866565 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feef8109f70 0x7feef810a350 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7feee8009990 tx=0x7feee8009ca0 comp rx=0 tx=0).stop 2023-12-11T18:04:34.599 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:34.597+0000 7feefd142700 1 -- 172.21.15.59:0/1622866565 shutdown_connections 2023-12-11T18:04:34.599 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:34.597+0000 7feefd142700 1 --2- 172.21.15.59:0/1622866565 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7feef8113520 0x7feef8115910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:04:34.599 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:34.597+0000 7feefd142700 1 --2- 172.21.15.59:0/1622866565 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feef8109f70 0x7feef810a350 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:04:34.600 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:34.597+0000 7feefd142700 1 --2- 172.21.15.59:0/1622866565 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feef810a890 0x7feef8112fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:04:34.600 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:34.597+0000 7feefd142700 1 -- 172.21.15.59:0/1622866565 >> 172.21.15.59:0/1622866565 conn(0x7feef806c540 msgr2=0x7feef806c950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T18:04:34.617 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:34.610+0000 7feefd142700 1 -- 172.21.15.59:0/1622866565 shutdown_connections 2023-12-11T18:04:34.617 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:34.612+0000 7feefd142700 1 -- 172.21.15.59:0/1622866565 wait complete. 2023-12-11T18:04:34.617 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:34.613+0000 7feefd142700 1 Processor -- start 2023-12-11T18:04:34.617 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:34.613+0000 7feefd142700 1 -- start start 2023-12-11T18:04:34.617 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:34.613+0000 7feefd142700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feef8109f70 0x7feef81a0170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T18:04:34.617 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:34.613+0000 7feefd142700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feef810a890 0x7feef81a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T18:04:34.617 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:34.613+0000 7feefd142700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7feef8113520 0x7feef81a4a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T18:04:34.617 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:34.613+0000 7feefd142700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feef81a5040 con 0x7feef8113520 2023-12-11T18:04:34.617 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:34.613+0000 7feefd142700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feef81a51b0 con 0x7feef8109f70 2023-12-11T18:04:34.617 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:34.613+0000 7feefd142700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feef81a5370 con 0x7feef810a890 2023-12-11T18:04:34.617 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:34.614+0000 7feef6d9d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feef8109f70 0x7feef81a0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T18:04:34.617 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:34.614+0000 7feef6d9d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feef8109f70 0x7feef81a0170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.59:60130/0 (socket says 172.21.15.59:60130) 2023-12-11T18:04:34.617 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:34.614+0000 7feef6d9d700 1 -- 172.21.15.59:0/3038073661 learned_addr learned my addr 172.21.15.59:0/3038073661 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T18:04:34.630 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:34.619+0000 7feef659c700 1 --2- 172.21.15.59:0/3038073661 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feef810a890 0x7feef81a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T18:04:34.630 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:34.623+0000 7feef759e700 1 --2- 172.21.15.59:0/3038073661 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7feef8113520 0x7feef81a4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T18:04:34.630 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:34.623+0000 7feef759e700 1 -- 172.21.15.59:0/3038073661 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feef810a890 msgr2=0x7feef81a06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T18:04:34.630 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:34.623+0000 7feef759e700 1 --2- 172.21.15.59:0/3038073661 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feef810a890 0x7feef81a06b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:04:34.630 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:34.623+0000 7feef759e700 1 -- 172.21.15.59:0/3038073661 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feef8109f70 msgr2=0x7feef81a0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T18:04:34.630 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:34.623+0000 7feef759e700 1 --2- 172.21.15.59:0/3038073661 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feef8109f70 0x7feef81a0170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:04:34.630 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:34.623+0000 7feef759e700 1 -- 172.21.15.59:0/3038073661 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feee80095f0 con 0x7feef8113520 2023-12-11T18:04:34.630 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:34.623+0000 7feef759e700 1 --2- 172.21.15.59:0/3038073661 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7feef8113520 0x7feef81a4a00 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7feeec00dfa0 tx=0x7feeec00fa60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T18:04:34.630 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:34.624+0000 7feedffff700 1 -- 172.21.15.59:0/3038073661 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeec018ce0 con 0x7feef8113520 2023-12-11T18:04:34.630 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:34.624+0000 7feedffff700 1 -- 172.21.15.59:0/3038073661 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feeec018e40 con 0x7feef8113520 2023-12-11T18:04:34.630 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:34.624+0000 7feedffff700 1 -- 172.21.15.59:0/3038073661 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeec01e5b0 con 0x7feef8113520 2023-12-11T18:04:34.631 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:34.624+0000 7feefd142700 1 -- 172.21.15.59:0/3038073661 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feef81a5650 con 0x7feef8113520 2023-12-11T18:04:34.631 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:34.624+0000 7feefd142700 1 -- 172.21.15.59:0/3038073661 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feef81161f0 con 0x7feef8113520 2023-12-11T18:04:34.631 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:34.626+0000 7feefd142700 1 -- 172.21.15.59:0/3038073661 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feee40052f0 con 0x7feef8113520 2023-12-11T18:04:34.639 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:34.628+0000 7feedffff700 1 -- 172.21.15.59:0/3038073661 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 28) v1 ==== 136137+0+0 (secure 0 0 0) 0x7feeec018440 con 0x7feef8113520 2023-12-11T18:04:34.639 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:34.629+0000 7feedffff700 1 --2- 172.21.15.59:0/3038073661 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7feee00b2750 0x7feee00b4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T18:04:34.639 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:34.629+0000 7feedffff700 1 -- 172.21.15.59:0/3038073661 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 9851+0+0 (secure 0 0 0) 0x7feeec009070 con 0x7feef8113520 2023-12-11T18:04:34.639 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:34.631+0000 7feef6d9d700 1 --2- 172.21.15.59:0/3038073661 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7feee00b2750 0x7feee00b4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T18:04:34.639 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:34.632+0000 7feef6d9d700 1 --2- 172.21.15.59:0/3038073661 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7feee00b2750 0x7feee00b4c10 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7feee8003760 tx=0x7feee800b560 comp rx=0 tx=0).ready entity=mgr.15414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T18:04:34.639 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:34.635+0000 7feedffff700 1 -- 172.21.15.59:0/3038073661 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7feeec096270 con 0x7feef8113520 2023-12-11T18:04:35.013 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:35.011+0000 7feefd142700 1 -- 172.21.15.59:0/3038073661 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7feee4005870 con 0x7feef8113520 2023-12-11T18:04:35.016 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:35.014+0000 7feedffff700 1 -- 172.21.15.59:0/3038073661 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v119) v1 ==== 74+0+25923 (secure 0 0 0) 0x7feeec099890 con 0x7feef8113520 2023-12-11T18:04:35.019 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-12-11T18:04:35.019 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":119,"fsid":"d0a07eaa-984b-11ee-95a3-87774f69a715","created":"2023-12-11T17:37:14.526539+0000","modified":"2023-12-11T18:04:11.237528+0000","last_up_change":"2023-12-11T17:58:28.975845+0000","last_in_change":"2023-12-11T17:43:21.014888+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":30,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-11T17:40:26.596029+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-11T17:44:18.586290+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-11T17:44:19.432952+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":27,"snap_epoch":119,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-11T17:44:23.097308+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":27,"snap_epoch":119,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3c0b3218-4b2b-40f8-8bef-ed8cd5633607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":1040759398},{"type":"v1","addr":"172.21.15.59:6803","nonce":1040759398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":1040759398},{"type":"v1","addr":"172.21.15.59:6805","nonce":1040759398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":1040759398},{"type":"v1","addr":"172.21.15.59:6809","nonce":1040759398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":1040759398},{"type":"v1","addr":"172.21.15.59:6807","nonce":1040759398}]},"public_addr":"172.21.15.59:6803/1040759398","cluster_addr":"172.21.15.59:6805/1040759398","heartbeat_back_addr":"172.21.15.59:6809/1040759398","heartbeat_front_addr":"172.21.15.59:6807/1040759398","state":["exists","up"]},{"osd":1,"uuid":"2ead7578-765b-4f5a-a72c-863731ed69e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":2776354245},{"type":"v1","addr":"172.21.15.115:6801","nonce":2776354245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":2776354245},{"type":"v1","addr":"172.21.15.115:6803","nonce":2776354245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":2776354245},{"type":"v1","addr":"172.21.15.115:6807","nonce":2776354245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":2776354245},{"type":"v1","addr":"172.21.15.115:6805","nonce":2776354245}]},"public_addr":"172.21.15.115:6801/2776354245","cluster_addr":"172.21.15.115:6803/2776354245","heartbeat_back_addr":"172.21.15.115:6807/2776354245","heartbeat_front_addr":"172.21.15.115:6805/2776354245","state":["exists","up"]},{"osd":2,"uuid":"8cb1d7ec-53cc-4672-b0b6-70d73797c44f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":18,"last_clean_end":97,"up_from":102,"up_thru":104,"down_at":98,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1449149834},{"type":"v1","addr":"172.21.15.181:6810","nonce":1449149834}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":1449149834},{"type":"v1","addr":"172.21.15.181:6814","nonce":1449149834}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":1449149834},{"type":"v1","addr":"172.21.15.181:6823","nonce":1449149834}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1449149834},{"type":"v1","addr":"172.21.15.181:6818","nonce":1449149834}]},"public_addr":"172.21.15.181:6810/1449149834","cluster_addr":"172.21.15.181:6814/1449149834","heartbeat_back_addr":"172.21.15.181:6823/1449149834","heartbeat_front_addr":"172.21.15.181:6818/1449149834","state":["exists","up"]},{"osd":3,"uuid":"5d64d9af-831b-4d0a-8ace-a452169351d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":219909995},{"type":"v1","addr":"172.21.15.59:6811","nonce":219909995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":219909995},{"type":"v1","addr":"172.21.15.59:6813","nonce":219909995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":219909995},{"type":"v1","addr":"172.21.15.59:6817","nonce":219909995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":219909995},{"type":"v1","addr":"172.21.15.59:6815","nonce":219909995}]},"public_addr":"172.21.15.59:6811/219909995","cluster_addr":"172.21.15.59:6813/219909995","heartbeat_back_addr":"172.21.15.59:6817/219909995","heartbeat_front_addr":"172.21.15.59:6815/219909995","state":["exists","up"]},{"osd":4,"uuid":"ef8fad28-1db5-4cba-b5f2-10e54e8e1549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":1021473826},{"type":"v1","addr":"172.21.15.115:6809","nonce":1021473826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":1021473826},{"type":"v1","addr":"172.21.15.115:6811","nonce":1021473826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":1021473826},{"type":"v1","addr":"172.21.15.115:6815","nonce":1021473826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":1021473826},{"type":"v1","addr":"172.21.15.115:6813","nonce":1021473826}]},"public_addr":"172.21.15.115:6809/1021473826","cluster_addr":"172.21.15.115:6811/1021473826","heartbeat_back_addr":"172.21.15.115:6815/1021473826","heartbeat_front_addr":"172.21.15.115:6813/1021473826","state":["exists","up"]},{"osd":5,"uuid":"4233a512-9e8f-4db8-aed2-6c3af706e589","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":37,"last_clean_end":97,"up_from":104,"up_thru":104,"down_at":98,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6809","nonce":3565629391},{"type":"v1","addr":"172.21.15.181:6811","nonce":3565629391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6813","nonce":3565629391},{"type":"v1","addr":"172.21.15.181:6815","nonce":3565629391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":3565629391},{"type":"v1","addr":"172.21.15.181:6822","nonce":3565629391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":3565629391},{"type":"v1","addr":"172.21.15.181:6819","nonce":3565629391}]},"public_addr":"172.21.15.181:6811/3565629391","cluster_addr":"172.21.15.181:6815/3565629391","heartbeat_back_addr":"172.21.15.181:6822/3565629391","heartbeat_front_addr":"172.21.15.181:6819/3565629391","state":["exists","up"]},{"osd":6,"uuid":"8faab332-016e-4c77-8370-8bef95fc6f34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":815374853},{"type":"v1","addr":"172.21.15.59:6819","nonce":815374853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":815374853},{"type":"v1","addr":"172.21.15.59:6821","nonce":815374853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":815374853},{"type":"v1","addr":"172.21.15.59:6825","nonce":815374853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":815374853},{"type":"v1","addr":"172.21.15.59:6823","nonce":815374853}]},"public_addr":"172.21.15.59:6819/815374853","cluster_addr":"172.21.15.59:6821/815374853","heartbeat_back_addr":"172.21.15.59:6825/815374853","heartbeat_front_addr":"172.21.15.59:6823/815374853","state":["exists","up"]},{"osd":7,"uuid":"f20f470a-0d40-4ed3-8e0e-fd58675cf143","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":1314971870},{"type":"v1","addr":"172.21.15.115:6817","nonce":1314971870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":1314971870},{"type":"v1","addr":"172.21.15.115:6819","nonce":1314971870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":1314971870},{"type":"v1","addr":"172.21.15.115:6823","nonce":1314971870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":1314971870},{"type":"v1","addr":"172.21.15.115:6821","nonce":1314971870}]},"public_addr":"172.21.15.115:6817/1314971870","cluster_addr":"172.21.15.115:6819/1314971870","heartbeat_back_addr":"172.21.15.115:6823/1314971870","heartbeat_front_addr":"172.21.15.115:6821/1314971870","state":["exists","up"]},{"osd":8,"uuid":"eb657d32-1a95-4fd7-a293-d0a970837fc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":57,"last_clean_end":97,"up_from":103,"up_thru":104,"down_at":98,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":3914011605},{"type":"v1","addr":"172.21.15.181:6833","nonce":3914011605}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3914011605},{"type":"v1","addr":"172.21.15.181:6835","nonce":3914011605}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":3914011605},{"type":"v1","addr":"172.21.15.181:6839","nonce":3914011605}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":3914011605},{"type":"v1","addr":"172.21.15.181:6837","nonce":3914011605}]},"public_addr":"172.21.15.181:6833/3914011605","cluster_addr":"172.21.15.181:6835/3914011605","heartbeat_back_addr":"172.21.15.181:6839/3914011605","heartbeat_front_addr":"172.21.15.181:6837/3914011605","state":["exists","up"]},{"osd":9,"uuid":"8f7717d4-b6f2-4ec1-9b58-62feb0c9a146","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":1163095677},{"type":"v1","addr":"172.21.15.59:6827","nonce":1163095677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":1163095677},{"type":"v1","addr":"172.21.15.59:6829","nonce":1163095677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":1163095677},{"type":"v1","addr":"172.21.15.59:6833","nonce":1163095677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":1163095677},{"type":"v1","addr":"172.21.15.59:6831","nonce":1163095677}]},"public_addr":"172.21.15.59:6827/1163095677","cluster_addr":"172.21.15.59:6829/1163095677","heartbeat_back_addr":"172.21.15.59:6833/1163095677","heartbeat_front_addr":"172.21.15.59:6831/1163095677","state":["exists","up"]},{"osd":10,"uuid":"b75c179a-875d-43bc-b4c1-e8eb80fb64fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":2199610269},{"type":"v1","addr":"172.21.15.115:6825","nonce":2199610269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":2199610269},{"type":"v1","addr":"172.21.15.115:6827","nonce":2199610269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":2199610269},{"type":"v1","addr":"172.21.15.115:6831","nonce":2199610269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":2199610269},{"type":"v1","addr":"172.21.15.115:6829","nonce":2199610269}]},"public_addr":"172.21.15.115:6825/2199610269","cluster_addr":"172.21.15.115:6827/2199610269","heartbeat_back_addr":"172.21.15.115:6831/2199610269","heartbeat_front_addr":"172.21.15.115:6829/2199610269","state":["exists","up"]},{"osd":11,"uuid":"5e141dfe-8a00-4bbc-8204-2dc9be24e18d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":77,"last_clean_end":97,"up_from":102,"up_thru":104,"down_at":98,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":540621004},{"type":"v1","addr":"172.21.15.181:6825","nonce":540621004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":540621004},{"type":"v1","addr":"172.21.15.181:6827","nonce":540621004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":540621004},{"type":"v1","addr":"172.21.15.181:6831","nonce":540621004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":540621004},{"type":"v1","addr":"172.21.15.181:6829","nonce":540621004}]},"public_addr":"172.21.15.181:6825/540621004","cluster_addr":"172.21.15.181:6827/540621004","heartbeat_back_addr":"172.21.15.181:6831/540621004","heartbeat_front_addr":"172.21.15.181:6829/540621004","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:39:42.687084+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:40:03.903448+0000","dead_epoch":0},{"osd":2,"down_stamp":"2023-12-11T17:56:22.121381+0000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:40:24.899908+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:40:46.600251+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:41:07.690037+0000","dead_epoch":0},{"osd":5,"down_stamp":"2023-12-11T17:56:22.121381+0000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:41:28.488855+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:41:49.459274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:42:10.863881+0000","dead_epoch":0},{"osd":8,"down_stamp":"2023-12-11T17:56:22.121381+0000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:42:32.248391+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:42:54.112265+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:43:15.634872+0000","dead_epoch":0},{"osd":11,"down_stamp":"2023-12-11T17:56:22.121381+0000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:43:36.922534+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":7}]},{"pgid":"3.28","mappings":[{"from":5,"to":6},{"from":3,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.37","mappings":[{"from":1,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"4.6","mappings":[{"from":9,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:6839/2214384806":"2023-12-12T17:57:14.937455+0000","172.21.15.59:6801/3627728353":"2023-12-12T17:46:17.753068+0000","172.21.15.59:0/2192742819":"2023-12-12T17:37:44.773925+0000","172.21.15.181:6838/2214384806":"2023-12-12T17:57:14.937455+0000","172.21.15.59:0/2859415380":"2023-12-12T17:38:14.037922+0000","172.21.15.59:0/644616907":"2023-12-12T17:46:17.753068+0000","172.21.15.59:0/3853462899":"2023-12-12T17:37:44.773925+0000","172.21.15.59:0/4159306605":"2023-12-12T17:46:17.753068+0000","172.21.15.59:6800/3627728353":"2023-12-12T17:46:17.753068+0000","172.21.15.59:0/3570767650":"2023-12-12T17:37:44.773925+0000","172.21.15.59:6800/1672530462":"2023-12-12T17:37:44.773925+0000","172.21.15.59:6801/1259916482":"2023-12-12T17:38:14.037922+0000","172.21.15.59:0/1053944547":"2023-12-12T17:38:14.037922+0000","172.21.15.59:6801/1672530462":"2023-12-12T17:37:44.773925+0000","172.21.15.59:0/2683787475":"2023-12-12T17:46:17.753068+0000","172.21.15.59:0/17457774":"2023-12-12T17:38:14.037922+0000","172.21.15.59:6800/1259916482":"2023-12-12T17:38:14.037922+0000","172.21.15.59:0/2112420116":"2023-12-12T17:46:17.753068+0000","172.21.15.59:0/1373005911":"2023-12-12T17:46:17.753068+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":12,"length":1},{"begin":14,"length":1},{"begin":21,"length":1},{"begin":23,"length":1},{"begin":25,"length":1},{"begin":27,"length":1}]},{"pool":4,"snaps":[{"begin":10,"length":1},{"begin":12,"length":1},{"begin":14,"length":1},{"begin":21,"length":1},{"begin":23,"length":1},{"begin":25,"length":1},{"begin":27,"length":1}]}],"new_removed_snaps":[{"pool":3,"snaps":[{"begin":14,"length":1},{"begin":27,"length":1}]},{"pool":4,"snaps":[{"begin":14,"length":1},{"begin":27,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-11T18:04:35.039 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:35.029+0000 7feefd142700 1 -- 172.21.15.59:0/3038073661 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7feee00b2750 msgr2=0x7feee00b4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T18:04:35.039 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:35.029+0000 7feefd142700 1 --2- 172.21.15.59:0/3038073661 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7feee00b2750 0x7feee00b4c10 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7feee8003760 tx=0x7feee800b560 comp rx=0 tx=0).stop 2023-12-11T18:04:35.039 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:35.029+0000 7feefd142700 1 -- 172.21.15.59:0/3038073661 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7feef8113520 msgr2=0x7feef81a4a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T18:04:35.039 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:35.029+0000 7feefd142700 1 --2- 172.21.15.59:0/3038073661 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7feef8113520 0x7feef81a4a00 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7feeec00dfa0 tx=0x7feeec00fa60 comp rx=0 tx=0).stop 2023-12-11T18:04:35.039 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:35.035+0000 7feefd142700 1 -- 172.21.15.59:0/3038073661 shutdown_connections 2023-12-11T18:04:35.039 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:35.035+0000 7feefd142700 1 --2- 172.21.15.59:0/3038073661 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7feee00b2750 0x7feee00b4c10 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:04:35.039 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:35.035+0000 7feefd142700 1 --2- 172.21.15.59:0/3038073661 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7feef8113520 0x7feef81a4a00 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:04:35.039 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:35.035+0000 7feefd142700 1 --2- 172.21.15.59:0/3038073661 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feef8109f70 0x7feef81a0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:04:35.039 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:35.035+0000 7feefd142700 1 --2- 172.21.15.59:0/3038073661 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feef810a890 0x7feef81a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:04:35.040 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:35.035+0000 7feefd142700 1 -- 172.21.15.59:0/3038073661 >> 172.21.15.59:0/3038073661 conn(0x7feef806c540 msgr2=0x7feef8070170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T18:04:35.040 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:35.038+0000 7feefd142700 1 -- 172.21.15.59:0/3038073661 shutdown_connections 2023-12-11T18:04:35.047 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:35.045+0000 7feefd142700 1 -- 172.21.15.59:0/3038073661 wait complete. 2023-12-11T18:04:35.087 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-11T17:40:26.596029+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-11T17:44:18.586290+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-11T17:44:19.432952+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '119', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 27, 'snap_epoch': 119, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-12-11T17:44:23.097308+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '119', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 27, 'snap_epoch': 119, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-12-11T18:04:35.087 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-11T18:04:35.427 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:35.424+0000 7fdb6cb99700 1 -- 172.21.15.59:0/1007077725 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdb6811adb0 msgr2=0x7fdb6811d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T18:04:35.427 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:35.424+0000 7fdb6cb99700 1 --2- 172.21.15.59:0/1007077725 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdb6811adb0 0x7fdb6811d1a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fdb580098f0 tx=0x7fdb58009c00 comp rx=0 tx=0).stop 2023-12-11T18:04:35.435 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:35.432+0000 7fdb6cb99700 1 -- 172.21.15.59:0/1007077725 shutdown_connections 2023-12-11T18:04:35.435 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:35.432+0000 7fdb6cb99700 1 --2- 172.21.15.59:0/1007077725 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdb6811adb0 0x7fdb6811d1a0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:04:35.435 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:35.432+0000 7fdb6cb99700 1 --2- 172.21.15.59:0/1007077725 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdb6810fd60 0x7fdb681085d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:04:35.435 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:35.432+0000 7fdb6cb99700 1 --2- 172.21.15.59:0/1007077725 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb68108b10 0x7fdb68108f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:04:35.435 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:35.432+0000 7fdb6cb99700 1 -- 172.21.15.59:0/1007077725 >> 172.21.15.59:0/1007077725 conn(0x7fdb6806c610 msgr2=0x7fdb6806ca20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T18:04:35.437 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:35.435+0000 7fdb6cb99700 1 -- 172.21.15.59:0/1007077725 shutdown_connections 2023-12-11T18:04:35.444 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:35.441+0000 7fdb6cb99700 1 -- 172.21.15.59:0/1007077725 wait complete. 2023-12-11T18:04:35.444 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:35.442+0000 7fdb6cb99700 1 Processor -- start 2023-12-11T18:04:35.455 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:35.453+0000 7fdb6cb99700 1 -- start start 2023-12-11T18:04:35.455 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:35.453+0000 7fdb6cb99700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdb68108b10 0x7fdb681191b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T18:04:35.455 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:35.453+0000 7fdb6cb99700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb6810fd60 0x7fdb68119730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T18:04:35.455 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:35.453+0000 7fdb6cb99700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdb6811adb0 0x7fdb681123c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T18:04:35.455 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:35.453+0000 7fdb6cb99700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb68112b90 con 0x7fdb6810fd60 2023-12-11T18:04:35.455 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:35.453+0000 7fdb6cb99700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb68112d00 con 0x7fdb6811adb0 2023-12-11T18:04:35.455 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:35.453+0000 7fdb6cb99700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb68112ec0 con 0x7fdb68108b10 2023-12-11T18:04:35.460 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:35.458+0000 7fdb66d9d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdb6811adb0 0x7fdb681123c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T18:04:35.460 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:35.458+0000 7fdb66d9d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdb6811adb0 0x7fdb681123c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.59:60148/0 (socket says 172.21.15.59:60148) 2023-12-11T18:04:35.460 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:35.458+0000 7fdb66d9d700 1 -- 172.21.15.59:0/2926709740 learned_addr learned my addr 172.21.15.59:0/2926709740 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T18:04:35.462 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:35.460+0000 7fdb6659c700 1 --2- 172.21.15.59:0/2926709740 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdb68108b10 0x7fdb681191b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T18:04:35.462 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:35.460+0000 7fdb66d9d700 1 -- 172.21.15.59:0/2926709740 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdb68108b10 msgr2=0x7fdb681191b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T18:04:35.463 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:35.460+0000 7fdb66d9d700 1 --2- 172.21.15.59:0/2926709740 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdb68108b10 0x7fdb681191b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:04:35.463 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:35.460+0000 7fdb66d9d700 1 -- 172.21.15.59:0/2926709740 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb6810fd60 msgr2=0x7fdb68119730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T18:04:35.463 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:35.460+0000 7fdb66d9d700 1 --2- 172.21.15.59:0/2926709740 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb6810fd60 0x7fdb68119730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:04:35.463 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:35.460+0000 7fdb66d9d700 1 -- 172.21.15.59:0/2926709740 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb58009520 con 0x7fdb6811adb0 2023-12-11T18:04:35.463 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:35.460+0000 7fdb6659c700 1 --2- 172.21.15.59:0/2926709740 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdb68108b10 0x7fdb681191b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T18:04:35.463 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:35.461+0000 7fdb66d9d700 1 --2- 172.21.15.59:0/2926709740 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdb6811adb0 0x7fdb681123c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fdb5800fa00 tx=0x7fdb5800b560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T18:04:35.463 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:35.461+0000 7fdb5f7fe700 1 -- 172.21.15.59:0/2926709740 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb5800bc10 con 0x7fdb6811adb0 2023-12-11T18:04:35.463 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:35.461+0000 7fdb6cb99700 1 -- 172.21.15.59:0/2926709740 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb68113140 con 0x7fdb6811adb0 2023-12-11T18:04:35.464 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:35.461+0000 7fdb6cb99700 1 -- 172.21.15.59:0/2926709740 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb68116fd0 con 0x7fdb6811adb0 2023-12-11T18:04:35.464 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:35.461+0000 7fdb5f7fe700 1 -- 172.21.15.59:0/2926709740 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdb58005500 con 0x7fdb6811adb0 2023-12-11T18:04:35.464 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:35.461+0000 7fdb5f7fe700 1 -- 172.21.15.59:0/2926709740 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb58017400 con 0x7fdb6811adb0 2023-12-11T18:04:35.467 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:35.464+0000 7fdb6cb99700 1 -- 172.21.15.59:0/2926709740 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb480052f0 con 0x7fdb6811adb0 2023-12-11T18:04:35.468 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:35.465+0000 7fdb5f7fe700 1 -- 172.21.15.59:0/2926709740 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 28) v1 ==== 136137+0+0 (secure 0 0 0) 0x7fdb58003680 con 0x7fdb6811adb0 2023-12-11T18:04:35.468 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:35.465+0000 7fdb5f7fe700 1 --2- 172.21.15.59:0/2926709740 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7fdb540a16f0 0x7fdb540a3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T18:04:35.468 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:35.466+0000 7fdb5f7fe700 1 -- 172.21.15.59:0/2926709740 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 9851+0+0 (secure 0 0 0) 0x7fdb580ce950 con 0x7fdb6811adb0 2023-12-11T18:04:35.468 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:35.466+0000 7fdb6659c700 1 --2- 172.21.15.59:0/2926709740 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7fdb540a16f0 0x7fdb540a3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T18:04:35.468 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:35.466+0000 7fdb6659c700 1 --2- 172.21.15.59:0/2926709740 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7fdb540a16f0 0x7fdb540a3bb0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fdb500099d0 tx=0x7fdb50008040 comp rx=0 tx=0).ready entity=mgr.15414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T18:04:35.478 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:35.476+0000 7fdb5f7fe700 1 -- 172.21.15.59:0/2926709740 <== mon.2 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fdb58097770 con 0x7fdb6811adb0 2023-12-11T18:04:35.846 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:35.844+0000 7fdb6cb99700 1 -- 172.21.15.59:0/2926709740 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdb48005b20 con 0x7fdb6811adb0 2023-12-11T18:04:35.847 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:35.845+0000 7fdb5f7fe700 1 -- 172.21.15.59:0/2926709740 <== mon.2 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v119) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdb5809ad90 con 0x7fdb6811adb0 2023-12-11T18:04:35.852 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2023-12-11T18:04:35.853 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:35.850+0000 7fdb6cb99700 1 -- 172.21.15.59:0/2926709740 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7fdb540a16f0 msgr2=0x7fdb540a3bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T18:04:35.853 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:35.850+0000 7fdb6cb99700 1 --2- 172.21.15.59:0/2926709740 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7fdb540a16f0 0x7fdb540a3bb0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fdb500099d0 tx=0x7fdb50008040 comp rx=0 tx=0).stop 2023-12-11T18:04:35.853 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:35.850+0000 7fdb6cb99700 1 -- 172.21.15.59:0/2926709740 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdb6811adb0 msgr2=0x7fdb681123c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T18:04:35.853 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:35.850+0000 7fdb6cb99700 1 --2- 172.21.15.59:0/2926709740 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdb6811adb0 0x7fdb681123c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fdb5800fa00 tx=0x7fdb5800b560 comp rx=0 tx=0).stop 2023-12-11T18:04:35.853 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:35.850+0000 7fdb6cb99700 1 -- 172.21.15.59:0/2926709740 shutdown_connections 2023-12-11T18:04:35.853 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:35.850+0000 7fdb6cb99700 1 --2- 172.21.15.59:0/2926709740 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7fdb540a16f0 0x7fdb540a3bb0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:04:35.854 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:35.850+0000 7fdb6cb99700 1 --2- 172.21.15.59:0/2926709740 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdb6811adb0 0x7fdb681123c0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:04:35.854 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:35.850+0000 7fdb6cb99700 1 --2- 172.21.15.59:0/2926709740 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdb68108b10 0x7fdb681191b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:04:35.854 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:35.850+0000 7fdb6cb99700 1 --2- 172.21.15.59:0/2926709740 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb6810fd60 0x7fdb68119730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:04:35.854 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:35.850+0000 7fdb6cb99700 1 -- 172.21.15.59:0/2926709740 >> 172.21.15.59:0/2926709740 conn(0x7fdb6806c610 msgr2=0x7fdb6806ffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T18:04:35.854 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:35.850+0000 7fdb6cb99700 1 -- 172.21.15.59:0/2926709740 shutdown_connections 2023-12-11T18:04:35.854 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:35.850+0000 7fdb6cb99700 1 -- 172.21.15.59:0/2926709740 wait complete. 2023-12-11T18:04:35.873 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-11T18:04:36.040 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:04:35 smithi115 ceph-mon[102748]: pgmap v582: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.7 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-12-11T18:04:36.040 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:04:35 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/3038073661' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-11T18:04:36.151 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:36.149+0000 7f4a128c3700 1 -- 172.21.15.59:0/3174192881 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4a0c100fa0 msgr2=0x7f4a0c101420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T18:04:36.151 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:36.149+0000 7f4a128c3700 1 --2- 172.21.15.59:0/3174192881 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4a0c100fa0 0x7f4a0c101420 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f4a0000faf0 tx=0x7f4a0000fe00 comp rx=0 tx=0).stop 2023-12-11T18:04:36.156 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:04:35 smithi059 ceph-mon[98847]: pgmap v582: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.7 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-12-11T18:04:36.156 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:04:35 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/3038073661' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-11T18:04:36.161 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:36.159+0000 7f4a128c3700 1 -- 172.21.15.59:0/3174192881 shutdown_connections 2023-12-11T18:04:36.161 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:36.159+0000 7f4a128c3700 1 --2- 172.21.15.59:0/3174192881 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4a0c101960 0x7f4a0c10d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:04:36.161 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:36.159+0000 7f4a128c3700 1 --2- 172.21.15.59:0/3174192881 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4a0c105410 0x7f4a0c1057f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:04:36.161 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:36.159+0000 7f4a128c3700 1 --2- 172.21.15.59:0/3174192881 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4a0c100fa0 0x7f4a0c101420 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:04:36.161 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:36.159+0000 7f4a128c3700 1 -- 172.21.15.59:0/3174192881 >> 172.21.15.59:0/3174192881 conn(0x7f4a0c075740 msgr2=0x7f4a0c075b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T18:04:36.180 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:36.172+0000 7f4a128c3700 1 -- 172.21.15.59:0/3174192881 shutdown_connections 2023-12-11T18:04:36.186 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:36.180+0000 7f4a128c3700 1 -- 172.21.15.59:0/3174192881 wait complete. 2023-12-11T18:04:36.186 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:36.181+0000 7f4a128c3700 1 Processor -- start 2023-12-11T18:04:36.186 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:36.181+0000 7f4a128c3700 1 -- start start 2023-12-11T18:04:36.186 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:36.182+0000 7f4a128c3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4a0c100fa0 0x7f4a0c072c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T18:04:36.186 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:36.182+0000 7f4a128c3700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4a0c101960 0x7f4a0c073190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T18:04:36.186 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:36.182+0000 7f4a128c3700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4a0c105410 0x7f4a0c06bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T18:04:36.186 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:36.182+0000 7f4a128c3700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a0c06c550 con 0x7f4a0c101960 2023-12-11T18:04:36.186 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:36.182+0000 7f4a128c3700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a0c06c6c0 con 0x7f4a0c105410 2023-12-11T18:04:36.186 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:36.182+0000 7f4a128c3700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a0c06c880 con 0x7f4a0c100fa0 2023-12-11T18:04:36.187 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:36.183+0000 7f4a0b7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4a0c101960 0x7f4a0c073190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T18:04:36.187 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:36.183+0000 7f4a0b7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4a0c101960 0x7f4a0c073190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35430/0 (socket says 172.21.15.59:35430) 2023-12-11T18:04:36.187 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:36.183+0000 7f4a0b7fe700 1 -- 172.21.15.59:0/575415052 learned_addr learned my addr 172.21.15.59:0/575415052 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T18:04:36.187 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:36.183+0000 7f4a10e60700 1 --2- 172.21.15.59:0/575415052 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4a0c105410 0x7f4a0c06bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T18:04:36.187 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:36.183+0000 7f4a0b7fe700 1 -- 172.21.15.59:0/575415052 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4a0c100fa0 msgr2=0x7f4a0c072c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T18:04:36.187 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:36.183+0000 7f4a0b7fe700 1 --2- 172.21.15.59:0/575415052 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4a0c100fa0 0x7f4a0c072c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:04:36.187 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:36.183+0000 7f4a0b7fe700 1 -- 172.21.15.59:0/575415052 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4a0c105410 msgr2=0x7f4a0c06bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T18:04:36.187 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:36.183+0000 7f4a0b7fe700 1 --2- 172.21.15.59:0/575415052 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4a0c105410 0x7f4a0c06bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:04:36.187 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:36.183+0000 7f4a0b7fe700 1 -- 172.21.15.59:0/575415052 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a0000f750 con 0x7f4a0c101960 2023-12-11T18:04:36.193 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:36.191+0000 7f4a0b7fe700 1 --2- 172.21.15.59:0/575415052 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4a0c101960 0x7f4a0c073190 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f4a00019040 tx=0x7f4a000043a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T18:04:36.193 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:36.191+0000 7f4a097fa700 1 -- 172.21.15.59:0/575415052 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a00003a90 con 0x7f4a0c101960 2023-12-11T18:04:36.194 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:36.192+0000 7f4a097fa700 1 -- 172.21.15.59:0/575415052 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a000054f0 con 0x7f4a0c101960 2023-12-11T18:04:36.194 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:36.192+0000 7f4a097fa700 1 -- 172.21.15.59:0/575415052 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a000048e0 con 0x7f4a0c101960 2023-12-11T18:04:36.196 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:36.192+0000 7f4a128c3700 1 -- 172.21.15.59:0/575415052 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a0c06cb00 con 0x7f4a0c101960 2023-12-11T18:04:36.196 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:36.192+0000 7f4a128c3700 1 -- 172.21.15.59:0/575415052 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a0c070a10 con 0x7f4a0c101960 2023-12-11T18:04:36.204 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:36.195+0000 7f4a097fa700 1 -- 172.21.15.59:0/575415052 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 28) v1 ==== 136137+0+0 (secure 0 0 0) 0x7f4a00005030 con 0x7f4a0c101960 2023-12-11T18:04:36.205 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:36.196+0000 7f4a097fa700 1 --2- 172.21.15.59:0/575415052 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7f49f40a1740 0x7f49f40a3c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T18:04:36.205 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:36.196+0000 7f4a097fa700 1 -- 172.21.15.59:0/575415052 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 9851+0+0 (secure 0 0 0) 0x7f4a000ce3c0 con 0x7f4a0c101960 2023-12-11T18:04:36.205 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:36.199+0000 7f4a0bfff700 1 --2- 172.21.15.59:0/575415052 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7f49f40a1740 0x7f49f40a3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T18:04:36.205 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:36.200+0000 7f4a0bfff700 1 --2- 172.21.15.59:0/575415052 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7f49f40a1740 0x7f49f40a3c00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c073aa0 tx=0x7f49fc009260 comp rx=0 tx=0).ready entity=mgr.15414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T18:04:36.205 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:36.203+0000 7f4a128c3700 1 -- 172.21.15.59:0/575415052 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49f80052f0 con 0x7f4a0c101960 2023-12-11T18:04:36.220 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:36.214+0000 7f4a097fa700 1 -- 172.21.15.59:0/575415052 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f4a00097240 con 0x7f4a0c101960 2023-12-11T18:04:36.577 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:36.571+0000 7f4a128c3700 1 -- 172.21.15.59:0/575415052 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f49f8005b20 con 0x7f4a0c101960 2023-12-11T18:04:36.577 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:36.573+0000 7f4a097fa700 1 -- 172.21.15.59:0/575415052 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v119) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4a00024020 con 0x7f4a0c101960 2023-12-11T18:04:36.585 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2023-12-11T18:04:36.591 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:36.589+0000 7f4a128c3700 1 -- 172.21.15.59:0/575415052 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7f49f40a1740 msgr2=0x7f49f40a3c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T18:04:36.591 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:36.589+0000 7f4a128c3700 1 --2- 172.21.15.59:0/575415052 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7f49f40a1740 0x7f49f40a3c00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c073aa0 tx=0x7f49fc009260 comp rx=0 tx=0).stop 2023-12-11T18:04:36.591 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:36.589+0000 7f4a128c3700 1 -- 172.21.15.59:0/575415052 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4a0c101960 msgr2=0x7f4a0c073190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T18:04:36.591 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:36.589+0000 7f4a128c3700 1 --2- 172.21.15.59:0/575415052 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4a0c101960 0x7f4a0c073190 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f4a00019040 tx=0x7f4a000043a0 comp rx=0 tx=0).stop 2023-12-11T18:04:36.603 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:36.598+0000 7f4a128c3700 1 -- 172.21.15.59:0/575415052 shutdown_connections 2023-12-11T18:04:36.603 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:36.598+0000 7f4a128c3700 1 --2- 172.21.15.59:0/575415052 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7f49f40a1740 0x7f49f40a3c00 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:04:36.603 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:36.598+0000 7f4a128c3700 1 --2- 172.21.15.59:0/575415052 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4a0c105410 0x7f4a0c06bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:04:36.603 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:36.598+0000 7f4a128c3700 1 --2- 172.21.15.59:0/575415052 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4a0c100fa0 0x7f4a0c072c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:04:36.603 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:36.598+0000 7f4a128c3700 1 --2- 172.21.15.59:0/575415052 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4a0c101960 0x7f4a0c073190 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:04:36.603 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:36.598+0000 7f4a128c3700 1 -- 172.21.15.59:0/575415052 >> 172.21.15.59:0/575415052 conn(0x7f4a0c075740 msgr2=0x7f4a0c0fdf40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T18:04:36.603 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:36.598+0000 7f4a128c3700 1 -- 172.21.15.59:0/575415052 shutdown_connections 2023-12-11T18:04:36.603 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:36.598+0000 7f4a128c3700 1 -- 172.21.15.59:0/575415052 wait complete. 2023-12-11T18:04:36.624 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-11T18:04:36.913 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:04:36 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/2926709740' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-11T18:04:36.913 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:04:36 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/575415052' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-11T18:04:36.934 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:36.932+0000 7faa2cde5700 1 -- 172.21.15.59:0/682401353 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa28108b10 msgr2=0x7faa28108f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T18:04:36.934 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:36.932+0000 7faa2cde5700 1 --2- 172.21.15.59:0/682401353 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa28108b10 0x7faa28108f90 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7faa1c009920 tx=0x7faa1c009c30 comp rx=0 tx=0).stop 2023-12-11T18:04:36.939 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:36.936+0000 7faa2cde5700 1 -- 172.21.15.59:0/682401353 shutdown_connections 2023-12-11T18:04:36.939 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:36.936+0000 7faa2cde5700 1 --2- 172.21.15.59:0/682401353 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faa2811adb0 0x7faa2811d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:04:36.939 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:36.936+0000 7faa2cde5700 1 --2- 172.21.15.59:0/682401353 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa2810fd60 0x7faa281085d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:04:36.939 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:36.936+0000 7faa2cde5700 1 --2- 172.21.15.59:0/682401353 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa28108b10 0x7faa28108f90 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:04:36.939 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:36.936+0000 7faa2cde5700 1 -- 172.21.15.59:0/682401353 >> 172.21.15.59:0/682401353 conn(0x7faa2806c610 msgr2=0x7faa2806ca20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T18:04:36.943 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:36.940+0000 7faa2cde5700 1 -- 172.21.15.59:0/682401353 shutdown_connections 2023-12-11T18:04:36.943 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:36.940+0000 7faa2cde5700 1 -- 172.21.15.59:0/682401353 wait complete. 2023-12-11T18:04:36.943 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:36.941+0000 7faa2cde5700 1 Processor -- start 2023-12-11T18:04:36.944 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:36.941+0000 7faa2cde5700 1 -- start start 2023-12-11T18:04:36.944 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:36.941+0000 7faa2cde5700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa28108b10 0x7faa281ac560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T18:04:36.944 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:36.941+0000 7faa2cde5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa2810fd60 0x7faa281acaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T18:04:36.944 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:36.941+0000 7faa2cde5700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faa2811adb0 0x7faa281a6690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T18:04:36.944 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:36.941+0000 7faa2cde5700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa281a6cf0 con 0x7faa28108b10 2023-12-11T18:04:36.944 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:36.941+0000 7faa2cde5700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa281a6e60 con 0x7faa2811adb0 2023-12-11T18:04:36.944 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:36.941+0000 7faa2cde5700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa281a7020 con 0x7faa2810fd60 2023-12-11T18:04:36.944 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:36.942+0000 7faa2659c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa28108b10 0x7faa281ac560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T18:04:36.944 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:36.942+0000 7faa2659c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa28108b10 0x7faa281ac560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35444/0 (socket says 172.21.15.59:35444) 2023-12-11T18:04:36.944 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:36.942+0000 7faa2659c700 1 -- 172.21.15.59:0/3101581180 learned_addr learned my addr 172.21.15.59:0/3101581180 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T18:04:36.952 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:36.942+0000 7faa2659c700 1 -- 172.21.15.59:0/3101581180 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa2810fd60 msgr2=0x7faa281acaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T18:04:36.952 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:36.942+0000 7faa2659c700 1 --2- 172.21.15.59:0/3101581180 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa2810fd60 0x7faa281acaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:04:36.952 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:36.942+0000 7faa2659c700 1 -- 172.21.15.59:0/3101581180 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faa2811adb0 msgr2=0x7faa281a6690 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-12-11T18:04:36.952 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:36.942+0000 7faa2659c700 1 --2- 172.21.15.59:0/3101581180 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faa2811adb0 0x7faa281a6690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:04:36.952 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:36.942+0000 7faa2659c700 1 -- 172.21.15.59:0/3101581180 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa1c009520 con 0x7faa28108b10 2023-12-11T18:04:36.952 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:36.943+0000 7faa2659c700 1 --2- 172.21.15.59:0/3101581180 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa28108b10 0x7faa281ac560 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7faa20007fa0 tx=0x7faa2000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T18:04:36.954 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:36.950+0000 7faa177fe700 1 -- 172.21.15.59:0/3101581180 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa2000de60 con 0x7faa28108b10 2023-12-11T18:04:36.954 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:36.950+0000 7faa177fe700 1 -- 172.21.15.59:0/3101581180 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faa2000f040 con 0x7faa28108b10 2023-12-11T18:04:36.954 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:36.950+0000 7faa2cde5700 1 -- 172.21.15.59:0/3101581180 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa281a7300 con 0x7faa28108b10 2023-12-11T18:04:36.954 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:36.950+0000 7faa2cde5700 1 -- 172.21.15.59:0/3101581180 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa28110770 con 0x7faa28108b10 2023-12-11T18:04:36.954 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:36.950+0000 7faa177fe700 1 -- 172.21.15.59:0/3101581180 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa20014720 con 0x7faa28108b10 2023-12-11T18:04:36.958 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:36.956+0000 7faa2cde5700 1 -- 172.21.15.59:0/3101581180 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa080052f0 con 0x7faa28108b10 2023-12-11T18:04:36.970 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:36.968+0000 7faa177fe700 1 -- 172.21.15.59:0/3101581180 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 28) v1 ==== 136137+0+0 (secure 0 0 0) 0x7faa20015a50 con 0x7faa28108b10 2023-12-11T18:04:36.972 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:36.969+0000 7faa177fe700 1 --2- 172.21.15.59:0/3101581180 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7faa100a17c0 0x7faa100a3c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T18:04:36.972 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:36.969+0000 7faa177fe700 1 -- 172.21.15.59:0/3101581180 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 9851+0+0 (secure 0 0 0) 0x7faa200cd250 con 0x7faa28108b10 2023-12-11T18:04:36.973 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:36.971+0000 7faa25d9b700 1 --2- 172.21.15.59:0/3101581180 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7faa100a17c0 0x7faa100a3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T18:04:36.974 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:36.972+0000 7faa25d9b700 1 --2- 172.21.15.59:0/3101581180 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7faa100a17c0 0x7faa100a3c80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7faa1c00c010 tx=0x7faa1c009f80 comp rx=0 tx=0).ready entity=mgr.15414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T18:04:36.978 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:36.976+0000 7faa177fe700 1 -- 172.21.15.59:0/3101581180 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7faa20095fe0 con 0x7faa28108b10 2023-12-11T18:04:37.033 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:04:36 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/2926709740' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-11T18:04:37.034 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:04:36 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/575415052' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-11T18:04:37.357 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:37.355+0000 7faa2cde5700 1 -- 172.21.15.59:0/3101581180 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7faa08005b20 con 0x7faa28108b10 2023-12-11T18:04:37.357 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:37.355+0000 7faa177fe700 1 -- 172.21.15.59:0/3101581180 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v119) v1 ==== 100+0+11 (secure 0 0 0) 0x7faa20099600 con 0x7faa28108b10 2023-12-11T18:04:37.358 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2023-12-11T18:04:37.363 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:37.361+0000 7faa2cde5700 1 -- 172.21.15.59:0/3101581180 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7faa100a17c0 msgr2=0x7faa100a3c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T18:04:37.363 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:37.361+0000 7faa2cde5700 1 --2- 172.21.15.59:0/3101581180 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7faa100a17c0 0x7faa100a3c80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7faa1c00c010 tx=0x7faa1c009f80 comp rx=0 tx=0).stop 2023-12-11T18:04:37.363 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:37.361+0000 7faa2cde5700 1 -- 172.21.15.59:0/3101581180 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa28108b10 msgr2=0x7faa281ac560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T18:04:37.363 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:37.361+0000 7faa2cde5700 1 --2- 172.21.15.59:0/3101581180 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa28108b10 0x7faa281ac560 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7faa20007fa0 tx=0x7faa2000d3b0 comp rx=0 tx=0).stop 2023-12-11T18:04:37.377 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:37.375+0000 7faa2cde5700 1 -- 172.21.15.59:0/3101581180 shutdown_connections 2023-12-11T18:04:37.377 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:37.375+0000 7faa2cde5700 1 --2- 172.21.15.59:0/3101581180 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7faa100a17c0 0x7faa100a3c80 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:04:37.377 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:37.375+0000 7faa2cde5700 1 --2- 172.21.15.59:0/3101581180 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faa2811adb0 0x7faa281a6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:04:37.377 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:37.375+0000 7faa2cde5700 1 --2- 172.21.15.59:0/3101581180 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa28108b10 0x7faa281ac560 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:04:37.377 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:37.375+0000 7faa2cde5700 1 --2- 172.21.15.59:0/3101581180 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa2810fd60 0x7faa281acaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:04:37.378 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:37.375+0000 7faa2cde5700 1 -- 172.21.15.59:0/3101581180 >> 172.21.15.59:0/3101581180 conn(0x7faa2806c610 msgr2=0x7faa28118da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T18:04:37.378 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:37.375+0000 7faa2cde5700 1 -- 172.21.15.59:0/3101581180 shutdown_connections 2023-12-11T18:04:37.378 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:37.375+0000 7faa2cde5700 1 -- 172.21.15.59:0/3101581180 wait complete. 2023-12-11T18:04:37.399 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-12-11T18:04:37.686 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:37.674+0000 7f8e6f547700 1 -- 172.21.15.59:0/1878779070 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e68103f70 msgr2=0x7f8e681043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T18:04:37.686 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:37.674+0000 7f8e6f547700 1 --2- 172.21.15.59:0/1878779070 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e68103f70 0x7f8e681043d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8e6000b0a0 tx=0x7f8e6000b3b0 comp rx=0 tx=0).stop 2023-12-11T18:04:37.686 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:37.676+0000 7f8e6f547700 1 -- 172.21.15.59:0/1878779070 shutdown_connections 2023-12-11T18:04:37.686 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:37.676+0000 7f8e6f547700 1 --2- 172.21.15.59:0/1878779070 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e68103f70 0x7f8e681043d0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:04:37.686 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:37.676+0000 7f8e6f547700 1 --2- 172.21.15.59:0/1878779070 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e68106470 0x7f8e680697d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:04:37.686 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:37.676+0000 7f8e6f547700 1 --2- 172.21.15.59:0/1878779070 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8e68069d10 0x7f8e6806a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:04:37.686 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:37.676+0000 7f8e6f547700 1 -- 172.21.15.59:0/1878779070 >> 172.21.15.59:0/1878779070 conn(0x7f8e680775a0 msgr2=0x7f8e680779b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T18:04:37.707 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:37.697+0000 7f8e6f547700 1 -- 172.21.15.59:0/1878779070 shutdown_connections 2023-12-11T18:04:37.707 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:37.705+0000 7f8e6f547700 1 -- 172.21.15.59:0/1878779070 wait complete. 2023-12-11T18:04:37.710 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:37.705+0000 7f8e6f547700 1 Processor -- start 2023-12-11T18:04:37.726 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:37.716+0000 7f8e6f547700 1 -- start start 2023-12-11T18:04:37.726 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:37.717+0000 7f8e6f547700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e68069d10 0x7f8e6810adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T18:04:37.726 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:37.717+0000 7f8e6f547700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e68103f70 0x7f8e6810b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T18:04:37.726 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:37.717+0000 7f8e6f547700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8e68106470 0x7f8e681069e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T18:04:37.726 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:37.717+0000 7f8e6f547700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e68107170 con 0x7f8e68106470 2023-12-11T18:04:37.726 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:37.717+0000 7f8e6f547700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e681072e0 con 0x7f8e68069d10 2023-12-11T18:04:37.726 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:37.717+0000 7f8e6f547700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e681074a0 con 0x7f8e68103f70 2023-12-11T18:04:37.726 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:37.718+0000 7f8e6cae2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e68103f70 0x7f8e6810b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T18:04:37.726 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:37.718+0000 7f8e6cae2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e68103f70 0x7f8e6810b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.59:40274/0 (socket says 172.21.15.59:40274) 2023-12-11T18:04:37.726 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:37.718+0000 7f8e6cae2700 1 -- 172.21.15.59:0/2023274003 learned_addr learned my addr 172.21.15.59:0/2023274003 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T18:04:37.727 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:37.719+0000 7f8e6dae4700 1 --2- 172.21.15.59:0/2023274003 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8e68106470 0x7f8e681069e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T18:04:37.727 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:37.724+0000 7f8e6d2e3700 1 --2- 172.21.15.59:0/2023274003 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e68069d10 0x7f8e6810adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T18:04:37.733 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:37.726+0000 7f8e6dae4700 1 -- 172.21.15.59:0/2023274003 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e68103f70 msgr2=0x7f8e6810b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T18:04:37.733 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:37.726+0000 7f8e6dae4700 1 --2- 172.21.15.59:0/2023274003 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e68103f70 0x7f8e6810b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:04:37.733 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:37.726+0000 7f8e6dae4700 1 -- 172.21.15.59:0/2023274003 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e68069d10 msgr2=0x7f8e6810adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T18:04:37.733 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:37.726+0000 7f8e6dae4700 1 --2- 172.21.15.59:0/2023274003 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e68069d10 0x7f8e6810adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:04:37.733 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:37.726+0000 7f8e6dae4700 1 -- 172.21.15.59:0/2023274003 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e60009d00 con 0x7f8e68106470 2023-12-11T18:04:37.733 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:37.726+0000 7f8e6dae4700 1 --2- 172.21.15.59:0/2023274003 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8e68106470 0x7f8e681069e0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f8e6000b0a0 tx=0x7f8e60007b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T18:04:37.733 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:37.728+0000 7f8e5e7fc700 1 -- 172.21.15.59:0/2023274003 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e6000e040 con 0x7f8e68106470 2023-12-11T18:04:37.733 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:37.728+0000 7f8e5e7fc700 1 -- 172.21.15.59:0/2023274003 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8e6001de10 con 0x7f8e68106470 2023-12-11T18:04:37.733 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:37.728+0000 7f8e6f547700 1 -- 172.21.15.59:0/2023274003 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e681076c0 con 0x7f8e68106470 2023-12-11T18:04:37.733 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:37.728+0000 7f8e5e7fc700 1 -- 172.21.15.59:0/2023274003 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e6001be60 con 0x7f8e68106470 2023-12-11T18:04:37.734 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:37.728+0000 7f8e6f547700 1 -- 172.21.15.59:0/2023274003 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e68112ec0 con 0x7f8e68106470 2023-12-11T18:04:37.734 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:37.729+0000 7f8e6f547700 1 -- 172.21.15.59:0/2023274003 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e68100a50 con 0x7f8e68106470 2023-12-11T18:04:37.755 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:37.747+0000 7f8e5e7fc700 1 -- 172.21.15.59:0/2023274003 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 28) v1 ==== 136137+0+0 (secure 0 0 0) 0x7f8e6001b550 con 0x7f8e68106470 2023-12-11T18:04:37.755 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:37.747+0000 7f8e5e7fc700 1 --2- 172.21.15.59:0/2023274003 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7f8e540a17c0 0x7f8e540a3c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T18:04:37.755 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:37.748+0000 7f8e5e7fc700 1 -- 172.21.15.59:0/2023274003 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 9851+0+0 (secure 0 0 0) 0x7f8e600cf920 con 0x7f8e68106470 2023-12-11T18:04:37.755 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:37.748+0000 7f8e5e7fc700 1 -- 172.21.15.59:0/2023274003 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f8e600fc390 con 0x7f8e68106470 2023-12-11T18:04:37.755 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:37.749+0000 7f8e6d2e3700 1 --2- 172.21.15.59:0/2023274003 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7f8e540a17c0 0x7f8e540a3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T18:04:37.755 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:37.751+0000 7f8e6d2e3700 1 --2- 172.21.15.59:0/2023274003 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7f8e540a17c0 0x7f8e540a3c80 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8e64005950 tx=0x7f8e6400b410 comp rx=0 tx=0).ready entity=mgr.15414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T18:04:38.124 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:38.122+0000 7f8e6f547700 1 -- 172.21.15.59:0/2023274003 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f8e68073290 con 0x7f8e68106470 2023-12-11T18:04:38.129 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:38.127+0000 7f8e5e7fc700 1 -- 172.21.15.59:0/2023274003 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v119) v1 ==== 103+0+11 (secure 0 0 0) 0x7f8e600210a0 con 0x7f8e68106470 2023-12-11T18:04:38.130 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2023-12-11T18:04:38.145 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:38.141+0000 7f8e6f547700 1 -- 172.21.15.59:0/2023274003 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7f8e540a17c0 msgr2=0x7f8e540a3c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T18:04:38.145 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:38.141+0000 7f8e6f547700 1 --2- 172.21.15.59:0/2023274003 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7f8e540a17c0 0x7f8e540a3c80 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8e64005950 tx=0x7f8e6400b410 comp rx=0 tx=0).stop 2023-12-11T18:04:38.145 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:38.141+0000 7f8e6f547700 1 -- 172.21.15.59:0/2023274003 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8e68106470 msgr2=0x7f8e681069e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T18:04:38.145 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:38.141+0000 7f8e6f547700 1 --2- 172.21.15.59:0/2023274003 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8e68106470 0x7f8e681069e0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f8e6000b0a0 tx=0x7f8e60007b60 comp rx=0 tx=0).stop 2023-12-11T18:04:38.145 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:38.142+0000 7f8e6f547700 1 -- 172.21.15.59:0/2023274003 shutdown_connections 2023-12-11T18:04:38.145 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:38.142+0000 7f8e6f547700 1 --2- 172.21.15.59:0/2023274003 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7f8e540a17c0 0x7f8e540a3c80 secure :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8e64005950 tx=0x7f8e6400b410 comp rx=0 tx=0).stop 2023-12-11T18:04:38.145 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:38.142+0000 7f8e6f547700 1 --2- 172.21.15.59:0/2023274003 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8e68106470 0x7f8e681069e0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:04:38.145 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:38.142+0000 7f8e6f547700 1 --2- 172.21.15.59:0/2023274003 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e68069d10 0x7f8e6810adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:04:38.145 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:38.142+0000 7f8e6f547700 1 --2- 172.21.15.59:0/2023274003 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e68103f70 0x7f8e6810b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:04:38.145 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:38.142+0000 7f8e6f547700 1 -- 172.21.15.59:0/2023274003 >> 172.21.15.59:0/2023274003 conn(0x7f8e680775a0 msgr2=0x7f8e68102f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T18:04:38.149 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:38.143+0000 7f8e6f547700 1 -- 172.21.15.59:0/2023274003 shutdown_connections 2023-12-11T18:04:38.149 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:38.143+0000 7f8e6f547700 1 -- 172.21.15.59:0/2023274003 wait complete. 2023-12-11T18:04:38.157 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:04:37 smithi059 ceph-mon[98847]: pgmap v583: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.7 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-12-11T18:04:38.157 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:04:37 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/3101581180' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-11T18:04:38.176 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-11T18:04:38.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:04:37 smithi115 ceph-mon[102748]: pgmap v583: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.7 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-12-11T18:04:38.283 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:04:37 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/3101581180' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-11T18:04:38.479 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:38.477+0000 7ff4f0459700 1 -- 172.21.15.59:0/2465760685 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4e81040d0 msgr2=0x7ff4e81105e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T18:04:38.479 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:38.477+0000 7ff4f0459700 1 --2- 172.21.15.59:0/2465760685 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4e81040d0 0x7ff4e81105e0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ff4e000b0a0 tx=0x7ff4e000b3b0 comp rx=0 tx=0).stop 2023-12-11T18:04:38.480 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:38.478+0000 7ff4f0459700 1 -- 172.21.15.59:0/2465760685 shutdown_connections 2023-12-11T18:04:38.480 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:38.478+0000 7ff4f0459700 1 --2- 172.21.15.59:0/2465760685 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4e81040d0 0x7ff4e81105e0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:04:38.481 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:38.478+0000 7ff4f0459700 1 --2- 172.21.15.59:0/2465760685 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff4e8109720 0x7ff4e8109b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:04:38.481 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:38.478+0000 7ff4f0459700 1 --2- 172.21.15.59:0/2465760685 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff4e8103710 0x7ff4e8103b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:04:38.481 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:38.478+0000 7ff4f0459700 1 -- 172.21.15.59:0/2465760685 >> 172.21.15.59:0/2465760685 conn(0x7ff4e80ff1e0 msgr2=0x7ff4e8101600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T18:04:38.491 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:38.489+0000 7ff4f0459700 1 -- 172.21.15.59:0/2465760685 shutdown_connections 2023-12-11T18:04:38.493 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:38.491+0000 7ff4f0459700 1 -- 172.21.15.59:0/2465760685 wait complete. 2023-12-11T18:04:38.494 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:38.492+0000 7ff4f0459700 1 Processor -- start 2023-12-11T18:04:38.497 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:38.495+0000 7ff4f0459700 1 -- start start 2023-12-11T18:04:38.497 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:38.495+0000 7ff4f0459700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff4e8103710 0x7ff4e810eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T18:04:38.497 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:38.495+0000 7ff4f0459700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4e81040d0 0x7ff4e810f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T18:04:38.497 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:38.495+0000 7ff4f0459700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff4e8109720 0x7ff4e81a9c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T18:04:38.497 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:38.495+0000 7ff4f0459700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4e81aa2b0 con 0x7ff4e8109720 2023-12-11T18:04:38.498 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:38.495+0000 7ff4f0459700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4e8112710 con 0x7ff4e8103710 2023-12-11T18:04:38.498 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:38.495+0000 7ff4f0459700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4e81128d0 con 0x7ff4e81040d0 2023-12-11T18:04:38.503 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:38.501+0000 7ff4ed9f4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4e81040d0 0x7ff4e810f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T18:04:38.504 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:38.502+0000 7ff4ed9f4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4e81040d0 0x7ff4e810f530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.59:40284/0 (socket says 172.21.15.59:40284) 2023-12-11T18:04:38.504 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:38.502+0000 7ff4ed9f4700 1 -- 172.21.15.59:0/3293407323 learned_addr learned my addr 172.21.15.59:0/3293407323 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T18:04:38.504 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:38.502+0000 7ff4ee1f5700 1 --2- 172.21.15.59:0/3293407323 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff4e8103710 0x7ff4e810eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T18:04:38.505 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:38.503+0000 7ff4ee9f6700 1 --2- 172.21.15.59:0/3293407323 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff4e8109720 0x7ff4e81a9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T18:04:38.509 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:38.507+0000 7ff4ee1f5700 1 -- 172.21.15.59:0/3293407323 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4e81040d0 msgr2=0x7ff4e810f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T18:04:38.509 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:38.507+0000 7ff4ee1f5700 1 --2- 172.21.15.59:0/3293407323 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4e81040d0 0x7ff4e810f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:04:38.509 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:38.507+0000 7ff4ee1f5700 1 -- 172.21.15.59:0/3293407323 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff4e8109720 msgr2=0x7ff4e81a9c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T18:04:38.509 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:38.507+0000 7ff4ee1f5700 1 --2- 172.21.15.59:0/3293407323 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff4e8109720 0x7ff4e81a9c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:04:38.509 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:38.507+0000 7ff4ee1f5700 1 -- 172.21.15.59:0/3293407323 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4e0009d00 con 0x7ff4e8103710 2023-12-11T18:04:38.510 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:38.507+0000 7ff4ee1f5700 1 --2- 172.21.15.59:0/3293407323 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff4e8103710 0x7ff4e810eff0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff4d800b450 tx=0x7ff4d800b810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T18:04:38.510 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:38.508+0000 7ff4df7fe700 1 -- 172.21.15.59:0/3293407323 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4d8011830 con 0x7ff4e8103710 2023-12-11T18:04:38.510 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:38.508+0000 7ff4f0459700 1 -- 172.21.15.59:0/3293407323 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4e8112bb0 con 0x7ff4e8103710 2023-12-11T18:04:38.511 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:38.508+0000 7ff4f0459700 1 -- 172.21.15.59:0/3293407323 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4e8113110 con 0x7ff4e8103710 2023-12-11T18:04:38.511 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:38.509+0000 7ff4ee9f6700 1 --2- 172.21.15.59:0/3293407323 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff4e8109720 0x7ff4e81a9c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-11T18:04:38.513 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:38.511+0000 7ff4df7fe700 1 -- 172.21.15.59:0/3293407323 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff4d8011e50 con 0x7ff4e8103710 2023-12-11T18:04:38.513 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:38.511+0000 7ff4df7fe700 1 -- 172.21.15.59:0/3293407323 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4d800f2e0 con 0x7ff4e8103710 2023-12-11T18:04:38.516 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:38.514+0000 7ff4f0459700 1 -- 172.21.15.59:0/3293407323 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4cc0052f0 con 0x7ff4e8103710 2023-12-11T18:04:38.525 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:38.521+0000 7ff4df7fe700 1 -- 172.21.15.59:0/3293407323 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 28) v1 ==== 136137+0+0 (secure 0 0 0) 0x7ff4d8011990 con 0x7ff4e8103710 2023-12-11T18:04:38.525 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:38.522+0000 7ff4df7fe700 1 --2- 172.21.15.59:0/3293407323 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7ff4d40a1630 0x7ff4d40a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T18:04:38.525 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:38.523+0000 7ff4df7fe700 1 -- 172.21.15.59:0/3293407323 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 9851+0+0 (secure 0 0 0) 0x7ff4d80cd4a0 con 0x7ff4e8103710 2023-12-11T18:04:38.533 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:38.530+0000 7ff4ed9f4700 1 --2- 172.21.15.59:0/3293407323 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7ff4d40a1630 0x7ff4d40a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T18:04:38.533 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:38.531+0000 7ff4ed9f4700 1 --2- 172.21.15.59:0/3293407323 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7ff4d40a1630 0x7ff4d40a3af0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff4e8110870 tx=0x7ff4e4008040 comp rx=0 tx=0).ready entity=mgr.15414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T18:04:38.536 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:38.534+0000 7ff4df7fe700 1 -- 172.21.15.59:0/3293407323 <== mon.2 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7ff4d80963a0 con 0x7ff4e8103710 2023-12-11T18:04:38.943 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:38.941+0000 7ff4f0459700 1 -- 172.21.15.59:0/3293407323 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7ff4cc005870 con 0x7ff4e8103710 2023-12-11T18:04:38.947 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:38.945+0000 7ff4df7fe700 1 -- 172.21.15.59:0/3293407323 <== mon.2 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 19 entries v119) v1 ==== 81+0+1125 (secure 0 0 0) 0x7ff4d80999c0 con 0x7ff4e8103710 2023-12-11T18:04:38.948 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.181:6839/2214384806 2023-12-12T17:57:14.937455+0000 2023-12-11T18:04:38.948 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:6801/3627728353 2023-12-12T17:46:17.753068+0000 2023-12-11T18:04:38.948 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:0/2192742819 2023-12-12T17:37:44.773925+0000 2023-12-11T18:04:38.948 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.181:6838/2214384806 2023-12-12T17:57:14.937455+0000 2023-12-11T18:04:38.948 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:0/2859415380 2023-12-12T17:38:14.037922+0000 2023-12-11T18:04:38.948 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:0/644616907 2023-12-12T17:46:17.753068+0000 2023-12-11T18:04:38.948 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:0/3853462899 2023-12-12T17:37:44.773925+0000 2023-12-11T18:04:38.948 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:0/4159306605 2023-12-12T17:46:17.753068+0000 2023-12-11T18:04:38.948 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:6800/3627728353 2023-12-12T17:46:17.753068+0000 2023-12-11T18:04:38.948 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:0/3570767650 2023-12-12T17:37:44.773925+0000 2023-12-11T18:04:38.948 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:6800/1672530462 2023-12-12T17:37:44.773925+0000 2023-12-11T18:04:38.948 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:6801/1259916482 2023-12-12T17:38:14.037922+0000 2023-12-11T18:04:38.949 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:0/1053944547 2023-12-12T17:38:14.037922+0000 2023-12-11T18:04:38.949 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:6801/1672530462 2023-12-12T17:37:44.773925+0000 2023-12-11T18:04:38.949 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:0/2683787475 2023-12-12T17:46:17.753068+0000 2023-12-11T18:04:38.949 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:0/17457774 2023-12-12T17:38:14.037922+0000 2023-12-11T18:04:38.949 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:6800/1259916482 2023-12-12T17:38:14.037922+0000 2023-12-11T18:04:38.949 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:0/2112420116 2023-12-12T17:46:17.753068+0000 2023-12-11T18:04:38.949 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:0/1373005911 2023-12-12T17:46:17.753068+0000 2023-12-11T18:04:38.959 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:38.953+0000 7ff4f0459700 1 -- 172.21.15.59:0/3293407323 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7ff4d40a1630 msgr2=0x7ff4d40a3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T18:04:38.959 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:38.953+0000 7ff4f0459700 1 --2- 172.21.15.59:0/3293407323 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7ff4d40a1630 0x7ff4d40a3af0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff4e8110870 tx=0x7ff4e4008040 comp rx=0 tx=0).stop 2023-12-11T18:04:38.959 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:38.953+0000 7ff4f0459700 1 -- 172.21.15.59:0/3293407323 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff4e8103710 msgr2=0x7ff4e810eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T18:04:38.959 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:38.953+0000 7ff4f0459700 1 --2- 172.21.15.59:0/3293407323 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff4e8103710 0x7ff4e810eff0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff4d800b450 tx=0x7ff4d800b810 comp rx=0 tx=0).stop 2023-12-11T18:04:38.975 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:38.968+0000 7ff4f0459700 1 -- 172.21.15.59:0/3293407323 shutdown_connections 2023-12-11T18:04:38.976 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:38.968+0000 7ff4f0459700 1 --2- 172.21.15.59:0/3293407323 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7ff4d40a1630 0x7ff4d40a3af0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:04:38.976 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:38.968+0000 7ff4f0459700 1 --2- 172.21.15.59:0/3293407323 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff4e8109720 0x7ff4e81a9c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:04:38.976 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:38.968+0000 7ff4f0459700 1 --2- 172.21.15.59:0/3293407323 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff4e8103710 0x7ff4e810eff0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:04:38.976 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:38.968+0000 7ff4f0459700 1 --2- 172.21.15.59:0/3293407323 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4e81040d0 0x7ff4e810f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:04:38.976 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:38.968+0000 7ff4f0459700 1 -- 172.21.15.59:0/3293407323 >> 172.21.15.59:0/3293407323 conn(0x7ff4e80ff1e0 msgr2=0x7ff4e8108a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T18:04:38.981 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:38.978+0000 7ff4f0459700 1 -- 172.21.15.59:0/3293407323 shutdown_connections 2023-12-11T18:04:38.983 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:38.981+0000 7ff4f0459700 1 -- 172.21.15.59:0/3293407323 wait complete. 2023-12-11T18:04:38.997 INFO:teuthology.orchestra.run.smithi059.stderr:listed 19 entries 2023-12-11T18:04:39.021 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-12-11T18:04:39.021 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-11T18:04:39.021 DEBUG:teuthology.orchestra.run.smithi059:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-12-11T18:04:39.115 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:04:38 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/2023274003' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-11T18:04:39.230 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi059.front.sepia.ceph.com 2023-12-11T18:04:39.230 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-11T18:04:39.230 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-12-11T18:04:39.289 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:04:38 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/2023274003' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-11T18:04:39.347 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-11T18:04:39.348 DEBUG:teuthology.orchestra.run.smithi059:> ip netns list 2023-12-11T18:04:39.440 INFO:teuthology.orchestra.run.smithi059.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-12-11T18:04:39.440 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-11T18:04:39.440 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-11T18:04:39.563 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-11T18:04:39.563 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link delete ceph-brx 2023-12-11T18:04:39.751 INFO:tasks.kclient:Unmounting kernel clients... 2023-12-11T18:04:39.751 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2023-12-11T18:04:39.751 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-11T18:04:39.829 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16287360k,nr_inodes=4071840,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=40,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13125 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3261172k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/sda1 /var/lib/containers/storage/overlay ext4 rw,seclabel,relatime 0 0 overlay /var/lib/containers/storage/overlay/2455e591f6739cbbbcaacd0827bf7d505a85a6fd5399d3dbeb16e7bf9c8caf31/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/2F3BGALN3VX65OZDWU63WNOZKA:/var/lib/containers/storage/overlay/l/56NMGCXRKDKXOTMBHN4IIKJ2CJ,upperdir=/var/lib/containers/storage/overlay/2455e591f6739cbbbcaacd0827bf7d505a85a6fd5399d3dbeb16e7bf9c8caf31/diff,workdir=/var/lib/containers/storage/overlay/2455e591f6739cbbbcaacd0827bf7d505a85a6fd5399d3dbeb16e7bf9c8caf31/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/7d363dfe8ee758db94cc885ec1a5345a5cc3214bc9a83b07bba8f871c61a954a/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/2F3BGALN3VX65OZDWU63WNOZKA:/var/lib/containers/storage/overlay/l/56NMGCXRKDKXOTMBHN4IIKJ2CJ,upperdir=/var/lib/containers/storage/overlay/7d363dfe8ee758db94cc885ec1a5345a5cc3214bc9a83b07bba8f871c61a954a/diff,workdir=/var/lib/containers/storage/overlay/7d363dfe8ee758db94cc885ec1a5345a5cc3214bc9a83b07bba8f871c61a954a/work,metacopy=on,volatile 0 0 tmpfs /run/user/0 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3261172k,mode=700 0 0 binfmt_misc /proc/sys/fs/binfmt_misc binfmt_misc rw,relatime 0 0 overlay /var/lib/containers/storage/overlay/158893c42a76fbe340d855bb5af1f85abd6c72ea2e136f335db666b3d4fa8336/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/2F3BGALN3VX65OZDWU63WNOZKA:/var/lib/containers/storage/overlay/l/56NMGCXRKDKXOTMBHN4IIKJ2CJ,upperdir=/var/lib/containers/storage/overlay/158893c42a76fbe340d855bb5af1f85abd6c72ea2e136f335db666b3d4fa8336/diff,workdir=/var/lib/containers/storage/overlay/158893c42a76fbe340d855bb5af1f85abd6c72ea2e136f335db666b3d4fa8336/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/9d30922a22bf35fa43b8469febb855d6837bd073531aea9e11c65e92843550b5/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/2F3BGALN3VX65OZDWU63WNOZKA:/var/lib/containers/storage/overlay/l/56NMGCXRKDKXOTMBHN4IIKJ2CJ,upperdir=/var/lib/containers/storage/overlay/9d30922a22bf35fa43b8469febb855d6837bd073531aea9e11c65e92843550b5/diff,workdir=/var/lib/containers/storage/overlay/9d30922a22bf35fa43b8469febb855d6837bd073531aea9e11c65e92843550b5/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/c1a3277f2167ca817995c42be034555f45b8a82144e3887ac238f4ead92bdc4c/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/2F3BGALN3VX65OZDWU63WNOZKA:/var/lib/containers/storage/overlay/l/56NMGCXRKDKXOTMBHN4IIKJ2CJ,upperdir=/var/lib/containers/storage/overlay/c1a3277f2167ca817995c42be034555f45b8a82144e3887ac238f4ead92bdc4c/diff,workdir=/var/lib/containers/storage/overlay/c1a3277f2167ca817995c42be034555f45b8a82144e3887ac238f4ead92bdc4c/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/5b9f2a294febc5f83a642cc187ff9fcb103bee1c91192e1a6b901bb9b92116b2/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/2F3BGALN3VX65OZDWU63WNOZKA:/var/lib/containers/storage/overlay/l/56NMGCXRKDKXOTMBHN4IIKJ2CJ,upperdir=/var/lib/containers/storage/overlay/5b9f2a294febc5f83a642cc187ff9fcb103bee1c91192e1a6b901bb9b92116b2/diff,workdir=/var/lib/containers/storage/overlay/5b9f2a294febc5f83a642cc187ff9fcb103bee1c91192e1a6b901bb9b92116b2/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/f3799e0bd998885d8fa7f7e1f3ced2e24765baa1a4e11187d961193cf4b90bcd/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/2F3BGALN3VX65OZDWU63WNOZKA:/var/lib/containers/storage/overlay/l/56NMGCXRKDKXOTMBHN4IIKJ2CJ,upperdir=/var/lib/containers/storage/overlay/f3799e0bd998885d8fa7f7e1f3ced2e24765baa1a4e11187d961193cf4b90bcd/diff,workdir=/var/lib/containers/storage/overlay/f3799e0bd998885d8fa7f7e1f3ced2e24765baa1a4e11187d961193cf4b90bcd/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/0bb37326e5761e4012444235fe6d29fae399919297b684b8f704ed0aaa98cbdc/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/2F3BGALN3VX65OZDWU63WNOZKA:/var/lib/containers/storage/overlay/l/56NMGCXRKDKXOTMBHN4IIKJ2CJ,upperdir=/var/lib/containers/storage/overlay/0bb37326e5761e4012444235fe6d29fae399919297b684b8f704ed0aaa98cbdc/diff,workdir=/var/lib/containers/storage/overlay/0bb37326e5761e4012444235fe6d29fae399919297b684b8f704ed0aaa98cbdc/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/83fbb5a430291679c6b087a9c3715297825f25f4777ca051d6ba9fb0f4cd0f39/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/2F3BGALN3VX65OZDWU63WNOZKA:/var/lib/containers/storage/overlay/l/56NMGCXRKDKXOTMBHN4IIKJ2CJ,upperdir=/var/lib/containers/storage/overlay/83fbb5a430291679c6b087a9c3715297825f25f4777ca051d6ba9fb0f4cd0f39/diff,workdir=/var/lib/containers/storage/overlay/83fbb5a430291679c6b087a9c3715297825f25f4777ca051d6ba9fb0f4cd0f39/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/1e65e5d92631d33b15a5ef793701103b4ef4092fce46241b6d452048aef17302/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/2F3BGALN3VX65OZDWU63WNOZKA:/var/lib/containers/storage/overlay/l/56NMGCXRKDKXOTMBHN4IIKJ2CJ,upperdir=/var/lib/containers/storage/overlay/1e65e5d92631d33b15a5ef793701103b4ef4092fce46241b6d452048aef17302/diff,workdir=/var/lib/containers/storage/overlay/1e65e5d92631d33b15a5ef793701103b4ef4092fce46241b6d452048aef17302/work,metacopy=on,volatile 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 2023-12-11T18:04:39.829 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-11T18:04:39.829 DEBUG:teuthology.orchestra.run.smithi059:> ip netns list 2023-12-11T18:04:39.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:04:39 smithi059 ceph-mon[98847]: pgmap v584: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.7 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-11T18:04:39.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:04:39 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/3293407323' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-12-11T18:04:39.892 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-11T18:04:39.892 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link delete ceph-brx 2023-12-11T18:04:40.018 INFO:teuthology.orchestra.run.smithi059.stderr:Cannot find device "ceph-brx" 2023-12-11T18:04:40.020 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-11T18:04:40.021 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2023-12-11T18:04:40.031 INFO:tasks.cephadm:Teardown begin 2023-12-11T18:04:40.031 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-12-11T18:04:40.031 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-11T18:04:40.284 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:04:39 smithi115 ceph-mon[102748]: pgmap v584: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.7 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-11T18:04:40.284 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:04:39 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/3293407323' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-12-11T18:04:40.334 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:40.331+0000 7fe79390a700 1 -- 172.21.15.59:0/18239756 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe78c1085d0 msgr2=0x7fe78c108a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T18:04:40.334 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:40.331+0000 7fe79390a700 1 --2- 172.21.15.59:0/18239756 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe78c1085d0 0x7fe78c108a50 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fe78400b0d0 tx=0x7fe78400b3e0 comp rx=0 tx=0).stop 2023-12-11T18:04:40.334 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:40.331+0000 7fe79390a700 1 -- 172.21.15.59:0/18239756 shutdown_connections 2023-12-11T18:04:40.334 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:40.331+0000 7fe79390a700 1 --2- 172.21.15.59:0/18239756 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe78c108f90 0x7fe78c114d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:04:40.334 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:40.331+0000 7fe79390a700 1 --2- 172.21.15.59:0/18239756 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe78c10ceb0 0x7fe78c10d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:04:40.334 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:40.331+0000 7fe79390a700 1 --2- 172.21.15.59:0/18239756 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe78c1085d0 0x7fe78c108a50 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:04:40.335 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:40.331+0000 7fe79390a700 1 -- 172.21.15.59:0/18239756 >> 172.21.15.59:0/18239756 conn(0x7fe78c06c990 msgr2=0x7fe78c06cda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T18:04:40.336 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:40.333+0000 7fe79390a700 1 -- 172.21.15.59:0/18239756 shutdown_connections 2023-12-11T18:04:40.336 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:40.333+0000 7fe79390a700 1 -- 172.21.15.59:0/18239756 wait complete. 2023-12-11T18:04:40.336 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:40.334+0000 7fe79390a700 1 Processor -- start 2023-12-11T18:04:40.337 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:40.334+0000 7fe79390a700 1 -- start start 2023-12-11T18:04:40.337 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:40.335+0000 7fe79390a700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe78c1085d0 0x7fe78c113250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T18:04:40.337 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:40.335+0000 7fe79390a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe78c108f90 0x7fe78c1233c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T18:04:40.338 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:40.335+0000 7fe79390a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe78c10ceb0 0x7fe78c113790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T18:04:40.338 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:40.335+0000 7fe79390a700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe78c113d90 con 0x7fe78c10ceb0 2023-12-11T18:04:40.338 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:40.335+0000 7fe79390a700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe78c113f00 con 0x7fe78c1085d0 2023-12-11T18:04:40.338 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:40.335+0000 7fe79390a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe78c114070 con 0x7fe78c108f90 2023-12-11T18:04:40.338 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:40.335+0000 7fe7916a6700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe78c1085d0 0x7fe78c113250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T18:04:40.338 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:40.335+0000 7fe791ea7700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe78c10ceb0 0x7fe78c113790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T18:04:40.338 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:40.335+0000 7fe791ea7700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe78c10ceb0 0x7fe78c113790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35480/0 (socket says 172.21.15.59:35480) 2023-12-11T18:04:40.338 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:40.335+0000 7fe791ea7700 1 -- 172.21.15.59:0/2476232404 learned_addr learned my addr 172.21.15.59:0/2476232404 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T18:04:40.339 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:40.336+0000 7fe791ea7700 1 -- 172.21.15.59:0/2476232404 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe78c108f90 msgr2=0x7fe78c1233c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T18:04:40.339 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:40.336+0000 7fe791ea7700 1 --2- 172.21.15.59:0/2476232404 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe78c108f90 0x7fe78c1233c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:04:40.339 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:40.336+0000 7fe791ea7700 1 -- 172.21.15.59:0/2476232404 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe78c1085d0 msgr2=0x7fe78c113250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T18:04:40.339 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:40.336+0000 7fe791ea7700 1 --2- 172.21.15.59:0/2476232404 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe78c1085d0 0x7fe78c113250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:04:40.339 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:40.336+0000 7fe791ea7700 1 -- 172.21.15.59:0/2476232404 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe784009d00 con 0x7fe78c10ceb0 2023-12-11T18:04:40.339 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:40.336+0000 7fe791ea7700 1 --2- 172.21.15.59:0/2476232404 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe78c10ceb0 0x7fe78c113790 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fe78800b550 tx=0x7fe78800b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T18:04:40.339 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:40.337+0000 7fe7827fc700 1 -- 172.21.15.59:0/2476232404 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe788011830 con 0x7fe78c10ceb0 2023-12-11T18:04:40.340 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:40.337+0000 7fe7827fc700 1 -- 172.21.15.59:0/2476232404 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe788011e50 con 0x7fe78c10ceb0 2023-12-11T18:04:40.340 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:40.337+0000 7fe7827fc700 1 -- 172.21.15.59:0/2476232404 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe78800f2e0 con 0x7fe78c10ceb0 2023-12-11T18:04:40.350 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:40.342+0000 7fe79390a700 1 -- 172.21.15.59:0/2476232404 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe78c1bba10 con 0x7fe78c10ceb0 2023-12-11T18:04:40.350 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:40.342+0000 7fe79390a700 1 -- 172.21.15.59:0/2476232404 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe78c1bbea0 con 0x7fe78c10ceb0 2023-12-11T18:04:40.352 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:40.349+0000 7fe79390a700 1 -- 172.21.15.59:0/2476232404 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe78c10b270 con 0x7fe78c10ceb0 2023-12-11T18:04:40.355 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:40.352+0000 7fe7827fc700 1 -- 172.21.15.59:0/2476232404 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 28) v1 ==== 136137+0+0 (secure 0 0 0) 0x7fe7880105a0 con 0x7fe78c10ceb0 2023-12-11T18:04:40.355 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:40.353+0000 7fe7827fc700 1 --2- 172.21.15.59:0/2476232404 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7fe7780a17a0 0x7fe7780a3c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T18:04:40.355 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:40.353+0000 7fe7827fc700 1 -- 172.21.15.59:0/2476232404 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 9851+0+0 (secure 0 0 0) 0x7fe7880ccf10 con 0x7fe78c10ceb0 2023-12-11T18:04:40.365 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:40.363+0000 7fe7916a6700 1 --2- 172.21.15.59:0/2476232404 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7fe7780a17a0 0x7fe7780a3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T18:04:40.365 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:40.363+0000 7fe7827fc700 1 -- 172.21.15.59:0/2476232404 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fe788095db0 con 0x7fe78c10ceb0 2023-12-11T18:04:40.371 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:40.369+0000 7fe7916a6700 1 --2- 172.21.15.59:0/2476232404 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7fe7780a17a0 0x7fe7780a3c60 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe77c006fd0 tx=0x7fe77c009190 comp rx=0 tx=0).ready entity=mgr.15414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T18:04:40.821 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-12-11T18:04:40.822 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":55,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/2184151593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":2184151593},{"type":"v1","addr":"172.21.15.59:6835","nonce":2184151593}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/2817614886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2817614886},{"type":"v1","addr":"172.21.15.59:6837","nonce":2817614886}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6839/229873490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":229873490},{"type":"v1","addr":"172.21.15.59:6839","nonce":229873490}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6841/967998755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":967998755},{"type":"v1","addr":"172.21.15.59:6841","nonce":967998755}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":15528,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6801/871235475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":871235475},{"type":"v1","addr":"172.21.15.181:6801","nonce":871235475}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":15531,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6803/3956691713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":3956691713},{"type":"v1","addr":"172.21.15.181:6803","nonce":3956691713}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":55,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-12-11T17:44:20.076660+0000","modified":"2023-12-11T18:04:16.341389+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":100,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24521,"mds_1":24422,"mds_2":24482},"failed":[],"damaged":[],"stopped":[],"info":{"gid_15537":{"gid":15537,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6807/3191405015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":3191405015},{"type":"v1","addr":"172.21.15.181:6807","nonce":3191405015}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24422":{"gid":24422,"name":"k","rank":1,"incarnation":40,"state":"up:active","state_seq":227,"addr":"172.21.15.115:6833/3214606191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3214606191},{"type":"v1","addr":"172.21.15.115:6833","nonce":3214606191}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/1874159657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1874159657},{"type":"v1","addr":"172.21.15.115:6835","nonce":1874159657}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24482":{"gid":24482,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.115:6837/1209365460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":1209365460},{"type":"v1","addr":"172.21.15.115:6837","nonce":1209365460}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.115:6839/4122922866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":4122922866},{"type":"v1","addr":"172.21.15.115:6839","nonce":4122922866}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_25256":{"gid":25256,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6806/166936948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":166936948},{"type":"v1","addr":"172.21.15.181:6806","nonce":166936948}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-11T18:04:40.822 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:40.814+0000 7fe79390a700 1 -- 172.21.15.59:0/2476232404 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe78c1095e0 con 0x7fe78c10ceb0 2023-12-11T18:04:40.822 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:40.816+0000 7fe7827fc700 1 -- 172.21.15.59:0/2476232404 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) v1 ==== 94+0+11334 (secure 0 0 0) 0x7fe788015090 con 0x7fe78c10ceb0 2023-12-11T18:04:40.827 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:40.825+0000 7fe79390a700 1 -- 172.21.15.59:0/2476232404 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7fe7780a17a0 msgr2=0x7fe7780a3c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T18:04:40.827 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:40.825+0000 7fe79390a700 1 --2- 172.21.15.59:0/2476232404 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7fe7780a17a0 0x7fe7780a3c60 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe77c006fd0 tx=0x7fe77c009190 comp rx=0 tx=0).stop 2023-12-11T18:04:40.828 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:40.825+0000 7fe79390a700 1 -- 172.21.15.59:0/2476232404 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe78c10ceb0 msgr2=0x7fe78c113790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T18:04:40.828 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:40.825+0000 7fe79390a700 1 --2- 172.21.15.59:0/2476232404 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe78c10ceb0 0x7fe78c113790 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fe78800b550 tx=0x7fe78800b860 comp rx=0 tx=0).stop 2023-12-11T18:04:40.830 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:40.828+0000 7fe79390a700 1 -- 172.21.15.59:0/2476232404 shutdown_connections 2023-12-11T18:04:40.830 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:40.828+0000 7fe79390a700 1 --2- 172.21.15.59:0/2476232404 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7fe7780a17a0 0x7fe7780a3c60 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:04:40.830 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:40.828+0000 7fe79390a700 1 --2- 172.21.15.59:0/2476232404 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe78c10ceb0 0x7fe78c113790 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:04:40.830 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:40.828+0000 7fe79390a700 1 --2- 172.21.15.59:0/2476232404 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe78c1085d0 0x7fe78c113250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:04:40.830 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:40.828+0000 7fe79390a700 1 --2- 172.21.15.59:0/2476232404 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe78c108f90 0x7fe78c1233c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:04:40.830 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:40.828+0000 7fe79390a700 1 -- 172.21.15.59:0/2476232404 >> 172.21.15.59:0/2476232404 conn(0x7fe78c06c990 msgr2=0x7fe78c072590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T18:04:40.834 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:40.832+0000 7fe79390a700 1 -- 172.21.15.59:0/2476232404 shutdown_connections 2023-12-11T18:04:40.834 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:40.832+0000 7fe79390a700 1 -- 172.21.15.59:0/2476232404 wait complete. 2023-12-11T18:04:40.835 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 55 2023-12-11T18:04:40.855 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-11T18:04:41.103 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:41.101+0000 7ffbbb4eb700 1 -- 172.21.15.59:0/4268177926 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffbb41074b0 msgr2=0x7ffbb4107890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T18:04:41.104 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:41.101+0000 7ffbbb4eb700 1 --2- 172.21.15.59:0/4268177926 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffbb41074b0 0x7ffbb4107890 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7ffba4009990 tx=0x7ffba4009ca0 comp rx=0 tx=0).stop 2023-12-11T18:04:41.106 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:41.104+0000 7ffbbb4eb700 1 -- 172.21.15.59:0/4268177926 shutdown_connections 2023-12-11T18:04:41.106 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:41.104+0000 7ffbbb4eb700 1 --2- 172.21.15.59:0/4268177926 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffbb4101dd0 0x7ffbb410e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:04:41.106 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:41.104+0000 7ffbbb4eb700 1 --2- 172.21.15.59:0/4268177926 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffbb41074b0 0x7ffbb4107890 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:04:41.106 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:41.104+0000 7ffbbb4eb700 1 --2- 172.21.15.59:0/4268177926 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffbb4101410 0x7ffbb4101890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:04:41.106 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:41.104+0000 7ffbbb4eb700 1 -- 172.21.15.59:0/4268177926 >> 172.21.15.59:0/4268177926 conn(0x7ffbb40fcf40 msgr2=0x7ffbb40ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T18:04:41.106 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:41.104+0000 7ffbbb4eb700 1 -- 172.21.15.59:0/4268177926 shutdown_connections 2023-12-11T18:04:41.107 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:41.104+0000 7ffbbb4eb700 1 -- 172.21.15.59:0/4268177926 wait complete. 2023-12-11T18:04:41.107 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:41.105+0000 7ffbbb4eb700 1 Processor -- start 2023-12-11T18:04:41.107 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:41.105+0000 7ffbbb4eb700 1 -- start start 2023-12-11T18:04:41.107 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:41.105+0000 7ffbbb4eb700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffbb4101410 0x7ffbb4072c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T18:04:41.108 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:41.105+0000 7ffbbb4eb700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffbb4101dd0 0x7ffbb40731c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T18:04:41.108 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:41.105+0000 7ffbbb4eb700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffbb41074b0 0x7ffbb406be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T18:04:41.108 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:41.105+0000 7ffbbb4eb700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb406c610 con 0x7ffbb41074b0 2023-12-11T18:04:41.108 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:41.105+0000 7ffbbb4eb700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb406c780 con 0x7ffbb4101410 2023-12-11T18:04:41.108 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:41.105+0000 7ffbbb4eb700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb406c940 con 0x7ffbb4101dd0 2023-12-11T18:04:41.108 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:41.106+0000 7ffbb8a86700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffbb4101dd0 0x7ffbb40731c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T18:04:41.108 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:41.106+0000 7ffbb8a86700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffbb4101dd0 0x7ffbb40731c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.59:40312/0 (socket says 172.21.15.59:40312) 2023-12-11T18:04:41.108 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:41.106+0000 7ffbb9a88700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffbb41074b0 0x7ffbb406be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T18:04:41.108 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:41.106+0000 7ffbb8a86700 1 -- 172.21.15.59:0/3638143799 learned_addr learned my addr 172.21.15.59:0/3638143799 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T18:04:41.109 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:41.106+0000 7ffbb8a86700 1 -- 172.21.15.59:0/3638143799 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffbb4101410 msgr2=0x7ffbb4072c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T18:04:41.109 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:41.106+0000 7ffbb8a86700 1 --2- 172.21.15.59:0/3638143799 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffbb4101410 0x7ffbb4072c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:04:41.109 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:41.106+0000 7ffbb8a86700 1 -- 172.21.15.59:0/3638143799 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffbb41074b0 msgr2=0x7ffbb406be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T18:04:41.109 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:41.106+0000 7ffbb8a86700 1 --2- 172.21.15.59:0/3638143799 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffbb41074b0 0x7ffbb406be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:04:41.109 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:41.106+0000 7ffbb8a86700 1 -- 172.21.15.59:0/3638143799 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffba40095f0 con 0x7ffbb4101dd0 2023-12-11T18:04:41.109 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:41.106+0000 7ffbb9a88700 1 --2- 172.21.15.59:0/3638143799 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffbb41074b0 0x7ffbb406be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-11T18:04:41.109 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:41.106+0000 7ffbb8a86700 1 --2- 172.21.15.59:0/3638143799 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffbb4101dd0 0x7ffbb40731c0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ffbb000b530 tx=0x7ffbb000b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T18:04:41.109 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:41.107+0000 7ffbaa7fc700 1 -- 172.21.15.59:0/3638143799 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbb0011830 con 0x7ffbb4101dd0 2023-12-11T18:04:41.110 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:41.107+0000 7ffbaa7fc700 1 -- 172.21.15.59:0/3638143799 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffbb0011e50 con 0x7ffbb4101dd0 2023-12-11T18:04:41.110 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:41.107+0000 7ffbaa7fc700 1 -- 172.21.15.59:0/3638143799 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbb000f2e0 con 0x7ffbb4101dd0 2023-12-11T18:04:41.111 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:41.108+0000 7ffbbb4eb700 1 -- 172.21.15.59:0/3638143799 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbb40707d0 con 0x7ffbb4101dd0 2023-12-11T18:04:41.111 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:41.108+0000 7ffbbb4eb700 1 -- 172.21.15.59:0/3638143799 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbb4070c60 con 0x7ffbb4101dd0 2023-12-11T18:04:41.112 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:41.110+0000 7ffbbb4eb700 1 -- 172.21.15.59:0/3638143799 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbb4102b50 con 0x7ffbb4101dd0 2023-12-11T18:04:41.114 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:41.111+0000 7ffbaa7fc700 1 -- 172.21.15.59:0/3638143799 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 28) v1 ==== 136137+0+0 (secure 0 0 0) 0x7ffbb0010380 con 0x7ffbb4101dd0 2023-12-11T18:04:41.114 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:41.112+0000 7ffbaa7fc700 1 --2- 172.21.15.59:0/3638143799 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7ffba00a16f0 0x7ffba00a3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T18:04:41.114 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:41.112+0000 7ffbaa7fc700 1 -- 172.21.15.59:0/3638143799 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 9851+0+0 (secure 0 0 0) 0x7ffbb00cd0a0 con 0x7ffbb4101dd0 2023-12-11T18:04:41.115 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:41.113+0000 7ffbb9287700 1 --2- 172.21.15.59:0/3638143799 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7ffba00a16f0 0x7ffba00a3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T18:04:41.115 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:41.113+0000 7ffbb9287700 1 --2- 172.21.15.59:0/3638143799 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7ffba00a16f0 0x7ffba00a3bb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ffba40095c0 tx=0x7ffba4009510 comp rx=0 tx=0).ready entity=mgr.15414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T18:04:41.123 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:41.121+0000 7ffbaa7fc700 1 -- 172.21.15.59:0/3638143799 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7ffbb0095f40 con 0x7ffbb4101dd0 2023-12-11T18:04:41.565 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:41.560+0000 7ffbbb4eb700 1 -- 172.21.15.59:0/3638143799 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffbb406d1f0 con 0x7ffbb4101dd0 2023-12-11T18:04:41.572 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:41.570+0000 7ffbaa7fc700 1 -- 172.21.15.59:0/3638143799 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) v1 ==== 94+0+11334 (secure 0 0 0) 0x7ffbb0099560 con 0x7ffbb4101dd0 2023-12-11T18:04:41.579 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-12-11T18:04:41.579 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":55,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/2184151593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":2184151593},{"type":"v1","addr":"172.21.15.59:6835","nonce":2184151593}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/2817614886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2817614886},{"type":"v1","addr":"172.21.15.59:6837","nonce":2817614886}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6839/229873490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":229873490},{"type":"v1","addr":"172.21.15.59:6839","nonce":229873490}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6841/967998755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":967998755},{"type":"v1","addr":"172.21.15.59:6841","nonce":967998755}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":15528,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6801/871235475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":871235475},{"type":"v1","addr":"172.21.15.181:6801","nonce":871235475}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":15531,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6803/3956691713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":3956691713},{"type":"v1","addr":"172.21.15.181:6803","nonce":3956691713}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":55,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-12-11T17:44:20.076660+0000","modified":"2023-12-11T18:04:16.341389+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":100,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24521,"mds_1":24422,"mds_2":24482},"failed":[],"damaged":[],"stopped":[],"info":{"gid_15537":{"gid":15537,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6807/3191405015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":3191405015},{"type":"v1","addr":"172.21.15.181:6807","nonce":3191405015}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24422":{"gid":24422,"name":"k","rank":1,"incarnation":40,"state":"up:active","state_seq":227,"addr":"172.21.15.115:6833/3214606191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":3214606191},{"type":"v1","addr":"172.21.15.115:6833","nonce":3214606191}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.115:6835/1874159657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1874159657},{"type":"v1","addr":"172.21.15.115:6835","nonce":1874159657}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24482":{"gid":24482,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.115:6837/1209365460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":1209365460},{"type":"v1","addr":"172.21.15.115:6837","nonce":1209365460}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.115:6839/4122922866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6838","nonce":4122922866},{"type":"v1","addr":"172.21.15.115:6839","nonce":4122922866}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_25256":{"gid":25256,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6806/166936948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":166936948},{"type":"v1","addr":"172.21.15.181:6806","nonce":166936948}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-11T18:04:41.591 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:41.589+0000 7ffbbb4eb700 1 -- 172.21.15.59:0/3638143799 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7ffba00a16f0 msgr2=0x7ffba00a3bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T18:04:41.592 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:41.589+0000 7ffbbb4eb700 1 --2- 172.21.15.59:0/3638143799 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7ffba00a16f0 0x7ffba00a3bb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ffba40095c0 tx=0x7ffba4009510 comp rx=0 tx=0).stop 2023-12-11T18:04:41.592 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:41.589+0000 7ffbbb4eb700 1 -- 172.21.15.59:0/3638143799 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffbb4101dd0 msgr2=0x7ffbb40731c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T18:04:41.592 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:41.589+0000 7ffbbb4eb700 1 --2- 172.21.15.59:0/3638143799 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffbb4101dd0 0x7ffbb40731c0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ffbb000b530 tx=0x7ffbb000b840 comp rx=0 tx=0).stop 2023-12-11T18:04:41.603 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:41.595+0000 7ffbbb4eb700 1 -- 172.21.15.59:0/3638143799 shutdown_connections 2023-12-11T18:04:41.603 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:41.595+0000 7ffbbb4eb700 1 --2- 172.21.15.59:0/3638143799 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7ffba00a16f0 0x7ffba00a3bb0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:04:41.603 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:41.595+0000 7ffbbb4eb700 1 --2- 172.21.15.59:0/3638143799 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffbb41074b0 0x7ffbb406be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:04:41.603 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:41.596+0000 7ffbbb4eb700 1 --2- 172.21.15.59:0/3638143799 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffbb4101410 0x7ffbb4072c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:04:41.603 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:41.596+0000 7ffbbb4eb700 1 --2- 172.21.15.59:0/3638143799 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffbb4101dd0 0x7ffbb40731c0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:04:41.603 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:41.596+0000 7ffbbb4eb700 1 -- 172.21.15.59:0/3638143799 >> 172.21.15.59:0/3638143799 conn(0x7ffbb40fcf40 msgr2=0x7ffbb40fd7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T18:04:41.609 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:41.605+0000 7ffbbb4eb700 1 -- 172.21.15.59:0/3638143799 shutdown_connections 2023-12-11T18:04:41.613 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:41.611+0000 7ffbbb4eb700 1 -- 172.21.15.59:0/3638143799 wait complete. 2023-12-11T18:04:41.630 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 55 2023-12-11T18:04:41.657 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:04:41 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/2476232404' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T18:04:41.658 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-11T18:04:41.784 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:04:41 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/2476232404' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T18:04:41.926 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:41.924+0000 7fc451b56700 1 -- 172.21.15.59:0/1601733711 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc44c100f30 msgr2=0x7fc44c104f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T18:04:41.926 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:41.924+0000 7fc451b56700 1 --2- 172.21.15.59:0/1601733711 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc44c100f30 0x7fc44c104f70 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc44001a9a0 tx=0x7fc44001acb0 comp rx=0 tx=0).stop 2023-12-11T18:04:41.927 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:41.925+0000 7fc451b56700 1 -- 172.21.15.59:0/1601733711 shutdown_connections 2023-12-11T18:04:41.927 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:41.925+0000 7fc451b56700 1 --2- 172.21.15.59:0/1601733711 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc44c1056a0 0x7fc44c10d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:04:41.927 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:41.925+0000 7fc451b56700 1 --2- 172.21.15.59:0/1601733711 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc44c100580 0x7fc44c100960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:04:41.927 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:41.925+0000 7fc451b56700 1 --2- 172.21.15.59:0/1601733711 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc44c100f30 0x7fc44c104f70 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:04:41.927 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:41.925+0000 7fc451b56700 1 -- 172.21.15.59:0/1601733711 >> 172.21.15.59:0/1601733711 conn(0x7fc44c0fbe40 msgr2=0x7fc44c0fe260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T18:04:41.927 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:41.925+0000 7fc451b56700 1 -- 172.21.15.59:0/1601733711 shutdown_connections 2023-12-11T18:04:41.927 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:41.925+0000 7fc451b56700 1 -- 172.21.15.59:0/1601733711 wait complete. 2023-12-11T18:04:41.936 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:41.926+0000 7fc451b56700 1 Processor -- start 2023-12-11T18:04:41.936 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:41.926+0000 7fc451b56700 1 -- start start 2023-12-11T18:04:41.936 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:41.926+0000 7fc451b56700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc44c100580 0x7fc44c1929f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T18:04:41.936 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:41.926+0000 7fc451b56700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc44c1056a0 0x7fc44c192f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T18:04:41.936 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:41.926+0000 7fc451b56700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc44c06a6c0 0x7fc44c193470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T18:04:41.936 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:41.926+0000 7fc451b56700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc44c10be90 con 0x7fc44c100580 2023-12-11T18:04:41.936 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:41.926+0000 7fc451b56700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc44c10c000 con 0x7fc44c1056a0 2023-12-11T18:04:41.936 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:41.926+0000 7fc451b56700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc44c10c1c0 con 0x7fc44c06a6c0 2023-12-11T18:04:41.936 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:41.926+0000 7fc44affd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc44c1056a0 0x7fc44c192f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T18:04:41.936 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:41.926+0000 7fc44affd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc44c1056a0 0x7fc44c192f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.59:60232/0 (socket says 172.21.15.59:60232) 2023-12-11T18:04:41.937 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:41.927+0000 7fc44affd700 1 -- 172.21.15.59:0/1082300140 learned_addr learned my addr 172.21.15.59:0/1082300140 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T18:04:41.937 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:41.928+0000 7fc44affd700 1 -- 172.21.15.59:0/1082300140 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc44c06a6c0 msgr2=0x7fc44c193470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T18:04:41.937 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:41.928+0000 7fc44affd700 1 --2- 172.21.15.59:0/1082300140 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc44c06a6c0 0x7fc44c193470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:04:41.937 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:41.928+0000 7fc44affd700 1 -- 172.21.15.59:0/1082300140 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc44c100580 msgr2=0x7fc44c1929f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T18:04:41.937 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:41.928+0000 7fc44affd700 1 --2- 172.21.15.59:0/1082300140 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc44c100580 0x7fc44c1929f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:04:41.937 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:41.928+0000 7fc44affd700 1 -- 172.21.15.59:0/1082300140 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc44001a600 con 0x7fc44c1056a0 2023-12-11T18:04:41.937 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:41.928+0000 7fc44affd700 1 --2- 172.21.15.59:0/1082300140 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc44c1056a0 0x7fc44c192f30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc440028620 tx=0x7fc440028700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T18:04:41.937 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:41.932+0000 7fc448ff9700 1 -- 172.21.15.59:0/1082300140 <== mon.2 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc44002d070 con 0x7fc44c1056a0 2023-12-11T18:04:41.937 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:41.933+0000 7fc448ff9700 1 -- 172.21.15.59:0/1082300140 <== mon.2 v2:172.21.15.115:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc440028b60 con 0x7fc44c1056a0 2023-12-11T18:04:41.937 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:41.933+0000 7fc448ff9700 1 -- 172.21.15.59:0/1082300140 <== mon.2 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc440031e80 con 0x7fc44c1056a0 2023-12-11T18:04:41.937 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:41.933+0000 7fc451b56700 1 -- 172.21.15.59:0/1082300140 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc44c10c3e0 con 0x7fc44c1056a0 2023-12-11T18:04:41.937 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:41.933+0000 7fc451b56700 1 -- 172.21.15.59:0/1082300140 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc44c10c8c0 con 0x7fc44c1056a0 2023-12-11T18:04:41.941 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:41.939+0000 7fc448ff9700 1 -- 172.21.15.59:0/1082300140 <== mon.2 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 28) v1 ==== 136137+0+0 (secure 0 0 0) 0x7fc440028cd0 con 0x7fc44c1056a0 2023-12-11T18:04:41.952 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:41.940+0000 7fc448ff9700 1 --2- 172.21.15.59:0/1082300140 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7fc4340c3750 0x7fc4340c5c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T18:04:41.952 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:41.940+0000 7fc451b56700 1 -- 172.21.15.59:0/1082300140 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc44c101a50 con 0x7fc44c1056a0 2023-12-11T18:04:41.952 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:41.944+0000 7fc44b7fe700 1 --2- 172.21.15.59:0/1082300140 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7fc4340c3750 0x7fc4340c5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T18:04:41.952 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:41.945+0000 7fc448ff9700 1 -- 172.21.15.59:0/1082300140 <== mon.2 v2:172.21.15.115:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 9851+0+0 (secure 0 0 0) 0x7fc4400dee20 con 0x7fc44c1056a0 2023-12-11T18:04:41.952 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:41.945+0000 7fc44b7fe700 1 --2- 172.21.15.59:0/1082300140 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7fc4340c3750 0x7fc4340c5c10 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc43c009770 tx=0x7fc43c006cd0 comp rx=0 tx=0).ready entity=mgr.15414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T18:04:41.952 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:41.947+0000 7fc448ff9700 1 -- 172.21.15.59:0/1082300140 <== mon.2 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fc4400ab730 con 0x7fc44c1056a0 2023-12-11T18:04:42.368 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:42.363+0000 7fc451b56700 1 -- 172.21.15.59:0/1082300140 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc44c100f30 con 0x7fc44c1056a0 2023-12-11T18:04:42.375 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:42.372+0000 7fc448ff9700 1 -- 172.21.15.59:0/1082300140 <== mon.2 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v119) v1 ==== 74+0+25923 (secure 0 0 0) 0x7fc440036090 con 0x7fc44c1056a0 2023-12-11T18:04:42.384 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-12-11T18:04:42.384 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":119,"fsid":"d0a07eaa-984b-11ee-95a3-87774f69a715","created":"2023-12-11T17:37:14.526539+0000","modified":"2023-12-11T18:04:11.237528+0000","last_up_change":"2023-12-11T17:58:28.975845+0000","last_in_change":"2023-12-11T17:43:21.014888+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":30,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-11T17:40:26.596029+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-11T17:44:18.586290+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-11T17:44:19.432952+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":27,"snap_epoch":119,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-11T17:44:23.097308+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":27,"snap_epoch":119,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3c0b3218-4b2b-40f8-8bef-ed8cd5633607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":1040759398},{"type":"v1","addr":"172.21.15.59:6803","nonce":1040759398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":1040759398},{"type":"v1","addr":"172.21.15.59:6805","nonce":1040759398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":1040759398},{"type":"v1","addr":"172.21.15.59:6809","nonce":1040759398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":1040759398},{"type":"v1","addr":"172.21.15.59:6807","nonce":1040759398}]},"public_addr":"172.21.15.59:6803/1040759398","cluster_addr":"172.21.15.59:6805/1040759398","heartbeat_back_addr":"172.21.15.59:6809/1040759398","heartbeat_front_addr":"172.21.15.59:6807/1040759398","state":["exists","up"]},{"osd":1,"uuid":"2ead7578-765b-4f5a-a72c-863731ed69e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":2776354245},{"type":"v1","addr":"172.21.15.115:6801","nonce":2776354245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":2776354245},{"type":"v1","addr":"172.21.15.115:6803","nonce":2776354245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":2776354245},{"type":"v1","addr":"172.21.15.115:6807","nonce":2776354245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":2776354245},{"type":"v1","addr":"172.21.15.115:6805","nonce":2776354245}]},"public_addr":"172.21.15.115:6801/2776354245","cluster_addr":"172.21.15.115:6803/2776354245","heartbeat_back_addr":"172.21.15.115:6807/2776354245","heartbeat_front_addr":"172.21.15.115:6805/2776354245","state":["exists","up"]},{"osd":2,"uuid":"8cb1d7ec-53cc-4672-b0b6-70d73797c44f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":18,"last_clean_end":97,"up_from":102,"up_thru":104,"down_at":98,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1449149834},{"type":"v1","addr":"172.21.15.181:6810","nonce":1449149834}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":1449149834},{"type":"v1","addr":"172.21.15.181:6814","nonce":1449149834}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6821","nonce":1449149834},{"type":"v1","addr":"172.21.15.181:6823","nonce":1449149834}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1449149834},{"type":"v1","addr":"172.21.15.181:6818","nonce":1449149834}]},"public_addr":"172.21.15.181:6810/1449149834","cluster_addr":"172.21.15.181:6814/1449149834","heartbeat_back_addr":"172.21.15.181:6823/1449149834","heartbeat_front_addr":"172.21.15.181:6818/1449149834","state":["exists","up"]},{"osd":3,"uuid":"5d64d9af-831b-4d0a-8ace-a452169351d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":219909995},{"type":"v1","addr":"172.21.15.59:6811","nonce":219909995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":219909995},{"type":"v1","addr":"172.21.15.59:6813","nonce":219909995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":219909995},{"type":"v1","addr":"172.21.15.59:6817","nonce":219909995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":219909995},{"type":"v1","addr":"172.21.15.59:6815","nonce":219909995}]},"public_addr":"172.21.15.59:6811/219909995","cluster_addr":"172.21.15.59:6813/219909995","heartbeat_back_addr":"172.21.15.59:6817/219909995","heartbeat_front_addr":"172.21.15.59:6815/219909995","state":["exists","up"]},{"osd":4,"uuid":"ef8fad28-1db5-4cba-b5f2-10e54e8e1549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":1021473826},{"type":"v1","addr":"172.21.15.115:6809","nonce":1021473826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":1021473826},{"type":"v1","addr":"172.21.15.115:6811","nonce":1021473826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":1021473826},{"type":"v1","addr":"172.21.15.115:6815","nonce":1021473826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":1021473826},{"type":"v1","addr":"172.21.15.115:6813","nonce":1021473826}]},"public_addr":"172.21.15.115:6809/1021473826","cluster_addr":"172.21.15.115:6811/1021473826","heartbeat_back_addr":"172.21.15.115:6815/1021473826","heartbeat_front_addr":"172.21.15.115:6813/1021473826","state":["exists","up"]},{"osd":5,"uuid":"4233a512-9e8f-4db8-aed2-6c3af706e589","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":37,"last_clean_end":97,"up_from":104,"up_thru":104,"down_at":98,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6809","nonce":3565629391},{"type":"v1","addr":"172.21.15.181:6811","nonce":3565629391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6813","nonce":3565629391},{"type":"v1","addr":"172.21.15.181:6815","nonce":3565629391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":3565629391},{"type":"v1","addr":"172.21.15.181:6822","nonce":3565629391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6817","nonce":3565629391},{"type":"v1","addr":"172.21.15.181:6819","nonce":3565629391}]},"public_addr":"172.21.15.181:6811/3565629391","cluster_addr":"172.21.15.181:6815/3565629391","heartbeat_back_addr":"172.21.15.181:6822/3565629391","heartbeat_front_addr":"172.21.15.181:6819/3565629391","state":["exists","up"]},{"osd":6,"uuid":"8faab332-016e-4c77-8370-8bef95fc6f34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":815374853},{"type":"v1","addr":"172.21.15.59:6819","nonce":815374853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":815374853},{"type":"v1","addr":"172.21.15.59:6821","nonce":815374853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":815374853},{"type":"v1","addr":"172.21.15.59:6825","nonce":815374853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":815374853},{"type":"v1","addr":"172.21.15.59:6823","nonce":815374853}]},"public_addr":"172.21.15.59:6819/815374853","cluster_addr":"172.21.15.59:6821/815374853","heartbeat_back_addr":"172.21.15.59:6825/815374853","heartbeat_front_addr":"172.21.15.59:6823/815374853","state":["exists","up"]},{"osd":7,"uuid":"f20f470a-0d40-4ed3-8e0e-fd58675cf143","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":1314971870},{"type":"v1","addr":"172.21.15.115:6817","nonce":1314971870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":1314971870},{"type":"v1","addr":"172.21.15.115:6819","nonce":1314971870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":1314971870},{"type":"v1","addr":"172.21.15.115:6823","nonce":1314971870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":1314971870},{"type":"v1","addr":"172.21.15.115:6821","nonce":1314971870}]},"public_addr":"172.21.15.115:6817/1314971870","cluster_addr":"172.21.15.115:6819/1314971870","heartbeat_back_addr":"172.21.15.115:6823/1314971870","heartbeat_front_addr":"172.21.15.115:6821/1314971870","state":["exists","up"]},{"osd":8,"uuid":"eb657d32-1a95-4fd7-a293-d0a970837fc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":57,"last_clean_end":97,"up_from":103,"up_thru":104,"down_at":98,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":3914011605},{"type":"v1","addr":"172.21.15.181:6833","nonce":3914011605}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3914011605},{"type":"v1","addr":"172.21.15.181:6835","nonce":3914011605}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":3914011605},{"type":"v1","addr":"172.21.15.181:6839","nonce":3914011605}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":3914011605},{"type":"v1","addr":"172.21.15.181:6837","nonce":3914011605}]},"public_addr":"172.21.15.181:6833/3914011605","cluster_addr":"172.21.15.181:6835/3914011605","heartbeat_back_addr":"172.21.15.181:6839/3914011605","heartbeat_front_addr":"172.21.15.181:6837/3914011605","state":["exists","up"]},{"osd":9,"uuid":"8f7717d4-b6f2-4ec1-9b58-62feb0c9a146","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":1163095677},{"type":"v1","addr":"172.21.15.59:6827","nonce":1163095677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":1163095677},{"type":"v1","addr":"172.21.15.59:6829","nonce":1163095677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":1163095677},{"type":"v1","addr":"172.21.15.59:6833","nonce":1163095677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":1163095677},{"type":"v1","addr":"172.21.15.59:6831","nonce":1163095677}]},"public_addr":"172.21.15.59:6827/1163095677","cluster_addr":"172.21.15.59:6829/1163095677","heartbeat_back_addr":"172.21.15.59:6833/1163095677","heartbeat_front_addr":"172.21.15.59:6831/1163095677","state":["exists","up"]},{"osd":10,"uuid":"b75c179a-875d-43bc-b4c1-e8eb80fb64fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":2199610269},{"type":"v1","addr":"172.21.15.115:6825","nonce":2199610269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":2199610269},{"type":"v1","addr":"172.21.15.115:6827","nonce":2199610269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":2199610269},{"type":"v1","addr":"172.21.15.115:6831","nonce":2199610269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":2199610269},{"type":"v1","addr":"172.21.15.115:6829","nonce":2199610269}]},"public_addr":"172.21.15.115:6825/2199610269","cluster_addr":"172.21.15.115:6827/2199610269","heartbeat_back_addr":"172.21.15.115:6831/2199610269","heartbeat_front_addr":"172.21.15.115:6829/2199610269","state":["exists","up"]},{"osd":11,"uuid":"5e141dfe-8a00-4bbc-8204-2dc9be24e18d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":77,"last_clean_end":97,"up_from":102,"up_thru":104,"down_at":98,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":540621004},{"type":"v1","addr":"172.21.15.181:6825","nonce":540621004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":540621004},{"type":"v1","addr":"172.21.15.181:6827","nonce":540621004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":540621004},{"type":"v1","addr":"172.21.15.181:6831","nonce":540621004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":540621004},{"type":"v1","addr":"172.21.15.181:6829","nonce":540621004}]},"public_addr":"172.21.15.181:6825/540621004","cluster_addr":"172.21.15.181:6827/540621004","heartbeat_back_addr":"172.21.15.181:6831/540621004","heartbeat_front_addr":"172.21.15.181:6829/540621004","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:39:42.687084+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:40:03.903448+0000","dead_epoch":0},{"osd":2,"down_stamp":"2023-12-11T17:56:22.121381+0000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:40:24.899908+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:40:46.600251+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:41:07.690037+0000","dead_epoch":0},{"osd":5,"down_stamp":"2023-12-11T17:56:22.121381+0000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:41:28.488855+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:41:49.459274+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:42:10.863881+0000","dead_epoch":0},{"osd":8,"down_stamp":"2023-12-11T17:56:22.121381+0000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:42:32.248391+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:42:54.112265+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:43:15.634872+0000","dead_epoch":0},{"osd":11,"down_stamp":"2023-12-11T17:56:22.121381+0000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-11T17:43:36.922534+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":7}]},{"pgid":"3.28","mappings":[{"from":5,"to":6},{"from":3,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.37","mappings":[{"from":1,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"4.6","mappings":[{"from":9,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:6839/2214384806":"2023-12-12T17:57:14.937455+0000","172.21.15.59:6801/3627728353":"2023-12-12T17:46:17.753068+0000","172.21.15.59:0/2192742819":"2023-12-12T17:37:44.773925+0000","172.21.15.181:6838/2214384806":"2023-12-12T17:57:14.937455+0000","172.21.15.59:0/2859415380":"2023-12-12T17:38:14.037922+0000","172.21.15.59:0/644616907":"2023-12-12T17:46:17.753068+0000","172.21.15.59:0/3853462899":"2023-12-12T17:37:44.773925+0000","172.21.15.59:0/4159306605":"2023-12-12T17:46:17.753068+0000","172.21.15.59:6800/3627728353":"2023-12-12T17:46:17.753068+0000","172.21.15.59:0/3570767650":"2023-12-12T17:37:44.773925+0000","172.21.15.59:6800/1672530462":"2023-12-12T17:37:44.773925+0000","172.21.15.59:6801/1259916482":"2023-12-12T17:38:14.037922+0000","172.21.15.59:0/1053944547":"2023-12-12T17:38:14.037922+0000","172.21.15.59:6801/1672530462":"2023-12-12T17:37:44.773925+0000","172.21.15.59:0/2683787475":"2023-12-12T17:46:17.753068+0000","172.21.15.59:0/17457774":"2023-12-12T17:38:14.037922+0000","172.21.15.59:6800/1259916482":"2023-12-12T17:38:14.037922+0000","172.21.15.59:0/2112420116":"2023-12-12T17:46:17.753068+0000","172.21.15.59:0/1373005911":"2023-12-12T17:46:17.753068+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":12,"length":1},{"begin":14,"length":1},{"begin":21,"length":1},{"begin":23,"length":1},{"begin":25,"length":1},{"begin":27,"length":1}]},{"pool":4,"snaps":[{"begin":10,"length":1},{"begin":12,"length":1},{"begin":14,"length":1},{"begin":21,"length":1},{"begin":23,"length":1},{"begin":25,"length":1},{"begin":27,"length":1}]}],"new_removed_snaps":[{"pool":3,"snaps":[{"begin":14,"length":1},{"begin":27,"length":1}]},{"pool":4,"snaps":[{"begin":14,"length":1},{"begin":27,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-11T18:04:42.385 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:42.382+0000 7fc451b56700 1 -- 172.21.15.59:0/1082300140 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7fc4340c3750 msgr2=0x7fc4340c5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T18:04:42.385 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:42.382+0000 7fc451b56700 1 --2- 172.21.15.59:0/1082300140 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7fc4340c3750 0x7fc4340c5c10 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc43c009770 tx=0x7fc43c006cd0 comp rx=0 tx=0).stop 2023-12-11T18:04:42.385 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:42.382+0000 7fc451b56700 1 -- 172.21.15.59:0/1082300140 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc44c1056a0 msgr2=0x7fc44c192f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T18:04:42.385 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:42.382+0000 7fc451b56700 1 --2- 172.21.15.59:0/1082300140 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc44c1056a0 0x7fc44c192f30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc440028620 tx=0x7fc440028700 comp rx=0 tx=0).stop 2023-12-11T18:04:42.391 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:42.388+0000 7fc451b56700 1 -- 172.21.15.59:0/1082300140 shutdown_connections 2023-12-11T18:04:42.391 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:42.388+0000 7fc451b56700 1 --2- 172.21.15.59:0/1082300140 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7fc4340c3750 0x7fc4340c5c10 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:04:42.391 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:42.388+0000 7fc451b56700 1 --2- 172.21.15.59:0/1082300140 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc44c06a6c0 0x7fc44c193470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:04:42.391 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:42.388+0000 7fc451b56700 1 --2- 172.21.15.59:0/1082300140 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc44c100580 0x7fc44c1929f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:04:42.391 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:42.388+0000 7fc451b56700 1 --2- 172.21.15.59:0/1082300140 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc44c1056a0 0x7fc44c192f30 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:04:42.391 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:42.388+0000 7fc451b56700 1 -- 172.21.15.59:0/1082300140 >> 172.21.15.59:0/1082300140 conn(0x7fc44c0fbe40 msgr2=0x7fc44c0fd190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T18:04:42.391 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:42.389+0000 7fc451b56700 1 -- 172.21.15.59:0/1082300140 shutdown_connections 2023-12-11T18:04:42.392 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:42.390+0000 7fc451b56700 1 -- 172.21.15.59:0/1082300140 wait complete. 2023-12-11T18:04:42.429 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-12-11T18:04:42.597 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:04:42 smithi059 ceph-mon[98847]: pgmap v585: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.7 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2023-12-11T18:04:42.598 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:04:42 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/3638143799' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T18:04:42.598 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:04:42 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T18:04:42.786 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:04:42 smithi115 ceph-mon[102748]: pgmap v585: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.7 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2023-12-11T18:04:42.786 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:04:42 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/3638143799' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-11T18:04:42.786 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:04:42 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-11T18:04:42.835 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:42.830+0000 7f7735744700 1 -- 172.21.15.59:0/502667994 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7730101540 msgr2=0x7f77301019c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T18:04:42.835 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:42.830+0000 7f7735744700 1 --2- 172.21.15.59:0/502667994 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7730101540 0x7f77301019c0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f772800b0a0 tx=0x7f772800b3b0 comp rx=0 tx=0).stop 2023-12-11T18:04:42.840 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:42.834+0000 7f7735744700 1 -- 172.21.15.59:0/502667994 shutdown_connections 2023-12-11T18:04:42.840 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:42.834+0000 7f7735744700 1 --2- 172.21.15.59:0/502667994 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7730101f00 0x7f773010e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:04:42.840 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:42.834+0000 7f7735744700 1 --2- 172.21.15.59:0/502667994 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7730107600 0x7f77301079e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:04:42.840 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:42.834+0000 7f7735744700 1 --2- 172.21.15.59:0/502667994 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7730101540 0x7f77301019c0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:04:42.840 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:42.834+0000 7f7735744700 1 -- 172.21.15.59:0/502667994 >> 172.21.15.59:0/502667994 conn(0x7f77300fd010 msgr2=0x7f77300ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T18:04:42.846 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:42.843+0000 7f7735744700 1 -- 172.21.15.59:0/502667994 shutdown_connections 2023-12-11T18:04:42.855 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:42.850+0000 7f7735744700 1 -- 172.21.15.59:0/502667994 wait complete. 2023-12-11T18:04:42.855 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:42.851+0000 7f7735744700 1 Processor -- start 2023-12-11T18:04:42.855 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:42.853+0000 7f7735744700 1 -- start start 2023-12-11T18:04:42.855 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:42.853+0000 7f7735744700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7730101540 0x7f7730072c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T18:04:42.855 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:42.853+0000 7f7735744700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7730101f00 0x7f77300731d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T18:04:42.855 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:42.853+0000 7f7735744700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7730107600 0x7f77300708d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T18:04:42.856 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:42.853+0000 7f7735744700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77300710a0 con 0x7f7730101f00 2023-12-11T18:04:42.856 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:42.853+0000 7f7735744700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7730071210 con 0x7f7730101540 2023-12-11T18:04:42.856 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:42.853+0000 7f7735744700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77300713d0 con 0x7f7730107600 2023-12-11T18:04:42.860 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:42.858+0000 7f772f7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7730107600 0x7f77300708d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T18:04:42.860 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:42.858+0000 7f772f7fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7730107600 0x7f77300708d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.59:40336/0 (socket says 172.21.15.59:40336) 2023-12-11T18:04:42.860 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:42.858+0000 7f772f7fe700 1 -- 172.21.15.59:0/3078122282 learned_addr learned my addr 172.21.15.59:0/3078122282 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T18:04:42.860 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:42.858+0000 7f772e7fc700 1 --2- 172.21.15.59:0/3078122282 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7730101f00 0x7f77300731d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T18:04:42.867 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:42.859+0000 7f772e7fc700 1 -- 172.21.15.59:0/3078122282 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7730107600 msgr2=0x7f77300708d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T18:04:42.867 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:42.859+0000 7f772e7fc700 1 --2- 172.21.15.59:0/3078122282 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7730107600 0x7f77300708d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:04:42.867 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:42.859+0000 7f772e7fc700 1 -- 172.21.15.59:0/3078122282 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7730101540 msgr2=0x7f7730072c90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-11T18:04:42.867 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:42.859+0000 7f772e7fc700 1 --2- 172.21.15.59:0/3078122282 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7730101540 0x7f7730072c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:04:42.867 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:42.859+0000 7f772e7fc700 1 -- 172.21.15.59:0/3078122282 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7728009d00 con 0x7f7730101f00 2023-12-11T18:04:42.871 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:42.869+0000 7f772e7fc700 1 --2- 172.21.15.59:0/3078122282 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7730101f00 0x7f77300731d0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f7728000f80 tx=0x7f7728004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T18:04:42.881 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:42.874+0000 7f7717fff700 1 -- 172.21.15.59:0/3078122282 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f772800e040 con 0x7f7730101f00 2023-12-11T18:04:42.881 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:42.874+0000 7f7717fff700 1 -- 172.21.15.59:0/3078122282 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f77280079d0 con 0x7f7730101f00 2023-12-11T18:04:42.881 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:42.874+0000 7f7717fff700 1 -- 172.21.15.59:0/3078122282 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77280052a0 con 0x7f7730101f00 2023-12-11T18:04:42.881 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:42.878+0000 7f7735744700 1 -- 172.21.15.59:0/3078122282 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7730071650 con 0x7f7730101f00 2023-12-11T18:04:42.881 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:42.878+0000 7f7735744700 1 -- 172.21.15.59:0/3078122282 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f773006c080 con 0x7f7730101f00 2023-12-11T18:04:42.882 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:42.880+0000 7f7735744700 1 -- 172.21.15.59:0/3078122282 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f771c0052f0 con 0x7f7730101f00 2023-12-11T18:04:42.884 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:42.880+0000 7f7717fff700 1 -- 172.21.15.59:0/3078122282 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 28) v1 ==== 136137+0+0 (secure 0 0 0) 0x7f7728005400 con 0x7f7730101f00 2023-12-11T18:04:42.884 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:42.881+0000 7f7717fff700 1 --2- 172.21.15.59:0/3078122282 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7f77180a16b0 0x7f77180a3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T18:04:42.884 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:42.881+0000 7f7717fff700 1 -- 172.21.15.59:0/3078122282 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 9851+0+0 (secure 0 0 0) 0x7f77280ce9e0 con 0x7f7730101f00 2023-12-11T18:04:42.896 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:42.891+0000 7f7717fff700 1 -- 172.21.15.59:0/3078122282 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f772809af40 con 0x7f7730101f00 2023-12-11T18:04:42.902 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:42.896+0000 7f772effd700 1 --2- 172.21.15.59:0/3078122282 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7f77180a16b0 0x7f77180a3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T18:04:42.915 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:42.913+0000 7f772effd700 1 --2- 172.21.15.59:0/3078122282 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7f77180a16b0 0x7f77180a3b70 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7730106580 tx=0x7f7720006cb0 comp rx=0 tx=0).ready entity=mgr.15414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T18:04:43.360 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:43.352+0000 7f7735744700 1 -- 172.21.15.59:0/3078122282 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f771c005160 con 0x7f7730101f00 2023-12-11T18:04:43.398 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:43.396+0000 7f7717fff700 1 -- 172.21.15.59:0/3078122282 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v56) v1 ==== 161+0+0 (secure 0 0 0) 0x7f772809af40 con 0x7f7730101f00 2023-12-11T18:04:43.414 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:43.407+0000 7f7735744700 1 -- 172.21.15.59:0/3078122282 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7f77180a16b0 msgr2=0x7f77180a3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T18:04:43.414 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:43.407+0000 7f7735744700 1 --2- 172.21.15.59:0/3078122282 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7f77180a16b0 0x7f77180a3b70 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7730106580 tx=0x7f7720006cb0 comp rx=0 tx=0).stop 2023-12-11T18:04:43.414 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:43.407+0000 7f7735744700 1 -- 172.21.15.59:0/3078122282 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7730101f00 msgr2=0x7f77300731d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T18:04:43.414 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:43.407+0000 7f7735744700 1 --2- 172.21.15.59:0/3078122282 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7730101f00 0x7f77300731d0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f7728000f80 tx=0x7f7728004c80 comp rx=0 tx=0).stop 2023-12-11T18:04:43.414 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:43.410+0000 7f7735744700 1 -- 172.21.15.59:0/3078122282 shutdown_connections 2023-12-11T18:04:43.414 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:43.410+0000 7f7735744700 1 --2- 172.21.15.59:0/3078122282 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7f77180a16b0 0x7f77180a3b70 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:04:43.414 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:43.410+0000 7f7735744700 1 --2- 172.21.15.59:0/3078122282 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7730107600 0x7f77300708d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:04:43.414 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:43.410+0000 7f7735744700 1 --2- 172.21.15.59:0/3078122282 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7730101540 0x7f7730072c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:04:43.414 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:43.410+0000 7f7735744700 1 --2- 172.21.15.59:0/3078122282 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7730101f00 0x7f77300731d0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:04:43.414 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:43.410+0000 7f7735744700 1 -- 172.21.15.59:0/3078122282 >> 172.21.15.59:0/3078122282 conn(0x7f77300fd010 msgr2=0x7f77300fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T18:04:43.420 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:43.418+0000 7f7735744700 1 -- 172.21.15.59:0/3078122282 shutdown_connections 2023-12-11T18:04:43.432 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:43.430+0000 7f7735744700 1 -- 172.21.15.59:0/3078122282 wait complete. 2023-12-11T18:04:43.433 INFO:teuthology.orchestra.run.smithi059.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-12-11T18:04:43.456 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-12-11T18:04:43.657 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:04:43 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mon-a[98823]: 2023-12-11T18:04:43.371+0000 7faaa0cf5700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-12-11T18:04:43.658 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:04:43 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/1082300140' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-11T18:04:43.658 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:04:43 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T18:04:43.658 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:04:43 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T18:04:43.658 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:04:43 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T18:04:43.691 INFO:journalctl@ceph.mds.h.smithi115.stdout:Dec 11 18:04:43 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mds-h[139354]: ignoring --setuser ceph since I am not root 2023-12-11T18:04:43.692 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:04:43 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/1082300140' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-11T18:04:43.692 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:04:43 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-11T18:04:43.692 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:04:43 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-11T18:04:43.692 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:04:43 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' 2023-12-11T18:04:43.750 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:43.747+0000 7f6fd63e9700 1 -- 172.21.15.59:0/146711301 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6fd0109730 msgr2=0x7f6fd0109b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T18:04:43.750 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:43.747+0000 7f6fd63e9700 1 --2- 172.21.15.59:0/146711301 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6fd0109730 0x7f6fd0109b10 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f6fc0009990 tx=0x7f6fc0009ca0 comp rx=0 tx=0).stop 2023-12-11T18:04:43.750 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:43.747+0000 7f6fd63e9700 1 -- 172.21.15.59:0/146711301 shutdown_connections 2023-12-11T18:04:43.750 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:43.747+0000 7f6fd63e9700 1 --2- 172.21.15.59:0/146711301 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6fd01040e0 0x7f6fd01105f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:04:43.750 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:43.747+0000 7f6fd63e9700 1 --2- 172.21.15.59:0/146711301 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6fd0109730 0x7f6fd0109b10 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:04:43.750 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:43.747+0000 7f6fd63e9700 1 --2- 172.21.15.59:0/146711301 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6fd0103720 0x7f6fd0103ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:04:43.750 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:43.747+0000 7f6fd63e9700 1 -- 172.21.15.59:0/146711301 >> 172.21.15.59:0/146711301 conn(0x7f6fd00ff1b0 msgr2=0x7f6fd01015d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T18:04:43.752 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:43.750+0000 7f6fd63e9700 1 -- 172.21.15.59:0/146711301 shutdown_connections 2023-12-11T18:04:43.759 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:43.756+0000 7f6fd63e9700 1 -- 172.21.15.59:0/146711301 wait complete. 2023-12-11T18:04:43.759 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:43.756+0000 7f6fd63e9700 1 Processor -- start 2023-12-11T18:04:43.759 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:43.757+0000 7f6fd63e9700 1 -- start start 2023-12-11T18:04:43.759 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:43.757+0000 7f6fd63e9700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6fd0103720 0x7f6fd0110370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T18:04:43.759 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:43.757+0000 7f6fd63e9700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6fd01040e0 0x7f6fd010d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T18:04:43.760 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:43.757+0000 7f6fd63e9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6fd0109730 0x7f6fd010d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T18:04:43.760 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:43.757+0000 7f6fd63e9700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fd010e100 con 0x7f6fd0103720 2023-12-11T18:04:43.760 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:43.757+0000 7f6fd63e9700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fd010e270 con 0x7f6fd01040e0 2023-12-11T18:04:43.760 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:43.757+0000 7f6fd63e9700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fd01a1df0 con 0x7f6fd0109730 2023-12-11T18:04:43.765 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:43.760+0000 7f6fcffff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6fd0103720 0x7f6fd0110370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T18:04:43.765 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:43.760+0000 7f6fcffff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6fd0103720 0x7f6fd0110370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35532/0 (socket says 172.21.15.59:35532) 2023-12-11T18:04:43.765 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:43.760+0000 7f6fcffff700 1 -- 172.21.15.59:0/704644296 learned_addr learned my addr 172.21.15.59:0/704644296 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T18:04:43.765 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:43.761+0000 7f6fcffff700 1 -- 172.21.15.59:0/704644296 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6fd0109730 msgr2=0x7f6fd010d980 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-12-11T18:04:43.765 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:43.761+0000 7f6fcffff700 1 --2- 172.21.15.59:0/704644296 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6fd0109730 0x7f6fd010d980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:04:43.765 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:43.761+0000 7f6fcffff700 1 -- 172.21.15.59:0/704644296 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6fd01040e0 msgr2=0x7f6fd010d440 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-11T18:04:43.765 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:43.761+0000 7f6fcffff700 1 --2- 172.21.15.59:0/704644296 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6fd01040e0 0x7f6fd010d440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:04:43.766 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:43.761+0000 7f6fcffff700 1 -- 172.21.15.59:0/704644296 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fc00095f0 con 0x7f6fd0103720 2023-12-11T18:04:43.766 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:43.761+0000 7f6fcffff700 1 --2- 172.21.15.59:0/704644296 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6fd0103720 0x7f6fd0110370 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f6fc0003ac0 tx=0x7f6fc00037c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T18:04:43.766 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:43.761+0000 7f6fcd7fa700 1 -- 172.21.15.59:0/704644296 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fc001d070 con 0x7f6fd0103720 2023-12-11T18:04:43.766 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:43.761+0000 7f6fcd7fa700 1 -- 172.21.15.59:0/704644296 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6fc0005580 con 0x7f6fd0103720 2023-12-11T18:04:43.766 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:43.761+0000 7f6fcd7fa700 1 -- 172.21.15.59:0/704644296 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fc000fd70 con 0x7f6fd0103720 2023-12-11T18:04:43.766 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:43.761+0000 7f6fd63e9700 1 -- 172.21.15.59:0/704644296 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fd006aff0 con 0x7f6fd0103720 2023-12-11T18:04:43.766 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:43.762+0000 7f6fd63e9700 1 -- 172.21.15.59:0/704644296 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fd006b4b0 con 0x7f6fd0103720 2023-12-11T18:04:43.769 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:43.765+0000 7f6fcd7fa700 1 -- 172.21.15.59:0/704644296 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 28) v1 ==== 136137+0+0 (secure 0 0 0) 0x7f6fc002b440 con 0x7f6fd0103720 2023-12-11T18:04:43.769 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:43.766+0000 7f6fcd7fa700 1 --2- 172.21.15.59:0/704644296 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7f6fb80a16b0 0x7f6fb80a3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T18:04:43.769 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:43.767+0000 7f6fcd7fa700 1 -- 172.21.15.59:0/704644296 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 10021+0+0 (secure 0 0 0) 0x7f6fc00ce800 con 0x7f6fd0103720 2023-12-11T18:04:43.779 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:43.770+0000 7f6fd63e9700 1 -- 172.21.15.59:0/704644296 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fd0077170 con 0x7f6fd0103720 2023-12-11T18:04:43.779 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:43.772+0000 7f6fcf7fe700 1 --2- 172.21.15.59:0/704644296 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7f6fb80a16b0 0x7f6fb80a3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T18:04:43.779 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:43.773+0000 7f6fcf7fe700 1 --2- 172.21.15.59:0/704644296 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7f6fb80a16b0 0x7f6fb80a3b70 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f6fd010eb70 tx=0x7f6fc4006c40 comp rx=0 tx=0).ready entity=mgr.15414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T18:04:43.780 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:43.777+0000 7f6fcd7fa700 1 -- 172.21.15.59:0/704644296 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f6fc00975d0 con 0x7f6fd0103720 2023-12-11T18:04:44.035 INFO:journalctl@ceph.mds.k.smithi115.stdout:Dec 11 18:04:43 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mds-k[138680]: ignoring --setuser ceph since I am not root 2023-12-11T18:04:44.035 INFO:journalctl@ceph.mds.k.smithi115.stdout:Dec 11 18:04:43 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mds-k[138680]: ignoring --setgroup ceph since I am not root 2023-12-11T18:04:44.035 INFO:journalctl@ceph.mds.k.smithi115.stdout:Dec 11 18:04:43 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mds-k[138680]: starting mds.k at 2023-12-11T18:04:44.040 INFO:journalctl@ceph.mds.b.smithi115.stdout:Dec 11 18:04:43 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mds-b[139691]: ignoring --setuser ceph since I am not root 2023-12-11T18:04:44.040 INFO:journalctl@ceph.mds.b.smithi115.stdout:Dec 11 18:04:43 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mds-b[139691]: ignoring --setgroup ceph since I am not root 2023-12-11T18:04:44.041 INFO:journalctl@ceph.mds.b.smithi115.stdout:Dec 11 18:04:43 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mds-b[139691]: starting mds.b at 2023-12-11T18:04:44.041 INFO:journalctl@ceph.mds.e.smithi115.stdout:Dec 11 18:04:43 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mds-e[139018]: ignoring --setuser ceph since I am not root 2023-12-11T18:04:44.041 INFO:journalctl@ceph.mds.e.smithi115.stdout:Dec 11 18:04:43 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mds-e[139018]: ignoring --setgroup ceph since I am not root 2023-12-11T18:04:44.041 INFO:journalctl@ceph.mds.e.smithi115.stdout:Dec 11 18:04:43 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mds-e[139018]: starting mds.e at 2023-12-11T18:04:44.041 INFO:journalctl@ceph.mds.h.smithi115.stdout:Dec 11 18:04:43 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mds-h[139354]: ignoring --setgroup ceph since I am not root 2023-12-11T18:04:44.041 INFO:journalctl@ceph.mds.h.smithi115.stdout:Dec 11 18:04:43 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mds-h[139354]: starting mds.h at 2023-12-11T18:04:44.249 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:44.245+0000 7f6fd63e9700 1 -- 172.21.15.59:0/704644296 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f6fd0104e60 con 0x7f6fd0103720 2023-12-11T18:04:44.417 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:44.415+0000 7f6fcd7fa700 1 -- 172.21.15.59:0/704644296 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v57) v1 ==== 104+0+0 (secure 0 0 0) 0x7f6fc009abf0 con 0x7f6fd0103720 2023-12-11T18:04:44.441 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:44.433+0000 7f6fd63e9700 1 -- 172.21.15.59:0/704644296 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7f6fb80a16b0 msgr2=0x7f6fb80a3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T18:04:44.441 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:44.433+0000 7f6fd63e9700 1 --2- 172.21.15.59:0/704644296 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7f6fb80a16b0 0x7f6fb80a3b70 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f6fd010eb70 tx=0x7f6fc4006c40 comp rx=0 tx=0).stop 2023-12-11T18:04:44.441 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:44.433+0000 7f6fd63e9700 1 -- 172.21.15.59:0/704644296 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6fd0103720 msgr2=0x7f6fd0110370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T18:04:44.441 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:44.433+0000 7f6fd63e9700 1 --2- 172.21.15.59:0/704644296 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6fd0103720 0x7f6fd0110370 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f6fc0003ac0 tx=0x7f6fc00037c0 comp rx=0 tx=0).stop 2023-12-11T18:04:44.441 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:44.437+0000 7f6fd63e9700 1 -- 172.21.15.59:0/704644296 shutdown_connections 2023-12-11T18:04:44.441 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:44.437+0000 7f6fd63e9700 1 --2- 172.21.15.59:0/704644296 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7f6fb80a16b0 0x7f6fb80a3b70 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:04:44.441 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:44.437+0000 7f6fd63e9700 1 --2- 172.21.15.59:0/704644296 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6fd0109730 0x7f6fd010d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:04:44.441 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:44.437+0000 7f6fd63e9700 1 --2- 172.21.15.59:0/704644296 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6fd0103720 0x7f6fd0110370 secure :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f6fc0003ac0 tx=0x7f6fc00037c0 comp rx=0 tx=0).stop 2023-12-11T18:04:44.441 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:44.437+0000 7f6fd63e9700 1 --2- 172.21.15.59:0/704644296 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6fd01040e0 0x7f6fd010d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:04:44.441 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:44.437+0000 7f6fd63e9700 1 -- 172.21.15.59:0/704644296 >> 172.21.15.59:0/704644296 conn(0x7f6fd00ff1b0 msgr2=0x7f6fd00ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T18:04:44.452 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:44.448+0000 7f6fd63e9700 1 -- 172.21.15.59:0/704644296 shutdown_connections 2023-12-11T18:04:44.458 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:44.454+0000 7f6fd63e9700 1 -- 172.21.15.59:0/704644296 wait complete. 2023-12-11T18:04:44.488 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-12-11T18:04:44.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:04:44 smithi059 ceph-mon[98847]: pgmap v586: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.7 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2023-12-11T18:04:44.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:04:44 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/3078122282' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-12-11T18:04:44.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:04:44 smithi059 ceph-mon[98847]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-12-11T18:04:44.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:04:44 smithi059 ceph-mon[98847]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-12-11T18:04:44.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:04:44 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/3078122282' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-12-11T18:04:44.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:04:44 smithi059 ceph-mon[98847]: osdmap e120: 12 total, 12 up, 12 in 2023-12-11T18:04:44.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:04:44 smithi059 ceph-mon[98847]: fsmap cephfs:0/3 6 up:standby, 3 failed 2023-12-11T18:04:44.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:04:44 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/704644296' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-12-11T18:04:44.785 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:04:44 smithi115 ceph-mon[102748]: pgmap v586: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.7 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2023-12-11T18:04:44.785 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:04:44 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/3078122282' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-12-11T18:04:44.785 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:04:44 smithi115 ceph-mon[102748]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-12-11T18:04:44.785 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:04:44 smithi115 ceph-mon[102748]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-12-11T18:04:44.785 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:04:44 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/3078122282' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-12-11T18:04:44.785 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:04:44 smithi115 ceph-mon[102748]: osdmap e120: 12 total, 12 up, 12 in 2023-12-11T18:04:44.785 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:04:44 smithi115 ceph-mon[102748]: fsmap cephfs:0/3 6 up:standby, 3 failed 2023-12-11T18:04:44.785 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:04:44 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/704644296' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-12-11T18:04:44.791 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:44.789+0000 7fdeb2d35700 1 -- 172.21.15.59:0/114226536 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdeac101510 msgr2=0x7fdeac10d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T18:04:44.791 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:44.789+0000 7fdeb2d35700 1 --2- 172.21.15.59:0/114226536 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdeac101510 0x7fdeac10d290 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fdea400b0a0 tx=0x7fdea400b3b0 comp rx=0 tx=0).stop 2023-12-11T18:04:44.794 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:44.792+0000 7fdeb2d35700 1 -- 172.21.15.59:0/114226536 shutdown_connections 2023-12-11T18:04:44.794 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:44.792+0000 7fdeb2d35700 1 --2- 172.21.15.59:0/114226536 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdeac101510 0x7fdeac10d290 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:04:44.794 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:44.792+0000 7fdeb2d35700 1 --2- 172.21.15.59:0/114226536 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdeac1064d0 0x7fdeac1068b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:04:44.794 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:44.792+0000 7fdeb2d35700 1 --2- 172.21.15.59:0/114226536 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdeac100b50 0x7fdeac100fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:04:44.794 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:44.792+0000 7fdeb2d35700 1 -- 172.21.15.59:0/114226536 >> 172.21.15.59:0/114226536 conn(0x7fdeac076a00 msgr2=0x7fdeac0fdd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T18:04:44.797 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:44.795+0000 7fdeb2d35700 1 -- 172.21.15.59:0/114226536 shutdown_connections 2023-12-11T18:04:44.802 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:44.799+0000 7fdeb2d35700 1 -- 172.21.15.59:0/114226536 wait complete. 2023-12-11T18:04:44.802 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:44.799+0000 7fdeb2d35700 1 Processor -- start 2023-12-11T18:04:44.813 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:44.802+0000 7fdeb2d35700 1 -- start start 2023-12-11T18:04:44.813 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:44.802+0000 7fdeb2d35700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdeac100b50 0x7fdeac10eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T18:04:44.813 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:44.802+0000 7fdeb2d35700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdeac101510 0x7fdeac10f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T18:04:44.813 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:44.802+0000 7fdeb2d35700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdeac1064d0 0x7fdeac19e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T18:04:44.813 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:44.802+0000 7fdeb2d35700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdeac19e980 con 0x7fdeac1064d0 2023-12-11T18:04:44.813 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:44.802+0000 7fdeb2d35700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdeac19eaf0 con 0x7fdeac101510 2023-12-11T18:04:44.813 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:44.802+0000 7fdeb2d35700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdeac19ecb0 con 0x7fdeac100b50 2023-12-11T18:04:44.813 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:44.803+0000 7fdeb12d2700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdeac1064d0 0x7fdeac19e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T18:04:44.813 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:44.803+0000 7fdeb0ad1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdeac100b50 0x7fdeac10eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T18:04:44.814 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:44.803+0000 7fdeb12d2700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdeac1064d0 0x7fdeac19e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35546/0 (socket says 172.21.15.59:35546) 2023-12-11T18:04:44.814 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:44.803+0000 7fdeb12d2700 1 -- 172.21.15.59:0/4206980712 learned_addr learned my addr 172.21.15.59:0/4206980712 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T18:04:44.814 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:44.803+0000 7fdeabfff700 1 --2- 172.21.15.59:0/4206980712 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdeac101510 0x7fdeac10f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T18:04:44.814 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:44.803+0000 7fdeb12d2700 1 -- 172.21.15.59:0/4206980712 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdeac100b50 msgr2=0x7fdeac10eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T18:04:44.814 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:44.803+0000 7fdeb12d2700 1 --2- 172.21.15.59:0/4206980712 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdeac100b50 0x7fdeac10eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:04:44.814 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:44.803+0000 7fdeb12d2700 1 -- 172.21.15.59:0/4206980712 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdeac101510 msgr2=0x7fdeac10f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T18:04:44.814 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:44.803+0000 7fdeb12d2700 1 --2- 172.21.15.59:0/4206980712 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdeac101510 0x7fdeac10f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:04:44.814 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:44.803+0000 7fdeb12d2700 1 -- 172.21.15.59:0/4206980712 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdea4009d00 con 0x7fdeac1064d0 2023-12-11T18:04:44.814 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:44.804+0000 7fdeb12d2700 1 --2- 172.21.15.59:0/4206980712 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdeac1064d0 0x7fdeac19e1b0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fdea4007fc0 tx=0x7fdea40094f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T18:04:44.814 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:44.806+0000 7fdea9ffb700 1 -- 172.21.15.59:0/4206980712 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea400e040 con 0x7fdeac1064d0 2023-12-11T18:04:44.814 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:44.807+0000 7fdea9ffb700 1 -- 172.21.15.59:0/4206980712 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdea40079b0 con 0x7fdeac1064d0 2023-12-11T18:04:44.814 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:44.807+0000 7fdea9ffb700 1 -- 172.21.15.59:0/4206980712 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea4005280 con 0x7fdeac1064d0 2023-12-11T18:04:44.814 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:44.810+0000 7fdeb2d35700 1 -- 172.21.15.59:0/4206980712 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdeac10a900 con 0x7fdeac1064d0 2023-12-11T18:04:44.815 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:44.810+0000 7fdeb2d35700 1 -- 172.21.15.59:0/4206980712 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdeac10ad90 con 0x7fdeac1064d0 2023-12-11T18:04:44.816 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:44.814+0000 7fdea9ffb700 1 -- 172.21.15.59:0/4206980712 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 28) v1 ==== 136137+0+0 (secure 0 0 0) 0x7fdea403d9b0 con 0x7fdeac1064d0 2023-12-11T18:04:44.817 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:44.814+0000 7fdea9ffb700 1 --2- 172.21.15.59:0/4206980712 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7fde9c0a1740 0x7fde9c0a3c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T18:04:44.817 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:44.815+0000 7fdea9ffb700 1 -- 172.21.15.59:0/4206980712 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 10021+0+0 (secure 0 0 0) 0x7fdea40f0880 con 0x7fdeac1064d0 2023-12-11T18:04:44.817 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:44.815+0000 7fdeb0ad1700 1 --2- 172.21.15.59:0/4206980712 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7fde9c0a1740 0x7fde9c0a3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T18:04:44.818 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:44.816+0000 7fdeb0ad1700 1 --2- 172.21.15.59:0/4206980712 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7fde9c0a1740 0x7fde9c0a3c00 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fdea0006fd0 tx=0x7fdea0008040 comp rx=0 tx=0).ready entity=mgr.15414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T18:04:44.826 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:44.822+0000 7fdeb2d35700 1 -- 172.21.15.59:0/4206980712 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdeac101b60 con 0x7fdeac1064d0 2023-12-11T18:04:44.838 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:44.836+0000 7fdea9ffb700 1 -- 172.21.15.59:0/4206980712 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fdea40b9650 con 0x7fdeac1064d0 2023-12-11T18:04:45.197 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:45.195+0000 7fdeb2d35700 1 -- 172.21.15.59:0/4206980712 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fdeac107330 con 0x7fdeac1064d0 2023-12-11T18:04:45.297 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:45.294+0000 7fdea9ffb700 1 -- 172.21.15.59:0/4206980712 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v121) v1 ==== 181+0+0 (secure 0 0 0) 0x7fdea4042070 con 0x7fdeac1064d0 2023-12-11T18:04:45.312 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:45.308+0000 7fdeb2d35700 1 -- 172.21.15.59:0/4206980712 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7fde9c0a1740 msgr2=0x7fde9c0a3c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T18:04:45.312 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:45.308+0000 7fdeb2d35700 1 --2- 172.21.15.59:0/4206980712 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7fde9c0a1740 0x7fde9c0a3c00 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fdea0006fd0 tx=0x7fdea0008040 comp rx=0 tx=0).stop 2023-12-11T18:04:45.312 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:45.308+0000 7fdeb2d35700 1 -- 172.21.15.59:0/4206980712 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdeac1064d0 msgr2=0x7fdeac19e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T18:04:45.312 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:45.308+0000 7fdeb2d35700 1 --2- 172.21.15.59:0/4206980712 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdeac1064d0 0x7fdeac19e1b0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fdea4007fc0 tx=0x7fdea40094f0 comp rx=0 tx=0).stop 2023-12-11T18:04:45.314 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:45.311+0000 7fdeb2d35700 1 -- 172.21.15.59:0/4206980712 shutdown_connections 2023-12-11T18:04:45.314 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:45.311+0000 7fdeb2d35700 1 --2- 172.21.15.59:0/4206980712 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7fde9c0a1740 0x7fde9c0a3c00 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:04:45.314 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:45.311+0000 7fdeb2d35700 1 --2- 172.21.15.59:0/4206980712 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdeac1064d0 0x7fdeac19e1b0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:04:45.314 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:45.311+0000 7fdeb2d35700 1 --2- 172.21.15.59:0/4206980712 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdeac100b50 0x7fdeac10eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:04:45.314 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:45.311+0000 7fdeb2d35700 1 --2- 172.21.15.59:0/4206980712 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdeac101510 0x7fdeac10f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:04:45.314 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:45.311+0000 7fdeb2d35700 1 -- 172.21.15.59:0/4206980712 >> 172.21.15.59:0/4206980712 conn(0x7fdeac076a00 msgr2=0x7fdeac0fe0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T18:04:45.327 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:45.323+0000 7fdeb2d35700 1 -- 172.21.15.59:0/4206980712 shutdown_connections 2023-12-11T18:04:45.333 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:45.328+0000 7fdeb2d35700 1 -- 172.21.15.59:0/4206980712 wait complete. 2023-12-11T18:04:45.349 INFO:teuthology.orchestra.run.smithi059.stderr:pool 'cephfs_metadata' removed 2023-12-11T18:04:45.367 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-12-11T18:04:45.624 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:45.622+0000 7fba7ce5a700 1 -- 172.21.15.59:0/2586353551 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fba780ffa40 msgr2=0x7fba780ffec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T18:04:45.625 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:45.622+0000 7fba7ce5a700 1 --2- 172.21.15.59:0/2586353551 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fba780ffa40 0x7fba780ffec0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fba680098f0 tx=0x7fba68009c00 comp rx=0 tx=0).stop 2023-12-11T18:04:45.630 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:45.627+0000 7fba7ce5a700 1 -- 172.21.15.59:0/2586353551 shutdown_connections 2023-12-11T18:04:45.630 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:45.627+0000 7fba7ce5a700 1 --2- 172.21.15.59:0/2586353551 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba78100400 0x7fba78103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:04:45.630 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:45.627+0000 7fba7ce5a700 1 --2- 172.21.15.59:0/2586353551 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba78106510 0x7fba781068f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:04:45.630 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:45.627+0000 7fba7ce5a700 1 --2- 172.21.15.59:0/2586353551 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fba780ffa40 0x7fba780ffec0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:04:45.630 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:45.627+0000 7fba7ce5a700 1 -- 172.21.15.59:0/2586353551 >> 172.21.15.59:0/2586353551 conn(0x7fba78076a00 msgr2=0x7fba780fdd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T18:04:45.643 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:45.638+0000 7fba7ce5a700 1 -- 172.21.15.59:0/2586353551 shutdown_connections 2023-12-11T18:04:45.643 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:45.640+0000 7fba7ce5a700 1 -- 172.21.15.59:0/2586353551 wait complete. 2023-12-11T18:04:45.649 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:45.641+0000 7fba7ce5a700 1 Processor -- start 2023-12-11T18:04:45.649 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:45.641+0000 7fba7ce5a700 1 -- start start 2023-12-11T18:04:45.650 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:45.642+0000 7fba7ce5a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba780ffa40 0x7fba780730d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T18:04:45.650 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:45.642+0000 7fba7ce5a700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fba78100400 0x7fba7806b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T18:04:45.650 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:45.642+0000 7fba7ce5a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba78106510 0x7fba7806b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T18:04:45.650 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:45.642+0000 7fba7ce5a700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba7806bdb0 con 0x7fba780ffa40 2023-12-11T18:04:45.650 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:45.642+0000 7fba7ce5a700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba7806bf20 con 0x7fba78100400 2023-12-11T18:04:45.650 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:45.642+0000 7fba7ce5a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba781b2db0 con 0x7fba78106510 2023-12-11T18:04:45.650 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:45.642+0000 7fba75d9b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fba78100400 0x7fba7806b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T18:04:45.650 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:45.642+0000 7fba76d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba78106510 0x7fba7806b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T18:04:45.650 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:45.642+0000 7fba76d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba78106510 0x7fba7806b620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.59:40374/0 (socket says 172.21.15.59:40374) 2023-12-11T18:04:45.650 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:45.642+0000 7fba75d9b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fba78100400 0x7fba7806b0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.59:60286/0 (socket says 172.21.15.59:60286) 2023-12-11T18:04:45.650 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:45.642+0000 7fba76d9d700 1 -- 172.21.15.59:0/4036288879 learned_addr learned my addr 172.21.15.59:0/4036288879 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T18:04:45.650 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:45.643+0000 7fba76d9d700 1 -- 172.21.15.59:0/4036288879 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fba78100400 msgr2=0x7fba7806b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T18:04:45.650 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:45.643+0000 7fba76d9d700 1 --2- 172.21.15.59:0/4036288879 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fba78100400 0x7fba7806b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:04:45.651 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:45.643+0000 7fba76d9d700 1 -- 172.21.15.59:0/4036288879 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba780ffa40 msgr2=0x7fba780730d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T18:04:45.651 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:45.643+0000 7fba76d9d700 1 --2- 172.21.15.59:0/4036288879 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba780ffa40 0x7fba780730d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:04:45.651 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:45.643+0000 7fba76d9d700 1 -- 172.21.15.59:0/4036288879 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba68009520 con 0x7fba78106510 2023-12-11T18:04:45.651 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:45.643+0000 7fba76d9d700 1 --2- 172.21.15.59:0/4036288879 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba78106510 0x7fba7806b620 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fba6c00b4d0 tx=0x7fba6c00b890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T18:04:45.651 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:45.645+0000 7fba677fe700 1 -- 172.21.15.59:0/4036288879 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba6c011830 con 0x7fba78106510 2023-12-11T18:04:45.651 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:45.645+0000 7fba677fe700 1 -- 172.21.15.59:0/4036288879 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fba6c011e50 con 0x7fba78106510 2023-12-11T18:04:45.651 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:45.645+0000 7fba677fe700 1 -- 172.21.15.59:0/4036288879 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba6c00f2e0 con 0x7fba78106510 2023-12-11T18:04:45.651 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:45.645+0000 7fba7ce5a700 1 -- 172.21.15.59:0/4036288879 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba7819dde0 con 0x7fba78106510 2023-12-11T18:04:45.651 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:45.646+0000 7fba7ce5a700 1 -- 172.21.15.59:0/4036288879 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba7819e270 con 0x7fba78106510 2023-12-11T18:04:45.653 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:45.648+0000 7fba7ce5a700 1 -- 172.21.15.59:0/4036288879 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba580052f0 con 0x7fba78106510 2023-12-11T18:04:45.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:04:45 smithi059 ceph-mon[98847]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-12-11T18:04:45.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:04:45 smithi059 ceph-mon[98847]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-12-11T18:04:45.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:04:45 smithi059 ceph-mon[98847]: Cluster is now healthy 2023-12-11T18:04:45.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:04:45 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/704644296' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-12-11T18:04:45.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:04:45 smithi059 ceph-mon[98847]: mds.? [v2:172.21.15.181:6806/361059264,v1:172.21.15.181:6807/361059264] up:boot 2023-12-11T18:04:45.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:04:45 smithi059 ceph-mon[98847]: mds.? [v2:172.21.15.181:6804/2447812160,v1:172.21.15.181:6805/2447812160] up:boot 2023-12-11T18:04:45.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:04:45 smithi059 ceph-mon[98847]: mds.? [v2:172.21.15.115:6832/2018449929,v1:172.21.15.115:6833/2018449929] up:boot 2023-12-11T18:04:45.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:04:45 smithi059 ceph-mon[98847]: mds.? [v2:172.21.15.115:6835/4283455912,v1:172.21.15.115:6837/4283455912] up:boot 2023-12-11T18:04:45.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:04:45 smithi059 ceph-mon[98847]: mds.? [v2:172.21.15.115:6834/3868201075,v1:172.21.15.115:6836/3868201075] up:boot 2023-12-11T18:04:45.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:04:45 smithi059 ceph-mon[98847]: mds.? [v2:172.21.15.115:6838/3860170944,v1:172.21.15.115:6839/3860170944] up:boot 2023-12-11T18:04:45.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:04:45 smithi059 ceph-mon[98847]: fsmap 12 up:standby 2023-12-11T18:04:45.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:04:45 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-12-11T18:04:45.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:04:45 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-12-11T18:04:45.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:04:45 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-12-11T18:04:45.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:04:45 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-12-11T18:04:45.656 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:04:45 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-12-11T18:04:45.656 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:04:45 smithi059 ceph-mon[98847]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-12-11T18:04:45.656 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:04:45 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/4206980712' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-12-11T18:04:45.656 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:04:45 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/4206980712' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-12-11T18:04:45.656 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:04:45 smithi059 ceph-mon[98847]: osdmap e121: 12 total, 12 up, 12 in 2023-12-11T18:04:45.664 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:45.661+0000 7fba677fe700 1 -- 172.21.15.59:0/4036288879 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 28) v1 ==== 136137+0+0 (secure 0 0 0) 0x7fba6c011990 con 0x7fba78106510 2023-12-11T18:04:45.674 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:45.662+0000 7fba677fe700 1 --2- 172.21.15.59:0/4036288879 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7fba600a1740 0x7fba600a3c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T18:04:45.674 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:45.662+0000 7fba677fe700 1 -- 172.21.15.59:0/4036288879 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 9479+0+0 (secure 0 0 0) 0x7fba6c0cd300 con 0x7fba78106510 2023-12-11T18:04:45.674 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:45.665+0000 7fba7659c700 1 --2- 172.21.15.59:0/4036288879 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7fba600a1740 0x7fba600a3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T18:04:45.674 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:45.665+0000 7fba7659c700 1 --2- 172.21.15.59:0/4036288879 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7fba600a1740 0x7fba600a3c00 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fba7000e350 tx=0x7fba7000b040 comp rx=0 tx=0).ready entity=mgr.15414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T18:04:45.674 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:45.667+0000 7fba677fe700 1 -- 172.21.15.59:0/4036288879 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fba6c0963f0 con 0x7fba78106510 2023-12-11T18:04:45.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:04:45 smithi115 ceph-mon[102748]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-12-11T18:04:45.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:04:45 smithi115 ceph-mon[102748]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-12-11T18:04:45.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:04:45 smithi115 ceph-mon[102748]: Cluster is now healthy 2023-12-11T18:04:45.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:04:45 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/704644296' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-12-11T18:04:45.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:04:45 smithi115 ceph-mon[102748]: mds.? [v2:172.21.15.181:6806/361059264,v1:172.21.15.181:6807/361059264] up:boot 2023-12-11T18:04:45.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:04:45 smithi115 ceph-mon[102748]: mds.? [v2:172.21.15.181:6804/2447812160,v1:172.21.15.181:6805/2447812160] up:boot 2023-12-11T18:04:45.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:04:45 smithi115 ceph-mon[102748]: mds.? [v2:172.21.15.115:6832/2018449929,v1:172.21.15.115:6833/2018449929] up:boot 2023-12-11T18:04:45.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:04:45 smithi115 ceph-mon[102748]: mds.? [v2:172.21.15.115:6835/4283455912,v1:172.21.15.115:6837/4283455912] up:boot 2023-12-11T18:04:45.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:04:45 smithi115 ceph-mon[102748]: mds.? [v2:172.21.15.115:6834/3868201075,v1:172.21.15.115:6836/3868201075] up:boot 2023-12-11T18:04:45.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:04:45 smithi115 ceph-mon[102748]: mds.? [v2:172.21.15.115:6838/3860170944,v1:172.21.15.115:6839/3860170944] up:boot 2023-12-11T18:04:45.784 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:04:45 smithi115 ceph-mon[102748]: fsmap 12 up:standby 2023-12-11T18:04:45.784 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:04:45 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-12-11T18:04:45.784 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:04:45 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-12-11T18:04:45.784 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:04:45 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-12-11T18:04:45.784 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:04:45 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-12-11T18:04:45.784 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:04:45 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-12-11T18:04:45.784 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:04:45 smithi115 ceph-mon[102748]: from='mgr.15414 172.21.15.59:0/1571949760' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-12-11T18:04:45.784 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:04:45 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/4206980712' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-12-11T18:04:45.784 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:04:45 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/4206980712' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-12-11T18:04:45.784 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:04:45 smithi115 ceph-mon[102748]: osdmap e121: 12 total, 12 up, 12 in 2023-12-11T18:04:46.023 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:46.019+0000 7fba7ce5a700 1 -- 172.21.15.59:0/4036288879 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fba580055a0 con 0x7fba78106510 2023-12-11T18:04:46.450 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:46.448+0000 7fba677fe700 1 -- 172.21.15.59:0/4036288879 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v122) v1 ==== 169+0+0 (secure 0 0 0) 0x7fba6c015070 con 0x7fba78106510 2023-12-11T18:04:46.457 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:46.455+0000 7fba7ce5a700 1 -- 172.21.15.59:0/4036288879 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7fba600a1740 msgr2=0x7fba600a3c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T18:04:46.458 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:46.455+0000 7fba7ce5a700 1 --2- 172.21.15.59:0/4036288879 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7fba600a1740 0x7fba600a3c00 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fba7000e350 tx=0x7fba7000b040 comp rx=0 tx=0).stop 2023-12-11T18:04:46.458 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:46.455+0000 7fba7ce5a700 1 -- 172.21.15.59:0/4036288879 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba78106510 msgr2=0x7fba7806b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T18:04:46.458 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:46.455+0000 7fba7ce5a700 1 --2- 172.21.15.59:0/4036288879 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba78106510 0x7fba7806b620 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fba6c00b4d0 tx=0x7fba6c00b890 comp rx=0 tx=0).stop 2023-12-11T18:04:46.458 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:46.455+0000 7fba7ce5a700 1 -- 172.21.15.59:0/4036288879 shutdown_connections 2023-12-11T18:04:46.458 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:46.455+0000 7fba7ce5a700 1 --2- 172.21.15.59:0/4036288879 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7fba600a1740 0x7fba600a3c00 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:04:46.458 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:46.455+0000 7fba7ce5a700 1 --2- 172.21.15.59:0/4036288879 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba78106510 0x7fba7806b620 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:04:46.458 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:46.455+0000 7fba7ce5a700 1 --2- 172.21.15.59:0/4036288879 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba780ffa40 0x7fba780730d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:04:46.458 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:46.455+0000 7fba7ce5a700 1 --2- 172.21.15.59:0/4036288879 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fba78100400 0x7fba7806b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:04:46.458 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:46.455+0000 7fba7ce5a700 1 -- 172.21.15.59:0/4036288879 >> 172.21.15.59:0/4036288879 conn(0x7fba78076a00 msgr2=0x7fba78101ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T18:04:46.462 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:46.459+0000 7fba7ce5a700 1 -- 172.21.15.59:0/4036288879 shutdown_connections 2023-12-11T18:04:46.466 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:46.464+0000 7fba7ce5a700 1 -- 172.21.15.59:0/4036288879 wait complete. 2023-12-11T18:04:46.474 INFO:teuthology.orchestra.run.smithi059.stderr:pool 'cephfs_data' removed 2023-12-11T18:04:46.491 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2023-12-11T18:04:46.749 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:46.747+0000 7f7dc4aaa700 1 -- 172.21.15.59:0/2725115980 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7dc010fd60 msgr2=0x7f7dc01085d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T18:04:46.749 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:46.747+0000 7f7dc4aaa700 1 --2- 172.21.15.59:0/2725115980 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7dc010fd60 0x7f7dc01085d0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f7db801c100 tx=0x7f7db801c410 comp rx=0 tx=0).stop 2023-12-11T18:04:46.749 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:46.747+0000 7f7dc4aaa700 1 -- 172.21.15.59:0/2725115980 shutdown_connections 2023-12-11T18:04:46.749 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:46.747+0000 7f7dc4aaa700 1 --2- 172.21.15.59:0/2725115980 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7dc011adb0 0x7f7dc011d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:04:46.749 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:46.747+0000 7f7dc4aaa700 1 --2- 172.21.15.59:0/2725115980 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7dc010fd60 0x7f7dc01085d0 secure :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f7db801c100 tx=0x7f7db801c410 comp rx=0 tx=0).stop 2023-12-11T18:04:46.749 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:46.747+0000 7f7dc4aaa700 1 --2- 172.21.15.59:0/2725115980 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7dc0108b10 0x7f7dc0108f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:04:46.749 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:46.747+0000 7f7dc4aaa700 1 -- 172.21.15.59:0/2725115980 >> 172.21.15.59:0/2725115980 conn(0x7f7dc006c610 msgr2=0x7f7dc006ca20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T18:04:46.755 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:46.753+0000 7f7dc4aaa700 1 -- 172.21.15.59:0/2725115980 shutdown_connections 2023-12-11T18:04:46.756 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:46.754+0000 7f7dc4aaa700 1 -- 172.21.15.59:0/2725115980 wait complete. 2023-12-11T18:04:46.757 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:46.755+0000 7f7dc4aaa700 1 Processor -- start 2023-12-11T18:04:46.758 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:46.756+0000 7f7dc4aaa700 1 -- start start 2023-12-11T18:04:46.758 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:46.756+0000 7f7dc4aaa700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7dc0108b10 0x7f7dc011a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T18:04:46.758 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:46.756+0000 7f7dc4aaa700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7dc011adb0 0x7f7dc01164e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T18:04:46.758 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:46.756+0000 7f7dc4aaa700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7dc0116a20 0x7f7dc0116ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T18:04:46.758 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:46.756+0000 7f7dc4aaa700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dc011abc0 con 0x7f7dc0108b10 2023-12-11T18:04:46.758 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:46.756+0000 7f7dc4aaa700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dc01aaa50 con 0x7f7dc011adb0 2023-12-11T18:04:46.758 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:46.756+0000 7f7dc4aaa700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dc01aab90 con 0x7f7dc0116a20 2023-12-11T18:04:46.759 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:46.756+0000 7f7dbed9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7dc0116a20 0x7f7dc0116ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T18:04:46.759 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:46.756+0000 7f7dbed9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7dc0116a20 0x7f7dc0116ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.59:40388/0 (socket says 172.21.15.59:40388) 2023-12-11T18:04:46.759 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:46.756+0000 7f7dbed9d700 1 -- 172.21.15.59:0/1760759977 learned_addr learned my addr 172.21.15.59:0/1760759977 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-12-11T18:04:46.759 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:46.757+0000 7f7dbed9d700 1 -- 172.21.15.59:0/1760759977 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7dc011adb0 msgr2=0x7f7dc01164e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T18:04:46.759 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:46.757+0000 7f7dbed9d700 1 --2- 172.21.15.59:0/1760759977 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7dc011adb0 0x7f7dc01164e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:04:46.759 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:46.757+0000 7f7dbed9d700 1 -- 172.21.15.59:0/1760759977 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7dc0108b10 msgr2=0x7f7dc011a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T18:04:46.759 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:46.757+0000 7f7dbed9d700 1 --2- 172.21.15.59:0/1760759977 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7dc0108b10 0x7f7dc011a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:04:46.760 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:46.757+0000 7f7dbed9d700 1 -- 172.21.15.59:0/1760759977 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7db801ad10 con 0x7f7dc0116a20 2023-12-11T18:04:46.760 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:46.757+0000 7f7dbed9d700 1 --2- 172.21.15.59:0/1760759977 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7dc0116a20 0x7f7dc0116ed0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f7db000b550 tx=0x7f7db000b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T18:04:46.760 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:46.758+0000 7f7daf7fe700 1 -- 172.21.15.59:0/1760759977 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7db0011830 con 0x7f7dc0116a20 2023-12-11T18:04:46.760 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:46.758+0000 7f7daf7fe700 1 -- 172.21.15.59:0/1760759977 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7db0011e50 con 0x7f7dc0116a20 2023-12-11T18:04:46.760 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:46.758+0000 7f7daf7fe700 1 -- 172.21.15.59:0/1760759977 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7db0010e70 con 0x7f7dc0116a20 2023-12-11T18:04:46.761 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:46.759+0000 7f7dc4aaa700 1 -- 172.21.15.59:0/1760759977 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7dc01aae70 con 0x7f7dc0116a20 2023-12-11T18:04:46.761 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:46.759+0000 7f7dc4aaa700 1 -- 172.21.15.59:0/1760759977 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7dc01ab300 con 0x7f7dc0116a20 2023-12-11T18:04:46.763 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:46.761+0000 7f7dc4aaa700 1 -- 172.21.15.59:0/1760759977 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7dc006da90 con 0x7f7dc0116a20 2023-12-11T18:04:46.769 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:46.765+0000 7f7daf7fe700 1 -- 172.21.15.59:0/1760759977 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 28) v1 ==== 136137+0+0 (secure 0 0 0) 0x7f7db0010510 con 0x7f7dc0116a20 2023-12-11T18:04:46.769 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:46.766+0000 7f7daf7fe700 1 --2- 172.21.15.59:0/1760759977 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7f7da80a1730 0x7f7da80a3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-11T18:04:46.769 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:46.766+0000 7f7daf7fe700 1 -- 172.21.15.59:0/1760759977 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 8855+0+0 (secure 0 0 0) 0x7f7db00cceb0 con 0x7f7dc0116a20 2023-12-11T18:04:46.769 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:46.767+0000 7f7dbe59c700 1 --2- 172.21.15.59:0/1760759977 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7f7da80a1730 0x7f7da80a3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-11T18:04:46.769 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:46.767+0000 7f7dbe59c700 1 --2- 172.21.15.59:0/1760759977 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7f7da80a1730 0x7f7da80a3bf0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f7db801c0d0 tx=0x7f7db801c040 comp rx=0 tx=0).ready entity=mgr.15414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-11T18:04:46.775 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:46.770+0000 7f7daf7fe700 1 -- 172.21.15.59:0/1760759977 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f7db0096130 con 0x7f7dc0116a20 2023-12-11T18:04:46.782 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:04:46 smithi115 ceph-mon[102748]: pgmap v588: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.7 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 818 B/s wr, 6 op/s 2023-12-11T18:04:46.782 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:04:46 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/4036288879' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-12-11T18:04:46.782 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:04:46 smithi115 ceph-mon[102748]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-12-11T18:04:46.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:04:46 smithi059 ceph-mon[98847]: pgmap v588: 129 pgs: 56 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 54 active+clean; 1.7 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 818 B/s wr, 6 op/s 2023-12-11T18:04:46.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:04:46 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/4036288879' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-12-11T18:04:46.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:04:46 smithi059 ceph-mon[98847]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-12-11T18:04:47.158 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:47.155+0000 7f7dc4aaa700 1 -- 172.21.15.59:0/1760759977 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f7dc0068310 con 0x7f7dc0116a20 2023-12-11T18:04:47.509 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:47.506+0000 7f7daf7fe700 1 -- 172.21.15.59:0/1760759977 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v123) v1 ==== 178+0+0 (secure 0 0 0) 0x7f7db0015020 con 0x7f7dc0116a20 2023-12-11T18:04:47.520 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:47.517+0000 7f7dc4aaa700 1 -- 172.21.15.59:0/1760759977 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7f7da80a1730 msgr2=0x7f7da80a3bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T18:04:47.520 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:47.517+0000 7f7dc4aaa700 1 --2- 172.21.15.59:0/1760759977 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7f7da80a1730 0x7f7da80a3bf0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f7db801c0d0 tx=0x7f7db801c040 comp rx=0 tx=0).stop 2023-12-11T18:04:47.520 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:47.517+0000 7f7dc4aaa700 1 -- 172.21.15.59:0/1760759977 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7dc0116a20 msgr2=0x7f7dc0116ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-11T18:04:47.520 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:47.517+0000 7f7dc4aaa700 1 --2- 172.21.15.59:0/1760759977 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7dc0116a20 0x7f7dc0116ed0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f7db000b550 tx=0x7f7db000b910 comp rx=0 tx=0).stop 2023-12-11T18:04:47.532 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:47.524+0000 7f7dc4aaa700 1 -- 172.21.15.59:0/1760759977 shutdown_connections 2023-12-11T18:04:47.532 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:47.524+0000 7f7dc4aaa700 1 --2- 172.21.15.59:0/1760759977 >> [v2:172.21.15.59:6800/4126604994,v1:172.21.15.59:6801/4126604994] conn(0x7f7da80a1730 0x7f7da80a3bf0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:04:47.532 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:47.524+0000 7f7dc4aaa700 1 --2- 172.21.15.59:0/1760759977 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7dc0116a20 0x7f7dc0116ed0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:04:47.532 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:47.524+0000 7f7dc4aaa700 1 --2- 172.21.15.59:0/1760759977 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7dc0108b10 0x7f7dc011a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:04:47.532 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:47.524+0000 7f7dc4aaa700 1 --2- 172.21.15.59:0/1760759977 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7dc011adb0 0x7f7dc01164e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-11T18:04:47.532 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:47.524+0000 7f7dc4aaa700 1 -- 172.21.15.59:0/1760759977 >> 172.21.15.59:0/1760759977 conn(0x7f7dc006c610 msgr2=0x7f7dc010c8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-11T18:04:47.532 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:47.528+0000 7f7dc4aaa700 1 -- 172.21.15.59:0/1760759977 shutdown_connections 2023-12-11T18:04:47.532 INFO:teuthology.orchestra.run.smithi059.stderr:2023-12-11T18:04:47.529+0000 7f7dc4aaa700 1 -- 172.21.15.59:0/1760759977 wait complete. 2023-12-11T18:04:47.544 INFO:teuthology.orchestra.run.smithi059.stderr:pool 'cephfs_data_ec' removed 2023-12-11T18:04:47.562 DEBUG:teuthology.orchestra.run.smithi059:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-12-11T18:04:47.620 DEBUG:teuthology.orchestra.run.smithi115:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-12-11T18:04:47.676 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi181.front.sepia.ceph.com' 2023-12-11T18:04:47.677 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-11T18:04:47.782 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:04:47 smithi115 ceph-mon[102748]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2023-12-11T18:04:47.782 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:04:47 smithi115 ceph-mon[102748]: osdmap e122: 12 total, 12 up, 12 in 2023-12-11T18:04:47.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:04:47 smithi115 ceph-mon[102748]: from='client.? 172.21.15.59:0/1760759977' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2023-12-11T18:04:47.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:04:47 smithi115 ceph-mon[102748]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2023-12-11T18:04:47.790 DEBUG:teuthology.orchestra.run.smithi181:> true 2023-12-11T18:04:47.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:04:47 smithi059 ceph-mon[98847]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2023-12-11T18:04:47.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:04:47 smithi059 ceph-mon[98847]: osdmap e122: 12 total, 12 up, 12 in 2023-12-11T18:04:47.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:04:47 smithi059 ceph-mon[98847]: from='client.? 172.21.15.59:0/1760759977' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2023-12-11T18:04:47.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:04:47 smithi059 ceph-mon[98847]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2023-12-11T18:04:47.923 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi181.front.sepia.ceph.com' 2023-12-11T18:04:47.923 DEBUG:teuthology.orchestra.run.smithi181:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-12-11T18:04:48.060 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2023-12-11T18:04:48.061 DEBUG:teuthology.orchestra.run.smithi059:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-12-11T18:04:48.079 DEBUG:teuthology.orchestra.run.smithi115:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-12-11T18:04:48.139 DEBUG:teuthology.orchestra.run.smithi181:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-12-11T18:04:48.155 INFO:tasks.cephadm:Stopping all daemons... 2023-12-11T18:04:48.156 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2023-12-11T18:04:48.156 DEBUG:teuthology.orchestra.run.smithi059:> sudo systemctl stop ceph-d0a07eaa-984b-11ee-95a3-87774f69a715@mon.a 2023-12-11T18:04:48.500 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:04:48 smithi059 systemd[1]: Stopping Ceph mon.a for d0a07eaa-984b-11ee-95a3-87774f69a715... 2023-12-11T18:04:48.776 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:04:48 smithi059 ceph-mon[98847]: pgmap v591: 33 pgs: 13 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 1 active+clean; 709 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T18:04:48.776 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:04:48 smithi059 ceph-mon[98847]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2023-12-11T18:04:48.776 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:04:48 smithi059 ceph-mon[98847]: osdmap e123: 12 total, 12 up, 12 in 2023-12-11T18:04:48.776 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:04:48 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mon-a[98823]: 2023-12-11T18:04:48.510+0000 7faaa6500700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-12-11T18:04:48.776 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:04:48 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mon-a[98823]: 2023-12-11T18:04:48.510+0000 7faaa6500700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2023-12-11T18:04:48.782 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:04:48 smithi115 ceph-mon[102748]: pgmap v591: 33 pgs: 13 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 1 active+clean; 709 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail 2023-12-11T18:04:48.782 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:04:48 smithi115 ceph-mon[102748]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2023-12-11T18:04:48.782 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:04:48 smithi115 ceph-mon[102748]: osdmap e123: 12 total, 12 up, 12 in 2023-12-11T18:04:49.065 INFO:journalctl@ceph.mon.a.smithi059.stdout:Dec 11 18:04:48 smithi059 bash[176851]: ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mon-a 2023-12-11T18:04:49.259 DEBUG:teuthology.orchestra.run.smithi059:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d0a07eaa-984b-11ee-95a3-87774f69a715@mon.a.service' 2023-12-11T18:04:49.306 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-11T18:04:49.307 INFO:tasks.cephadm.mon.a:Stopped mon.a 2023-12-11T18:04:49.307 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2023-12-11T18:04:49.307 DEBUG:teuthology.orchestra.run.smithi115:> sudo systemctl stop ceph-d0a07eaa-984b-11ee-95a3-87774f69a715@mon.b 2023-12-11T18:04:49.783 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:04:49 smithi115 systemd[1]: Stopping Ceph mon.b for d0a07eaa-984b-11ee-95a3-87774f69a715... 2023-12-11T18:04:50.235 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:04:49 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mon-b[102724]: 2023-12-11T18:04:49.811+0000 7f1f248fb700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-12-11T18:04:50.236 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:04:49 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mon-b[102724]: 2023-12-11T18:04:49.811+0000 7f1f248fb700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2023-12-11T18:04:50.513 INFO:journalctl@ceph.mon.b.smithi115.stdout:Dec 11 18:04:50 smithi115 bash[146742]: ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mon-b 2023-12-11T18:04:50.708 DEBUG:teuthology.orchestra.run.smithi115:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d0a07eaa-984b-11ee-95a3-87774f69a715@mon.b.service' 2023-12-11T18:04:50.761 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-11T18:04:50.761 INFO:tasks.cephadm.mon.c:Stopped mon.b 2023-12-11T18:04:50.761 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2023-12-11T18:04:50.761 DEBUG:teuthology.orchestra.run.smithi181:> sudo systemctl stop ceph-d0a07eaa-984b-11ee-95a3-87774f69a715@mon.c 2023-12-11T18:04:51.803 DEBUG:teuthology.orchestra.run.smithi181:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d0a07eaa-984b-11ee-95a3-87774f69a715@mon.c.service' 2023-12-11T18:04:51.850 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-11T18:04:51.850 INFO:tasks.cephadm.mon.c:Stopped mon.c 2023-12-11T18:04:51.850 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2023-12-11T18:04:51.850 DEBUG:teuthology.orchestra.run.smithi059:> sudo systemctl stop ceph-d0a07eaa-984b-11ee-95a3-87774f69a715@mgr.x 2023-12-11T18:04:52.154 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 18:04:51 smithi059 systemd[1]: Stopping Ceph mgr.x for d0a07eaa-984b-11ee-95a3-87774f69a715... 2023-12-11T18:04:52.525 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 18:04:52 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T18:04:52.224+0000 7f46a761b700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-11T18:04:52.525 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 18:04:52 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x[99092]: 2023-12-11T18:04:52.224+0000 7f46a761b700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2023-12-11T18:04:52.766 DEBUG:teuthology.orchestra.run.smithi059:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d0a07eaa-984b-11ee-95a3-87774f69a715@mgr.x.service' 2023-12-11T18:04:52.791 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 18:04:52 smithi059 bash[176989]: ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-x 2023-12-11T18:04:52.791 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 18:04:52 smithi059 systemd[1]: ceph-d0a07eaa-984b-11ee-95a3-87774f69a715@mgr.x.service: Succeeded. 2023-12-11T18:04:52.791 INFO:journalctl@ceph.mgr.x.smithi059.stdout:Dec 11 18:04:52 smithi059 systemd[1]: Stopped Ceph mgr.x for d0a07eaa-984b-11ee-95a3-87774f69a715. 2023-12-11T18:04:52.804 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-11T18:04:52.804 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2023-12-11T18:04:52.805 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2023-12-11T18:04:52.805 DEBUG:teuthology.orchestra.run.smithi115:> sudo systemctl stop ceph-d0a07eaa-984b-11ee-95a3-87774f69a715@mgr.y 2023-12-11T18:04:53.261 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Dec 11 18:04:52 smithi115 systemd[1]: Stopping Ceph mgr.y for d0a07eaa-984b-11ee-95a3-87774f69a715... 2023-12-11T18:04:53.515 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Dec 11 18:04:53 smithi115 bash[146902]: ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mgr-y 2023-12-11T18:04:53.711 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Dec 11 18:04:53 smithi115 systemd[1]: ceph-d0a07eaa-984b-11ee-95a3-87774f69a715@mgr.y.service: Main process exited, code=exited, status=143/n/a 2023-12-11T18:04:53.714 DEBUG:teuthology.orchestra.run.smithi115:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d0a07eaa-984b-11ee-95a3-87774f69a715@mgr.y.service' 2023-12-11T18:04:53.751 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-11T18:04:53.751 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2023-12-11T18:04:53.752 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2023-12-11T18:04:53.752 DEBUG:teuthology.orchestra.run.smithi181:> sudo systemctl stop ceph-d0a07eaa-984b-11ee-95a3-87774f69a715@mgr.z 2023-12-11T18:04:54.647 DEBUG:teuthology.orchestra.run.smithi181:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d0a07eaa-984b-11ee-95a3-87774f69a715@mgr.z.service' 2023-12-11T18:04:54.683 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-11T18:04:54.683 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2023-12-11T18:04:54.683 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2023-12-11T18:04:54.683 DEBUG:teuthology.orchestra.run.smithi059:> sudo systemctl stop ceph-d0a07eaa-984b-11ee-95a3-87774f69a715@osd.0 2023-12-11T18:04:55.043 INFO:journalctl@ceph.osd.0.smithi059.stdout:Dec 11 18:04:54 smithi059 systemd[1]: Stopping Ceph osd.0 for d0a07eaa-984b-11ee-95a3-87774f69a715... 2023-12-11T18:04:55.404 INFO:journalctl@ceph.osd.0.smithi059.stdout:Dec 11 18:04:55 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-0[117294]: 2023-12-11T18:04:55.040+0000 7efcfb3e2700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-11T18:04:55.404 INFO:journalctl@ceph.osd.0.smithi059.stdout:Dec 11 18:04:55 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-0[117294]: 2023-12-11T18:04:55.040+0000 7efcfb3e2700 -1 osd.0 123 *** Got signal Terminated *** 2023-12-11T18:04:55.404 INFO:journalctl@ceph.osd.0.smithi059.stdout:Dec 11 18:04:55 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-0[117294]: 2023-12-11T18:04:55.040+0000 7efcfb3e2700 -1 osd.0 123 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-12-11T18:05:01.154 INFO:journalctl@ceph.osd.0.smithi059.stdout:Dec 11 18:05:00 smithi059 bash[177134]: ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-0 2023-12-11T18:05:02.482 DEBUG:teuthology.orchestra.run.smithi059:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d0a07eaa-984b-11ee-95a3-87774f69a715@osd.0.service' 2023-12-11T18:05:02.524 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-11T18:05:02.524 INFO:tasks.cephadm.osd.0:Stopped osd.0 2023-12-11T18:05:02.525 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2023-12-11T18:05:02.525 DEBUG:teuthology.orchestra.run.smithi115:> sudo systemctl stop ceph-d0a07eaa-984b-11ee-95a3-87774f69a715@osd.1 2023-12-11T18:05:02.906 INFO:journalctl@ceph.osd.1.smithi115.stdout:Dec 11 18:05:02 smithi115 systemd[1]: Stopping Ceph osd.1 for d0a07eaa-984b-11ee-95a3-87774f69a715... 2023-12-11T18:05:03.282 INFO:journalctl@ceph.osd.1.smithi115.stdout:Dec 11 18:05:02 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-1[107812]: 2023-12-11T18:05:02.903+0000 7f3bf9b9a700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-11T18:05:03.283 INFO:journalctl@ceph.osd.1.smithi115.stdout:Dec 11 18:05:02 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-1[107812]: 2023-12-11T18:05:02.903+0000 7f3bf9b9a700 -1 osd.1 123 *** Got signal Terminated *** 2023-12-11T18:05:03.283 INFO:journalctl@ceph.osd.1.smithi115.stdout:Dec 11 18:05:02 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-1[107812]: 2023-12-11T18:05:02.903+0000 7f3bf9b9a700 -1 osd.1 123 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-12-11T18:05:09.782 INFO:journalctl@ceph.osd.1.smithi115.stdout:Dec 11 18:05:09 smithi115 bash[147049]: ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-1 2023-12-11T18:05:12.958 DEBUG:teuthology.orchestra.run.smithi115:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d0a07eaa-984b-11ee-95a3-87774f69a715@osd.1.service' 2023-12-11T18:05:12.998 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-11T18:05:12.999 INFO:tasks.cephadm.osd.1:Stopped osd.1 2023-12-11T18:05:12.999 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2023-12-11T18:05:12.999 DEBUG:teuthology.orchestra.run.smithi181:> sudo systemctl stop ceph-d0a07eaa-984b-11ee-95a3-87774f69a715@osd.2 2023-12-11T18:05:20.491 DEBUG:teuthology.orchestra.run.smithi181:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d0a07eaa-984b-11ee-95a3-87774f69a715@osd.2.service' 2023-12-11T18:05:20.530 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-11T18:05:20.530 INFO:tasks.cephadm.osd.2:Stopped osd.2 2023-12-11T18:05:20.531 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2023-12-11T18:05:20.531 DEBUG:teuthology.orchestra.run.smithi059:> sudo systemctl stop ceph-d0a07eaa-984b-11ee-95a3-87774f69a715@osd.3 2023-12-11T18:05:20.904 INFO:journalctl@ceph.osd.3.smithi059.stdout:Dec 11 18:05:20 smithi059 systemd[1]: Stopping Ceph osd.3 for d0a07eaa-984b-11ee-95a3-87774f69a715... 2023-12-11T18:05:21.404 INFO:journalctl@ceph.osd.3.smithi059.stdout:Dec 11 18:05:20 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-3[126028]: 2023-12-11T18:05:20.920+0000 7fe390fd1700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-11T18:05:21.404 INFO:journalctl@ceph.osd.3.smithi059.stdout:Dec 11 18:05:20 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-3[126028]: 2023-12-11T18:05:20.920+0000 7fe390fd1700 -1 osd.3 123 *** Got signal Terminated *** 2023-12-11T18:05:21.404 INFO:journalctl@ceph.osd.3.smithi059.stdout:Dec 11 18:05:20 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-3[126028]: 2023-12-11T18:05:20.920+0000 7fe390fd1700 -1 osd.3 123 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-12-11T18:05:22.404 INFO:journalctl@ceph.osd.6.smithi059.stdout:Dec 11 18:05:21 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-6[133999]: 2023-12-11T18:05:21.959+0000 7f6dafb58700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:58.323226+0000 front 2023-12-11T18:04:58.323260+0000 (oldest deadline 2023-12-11T18:05:21.223373+0000) 2023-12-11T18:05:22.532 INFO:journalctl@ceph.osd.4.smithi115.stdout:Dec 11 18:05:22 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-4[116520]: 2023-12-11T18:05:22.193+0000 7ff7103a4700 -1 osd.4 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:58.556065+0000 front 2023-12-11T18:04:58.556079+0000 (oldest deadline 2023-12-11T18:05:22.055902+0000) 2023-12-11T18:05:22.974 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:05:22 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:05:22.711+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:58.934598+0000 front 2023-12-11T18:04:58.934735+0000 (oldest deadline 2023-12-11T18:05:21.834661+0000) 2023-12-11T18:05:23.404 INFO:journalctl@ceph.osd.6.smithi059.stdout:Dec 11 18:05:22 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-6[133999]: 2023-12-11T18:05:22.971+0000 7f6dafb58700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:58.323226+0000 front 2023-12-11T18:04:58.323260+0000 (oldest deadline 2023-12-11T18:05:21.223373+0000) 2023-12-11T18:05:23.502 INFO:journalctl@ceph.osd.4.smithi115.stdout:Dec 11 18:05:23 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-4[116520]: 2023-12-11T18:05:23.157+0000 7ff7103a4700 -1 osd.4 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:58.556065+0000 front 2023-12-11T18:04:58.556079+0000 (oldest deadline 2023-12-11T18:05:22.055902+0000) 2023-12-11T18:05:23.782 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:05:23 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:05:23.500+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:59.875115+0000 front 2023-12-11T18:04:59.874965+0000 (oldest deadline 2023-12-11T18:05:22.775102+0000) 2023-12-11T18:05:24.154 INFO:journalctl@ceph.osd.6.smithi059.stdout:Dec 11 18:05:23 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-6[133999]: 2023-12-11T18:05:23.952+0000 7f6dafb58700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:58.323226+0000 front 2023-12-11T18:04:58.323260+0000 (oldest deadline 2023-12-11T18:05:21.223373+0000) 2023-12-11T18:05:24.154 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:05:23 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:05:23.728+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:58.934598+0000 front 2023-12-11T18:04:58.934735+0000 (oldest deadline 2023-12-11T18:05:21.834661+0000) 2023-12-11T18:05:24.459 INFO:journalctl@ceph.osd.4.smithi115.stdout:Dec 11 18:05:24 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-4[116520]: 2023-12-11T18:05:24.136+0000 7ff7103a4700 -1 osd.4 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:58.556065+0000 front 2023-12-11T18:04:58.556079+0000 (oldest deadline 2023-12-11T18:05:22.055902+0000) 2023-12-11T18:05:24.782 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:05:24 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:05:24.457+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:59.875115+0000 front 2023-12-11T18:04:59.874965+0000 (oldest deadline 2023-12-11T18:05:22.775102+0000) 2023-12-11T18:05:24.994 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:05:24 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:05:24.727+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:58.934598+0000 front 2023-12-11T18:04:58.934735+0000 (oldest deadline 2023-12-11T18:05:21.834661+0000) 2023-12-11T18:05:25.404 INFO:journalctl@ceph.osd.6.smithi059.stdout:Dec 11 18:05:24 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-6[133999]: 2023-12-11T18:05:24.991+0000 7f6dafb58700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:58.323226+0000 front 2023-12-11T18:04:58.323260+0000 (oldest deadline 2023-12-11T18:05:21.223373+0000) 2023-12-11T18:05:25.451 INFO:journalctl@ceph.osd.4.smithi115.stdout:Dec 11 18:05:25 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-4[116520]: 2023-12-11T18:05:25.115+0000 7ff7103a4700 -1 osd.4 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:58.556065+0000 front 2023-12-11T18:04:58.556079+0000 (oldest deadline 2023-12-11T18:05:22.055902+0000) 2023-12-11T18:05:25.782 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:05:25 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:05:25.449+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:59.875115+0000 front 2023-12-11T18:04:59.874965+0000 (oldest deadline 2023-12-11T18:05:22.775102+0000) 2023-12-11T18:05:25.980 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:05:25 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:05:25.683+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:58.934598+0000 front 2023-12-11T18:04:58.934735+0000 (oldest deadline 2023-12-11T18:05:21.834661+0000) 2023-12-11T18:05:26.404 INFO:journalctl@ceph.osd.6.smithi059.stdout:Dec 11 18:05:25 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-6[133999]: 2023-12-11T18:05:25.977+0000 7f6dafb58700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:58.323226+0000 front 2023-12-11T18:04:58.323260+0000 (oldest deadline 2023-12-11T18:05:21.223373+0000) 2023-12-11T18:05:26.495 INFO:journalctl@ceph.osd.4.smithi115.stdout:Dec 11 18:05:26 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-4[116520]: 2023-12-11T18:05:26.093+0000 7ff7103a4700 -1 osd.4 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:58.556065+0000 front 2023-12-11T18:04:58.556079+0000 (oldest deadline 2023-12-11T18:05:22.055902+0000) 2023-12-11T18:05:26.685 INFO:journalctl@ceph.osd.3.smithi059.stdout:Dec 11 18:05:26 smithi059 bash[177409]: ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-3 2023-12-11T18:05:26.782 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:05:26 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:05:26.493+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:59.875115+0000 front 2023-12-11T18:04:59.874965+0000 (oldest deadline 2023-12-11T18:05:22.775102+0000) 2023-12-11T18:05:26.978 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:05:26 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:05:26.700+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:58.934598+0000 front 2023-12-11T18:04:58.934735+0000 (oldest deadline 2023-12-11T18:05:21.834661+0000) 2023-12-11T18:05:27.242 INFO:journalctl@ceph.osd.6.smithi059.stdout:Dec 11 18:05:26 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-6[133999]: 2023-12-11T18:05:26.976+0000 7f6dafb58700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:58.323226+0000 front 2023-12-11T18:04:58.323260+0000 (oldest deadline 2023-12-11T18:05:21.223373+0000) 2023-12-11T18:05:27.459 INFO:journalctl@ceph.osd.4.smithi115.stdout:Dec 11 18:05:27 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-4[116520]: 2023-12-11T18:05:27.089+0000 7ff7103a4700 -1 osd.4 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:58.556065+0000 front 2023-12-11T18:04:58.556079+0000 (oldest deadline 2023-12-11T18:05:22.055902+0000) 2023-12-11T18:05:27.782 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:05:27 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:05:27.457+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:59.875115+0000 front 2023-12-11T18:04:59.874965+0000 (oldest deadline 2023-12-11T18:05:22.775102+0000) 2023-12-11T18:05:27.904 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:05:27 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:05:27.711+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:58.934598+0000 front 2023-12-11T18:04:58.934735+0000 (oldest deadline 2023-12-11T18:05:21.834661+0000) 2023-12-11T18:05:28.070 DEBUG:teuthology.orchestra.run.smithi059:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d0a07eaa-984b-11ee-95a3-87774f69a715@osd.3.service' 2023-12-11T18:05:28.112 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-11T18:05:28.112 INFO:tasks.cephadm.osd.3:Stopped osd.3 2023-12-11T18:05:28.112 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2023-12-11T18:05:28.113 DEBUG:teuthology.orchestra.run.smithi115:> sudo systemctl stop ceph-d0a07eaa-984b-11ee-95a3-87774f69a715@osd.4 2023-12-11T18:05:28.404 INFO:journalctl@ceph.osd.6.smithi059.stdout:Dec 11 18:05:27 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-6[133999]: 2023-12-11T18:05:27.929+0000 7f6dafb58700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:58.323226+0000 front 2023-12-11T18:04:58.323260+0000 (oldest deadline 2023-12-11T18:05:21.223373+0000) 2023-12-11T18:05:28.418 INFO:journalctl@ceph.osd.4.smithi115.stdout:Dec 11 18:05:28 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-4[116520]: 2023-12-11T18:05:28.066+0000 7ff7103a4700 -1 osd.4 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:58.556065+0000 front 2023-12-11T18:04:58.556079+0000 (oldest deadline 2023-12-11T18:05:22.055902+0000) 2023-12-11T18:05:28.418 INFO:journalctl@ceph.osd.4.smithi115.stdout:Dec 11 18:05:28 smithi115 systemd[1]: Stopping Ceph osd.4 for d0a07eaa-984b-11ee-95a3-87774f69a715... 2023-12-11T18:05:28.783 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:05:28 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:05:28.416+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:59.875115+0000 front 2023-12-11T18:04:59.874965+0000 (oldest deadline 2023-12-11T18:05:22.775102+0000) 2023-12-11T18:05:28.783 INFO:journalctl@ceph.osd.4.smithi115.stdout:Dec 11 18:05:28 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-4[116520]: 2023-12-11T18:05:28.473+0000 7ff715dad700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-11T18:05:28.783 INFO:journalctl@ceph.osd.4.smithi115.stdout:Dec 11 18:05:28 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-4[116520]: 2023-12-11T18:05:28.473+0000 7ff715dad700 -1 osd.4 123 *** Got signal Terminated *** 2023-12-11T18:05:28.783 INFO:journalctl@ceph.osd.4.smithi115.stdout:Dec 11 18:05:28 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-4[116520]: 2023-12-11T18:05:28.473+0000 7ff715dad700 -1 osd.4 123 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-12-11T18:05:29.154 INFO:journalctl@ceph.osd.6.smithi059.stdout:Dec 11 18:05:28 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-6[133999]: 2023-12-11T18:05:28.943+0000 7f6dafb58700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:58.323226+0000 front 2023-12-11T18:04:58.323260+0000 (oldest deadline 2023-12-11T18:05:21.223373+0000) 2023-12-11T18:05:29.154 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:05:28 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:05:28.695+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:58.934598+0000 front 2023-12-11T18:04:58.934735+0000 (oldest deadline 2023-12-11T18:05:21.834661+0000) 2023-12-11T18:05:29.433 INFO:journalctl@ceph.osd.4.smithi115.stdout:Dec 11 18:05:29 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-4[116520]: 2023-12-11T18:05:29.106+0000 7ff7103a4700 -1 osd.4 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:58.556065+0000 front 2023-12-11T18:04:58.556079+0000 (oldest deadline 2023-12-11T18:05:22.055902+0000) 2023-12-11T18:05:29.782 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:05:29 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:05:29.431+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:59.875115+0000 front 2023-12-11T18:04:59.874965+0000 (oldest deadline 2023-12-11T18:05:22.775102+0000) 2023-12-11T18:05:30.154 INFO:journalctl@ceph.osd.6.smithi059.stdout:Dec 11 18:05:29 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-6[133999]: 2023-12-11T18:05:29.945+0000 7f6dafb58700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:58.323226+0000 front 2023-12-11T18:04:58.323260+0000 (oldest deadline 2023-12-11T18:05:21.223373+0000) 2023-12-11T18:05:30.154 INFO:journalctl@ceph.osd.6.smithi059.stdout:Dec 11 18:05:29 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-6[133999]: 2023-12-11T18:05:29.945+0000 7f6dafb58700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:06.127953+0000 front 2023-12-11T18:05:06.127805+0000 (oldest deadline 2023-12-11T18:05:29.027582+0000) 2023-12-11T18:05:30.154 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:05:29 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:05:29.733+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:58.934598+0000 front 2023-12-11T18:04:58.934735+0000 (oldest deadline 2023-12-11T18:05:21.834661+0000) 2023-12-11T18:05:30.470 INFO:journalctl@ceph.osd.4.smithi115.stdout:Dec 11 18:05:30 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-4[116520]: 2023-12-11T18:05:30.138+0000 7ff7103a4700 -1 osd.4 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:58.556065+0000 front 2023-12-11T18:04:58.556079+0000 (oldest deadline 2023-12-11T18:05:22.055902+0000) 2023-12-11T18:05:30.783 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:05:30 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:05:30.469+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:59.875115+0000 front 2023-12-11T18:04:59.874965+0000 (oldest deadline 2023-12-11T18:05:22.775102+0000) 2023-12-11T18:05:31.154 INFO:journalctl@ceph.osd.6.smithi059.stdout:Dec 11 18:05:30 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-6[133999]: 2023-12-11T18:05:30.960+0000 7f6dafb58700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:58.323226+0000 front 2023-12-11T18:04:58.323260+0000 (oldest deadline 2023-12-11T18:05:21.223373+0000) 2023-12-11T18:05:31.154 INFO:journalctl@ceph.osd.6.smithi059.stdout:Dec 11 18:05:30 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-6[133999]: 2023-12-11T18:05:30.960+0000 7f6dafb58700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:06.127953+0000 front 2023-12-11T18:05:06.127805+0000 (oldest deadline 2023-12-11T18:05:29.027582+0000) 2023-12-11T18:05:31.155 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:05:30 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:05:30.768+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:58.934598+0000 front 2023-12-11T18:04:58.934735+0000 (oldest deadline 2023-12-11T18:05:21.834661+0000) 2023-12-11T18:05:31.422 INFO:journalctl@ceph.osd.4.smithi115.stdout:Dec 11 18:05:31 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-4[116520]: 2023-12-11T18:05:31.167+0000 7ff7103a4700 -1 osd.4 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:58.556065+0000 front 2023-12-11T18:04:58.556079+0000 (oldest deadline 2023-12-11T18:05:22.055902+0000) 2023-12-11T18:05:31.422 INFO:journalctl@ceph.osd.4.smithi115.stdout:Dec 11 18:05:31 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-4[116520]: 2023-12-11T18:05:31.167+0000 7ff7103a4700 -1 osd.4 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:05.556911+0000 front 2023-12-11T18:05:05.556884+0000 (oldest deadline 2023-12-11T18:05:30.257044+0000) 2023-12-11T18:05:31.782 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:05:31 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:05:31.420+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:59.875115+0000 front 2023-12-11T18:04:59.874965+0000 (oldest deadline 2023-12-11T18:05:22.775102+0000) 2023-12-11T18:05:32.154 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:05:31 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:05:31.752+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:58.934598+0000 front 2023-12-11T18:04:58.934735+0000 (oldest deadline 2023-12-11T18:05:21.834661+0000) 2023-12-11T18:05:32.154 INFO:journalctl@ceph.osd.6.smithi059.stdout:Dec 11 18:05:31 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-6[133999]: 2023-12-11T18:05:31.990+0000 7f6dafb58700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:58.323226+0000 front 2023-12-11T18:04:58.323260+0000 (oldest deadline 2023-12-11T18:05:21.223373+0000) 2023-12-11T18:05:32.154 INFO:journalctl@ceph.osd.6.smithi059.stdout:Dec 11 18:05:31 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-6[133999]: 2023-12-11T18:05:31.990+0000 7f6dafb58700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:06.127953+0000 front 2023-12-11T18:05:06.127805+0000 (oldest deadline 2023-12-11T18:05:29.027582+0000) 2023-12-11T18:05:32.214 INFO:journalctl@ceph.osd.7.smithi115.stdout:Dec 11 18:05:31 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-7[124665]: 2023-12-11T18:05:31.920+0000 7f9a71cae700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:05.284230+0000 front 2023-12-11T18:05:05.284410+0000 (oldest deadline 2023-12-11T18:05:31.184416+0000) 2023-12-11T18:05:32.533 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:05:32 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:05:32.410+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:59.875115+0000 front 2023-12-11T18:04:59.874965+0000 (oldest deadline 2023-12-11T18:05:22.775102+0000) 2023-12-11T18:05:32.533 INFO:journalctl@ceph.osd.4.smithi115.stdout:Dec 11 18:05:32 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-4[116520]: 2023-12-11T18:05:32.212+0000 7ff7103a4700 -1 osd.4 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:58.556065+0000 front 2023-12-11T18:04:58.556079+0000 (oldest deadline 2023-12-11T18:05:22.055902+0000) 2023-12-11T18:05:32.533 INFO:journalctl@ceph.osd.4.smithi115.stdout:Dec 11 18:05:32 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-4[116520]: 2023-12-11T18:05:32.212+0000 7ff7103a4700 -1 osd.4 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:05.556911+0000 front 2023-12-11T18:05:05.556884+0000 (oldest deadline 2023-12-11T18:05:30.257044+0000) 2023-12-11T18:05:33.005 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:05:32 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:05:32.725+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:58.934598+0000 front 2023-12-11T18:04:58.934735+0000 (oldest deadline 2023-12-11T18:05:21.834661+0000) 2023-12-11T18:05:33.005 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:05:32 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:05:32.725+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:06.536080+0000 front 2023-12-11T18:05:06.535905+0000 (oldest deadline 2023-12-11T18:05:31.835855+0000) 2023-12-11T18:05:33.223 INFO:journalctl@ceph.osd.7.smithi115.stdout:Dec 11 18:05:32 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-7[124665]: 2023-12-11T18:05:32.969+0000 7f9a71cae700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:05.284230+0000 front 2023-12-11T18:05:05.284410+0000 (oldest deadline 2023-12-11T18:05:31.184416+0000) 2023-12-11T18:05:33.404 INFO:journalctl@ceph.osd.6.smithi059.stdout:Dec 11 18:05:33 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-6[133999]: 2023-12-11T18:05:33.003+0000 7f6dafb58700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:58.323226+0000 front 2023-12-11T18:04:58.323260+0000 (oldest deadline 2023-12-11T18:05:21.223373+0000) 2023-12-11T18:05:33.404 INFO:journalctl@ceph.osd.6.smithi059.stdout:Dec 11 18:05:33 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-6[133999]: 2023-12-11T18:05:33.003+0000 7f6dafb58700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:06.127953+0000 front 2023-12-11T18:05:06.127805+0000 (oldest deadline 2023-12-11T18:05:29.027582+0000) 2023-12-11T18:05:33.532 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:05:33 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:05:33.406+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:59.875115+0000 front 2023-12-11T18:04:59.874965+0000 (oldest deadline 2023-12-11T18:05:22.775102+0000) 2023-12-11T18:05:33.533 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:05:33 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:05:33.406+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:02.775745+0000 front 2023-12-11T18:05:02.775784+0000 (oldest deadline 2023-12-11T18:05:33.376735+0000) 2023-12-11T18:05:33.533 INFO:journalctl@ceph.osd.4.smithi115.stdout:Dec 11 18:05:33 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-4[116520]: 2023-12-11T18:05:33.220+0000 7ff7103a4700 -1 osd.4 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:58.556065+0000 front 2023-12-11T18:04:58.556079+0000 (oldest deadline 2023-12-11T18:05:22.055902+0000) 2023-12-11T18:05:33.533 INFO:journalctl@ceph.osd.4.smithi115.stdout:Dec 11 18:05:33 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-4[116520]: 2023-12-11T18:05:33.220+0000 7ff7103a4700 -1 osd.4 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:05.556911+0000 front 2023-12-11T18:05:05.556884+0000 (oldest deadline 2023-12-11T18:05:30.257044+0000) 2023-12-11T18:05:34.052 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:05:33 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:05:33.759+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:58.934598+0000 front 2023-12-11T18:04:58.934735+0000 (oldest deadline 2023-12-11T18:05:21.834661+0000) 2023-12-11T18:05:34.053 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:05:33 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:05:33.759+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:06.536080+0000 front 2023-12-11T18:05:06.535905+0000 (oldest deadline 2023-12-11T18:05:31.835855+0000) 2023-12-11T18:05:34.283 INFO:journalctl@ceph.osd.4.smithi115.stdout:Dec 11 18:05:34 smithi115 bash[147317]: ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-4 2023-12-11T18:05:34.283 INFO:journalctl@ceph.osd.7.smithi115.stdout:Dec 11 18:05:34 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-7[124665]: 2023-12-11T18:05:34.013+0000 7f9a71cae700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:05.284230+0000 front 2023-12-11T18:05:05.284410+0000 (oldest deadline 2023-12-11T18:05:31.184416+0000) 2023-12-11T18:05:34.404 INFO:journalctl@ceph.osd.6.smithi059.stdout:Dec 11 18:05:34 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-6[133999]: 2023-12-11T18:05:34.050+0000 7f6dafb58700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:58.323226+0000 front 2023-12-11T18:04:58.323260+0000 (oldest deadline 2023-12-11T18:05:21.223373+0000) 2023-12-11T18:05:34.404 INFO:journalctl@ceph.osd.6.smithi059.stdout:Dec 11 18:05:34 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-6[133999]: 2023-12-11T18:05:34.050+0000 7f6dafb58700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:06.127953+0000 front 2023-12-11T18:05:06.127805+0000 (oldest deadline 2023-12-11T18:05:29.027582+0000) 2023-12-11T18:05:34.782 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:05:34 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:05:34.413+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:59.875115+0000 front 2023-12-11T18:04:59.874965+0000 (oldest deadline 2023-12-11T18:05:22.775102+0000) 2023-12-11T18:05:34.783 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:05:34 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:05:34.413+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:02.775745+0000 front 2023-12-11T18:05:02.775784+0000 (oldest deadline 2023-12-11T18:05:33.376735+0000) 2023-12-11T18:05:35.025 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:05:34 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:05:34.758+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:58.934598+0000 front 2023-12-11T18:04:58.934735+0000 (oldest deadline 2023-12-11T18:05:21.834661+0000) 2023-12-11T18:05:35.026 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:05:34 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:05:34.758+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:06.536080+0000 front 2023-12-11T18:05:06.535905+0000 (oldest deadline 2023-12-11T18:05:31.835855+0000) 2023-12-11T18:05:35.282 INFO:journalctl@ceph.osd.7.smithi115.stdout:Dec 11 18:05:35 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-7[124665]: 2023-12-11T18:05:35.045+0000 7f9a71cae700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:05.284230+0000 front 2023-12-11T18:05:05.284410+0000 (oldest deadline 2023-12-11T18:05:31.184416+0000) 2023-12-11T18:05:35.404 INFO:journalctl@ceph.osd.6.smithi059.stdout:Dec 11 18:05:35 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-6[133999]: 2023-12-11T18:05:35.022+0000 7f6dafb58700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:58.323226+0000 front 2023-12-11T18:04:58.323260+0000 (oldest deadline 2023-12-11T18:05:21.223373+0000) 2023-12-11T18:05:35.404 INFO:journalctl@ceph.osd.6.smithi059.stdout:Dec 11 18:05:35 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-6[133999]: 2023-12-11T18:05:35.022+0000 7f6dafb58700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:06.127953+0000 front 2023-12-11T18:05:06.127805+0000 (oldest deadline 2023-12-11T18:05:29.027582+0000) 2023-12-11T18:05:35.766 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:05:35 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:05:35.426+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:59.875115+0000 front 2023-12-11T18:04:59.874965+0000 (oldest deadline 2023-12-11T18:05:22.775102+0000) 2023-12-11T18:05:35.766 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:05:35 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:05:35.426+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:02.775745+0000 front 2023-12-11T18:05:02.775784+0000 (oldest deadline 2023-12-11T18:05:33.376735+0000) 2023-12-11T18:05:35.772 DEBUG:teuthology.orchestra.run.smithi115:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d0a07eaa-984b-11ee-95a3-87774f69a715@osd.4.service' 2023-12-11T18:05:35.812 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-11T18:05:35.812 INFO:tasks.cephadm.osd.4:Stopped osd.4 2023-12-11T18:05:35.812 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2023-12-11T18:05:35.812 DEBUG:teuthology.orchestra.run.smithi181:> sudo systemctl stop ceph-d0a07eaa-984b-11ee-95a3-87774f69a715@osd.5 2023-12-11T18:05:36.039 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:05:35 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:05:35.714+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:58.934598+0000 front 2023-12-11T18:04:58.934735+0000 (oldest deadline 2023-12-11T18:05:21.834661+0000) 2023-12-11T18:05:36.039 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:05:35 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:05:35.714+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:06.536080+0000 front 2023-12-11T18:05:06.535905+0000 (oldest deadline 2023-12-11T18:05:31.835855+0000) 2023-12-11T18:05:36.404 INFO:journalctl@ceph.osd.6.smithi059.stdout:Dec 11 18:05:36 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-6[133999]: 2023-12-11T18:05:36.036+0000 7f6dafb58700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:58.323226+0000 front 2023-12-11T18:04:58.323260+0000 (oldest deadline 2023-12-11T18:05:21.223373+0000) 2023-12-11T18:05:36.404 INFO:journalctl@ceph.osd.6.smithi059.stdout:Dec 11 18:05:36 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-6[133999]: 2023-12-11T18:05:36.036+0000 7f6dafb58700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:06.127953+0000 front 2023-12-11T18:05:06.127805+0000 (oldest deadline 2023-12-11T18:05:29.027582+0000) 2023-12-11T18:05:36.472 INFO:journalctl@ceph.osd.7.smithi115.stdout:Dec 11 18:05:36 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-7[124665]: 2023-12-11T18:05:36.089+0000 7f9a71cae700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:05.284230+0000 front 2023-12-11T18:05:05.284410+0000 (oldest deadline 2023-12-11T18:05:31.184416+0000) 2023-12-11T18:05:36.782 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:05:36 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:05:36.469+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:59.875115+0000 front 2023-12-11T18:04:59.874965+0000 (oldest deadline 2023-12-11T18:05:22.775102+0000) 2023-12-11T18:05:36.783 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:05:36 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:05:36.469+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:02.775745+0000 front 2023-12-11T18:05:02.775784+0000 (oldest deadline 2023-12-11T18:05:33.376735+0000) 2023-12-11T18:05:37.077 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:05:36 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:05:36.706+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:58.934598+0000 front 2023-12-11T18:04:58.934735+0000 (oldest deadline 2023-12-11T18:05:21.834661+0000) 2023-12-11T18:05:37.078 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:05:36 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:05:36.706+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:06.536080+0000 front 2023-12-11T18:05:06.535905+0000 (oldest deadline 2023-12-11T18:05:31.835855+0000) 2023-12-11T18:05:37.404 INFO:journalctl@ceph.osd.6.smithi059.stdout:Dec 11 18:05:37 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-6[133999]: 2023-12-11T18:05:37.075+0000 7f6dafb58700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:58.323226+0000 front 2023-12-11T18:04:58.323260+0000 (oldest deadline 2023-12-11T18:05:21.223373+0000) 2023-12-11T18:05:37.404 INFO:journalctl@ceph.osd.6.smithi059.stdout:Dec 11 18:05:37 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-6[133999]: 2023-12-11T18:05:37.075+0000 7f6dafb58700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:06.127953+0000 front 2023-12-11T18:05:06.127805+0000 (oldest deadline 2023-12-11T18:05:29.027582+0000) 2023-12-11T18:05:37.521 INFO:journalctl@ceph.osd.7.smithi115.stdout:Dec 11 18:05:37 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-7[124665]: 2023-12-11T18:05:37.101+0000 7f9a71cae700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:05.284230+0000 front 2023-12-11T18:05:05.284410+0000 (oldest deadline 2023-12-11T18:05:31.184416+0000) 2023-12-11T18:05:37.782 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:05:37 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:05:37.518+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:59.875115+0000 front 2023-12-11T18:04:59.874965+0000 (oldest deadline 2023-12-11T18:05:22.775102+0000) 2023-12-11T18:05:37.783 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:05:37 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:05:37.518+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:02.775745+0000 front 2023-12-11T18:05:02.775784+0000 (oldest deadline 2023-12-11T18:05:33.376735+0000) 2023-12-11T18:05:38.108 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:05:37 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:05:37.696+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:58.934598+0000 front 2023-12-11T18:04:58.934735+0000 (oldest deadline 2023-12-11T18:05:21.834661+0000) 2023-12-11T18:05:38.108 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:05:37 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:05:37.696+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:06.536080+0000 front 2023-12-11T18:05:06.535905+0000 (oldest deadline 2023-12-11T18:05:31.835855+0000) 2023-12-11T18:05:38.404 INFO:journalctl@ceph.osd.6.smithi059.stdout:Dec 11 18:05:38 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-6[133999]: 2023-12-11T18:05:38.106+0000 7f6dafb58700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:58.323226+0000 front 2023-12-11T18:04:58.323260+0000 (oldest deadline 2023-12-11T18:05:21.223373+0000) 2023-12-11T18:05:38.404 INFO:journalctl@ceph.osd.6.smithi059.stdout:Dec 11 18:05:38 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-6[133999]: 2023-12-11T18:05:38.106+0000 7f6dafb58700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:06.127953+0000 front 2023-12-11T18:05:06.127805+0000 (oldest deadline 2023-12-11T18:05:29.027582+0000) 2023-12-11T18:05:38.532 INFO:journalctl@ceph.osd.7.smithi115.stdout:Dec 11 18:05:38 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-7[124665]: 2023-12-11T18:05:38.069+0000 7f9a71cae700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:05.284230+0000 front 2023-12-11T18:05:05.284410+0000 (oldest deadline 2023-12-11T18:05:31.184416+0000) 2023-12-11T18:05:39.032 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:05:38 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:05:38.543+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:59.875115+0000 front 2023-12-11T18:04:59.874965+0000 (oldest deadline 2023-12-11T18:05:22.775102+0000) 2023-12-11T18:05:39.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:05:38 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:05:38.544+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:02.775745+0000 front 2023-12-11T18:05:02.775784+0000 (oldest deadline 2023-12-11T18:05:33.376735+0000) 2023-12-11T18:05:39.098 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:05:38 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:05:38.679+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:58.934598+0000 front 2023-12-11T18:04:58.934735+0000 (oldest deadline 2023-12-11T18:05:21.834661+0000) 2023-12-11T18:05:39.098 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:05:38 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:05:38.679+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:06.536080+0000 front 2023-12-11T18:05:06.535905+0000 (oldest deadline 2023-12-11T18:05:31.835855+0000) 2023-12-11T18:05:39.404 INFO:journalctl@ceph.osd.6.smithi059.stdout:Dec 11 18:05:39 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-6[133999]: 2023-12-11T18:05:39.095+0000 7f6dafb58700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:58.323226+0000 front 2023-12-11T18:04:58.323260+0000 (oldest deadline 2023-12-11T18:05:21.223373+0000) 2023-12-11T18:05:39.404 INFO:journalctl@ceph.osd.6.smithi059.stdout:Dec 11 18:05:39 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-6[133999]: 2023-12-11T18:05:39.095+0000 7f6dafb58700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:06.127953+0000 front 2023-12-11T18:05:06.127805+0000 (oldest deadline 2023-12-11T18:05:29.027582+0000) 2023-12-11T18:05:39.532 INFO:journalctl@ceph.osd.7.smithi115.stdout:Dec 11 18:05:39 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-7[124665]: 2023-12-11T18:05:39.042+0000 7f9a71cae700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:05.284230+0000 front 2023-12-11T18:05:05.284410+0000 (oldest deadline 2023-12-11T18:05:31.184416+0000) 2023-12-11T18:05:40.032 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:05:39 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:05:39.585+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:59.875115+0000 front 2023-12-11T18:04:59.874965+0000 (oldest deadline 2023-12-11T18:05:22.775102+0000) 2023-12-11T18:05:40.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:05:39 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:05:39.585+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:02.775745+0000 front 2023-12-11T18:05:02.775784+0000 (oldest deadline 2023-12-11T18:05:33.376735+0000) 2023-12-11T18:05:40.115 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:05:39 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:05:39.662+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:58.934598+0000 front 2023-12-11T18:04:58.934735+0000 (oldest deadline 2023-12-11T18:05:21.834661+0000) 2023-12-11T18:05:40.115 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:05:39 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:05:39.662+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:06.536080+0000 front 2023-12-11T18:05:06.535905+0000 (oldest deadline 2023-12-11T18:05:31.835855+0000) 2023-12-11T18:05:40.404 INFO:journalctl@ceph.osd.6.smithi059.stdout:Dec 11 18:05:40 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-6[133999]: 2023-12-11T18:05:40.112+0000 7f6dafb58700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:58.323226+0000 front 2023-12-11T18:04:58.323260+0000 (oldest deadline 2023-12-11T18:05:21.223373+0000) 2023-12-11T18:05:40.404 INFO:journalctl@ceph.osd.6.smithi059.stdout:Dec 11 18:05:40 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-6[133999]: 2023-12-11T18:05:40.112+0000 7f6dafb58700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:06.127953+0000 front 2023-12-11T18:05:06.127805+0000 (oldest deadline 2023-12-11T18:05:29.027582+0000) 2023-12-11T18:05:40.532 INFO:journalctl@ceph.osd.7.smithi115.stdout:Dec 11 18:05:40 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-7[124665]: 2023-12-11T18:05:40.077+0000 7f9a71cae700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:05.284230+0000 front 2023-12-11T18:05:05.284410+0000 (oldest deadline 2023-12-11T18:05:31.184416+0000) 2023-12-11T18:05:40.904 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:05:40 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:05:40.633+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:58.934598+0000 front 2023-12-11T18:04:58.934735+0000 (oldest deadline 2023-12-11T18:05:21.834661+0000) 2023-12-11T18:05:40.904 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:05:40 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:05:40.633+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:06.536080+0000 front 2023-12-11T18:05:06.535905+0000 (oldest deadline 2023-12-11T18:05:31.835855+0000) 2023-12-11T18:05:41.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:05:40 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:05:40.625+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:59.875115+0000 front 2023-12-11T18:04:59.874965+0000 (oldest deadline 2023-12-11T18:05:22.775102+0000) 2023-12-11T18:05:41.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:05:40 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:05:40.625+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:02.775745+0000 front 2023-12-11T18:05:02.775784+0000 (oldest deadline 2023-12-11T18:05:33.376735+0000) 2023-12-11T18:05:41.404 INFO:journalctl@ceph.osd.6.smithi059.stdout:Dec 11 18:05:41 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-6[133999]: 2023-12-11T18:05:41.121+0000 7f6dafb58700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:58.323226+0000 front 2023-12-11T18:04:58.323260+0000 (oldest deadline 2023-12-11T18:05:21.223373+0000) 2023-12-11T18:05:41.404 INFO:journalctl@ceph.osd.6.smithi059.stdout:Dec 11 18:05:41 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-6[133999]: 2023-12-11T18:05:41.121+0000 7f6dafb58700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:06.127953+0000 front 2023-12-11T18:05:06.127805+0000 (oldest deadline 2023-12-11T18:05:29.027582+0000) 2023-12-11T18:05:41.532 INFO:journalctl@ceph.osd.7.smithi115.stdout:Dec 11 18:05:41 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-7[124665]: 2023-12-11T18:05:41.071+0000 7f9a71cae700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:05.284230+0000 front 2023-12-11T18:05:05.284410+0000 (oldest deadline 2023-12-11T18:05:31.184416+0000) 2023-12-11T18:05:41.904 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:05:41 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:05:41.643+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:58.934598+0000 front 2023-12-11T18:04:58.934735+0000 (oldest deadline 2023-12-11T18:05:21.834661+0000) 2023-12-11T18:05:41.905 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:05:41 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:05:41.643+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:06.536080+0000 front 2023-12-11T18:05:06.535905+0000 (oldest deadline 2023-12-11T18:05:31.835855+0000) 2023-12-11T18:05:42.032 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:05:41 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:05:41.670+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:59.875115+0000 front 2023-12-11T18:04:59.874965+0000 (oldest deadline 2023-12-11T18:05:22.775102+0000) 2023-12-11T18:05:42.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:05:41 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:05:41.670+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:02.775745+0000 front 2023-12-11T18:05:02.775784+0000 (oldest deadline 2023-12-11T18:05:33.376735+0000) 2023-12-11T18:05:42.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:05:41 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:05:41.670+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.181:6816 osd.2 since back 2023-12-11T18:05:16.778632+0000 front 2023-12-11T18:05:16.778462+0000 (oldest deadline 2023-12-11T18:05:41.478412+0000) 2023-12-11T18:05:42.404 INFO:journalctl@ceph.osd.6.smithi059.stdout:Dec 11 18:05:42 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-6[133999]: 2023-12-11T18:05:42.116+0000 7f6dafb58700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:58.323226+0000 front 2023-12-11T18:04:58.323260+0000 (oldest deadline 2023-12-11T18:05:21.223373+0000) 2023-12-11T18:05:42.404 INFO:journalctl@ceph.osd.6.smithi059.stdout:Dec 11 18:05:42 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-6[133999]: 2023-12-11T18:05:42.116+0000 7f6dafb58700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:06.127953+0000 front 2023-12-11T18:05:06.127805+0000 (oldest deadline 2023-12-11T18:05:29.027582+0000) 2023-12-11T18:05:42.532 INFO:journalctl@ceph.osd.7.smithi115.stdout:Dec 11 18:05:42 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-7[124665]: 2023-12-11T18:05:42.032+0000 7f9a71cae700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:05.284230+0000 front 2023-12-11T18:05:05.284410+0000 (oldest deadline 2023-12-11T18:05:31.184416+0000) 2023-12-11T18:05:42.904 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:05:42 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:05:42.616+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:58.934598+0000 front 2023-12-11T18:04:58.934735+0000 (oldest deadline 2023-12-11T18:05:21.834661+0000) 2023-12-11T18:05:42.904 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:05:42 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:05:42.616+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:06.536080+0000 front 2023-12-11T18:05:06.535905+0000 (oldest deadline 2023-12-11T18:05:31.835855+0000) 2023-12-11T18:05:42.904 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:05:42 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:05:42.616+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.181:6816 osd.2 since back 2023-12-11T18:05:16.536946+0000 front 2023-12-11T18:05:16.537012+0000 (oldest deadline 2023-12-11T18:05:42.436997+0000) 2023-12-11T18:05:42.993 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:05:42 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:05:42.645+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:59.875115+0000 front 2023-12-11T18:04:59.874965+0000 (oldest deadline 2023-12-11T18:05:22.775102+0000) 2023-12-11T18:05:42.993 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:05:42 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:05:42.645+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:02.775745+0000 front 2023-12-11T18:05:02.775784+0000 (oldest deadline 2023-12-11T18:05:33.376735+0000) 2023-12-11T18:05:42.993 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:05:42 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:05:42.645+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.181:6816 osd.2 since back 2023-12-11T18:05:16.778632+0000 front 2023-12-11T18:05:16.778462+0000 (oldest deadline 2023-12-11T18:05:41.478412+0000) 2023-12-11T18:05:43.282 INFO:journalctl@ceph.osd.7.smithi115.stdout:Dec 11 18:05:42 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-7[124665]: 2023-12-11T18:05:42.989+0000 7f9a71cae700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:05.284230+0000 front 2023-12-11T18:05:05.284410+0000 (oldest deadline 2023-12-11T18:05:31.184416+0000) 2023-12-11T18:05:43.283 INFO:journalctl@ceph.osd.7.smithi115.stdout:Dec 11 18:05:42 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-7[124665]: 2023-12-11T18:05:42.989+0000 7f9a71cae700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.181:6816 osd.2 since back 2023-12-11T18:05:18.186375+0000 front 2023-12-11T18:05:18.186338+0000 (oldest deadline 2023-12-11T18:05:42.886186+0000) 2023-12-11T18:05:43.404 INFO:journalctl@ceph.osd.6.smithi059.stdout:Dec 11 18:05:43 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-6[133999]: 2023-12-11T18:05:43.077+0000 7f6dafb58700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:58.323226+0000 front 2023-12-11T18:04:58.323260+0000 (oldest deadline 2023-12-11T18:05:21.223373+0000) 2023-12-11T18:05:43.404 INFO:journalctl@ceph.osd.6.smithi059.stdout:Dec 11 18:05:43 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-6[133999]: 2023-12-11T18:05:43.077+0000 7f6dafb58700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:06.127953+0000 front 2023-12-11T18:05:06.127805+0000 (oldest deadline 2023-12-11T18:05:29.027582+0000) 2023-12-11T18:05:43.404 INFO:journalctl@ceph.osd.6.smithi059.stdout:Dec 11 18:05:43 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-6[133999]: 2023-12-11T18:05:43.077+0000 7f6dafb58700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.181:6816 osd.2 since back 2023-12-11T18:05:16.529711+0000 front 2023-12-11T18:05:16.529552+0000 (oldest deadline 2023-12-11T18:05:42.429653+0000) 2023-12-11T18:05:43.904 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:05:43 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:05:43.614+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:58.934598+0000 front 2023-12-11T18:04:58.934735+0000 (oldest deadline 2023-12-11T18:05:21.834661+0000) 2023-12-11T18:05:43.904 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:05:43 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:05:43.614+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:06.536080+0000 front 2023-12-11T18:05:06.535905+0000 (oldest deadline 2023-12-11T18:05:31.835855+0000) 2023-12-11T18:05:43.904 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:05:43 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:05:43.614+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.181:6816 osd.2 since back 2023-12-11T18:05:16.536946+0000 front 2023-12-11T18:05:16.537012+0000 (oldest deadline 2023-12-11T18:05:42.436997+0000) 2023-12-11T18:05:43.990 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:05:43 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:05:43.617+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:59.875115+0000 front 2023-12-11T18:04:59.874965+0000 (oldest deadline 2023-12-11T18:05:22.775102+0000) 2023-12-11T18:05:43.990 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:05:43 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:05:43.617+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:02.775745+0000 front 2023-12-11T18:05:02.775784+0000 (oldest deadline 2023-12-11T18:05:33.376735+0000) 2023-12-11T18:05:43.990 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:05:43 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:05:43.617+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.181:6816 osd.2 since back 2023-12-11T18:05:16.778632+0000 front 2023-12-11T18:05:16.778462+0000 (oldest deadline 2023-12-11T18:05:41.478412+0000) 2023-12-11T18:05:44.282 INFO:journalctl@ceph.osd.7.smithi115.stdout:Dec 11 18:05:43 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-7[124665]: 2023-12-11T18:05:43.987+0000 7f9a71cae700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:05.284230+0000 front 2023-12-11T18:05:05.284410+0000 (oldest deadline 2023-12-11T18:05:31.184416+0000) 2023-12-11T18:05:44.283 INFO:journalctl@ceph.osd.7.smithi115.stdout:Dec 11 18:05:43 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-7[124665]: 2023-12-11T18:05:43.987+0000 7f9a71cae700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.181:6816 osd.2 since back 2023-12-11T18:05:18.186375+0000 front 2023-12-11T18:05:18.186338+0000 (oldest deadline 2023-12-11T18:05:42.886186+0000) 2023-12-11T18:05:44.404 INFO:journalctl@ceph.osd.6.smithi059.stdout:Dec 11 18:05:44 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-6[133999]: 2023-12-11T18:05:44.112+0000 7f6dafb58700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:58.323226+0000 front 2023-12-11T18:04:58.323260+0000 (oldest deadline 2023-12-11T18:05:21.223373+0000) 2023-12-11T18:05:44.404 INFO:journalctl@ceph.osd.6.smithi059.stdout:Dec 11 18:05:44 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-6[133999]: 2023-12-11T18:05:44.112+0000 7f6dafb58700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:06.127953+0000 front 2023-12-11T18:05:06.127805+0000 (oldest deadline 2023-12-11T18:05:29.027582+0000) 2023-12-11T18:05:44.404 INFO:journalctl@ceph.osd.6.smithi059.stdout:Dec 11 18:05:44 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-6[133999]: 2023-12-11T18:05:44.112+0000 7f6dafb58700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.181:6816 osd.2 since back 2023-12-11T18:05:16.529711+0000 front 2023-12-11T18:05:16.529552+0000 (oldest deadline 2023-12-11T18:05:42.429653+0000) 2023-12-11T18:05:44.419 DEBUG:teuthology.orchestra.run.smithi181:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d0a07eaa-984b-11ee-95a3-87774f69a715@osd.5.service' 2023-12-11T18:05:44.458 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-11T18:05:44.458 INFO:tasks.cephadm.osd.5:Stopped osd.5 2023-12-11T18:05:44.459 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2023-12-11T18:05:44.459 DEBUG:teuthology.orchestra.run.smithi059:> sudo systemctl stop ceph-d0a07eaa-984b-11ee-95a3-87774f69a715@osd.6 2023-12-11T18:05:44.829 INFO:journalctl@ceph.osd.6.smithi059.stdout:Dec 11 18:05:44 smithi059 systemd[1]: Stopping Ceph osd.6 for d0a07eaa-984b-11ee-95a3-87774f69a715... 2023-12-11T18:05:44.829 INFO:journalctl@ceph.osd.6.smithi059.stdout:Dec 11 18:05:44 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-6[133999]: 2023-12-11T18:05:44.826+0000 7f6db7565700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-11T18:05:44.829 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:05:44 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:05:44.567+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:58.934598+0000 front 2023-12-11T18:04:58.934735+0000 (oldest deadline 2023-12-11T18:05:21.834661+0000) 2023-12-11T18:05:44.829 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:05:44 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:05:44.567+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:06.536080+0000 front 2023-12-11T18:05:06.535905+0000 (oldest deadline 2023-12-11T18:05:31.835855+0000) 2023-12-11T18:05:44.829 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:05:44 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:05:44.567+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.181:6816 osd.2 since back 2023-12-11T18:05:16.536946+0000 front 2023-12-11T18:05:16.537012+0000 (oldest deadline 2023-12-11T18:05:42.436997+0000) 2023-12-11T18:05:44.991 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:05:44 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:05:44.663+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:59.875115+0000 front 2023-12-11T18:04:59.874965+0000 (oldest deadline 2023-12-11T18:05:22.775102+0000) 2023-12-11T18:05:44.991 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:05:44 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:05:44.663+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:02.775745+0000 front 2023-12-11T18:05:02.775784+0000 (oldest deadline 2023-12-11T18:05:33.376735+0000) 2023-12-11T18:05:44.991 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:05:44 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:05:44.663+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.181:6816 osd.2 since back 2023-12-11T18:05:16.778632+0000 front 2023-12-11T18:05:16.778462+0000 (oldest deadline 2023-12-11T18:05:41.478412+0000) 2023-12-11T18:05:45.154 INFO:journalctl@ceph.osd.6.smithi059.stdout:Dec 11 18:05:44 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-6[133999]: 2023-12-11T18:05:44.826+0000 7f6db7565700 -1 osd.6 123 *** Got signal Terminated *** 2023-12-11T18:05:45.154 INFO:journalctl@ceph.osd.6.smithi059.stdout:Dec 11 18:05:44 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-6[133999]: 2023-12-11T18:05:44.826+0000 7f6db7565700 -1 osd.6 123 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-12-11T18:05:45.282 INFO:journalctl@ceph.osd.7.smithi115.stdout:Dec 11 18:05:44 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-7[124665]: 2023-12-11T18:05:44.988+0000 7f9a71cae700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:05.284230+0000 front 2023-12-11T18:05:05.284410+0000 (oldest deadline 2023-12-11T18:05:31.184416+0000) 2023-12-11T18:05:45.282 INFO:journalctl@ceph.osd.7.smithi115.stdout:Dec 11 18:05:44 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-7[124665]: 2023-12-11T18:05:44.988+0000 7f9a71cae700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.181:6816 osd.2 since back 2023-12-11T18:05:18.186375+0000 front 2023-12-11T18:05:18.186338+0000 (oldest deadline 2023-12-11T18:05:42.886186+0000) 2023-12-11T18:05:45.597 INFO:journalctl@ceph.osd.6.smithi059.stdout:Dec 11 18:05:45 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-6[133999]: 2023-12-11T18:05:45.159+0000 7f6dafb58700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:58.323226+0000 front 2023-12-11T18:04:58.323260+0000 (oldest deadline 2023-12-11T18:05:21.223373+0000) 2023-12-11T18:05:45.598 INFO:journalctl@ceph.osd.6.smithi059.stdout:Dec 11 18:05:45 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-6[133999]: 2023-12-11T18:05:45.159+0000 7f6dafb58700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:06.127953+0000 front 2023-12-11T18:05:06.127805+0000 (oldest deadline 2023-12-11T18:05:29.027582+0000) 2023-12-11T18:05:45.598 INFO:journalctl@ceph.osd.6.smithi059.stdout:Dec 11 18:05:45 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-6[133999]: 2023-12-11T18:05:45.159+0000 7f6dafb58700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.181:6816 osd.2 since back 2023-12-11T18:05:16.529711+0000 front 2023-12-11T18:05:16.529552+0000 (oldest deadline 2023-12-11T18:05:42.429653+0000) 2023-12-11T18:05:45.904 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:05:45 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:05:45.595+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:58.934598+0000 front 2023-12-11T18:04:58.934735+0000 (oldest deadline 2023-12-11T18:05:21.834661+0000) 2023-12-11T18:05:45.904 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:05:45 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:05:45.595+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:06.536080+0000 front 2023-12-11T18:05:06.535905+0000 (oldest deadline 2023-12-11T18:05:31.835855+0000) 2023-12-11T18:05:45.904 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:05:45 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:05:45.595+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.181:6816 osd.2 since back 2023-12-11T18:05:16.536946+0000 front 2023-12-11T18:05:16.537012+0000 (oldest deadline 2023-12-11T18:05:42.436997+0000) 2023-12-11T18:05:45.963 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:05:45 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:05:45.709+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:59.875115+0000 front 2023-12-11T18:04:59.874965+0000 (oldest deadline 2023-12-11T18:05:22.775102+0000) 2023-12-11T18:05:45.963 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:05:45 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:05:45.709+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:02.775745+0000 front 2023-12-11T18:05:02.775784+0000 (oldest deadline 2023-12-11T18:05:33.376735+0000) 2023-12-11T18:05:45.964 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:05:45 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:05:45.709+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.181:6816 osd.2 since back 2023-12-11T18:05:16.778632+0000 front 2023-12-11T18:05:16.778462+0000 (oldest deadline 2023-12-11T18:05:41.478412+0000) 2023-12-11T18:05:46.282 INFO:journalctl@ceph.osd.7.smithi115.stdout:Dec 11 18:05:45 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-7[124665]: 2023-12-11T18:05:45.961+0000 7f9a71cae700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:05.284230+0000 front 2023-12-11T18:05:05.284410+0000 (oldest deadline 2023-12-11T18:05:31.184416+0000) 2023-12-11T18:05:46.282 INFO:journalctl@ceph.osd.7.smithi115.stdout:Dec 11 18:05:45 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-7[124665]: 2023-12-11T18:05:45.961+0000 7f9a71cae700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.181:6816 osd.2 since back 2023-12-11T18:05:18.186375+0000 front 2023-12-11T18:05:18.186338+0000 (oldest deadline 2023-12-11T18:05:42.886186+0000) 2023-12-11T18:05:46.404 INFO:journalctl@ceph.osd.6.smithi059.stdout:Dec 11 18:05:46 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-6[133999]: 2023-12-11T18:05:46.119+0000 7f6dafb58700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:58.323226+0000 front 2023-12-11T18:04:58.323260+0000 (oldest deadline 2023-12-11T18:05:21.223373+0000) 2023-12-11T18:05:46.404 INFO:journalctl@ceph.osd.6.smithi059.stdout:Dec 11 18:05:46 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-6[133999]: 2023-12-11T18:05:46.119+0000 7f6dafb58700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:06.127953+0000 front 2023-12-11T18:05:06.127805+0000 (oldest deadline 2023-12-11T18:05:29.027582+0000) 2023-12-11T18:05:46.404 INFO:journalctl@ceph.osd.6.smithi059.stdout:Dec 11 18:05:46 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-6[133999]: 2023-12-11T18:05:46.119+0000 7f6dafb58700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.181:6816 osd.2 since back 2023-12-11T18:05:16.529711+0000 front 2023-12-11T18:05:16.529552+0000 (oldest deadline 2023-12-11T18:05:42.429653+0000) 2023-12-11T18:05:46.904 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:05:46 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:05:46.563+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:58.934598+0000 front 2023-12-11T18:04:58.934735+0000 (oldest deadline 2023-12-11T18:05:21.834661+0000) 2023-12-11T18:05:46.904 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:05:46 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:05:46.563+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:06.536080+0000 front 2023-12-11T18:05:06.535905+0000 (oldest deadline 2023-12-11T18:05:31.835855+0000) 2023-12-11T18:05:46.904 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:05:46 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:05:46.563+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.181:6816 osd.2 since back 2023-12-11T18:05:16.536946+0000 front 2023-12-11T18:05:16.537012+0000 (oldest deadline 2023-12-11T18:05:42.436997+0000) 2023-12-11T18:05:46.964 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:05:46 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:05:46.675+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:59.875115+0000 front 2023-12-11T18:04:59.874965+0000 (oldest deadline 2023-12-11T18:05:22.775102+0000) 2023-12-11T18:05:46.964 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:05:46 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:05:46.675+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:02.775745+0000 front 2023-12-11T18:05:02.775784+0000 (oldest deadline 2023-12-11T18:05:33.376735+0000) 2023-12-11T18:05:46.964 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:05:46 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:05:46.675+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.181:6816 osd.2 since back 2023-12-11T18:05:16.778632+0000 front 2023-12-11T18:05:16.778462+0000 (oldest deadline 2023-12-11T18:05:41.478412+0000) 2023-12-11T18:05:47.282 INFO:journalctl@ceph.osd.7.smithi115.stdout:Dec 11 18:05:46 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-7[124665]: 2023-12-11T18:05:46.962+0000 7f9a71cae700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:05.284230+0000 front 2023-12-11T18:05:05.284410+0000 (oldest deadline 2023-12-11T18:05:31.184416+0000) 2023-12-11T18:05:47.282 INFO:journalctl@ceph.osd.7.smithi115.stdout:Dec 11 18:05:46 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-7[124665]: 2023-12-11T18:05:46.962+0000 7f9a71cae700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.181:6816 osd.2 since back 2023-12-11T18:05:18.186375+0000 front 2023-12-11T18:05:18.186338+0000 (oldest deadline 2023-12-11T18:05:42.886186+0000) 2023-12-11T18:05:47.404 INFO:journalctl@ceph.osd.6.smithi059.stdout:Dec 11 18:05:47 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-6[133999]: 2023-12-11T18:05:47.121+0000 7f6dafb58700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:58.323226+0000 front 2023-12-11T18:04:58.323260+0000 (oldest deadline 2023-12-11T18:05:21.223373+0000) 2023-12-11T18:05:47.404 INFO:journalctl@ceph.osd.6.smithi059.stdout:Dec 11 18:05:47 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-6[133999]: 2023-12-11T18:05:47.121+0000 7f6dafb58700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:06.127953+0000 front 2023-12-11T18:05:06.127805+0000 (oldest deadline 2023-12-11T18:05:29.027582+0000) 2023-12-11T18:05:47.404 INFO:journalctl@ceph.osd.6.smithi059.stdout:Dec 11 18:05:47 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-6[133999]: 2023-12-11T18:05:47.121+0000 7f6dafb58700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.181:6816 osd.2 since back 2023-12-11T18:05:16.529711+0000 front 2023-12-11T18:05:16.529552+0000 (oldest deadline 2023-12-11T18:05:42.429653+0000) 2023-12-11T18:05:47.904 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:05:47 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:05:47.522+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:58.934598+0000 front 2023-12-11T18:04:58.934735+0000 (oldest deadline 2023-12-11T18:05:21.834661+0000) 2023-12-11T18:05:47.904 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:05:47 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:05:47.522+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:06.536080+0000 front 2023-12-11T18:05:06.535905+0000 (oldest deadline 2023-12-11T18:05:31.835855+0000) 2023-12-11T18:05:47.905 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:05:47 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:05:47.522+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.181:6816 osd.2 since back 2023-12-11T18:05:16.536946+0000 front 2023-12-11T18:05:16.537012+0000 (oldest deadline 2023-12-11T18:05:42.436997+0000) 2023-12-11T18:05:48.032 INFO:journalctl@ceph.osd.7.smithi115.stdout:Dec 11 18:05:47 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-7[124665]: 2023-12-11T18:05:47.945+0000 7f9a71cae700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:05.284230+0000 front 2023-12-11T18:05:05.284410+0000 (oldest deadline 2023-12-11T18:05:31.184416+0000) 2023-12-11T18:05:48.033 INFO:journalctl@ceph.osd.7.smithi115.stdout:Dec 11 18:05:47 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-7[124665]: 2023-12-11T18:05:47.945+0000 7f9a71cae700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.181:6816 osd.2 since back 2023-12-11T18:05:18.186375+0000 front 2023-12-11T18:05:18.186338+0000 (oldest deadline 2023-12-11T18:05:42.886186+0000) 2023-12-11T18:05:48.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:05:47 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:05:47.712+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:59.875115+0000 front 2023-12-11T18:04:59.874965+0000 (oldest deadline 2023-12-11T18:05:22.775102+0000) 2023-12-11T18:05:48.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:05:47 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:05:47.712+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:02.775745+0000 front 2023-12-11T18:05:02.775784+0000 (oldest deadline 2023-12-11T18:05:33.376735+0000) 2023-12-11T18:05:48.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:05:47 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:05:47.712+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.181:6816 osd.2 since back 2023-12-11T18:05:16.778632+0000 front 2023-12-11T18:05:16.778462+0000 (oldest deadline 2023-12-11T18:05:41.478412+0000) 2023-12-11T18:05:48.404 INFO:journalctl@ceph.osd.6.smithi059.stdout:Dec 11 18:05:48 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-6[133999]: 2023-12-11T18:05:48.088+0000 7f6dafb58700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:58.323226+0000 front 2023-12-11T18:04:58.323260+0000 (oldest deadline 2023-12-11T18:05:21.223373+0000) 2023-12-11T18:05:48.404 INFO:journalctl@ceph.osd.6.smithi059.stdout:Dec 11 18:05:48 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-6[133999]: 2023-12-11T18:05:48.088+0000 7f6dafb58700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:06.127953+0000 front 2023-12-11T18:05:06.127805+0000 (oldest deadline 2023-12-11T18:05:29.027582+0000) 2023-12-11T18:05:48.404 INFO:journalctl@ceph.osd.6.smithi059.stdout:Dec 11 18:05:48 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-6[133999]: 2023-12-11T18:05:48.088+0000 7f6dafb58700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.181:6816 osd.2 since back 2023-12-11T18:05:16.529711+0000 front 2023-12-11T18:05:16.529552+0000 (oldest deadline 2023-12-11T18:05:42.429653+0000) 2023-12-11T18:05:48.904 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:05:48 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:05:48.486+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:58.934598+0000 front 2023-12-11T18:04:58.934735+0000 (oldest deadline 2023-12-11T18:05:21.834661+0000) 2023-12-11T18:05:48.904 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:05:48 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:05:48.486+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:06.536080+0000 front 2023-12-11T18:05:06.535905+0000 (oldest deadline 2023-12-11T18:05:31.835855+0000) 2023-12-11T18:05:48.904 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:05:48 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:05:48.486+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.181:6816 osd.2 since back 2023-12-11T18:05:16.536946+0000 front 2023-12-11T18:05:16.537012+0000 (oldest deadline 2023-12-11T18:05:42.436997+0000) 2023-12-11T18:05:48.972 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:05:48 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:05:48.667+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:59.875115+0000 front 2023-12-11T18:04:59.874965+0000 (oldest deadline 2023-12-11T18:05:22.775102+0000) 2023-12-11T18:05:48.972 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:05:48 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:05:48.667+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:02.775745+0000 front 2023-12-11T18:05:02.775784+0000 (oldest deadline 2023-12-11T18:05:33.376735+0000) 2023-12-11T18:05:48.973 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:05:48 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:05:48.667+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.181:6816 osd.2 since back 2023-12-11T18:05:16.778632+0000 front 2023-12-11T18:05:16.778462+0000 (oldest deadline 2023-12-11T18:05:41.478412+0000) 2023-12-11T18:05:48.973 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:05:48 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:05:48.667+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2023-12-11T18:05:21.479017+0000 front 2023-12-11T18:05:21.479156+0000 (oldest deadline 2023-12-11T18:05:47.879527+0000) 2023-12-11T18:05:49.282 INFO:journalctl@ceph.osd.7.smithi115.stdout:Dec 11 18:05:48 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-7[124665]: 2023-12-11T18:05:48.970+0000 7f9a71cae700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:05.284230+0000 front 2023-12-11T18:05:05.284410+0000 (oldest deadline 2023-12-11T18:05:31.184416+0000) 2023-12-11T18:05:49.282 INFO:journalctl@ceph.osd.7.smithi115.stdout:Dec 11 18:05:48 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-7[124665]: 2023-12-11T18:05:48.970+0000 7f9a71cae700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.181:6816 osd.2 since back 2023-12-11T18:05:18.186375+0000 front 2023-12-11T18:05:18.186338+0000 (oldest deadline 2023-12-11T18:05:42.886186+0000) 2023-12-11T18:05:49.283 INFO:journalctl@ceph.osd.7.smithi115.stdout:Dec 11 18:05:48 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-7[124665]: 2023-12-11T18:05:48.970+0000 7f9a71cae700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2023-12-11T18:05:24.587487+0000 front 2023-12-11T18:05:24.587500+0000 (oldest deadline 2023-12-11T18:05:48.687232+0000) 2023-12-11T18:05:49.404 INFO:journalctl@ceph.osd.6.smithi059.stdout:Dec 11 18:05:49 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-6[133999]: 2023-12-11T18:05:49.122+0000 7f6dafb58700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:58.323226+0000 front 2023-12-11T18:04:58.323260+0000 (oldest deadline 2023-12-11T18:05:21.223373+0000) 2023-12-11T18:05:49.404 INFO:journalctl@ceph.osd.6.smithi059.stdout:Dec 11 18:05:49 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-6[133999]: 2023-12-11T18:05:49.122+0000 7f6dafb58700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:06.127953+0000 front 2023-12-11T18:05:06.127805+0000 (oldest deadline 2023-12-11T18:05:29.027582+0000) 2023-12-11T18:05:49.404 INFO:journalctl@ceph.osd.6.smithi059.stdout:Dec 11 18:05:49 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-6[133999]: 2023-12-11T18:05:49.122+0000 7f6dafb58700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.181:6816 osd.2 since back 2023-12-11T18:05:16.529711+0000 front 2023-12-11T18:05:16.529552+0000 (oldest deadline 2023-12-11T18:05:42.429653+0000) 2023-12-11T18:05:49.904 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:05:49 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:05:49.458+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:58.934598+0000 front 2023-12-11T18:04:58.934735+0000 (oldest deadline 2023-12-11T18:05:21.834661+0000) 2023-12-11T18:05:49.904 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:05:49 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:05:49.458+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:06.536080+0000 front 2023-12-11T18:05:06.535905+0000 (oldest deadline 2023-12-11T18:05:31.835855+0000) 2023-12-11T18:05:49.904 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:05:49 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:05:49.458+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.181:6816 osd.2 since back 2023-12-11T18:05:16.536946+0000 front 2023-12-11T18:05:16.537012+0000 (oldest deadline 2023-12-11T18:05:42.436997+0000) 2023-12-11T18:05:49.904 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:05:49 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:05:49.458+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2023-12-11T18:05:23.538131+0000 front 2023-12-11T18:05:23.538151+0000 (oldest deadline 2023-12-11T18:05:48.838365+0000) 2023-12-11T18:05:50.010 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:05:49 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:05:49.694+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:59.875115+0000 front 2023-12-11T18:04:59.874965+0000 (oldest deadline 2023-12-11T18:05:22.775102+0000) 2023-12-11T18:05:50.011 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:05:49 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:05:49.694+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:02.775745+0000 front 2023-12-11T18:05:02.775784+0000 (oldest deadline 2023-12-11T18:05:33.376735+0000) 2023-12-11T18:05:50.011 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:05:49 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:05:49.694+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.181:6816 osd.2 since back 2023-12-11T18:05:16.778632+0000 front 2023-12-11T18:05:16.778462+0000 (oldest deadline 2023-12-11T18:05:41.478412+0000) 2023-12-11T18:05:50.011 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:05:49 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:05:49.694+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2023-12-11T18:05:21.479017+0000 front 2023-12-11T18:05:21.479156+0000 (oldest deadline 2023-12-11T18:05:47.879527+0000) 2023-12-11T18:05:50.282 INFO:journalctl@ceph.osd.7.smithi115.stdout:Dec 11 18:05:50 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-7[124665]: 2023-12-11T18:05:50.008+0000 7f9a71cae700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:05.284230+0000 front 2023-12-11T18:05:05.284410+0000 (oldest deadline 2023-12-11T18:05:31.184416+0000) 2023-12-11T18:05:50.283 INFO:journalctl@ceph.osd.7.smithi115.stdout:Dec 11 18:05:50 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-7[124665]: 2023-12-11T18:05:50.008+0000 7f9a71cae700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.181:6816 osd.2 since back 2023-12-11T18:05:18.186375+0000 front 2023-12-11T18:05:18.186338+0000 (oldest deadline 2023-12-11T18:05:42.886186+0000) 2023-12-11T18:05:50.283 INFO:journalctl@ceph.osd.7.smithi115.stdout:Dec 11 18:05:50 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-7[124665]: 2023-12-11T18:05:50.008+0000 7f9a71cae700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2023-12-11T18:05:24.587487+0000 front 2023-12-11T18:05:24.587500+0000 (oldest deadline 2023-12-11T18:05:48.687232+0000) 2023-12-11T18:05:50.768 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:05:50 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:05:50.417+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:58.934598+0000 front 2023-12-11T18:04:58.934735+0000 (oldest deadline 2023-12-11T18:05:21.834661+0000) 2023-12-11T18:05:50.768 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:05:50 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:05:50.417+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:06.536080+0000 front 2023-12-11T18:05:06.535905+0000 (oldest deadline 2023-12-11T18:05:31.835855+0000) 2023-12-11T18:05:50.768 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:05:50 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:05:50.417+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.181:6816 osd.2 since back 2023-12-11T18:05:16.536946+0000 front 2023-12-11T18:05:16.537012+0000 (oldest deadline 2023-12-11T18:05:42.436997+0000) 2023-12-11T18:05:50.768 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:05:50 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:05:50.417+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2023-12-11T18:05:23.538131+0000 front 2023-12-11T18:05:23.538151+0000 (oldest deadline 2023-12-11T18:05:48.838365+0000) 2023-12-11T18:05:51.023 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:05:50 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:05:50.700+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:59.875115+0000 front 2023-12-11T18:04:59.874965+0000 (oldest deadline 2023-12-11T18:05:22.775102+0000) 2023-12-11T18:05:51.024 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:05:50 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:05:50.700+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:02.775745+0000 front 2023-12-11T18:05:02.775784+0000 (oldest deadline 2023-12-11T18:05:33.376735+0000) 2023-12-11T18:05:51.024 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:05:50 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:05:50.700+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.181:6816 osd.2 since back 2023-12-11T18:05:16.778632+0000 front 2023-12-11T18:05:16.778462+0000 (oldest deadline 2023-12-11T18:05:41.478412+0000) 2023-12-11T18:05:51.024 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:05:50 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:05:50.700+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2023-12-11T18:05:21.479017+0000 front 2023-12-11T18:05:21.479156+0000 (oldest deadline 2023-12-11T18:05:47.879527+0000) 2023-12-11T18:05:51.154 INFO:journalctl@ceph.osd.6.smithi059.stdout:Dec 11 18:05:50 smithi059 bash[177677]: ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-6 2023-12-11T18:05:51.282 INFO:journalctl@ceph.osd.7.smithi115.stdout:Dec 11 18:05:51 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-7[124665]: 2023-12-11T18:05:51.021+0000 7f9a71cae700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:05.284230+0000 front 2023-12-11T18:05:05.284410+0000 (oldest deadline 2023-12-11T18:05:31.184416+0000) 2023-12-11T18:05:51.282 INFO:journalctl@ceph.osd.7.smithi115.stdout:Dec 11 18:05:51 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-7[124665]: 2023-12-11T18:05:51.021+0000 7f9a71cae700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.181:6816 osd.2 since back 2023-12-11T18:05:18.186375+0000 front 2023-12-11T18:05:18.186338+0000 (oldest deadline 2023-12-11T18:05:42.886186+0000) 2023-12-11T18:05:51.283 INFO:journalctl@ceph.osd.7.smithi115.stdout:Dec 11 18:05:51 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-7[124665]: 2023-12-11T18:05:51.021+0000 7f9a71cae700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2023-12-11T18:05:24.587487+0000 front 2023-12-11T18:05:24.587500+0000 (oldest deadline 2023-12-11T18:05:48.687232+0000) 2023-12-11T18:05:51.654 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:05:51 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:05:51.377+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:58.934598+0000 front 2023-12-11T18:04:58.934735+0000 (oldest deadline 2023-12-11T18:05:21.834661+0000) 2023-12-11T18:05:51.654 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:05:51 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:05:51.377+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:06.536080+0000 front 2023-12-11T18:05:06.535905+0000 (oldest deadline 2023-12-11T18:05:31.835855+0000) 2023-12-11T18:05:51.654 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:05:51 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:05:51.377+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.181:6816 osd.2 since back 2023-12-11T18:05:16.536946+0000 front 2023-12-11T18:05:16.537012+0000 (oldest deadline 2023-12-11T18:05:42.436997+0000) 2023-12-11T18:05:51.654 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:05:51 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:05:51.377+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2023-12-11T18:05:23.538131+0000 front 2023-12-11T18:05:23.538151+0000 (oldest deadline 2023-12-11T18:05:48.838365+0000) 2023-12-11T18:05:52.032 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:05:51 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:05:51.660+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:59.875115+0000 front 2023-12-11T18:04:59.874965+0000 (oldest deadline 2023-12-11T18:05:22.775102+0000) 2023-12-11T18:05:52.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:05:51 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:05:51.660+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:02.775745+0000 front 2023-12-11T18:05:02.775784+0000 (oldest deadline 2023-12-11T18:05:33.376735+0000) 2023-12-11T18:05:52.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:05:51 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:05:51.660+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.181:6816 osd.2 since back 2023-12-11T18:05:16.778632+0000 front 2023-12-11T18:05:16.778462+0000 (oldest deadline 2023-12-11T18:05:41.478412+0000) 2023-12-11T18:05:52.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:05:51 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:05:51.660+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2023-12-11T18:05:21.479017+0000 front 2023-12-11T18:05:21.479156+0000 (oldest deadline 2023-12-11T18:05:47.879527+0000) 2023-12-11T18:05:52.174 DEBUG:teuthology.orchestra.run.smithi059:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d0a07eaa-984b-11ee-95a3-87774f69a715@osd.6.service' 2023-12-11T18:05:52.217 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-11T18:05:52.217 INFO:tasks.cephadm.osd.6:Stopped osd.6 2023-12-11T18:05:52.217 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2023-12-11T18:05:52.217 DEBUG:teuthology.orchestra.run.smithi115:> sudo systemctl stop ceph-d0a07eaa-984b-11ee-95a3-87774f69a715@osd.7 2023-12-11T18:05:52.533 INFO:journalctl@ceph.osd.7.smithi115.stdout:Dec 11 18:05:52 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-7[124665]: 2023-12-11T18:05:52.051+0000 7f9a71cae700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:05.284230+0000 front 2023-12-11T18:05:05.284410+0000 (oldest deadline 2023-12-11T18:05:31.184416+0000) 2023-12-11T18:05:52.533 INFO:journalctl@ceph.osd.7.smithi115.stdout:Dec 11 18:05:52 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-7[124665]: 2023-12-11T18:05:52.051+0000 7f9a71cae700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.181:6816 osd.2 since back 2023-12-11T18:05:18.186375+0000 front 2023-12-11T18:05:18.186338+0000 (oldest deadline 2023-12-11T18:05:42.886186+0000) 2023-12-11T18:05:52.533 INFO:journalctl@ceph.osd.7.smithi115.stdout:Dec 11 18:05:52 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-7[124665]: 2023-12-11T18:05:52.051+0000 7f9a71cae700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2023-12-11T18:05:24.587487+0000 front 2023-12-11T18:05:24.587500+0000 (oldest deadline 2023-12-11T18:05:48.687232+0000) 2023-12-11T18:05:52.533 INFO:journalctl@ceph.osd.7.smithi115.stdout:Dec 11 18:05:52 smithi115 systemd[1]: Stopping Ceph osd.7 for d0a07eaa-984b-11ee-95a3-87774f69a715... 2023-12-11T18:05:52.654 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:05:52 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:05:52.413+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:58.934598+0000 front 2023-12-11T18:04:58.934735+0000 (oldest deadline 2023-12-11T18:05:21.834661+0000) 2023-12-11T18:05:52.654 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:05:52 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:05:52.413+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:06.536080+0000 front 2023-12-11T18:05:06.535905+0000 (oldest deadline 2023-12-11T18:05:31.835855+0000) 2023-12-11T18:05:52.654 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:05:52 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:05:52.413+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.181:6816 osd.2 since back 2023-12-11T18:05:16.536946+0000 front 2023-12-11T18:05:16.537012+0000 (oldest deadline 2023-12-11T18:05:42.436997+0000) 2023-12-11T18:05:52.654 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:05:52 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:05:52.413+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2023-12-11T18:05:23.538131+0000 front 2023-12-11T18:05:23.538151+0000 (oldest deadline 2023-12-11T18:05:48.838365+0000) 2023-12-11T18:05:53.007 INFO:journalctl@ceph.osd.7.smithi115.stdout:Dec 11 18:05:52 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-7[124665]: 2023-12-11T18:05:52.592+0000 7f9a76eb6700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-11T18:05:53.008 INFO:journalctl@ceph.osd.7.smithi115.stdout:Dec 11 18:05:52 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-7[124665]: 2023-12-11T18:05:52.592+0000 7f9a76eb6700 -1 osd.7 123 *** Got signal Terminated *** 2023-12-11T18:05:53.008 INFO:journalctl@ceph.osd.7.smithi115.stdout:Dec 11 18:05:52 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-7[124665]: 2023-12-11T18:05:52.592+0000 7f9a76eb6700 -1 osd.7 123 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-12-11T18:05:53.008 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:05:52 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:05:52.670+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:59.875115+0000 front 2023-12-11T18:04:59.874965+0000 (oldest deadline 2023-12-11T18:05:22.775102+0000) 2023-12-11T18:05:53.008 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:05:52 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:05:52.670+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:02.775745+0000 front 2023-12-11T18:05:02.775784+0000 (oldest deadline 2023-12-11T18:05:33.376735+0000) 2023-12-11T18:05:53.008 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:05:52 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:05:52.670+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.181:6816 osd.2 since back 2023-12-11T18:05:16.778632+0000 front 2023-12-11T18:05:16.778462+0000 (oldest deadline 2023-12-11T18:05:41.478412+0000) 2023-12-11T18:05:53.008 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:05:52 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:05:52.670+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2023-12-11T18:05:21.479017+0000 front 2023-12-11T18:05:21.479156+0000 (oldest deadline 2023-12-11T18:05:47.879527+0000) 2023-12-11T18:05:53.282 INFO:journalctl@ceph.osd.7.smithi115.stdout:Dec 11 18:05:53 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-7[124665]: 2023-12-11T18:05:53.005+0000 7f9a71cae700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:05.284230+0000 front 2023-12-11T18:05:05.284410+0000 (oldest deadline 2023-12-11T18:05:31.184416+0000) 2023-12-11T18:05:53.283 INFO:journalctl@ceph.osd.7.smithi115.stdout:Dec 11 18:05:53 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-7[124665]: 2023-12-11T18:05:53.005+0000 7f9a71cae700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.181:6816 osd.2 since back 2023-12-11T18:05:18.186375+0000 front 2023-12-11T18:05:18.186338+0000 (oldest deadline 2023-12-11T18:05:42.886186+0000) 2023-12-11T18:05:53.283 INFO:journalctl@ceph.osd.7.smithi115.stdout:Dec 11 18:05:53 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-7[124665]: 2023-12-11T18:05:53.005+0000 7f9a71cae700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2023-12-11T18:05:24.587487+0000 front 2023-12-11T18:05:24.587500+0000 (oldest deadline 2023-12-11T18:05:48.687232+0000) 2023-12-11T18:05:53.904 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:05:53 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:05:53.409+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:58.934598+0000 front 2023-12-11T18:04:58.934735+0000 (oldest deadline 2023-12-11T18:05:21.834661+0000) 2023-12-11T18:05:53.904 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:05:53 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:05:53.409+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:06.536080+0000 front 2023-12-11T18:05:06.535905+0000 (oldest deadline 2023-12-11T18:05:31.835855+0000) 2023-12-11T18:05:53.904 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:05:53 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:05:53.409+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.181:6816 osd.2 since back 2023-12-11T18:05:16.536946+0000 front 2023-12-11T18:05:16.537012+0000 (oldest deadline 2023-12-11T18:05:42.436997+0000) 2023-12-11T18:05:53.904 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:05:53 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:05:53.409+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2023-12-11T18:05:23.538131+0000 front 2023-12-11T18:05:23.538151+0000 (oldest deadline 2023-12-11T18:05:48.838365+0000) 2023-12-11T18:05:54.026 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:05:53 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:05:53.652+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:59.875115+0000 front 2023-12-11T18:04:59.874965+0000 (oldest deadline 2023-12-11T18:05:22.775102+0000) 2023-12-11T18:05:54.026 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:05:53 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:05:53.652+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:02.775745+0000 front 2023-12-11T18:05:02.775784+0000 (oldest deadline 2023-12-11T18:05:33.376735+0000) 2023-12-11T18:05:54.026 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:05:53 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:05:53.652+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.181:6816 osd.2 since back 2023-12-11T18:05:16.778632+0000 front 2023-12-11T18:05:16.778462+0000 (oldest deadline 2023-12-11T18:05:41.478412+0000) 2023-12-11T18:05:54.026 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:05:53 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:05:53.652+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2023-12-11T18:05:21.479017+0000 front 2023-12-11T18:05:21.479156+0000 (oldest deadline 2023-12-11T18:05:47.879527+0000) 2023-12-11T18:05:54.283 INFO:journalctl@ceph.osd.7.smithi115.stdout:Dec 11 18:05:54 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-7[124665]: 2023-12-11T18:05:54.023+0000 7f9a71cae700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:05.284230+0000 front 2023-12-11T18:05:05.284410+0000 (oldest deadline 2023-12-11T18:05:31.184416+0000) 2023-12-11T18:05:54.283 INFO:journalctl@ceph.osd.7.smithi115.stdout:Dec 11 18:05:54 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-7[124665]: 2023-12-11T18:05:54.023+0000 7f9a71cae700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.181:6816 osd.2 since back 2023-12-11T18:05:18.186375+0000 front 2023-12-11T18:05:18.186338+0000 (oldest deadline 2023-12-11T18:05:42.886186+0000) 2023-12-11T18:05:54.283 INFO:journalctl@ceph.osd.7.smithi115.stdout:Dec 11 18:05:54 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-7[124665]: 2023-12-11T18:05:54.023+0000 7f9a71cae700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2023-12-11T18:05:24.587487+0000 front 2023-12-11T18:05:24.587500+0000 (oldest deadline 2023-12-11T18:05:48.687232+0000) 2023-12-11T18:05:54.283 INFO:journalctl@ceph.osd.7.smithi115.stdout:Dec 11 18:05:54 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-7[124665]: 2023-12-11T18:05:54.023+0000 7f9a71cae700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2023-12-11T18:05:28.687579+0000 front 2023-12-11T18:05:28.687554+0000 (oldest deadline 2023-12-11T18:05:53.987711+0000) 2023-12-11T18:05:54.654 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:05:54 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:05:54.360+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:58.934598+0000 front 2023-12-11T18:04:58.934735+0000 (oldest deadline 2023-12-11T18:05:21.834661+0000) 2023-12-11T18:05:54.654 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:05:54 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:05:54.360+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:06.536080+0000 front 2023-12-11T18:05:06.535905+0000 (oldest deadline 2023-12-11T18:05:31.835855+0000) 2023-12-11T18:05:54.654 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:05:54 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:05:54.360+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.181:6816 osd.2 since back 2023-12-11T18:05:16.536946+0000 front 2023-12-11T18:05:16.537012+0000 (oldest deadline 2023-12-11T18:05:42.436997+0000) 2023-12-11T18:05:54.654 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:05:54 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:05:54.360+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2023-12-11T18:05:23.538131+0000 front 2023-12-11T18:05:23.538151+0000 (oldest deadline 2023-12-11T18:05:48.838365+0000) 2023-12-11T18:05:54.654 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:05:54 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:05:54.360+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2023-12-11T18:05:28.838946+0000 front 2023-12-11T18:05:28.838926+0000 (oldest deadline 2023-12-11T18:05:54.139025+0000) 2023-12-11T18:05:55.032 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:05:54 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:05:54.609+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:59.875115+0000 front 2023-12-11T18:04:59.874965+0000 (oldest deadline 2023-12-11T18:05:22.775102+0000) 2023-12-11T18:05:55.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:05:54 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:05:54.609+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:02.775745+0000 front 2023-12-11T18:05:02.775784+0000 (oldest deadline 2023-12-11T18:05:33.376735+0000) 2023-12-11T18:05:55.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:05:54 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:05:54.609+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.181:6816 osd.2 since back 2023-12-11T18:05:16.778632+0000 front 2023-12-11T18:05:16.778462+0000 (oldest deadline 2023-12-11T18:05:41.478412+0000) 2023-12-11T18:05:55.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:05:54 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:05:54.609+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2023-12-11T18:05:21.479017+0000 front 2023-12-11T18:05:21.479156+0000 (oldest deadline 2023-12-11T18:05:47.879527+0000) 2023-12-11T18:05:55.533 INFO:journalctl@ceph.osd.7.smithi115.stdout:Dec 11 18:05:55 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-7[124665]: 2023-12-11T18:05:55.062+0000 7f9a71cae700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:05.284230+0000 front 2023-12-11T18:05:05.284410+0000 (oldest deadline 2023-12-11T18:05:31.184416+0000) 2023-12-11T18:05:55.533 INFO:journalctl@ceph.osd.7.smithi115.stdout:Dec 11 18:05:55 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-7[124665]: 2023-12-11T18:05:55.062+0000 7f9a71cae700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.181:6816 osd.2 since back 2023-12-11T18:05:18.186375+0000 front 2023-12-11T18:05:18.186338+0000 (oldest deadline 2023-12-11T18:05:42.886186+0000) 2023-12-11T18:05:55.533 INFO:journalctl@ceph.osd.7.smithi115.stdout:Dec 11 18:05:55 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-7[124665]: 2023-12-11T18:05:55.062+0000 7f9a71cae700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2023-12-11T18:05:24.587487+0000 front 2023-12-11T18:05:24.587500+0000 (oldest deadline 2023-12-11T18:05:48.687232+0000) 2023-12-11T18:05:55.533 INFO:journalctl@ceph.osd.7.smithi115.stdout:Dec 11 18:05:55 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-7[124665]: 2023-12-11T18:05:55.062+0000 7f9a71cae700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2023-12-11T18:05:28.687579+0000 front 2023-12-11T18:05:28.687554+0000 (oldest deadline 2023-12-11T18:05:53.987711+0000) 2023-12-11T18:05:55.654 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:05:55 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:05:55.393+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:58.934598+0000 front 2023-12-11T18:04:58.934735+0000 (oldest deadline 2023-12-11T18:05:21.834661+0000) 2023-12-11T18:05:55.654 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:05:55 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:05:55.393+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:06.536080+0000 front 2023-12-11T18:05:06.535905+0000 (oldest deadline 2023-12-11T18:05:31.835855+0000) 2023-12-11T18:05:55.654 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:05:55 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:05:55.393+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.181:6816 osd.2 since back 2023-12-11T18:05:16.536946+0000 front 2023-12-11T18:05:16.537012+0000 (oldest deadline 2023-12-11T18:05:42.436997+0000) 2023-12-11T18:05:55.654 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:05:55 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:05:55.393+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2023-12-11T18:05:23.538131+0000 front 2023-12-11T18:05:23.538151+0000 (oldest deadline 2023-12-11T18:05:48.838365+0000) 2023-12-11T18:05:55.654 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:05:55 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:05:55.393+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2023-12-11T18:05:28.838946+0000 front 2023-12-11T18:05:28.838926+0000 (oldest deadline 2023-12-11T18:05:54.139025+0000) 2023-12-11T18:05:56.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:05:55 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:05:55.638+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:59.875115+0000 front 2023-12-11T18:04:59.874965+0000 (oldest deadline 2023-12-11T18:05:22.775102+0000) 2023-12-11T18:05:56.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:05:55 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:05:55.638+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:02.775745+0000 front 2023-12-11T18:05:02.775784+0000 (oldest deadline 2023-12-11T18:05:33.376735+0000) 2023-12-11T18:05:56.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:05:55 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:05:55.638+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.181:6816 osd.2 since back 2023-12-11T18:05:16.778632+0000 front 2023-12-11T18:05:16.778462+0000 (oldest deadline 2023-12-11T18:05:41.478412+0000) 2023-12-11T18:05:56.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:05:55 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:05:55.638+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2023-12-11T18:05:21.479017+0000 front 2023-12-11T18:05:21.479156+0000 (oldest deadline 2023-12-11T18:05:47.879527+0000) 2023-12-11T18:05:56.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:05:55 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:05:55.638+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2023-12-11T18:05:30.180654+0000 front 2023-12-11T18:05:30.180627+0000 (oldest deadline 2023-12-11T18:05:54.880677+0000) 2023-12-11T18:05:56.532 INFO:journalctl@ceph.osd.7.smithi115.stdout:Dec 11 18:05:56 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-7[124665]: 2023-12-11T18:05:56.108+0000 7f9a71cae700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:05.284230+0000 front 2023-12-11T18:05:05.284410+0000 (oldest deadline 2023-12-11T18:05:31.184416+0000) 2023-12-11T18:05:56.533 INFO:journalctl@ceph.osd.7.smithi115.stdout:Dec 11 18:05:56 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-7[124665]: 2023-12-11T18:05:56.108+0000 7f9a71cae700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.181:6816 osd.2 since back 2023-12-11T18:05:18.186375+0000 front 2023-12-11T18:05:18.186338+0000 (oldest deadline 2023-12-11T18:05:42.886186+0000) 2023-12-11T18:05:56.533 INFO:journalctl@ceph.osd.7.smithi115.stdout:Dec 11 18:05:56 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-7[124665]: 2023-12-11T18:05:56.108+0000 7f9a71cae700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2023-12-11T18:05:24.587487+0000 front 2023-12-11T18:05:24.587500+0000 (oldest deadline 2023-12-11T18:05:48.687232+0000) 2023-12-11T18:05:56.533 INFO:journalctl@ceph.osd.7.smithi115.stdout:Dec 11 18:05:56 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-7[124665]: 2023-12-11T18:05:56.108+0000 7f9a71cae700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2023-12-11T18:05:28.687579+0000 front 2023-12-11T18:05:28.687554+0000 (oldest deadline 2023-12-11T18:05:53.987711+0000) 2023-12-11T18:05:56.904 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:05:56 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:05:56.434+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:58.934598+0000 front 2023-12-11T18:04:58.934735+0000 (oldest deadline 2023-12-11T18:05:21.834661+0000) 2023-12-11T18:05:56.904 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:05:56 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:05:56.434+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:06.536080+0000 front 2023-12-11T18:05:06.535905+0000 (oldest deadline 2023-12-11T18:05:31.835855+0000) 2023-12-11T18:05:56.904 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:05:56 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:05:56.434+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.181:6816 osd.2 since back 2023-12-11T18:05:16.536946+0000 front 2023-12-11T18:05:16.537012+0000 (oldest deadline 2023-12-11T18:05:42.436997+0000) 2023-12-11T18:05:56.904 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:05:56 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:05:56.434+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2023-12-11T18:05:23.538131+0000 front 2023-12-11T18:05:23.538151+0000 (oldest deadline 2023-12-11T18:05:48.838365+0000) 2023-12-11T18:05:56.904 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:05:56 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:05:56.434+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2023-12-11T18:05:28.838946+0000 front 2023-12-11T18:05:28.838926+0000 (oldest deadline 2023-12-11T18:05:54.139025+0000) 2023-12-11T18:05:57.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:05:56 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:05:56.667+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:59.875115+0000 front 2023-12-11T18:04:59.874965+0000 (oldest deadline 2023-12-11T18:05:22.775102+0000) 2023-12-11T18:05:57.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:05:56 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:05:56.667+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:02.775745+0000 front 2023-12-11T18:05:02.775784+0000 (oldest deadline 2023-12-11T18:05:33.376735+0000) 2023-12-11T18:05:57.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:05:56 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:05:56.667+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.181:6816 osd.2 since back 2023-12-11T18:05:16.778632+0000 front 2023-12-11T18:05:16.778462+0000 (oldest deadline 2023-12-11T18:05:41.478412+0000) 2023-12-11T18:05:57.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:05:56 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:05:56.667+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2023-12-11T18:05:21.479017+0000 front 2023-12-11T18:05:21.479156+0000 (oldest deadline 2023-12-11T18:05:47.879527+0000) 2023-12-11T18:05:57.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:05:56 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:05:56.668+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2023-12-11T18:05:30.180654+0000 front 2023-12-11T18:05:30.180627+0000 (oldest deadline 2023-12-11T18:05:54.880677+0000) 2023-12-11T18:05:57.532 INFO:journalctl@ceph.osd.7.smithi115.stdout:Dec 11 18:05:57 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-7[124665]: 2023-12-11T18:05:57.059+0000 7f9a71cae700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:05.284230+0000 front 2023-12-11T18:05:05.284410+0000 (oldest deadline 2023-12-11T18:05:31.184416+0000) 2023-12-11T18:05:57.533 INFO:journalctl@ceph.osd.7.smithi115.stdout:Dec 11 18:05:57 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-7[124665]: 2023-12-11T18:05:57.059+0000 7f9a71cae700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.181:6816 osd.2 since back 2023-12-11T18:05:18.186375+0000 front 2023-12-11T18:05:18.186338+0000 (oldest deadline 2023-12-11T18:05:42.886186+0000) 2023-12-11T18:05:57.533 INFO:journalctl@ceph.osd.7.smithi115.stdout:Dec 11 18:05:57 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-7[124665]: 2023-12-11T18:05:57.059+0000 7f9a71cae700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2023-12-11T18:05:24.587487+0000 front 2023-12-11T18:05:24.587500+0000 (oldest deadline 2023-12-11T18:05:48.687232+0000) 2023-12-11T18:05:57.533 INFO:journalctl@ceph.osd.7.smithi115.stdout:Dec 11 18:05:57 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-7[124665]: 2023-12-11T18:05:57.059+0000 7f9a71cae700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2023-12-11T18:05:28.687579+0000 front 2023-12-11T18:05:28.687554+0000 (oldest deadline 2023-12-11T18:05:53.987711+0000) 2023-12-11T18:05:57.654 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:05:57 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:05:57.387+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:58.934598+0000 front 2023-12-11T18:04:58.934735+0000 (oldest deadline 2023-12-11T18:05:21.834661+0000) 2023-12-11T18:05:57.654 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:05:57 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:05:57.387+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:06.536080+0000 front 2023-12-11T18:05:06.535905+0000 (oldest deadline 2023-12-11T18:05:31.835855+0000) 2023-12-11T18:05:57.654 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:05:57 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:05:57.387+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.181:6816 osd.2 since back 2023-12-11T18:05:16.536946+0000 front 2023-12-11T18:05:16.537012+0000 (oldest deadline 2023-12-11T18:05:42.436997+0000) 2023-12-11T18:05:57.654 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:05:57 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:05:57.387+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2023-12-11T18:05:23.538131+0000 front 2023-12-11T18:05:23.538151+0000 (oldest deadline 2023-12-11T18:05:48.838365+0000) 2023-12-11T18:05:57.654 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:05:57 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:05:57.387+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2023-12-11T18:05:28.838946+0000 front 2023-12-11T18:05:28.838926+0000 (oldest deadline 2023-12-11T18:05:54.139025+0000) 2023-12-11T18:05:58.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:05:57 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:05:57.692+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:59.875115+0000 front 2023-12-11T18:04:59.874965+0000 (oldest deadline 2023-12-11T18:05:22.775102+0000) 2023-12-11T18:05:58.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:05:57 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:05:57.692+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:02.775745+0000 front 2023-12-11T18:05:02.775784+0000 (oldest deadline 2023-12-11T18:05:33.376735+0000) 2023-12-11T18:05:58.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:05:57 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:05:57.692+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.181:6816 osd.2 since back 2023-12-11T18:05:16.778632+0000 front 2023-12-11T18:05:16.778462+0000 (oldest deadline 2023-12-11T18:05:41.478412+0000) 2023-12-11T18:05:58.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:05:57 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:05:57.692+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2023-12-11T18:05:21.479017+0000 front 2023-12-11T18:05:21.479156+0000 (oldest deadline 2023-12-11T18:05:47.879527+0000) 2023-12-11T18:05:58.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:05:57 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:05:57.692+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2023-12-11T18:05:30.180654+0000 front 2023-12-11T18:05:30.180627+0000 (oldest deadline 2023-12-11T18:05:54.880677+0000) 2023-12-11T18:05:58.356 INFO:journalctl@ceph.osd.7.smithi115.stdout:Dec 11 18:05:58 smithi115 bash[147597]: ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-7 2023-12-11T18:05:58.654 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:05:58 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:05:58.373+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:58.934598+0000 front 2023-12-11T18:04:58.934735+0000 (oldest deadline 2023-12-11T18:05:21.834661+0000) 2023-12-11T18:05:58.654 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:05:58 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:05:58.373+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:06.536080+0000 front 2023-12-11T18:05:06.535905+0000 (oldest deadline 2023-12-11T18:05:31.835855+0000) 2023-12-11T18:05:58.654 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:05:58 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:05:58.373+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.181:6816 osd.2 since back 2023-12-11T18:05:16.536946+0000 front 2023-12-11T18:05:16.537012+0000 (oldest deadline 2023-12-11T18:05:42.436997+0000) 2023-12-11T18:05:58.654 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:05:58 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:05:58.373+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2023-12-11T18:05:23.538131+0000 front 2023-12-11T18:05:23.538151+0000 (oldest deadline 2023-12-11T18:05:48.838365+0000) 2023-12-11T18:05:58.654 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:05:58 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:05:58.373+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2023-12-11T18:05:28.838946+0000 front 2023-12-11T18:05:28.838926+0000 (oldest deadline 2023-12-11T18:05:54.139025+0000) 2023-12-11T18:05:58.947 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:05:58 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:05:58.677+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:59.875115+0000 front 2023-12-11T18:04:59.874965+0000 (oldest deadline 2023-12-11T18:05:22.775102+0000) 2023-12-11T18:05:58.947 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:05:58 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:05:58.678+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:02.775745+0000 front 2023-12-11T18:05:02.775784+0000 (oldest deadline 2023-12-11T18:05:33.376735+0000) 2023-12-11T18:05:58.947 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:05:58 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:05:58.678+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.181:6816 osd.2 since back 2023-12-11T18:05:16.778632+0000 front 2023-12-11T18:05:16.778462+0000 (oldest deadline 2023-12-11T18:05:41.478412+0000) 2023-12-11T18:05:58.947 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:05:58 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:05:58.678+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2023-12-11T18:05:21.479017+0000 front 2023-12-11T18:05:21.479156+0000 (oldest deadline 2023-12-11T18:05:47.879527+0000) 2023-12-11T18:05:58.947 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:05:58 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:05:58.678+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2023-12-11T18:05:30.180654+0000 front 2023-12-11T18:05:30.180627+0000 (oldest deadline 2023-12-11T18:05:54.880677+0000) 2023-12-11T18:05:59.654 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:05:59 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:05:59.374+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:58.934598+0000 front 2023-12-11T18:04:58.934735+0000 (oldest deadline 2023-12-11T18:05:21.834661+0000) 2023-12-11T18:05:59.654 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:05:59 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:05:59.374+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:06.536080+0000 front 2023-12-11T18:05:06.535905+0000 (oldest deadline 2023-12-11T18:05:31.835855+0000) 2023-12-11T18:05:59.654 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:05:59 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:05:59.374+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.181:6816 osd.2 since back 2023-12-11T18:05:16.536946+0000 front 2023-12-11T18:05:16.537012+0000 (oldest deadline 2023-12-11T18:05:42.436997+0000) 2023-12-11T18:05:59.654 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:05:59 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:05:59.374+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2023-12-11T18:05:23.538131+0000 front 2023-12-11T18:05:23.538151+0000 (oldest deadline 2023-12-11T18:05:48.838365+0000) 2023-12-11T18:05:59.654 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:05:59 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:05:59.374+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2023-12-11T18:05:28.838946+0000 front 2023-12-11T18:05:28.838926+0000 (oldest deadline 2023-12-11T18:05:54.139025+0000) 2023-12-11T18:05:59.714 DEBUG:teuthology.orchestra.run.smithi115:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d0a07eaa-984b-11ee-95a3-87774f69a715@osd.7.service' 2023-12-11T18:05:59.755 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-11T18:05:59.755 INFO:tasks.cephadm.osd.7:Stopped osd.7 2023-12-11T18:05:59.755 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2023-12-11T18:05:59.755 DEBUG:teuthology.orchestra.run.smithi181:> sudo systemctl stop ceph-d0a07eaa-984b-11ee-95a3-87774f69a715@osd.8 2023-12-11T18:06:00.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:05:59 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:05:59.720+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:59.875115+0000 front 2023-12-11T18:04:59.874965+0000 (oldest deadline 2023-12-11T18:05:22.775102+0000) 2023-12-11T18:06:00.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:05:59 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:05:59.720+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:02.775745+0000 front 2023-12-11T18:05:02.775784+0000 (oldest deadline 2023-12-11T18:05:33.376735+0000) 2023-12-11T18:06:00.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:05:59 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:05:59.720+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.181:6816 osd.2 since back 2023-12-11T18:05:16.778632+0000 front 2023-12-11T18:05:16.778462+0000 (oldest deadline 2023-12-11T18:05:41.478412+0000) 2023-12-11T18:06:00.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:05:59 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:05:59.720+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2023-12-11T18:05:21.479017+0000 front 2023-12-11T18:05:21.479156+0000 (oldest deadline 2023-12-11T18:05:47.879527+0000) 2023-12-11T18:06:00.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:05:59 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:05:59.720+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2023-12-11T18:05:30.180654+0000 front 2023-12-11T18:05:30.180627+0000 (oldest deadline 2023-12-11T18:05:54.880677+0000) 2023-12-11T18:06:00.654 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:06:00 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:06:00.332+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:58.934598+0000 front 2023-12-11T18:04:58.934735+0000 (oldest deadline 2023-12-11T18:05:21.834661+0000) 2023-12-11T18:06:00.654 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:06:00 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:06:00.332+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:06.536080+0000 front 2023-12-11T18:05:06.535905+0000 (oldest deadline 2023-12-11T18:05:31.835855+0000) 2023-12-11T18:06:00.654 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:06:00 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:06:00.332+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.181:6816 osd.2 since back 2023-12-11T18:05:16.536946+0000 front 2023-12-11T18:05:16.537012+0000 (oldest deadline 2023-12-11T18:05:42.436997+0000) 2023-12-11T18:06:00.654 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:06:00 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:06:00.332+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2023-12-11T18:05:23.538131+0000 front 2023-12-11T18:05:23.538151+0000 (oldest deadline 2023-12-11T18:05:48.838365+0000) 2023-12-11T18:06:00.654 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:06:00 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:06:00.332+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2023-12-11T18:05:28.838946+0000 front 2023-12-11T18:05:28.838926+0000 (oldest deadline 2023-12-11T18:05:54.139025+0000) 2023-12-11T18:06:01.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:06:00 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:06:00.693+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:59.875115+0000 front 2023-12-11T18:04:59.874965+0000 (oldest deadline 2023-12-11T18:05:22.775102+0000) 2023-12-11T18:06:01.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:06:00 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:06:00.693+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:02.775745+0000 front 2023-12-11T18:05:02.775784+0000 (oldest deadline 2023-12-11T18:05:33.376735+0000) 2023-12-11T18:06:01.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:06:00 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:06:00.693+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.181:6816 osd.2 since back 2023-12-11T18:05:16.778632+0000 front 2023-12-11T18:05:16.778462+0000 (oldest deadline 2023-12-11T18:05:41.478412+0000) 2023-12-11T18:06:01.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:06:00 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:06:00.693+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2023-12-11T18:05:21.479017+0000 front 2023-12-11T18:05:21.479156+0000 (oldest deadline 2023-12-11T18:05:47.879527+0000) 2023-12-11T18:06:01.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:06:00 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:06:00.693+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2023-12-11T18:05:30.180654+0000 front 2023-12-11T18:05:30.180627+0000 (oldest deadline 2023-12-11T18:05:54.880677+0000) 2023-12-11T18:06:01.654 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:06:01 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:06:01.333+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:58.934598+0000 front 2023-12-11T18:04:58.934735+0000 (oldest deadline 2023-12-11T18:05:21.834661+0000) 2023-12-11T18:06:01.654 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:06:01 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:06:01.333+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:06.536080+0000 front 2023-12-11T18:05:06.535905+0000 (oldest deadline 2023-12-11T18:05:31.835855+0000) 2023-12-11T18:06:01.654 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:06:01 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:06:01.333+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.181:6816 osd.2 since back 2023-12-11T18:05:16.536946+0000 front 2023-12-11T18:05:16.537012+0000 (oldest deadline 2023-12-11T18:05:42.436997+0000) 2023-12-11T18:06:01.654 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:06:01 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:06:01.333+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2023-12-11T18:05:23.538131+0000 front 2023-12-11T18:05:23.538151+0000 (oldest deadline 2023-12-11T18:05:48.838365+0000) 2023-12-11T18:06:01.654 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:06:01 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:06:01.333+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2023-12-11T18:05:28.838946+0000 front 2023-12-11T18:05:28.838926+0000 (oldest deadline 2023-12-11T18:05:54.139025+0000) 2023-12-11T18:06:02.032 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:06:01 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:06:01.723+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:59.875115+0000 front 2023-12-11T18:04:59.874965+0000 (oldest deadline 2023-12-11T18:05:22.775102+0000) 2023-12-11T18:06:02.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:06:01 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:06:01.723+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:02.775745+0000 front 2023-12-11T18:05:02.775784+0000 (oldest deadline 2023-12-11T18:05:33.376735+0000) 2023-12-11T18:06:02.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:06:01 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:06:01.723+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.181:6816 osd.2 since back 2023-12-11T18:05:16.778632+0000 front 2023-12-11T18:05:16.778462+0000 (oldest deadline 2023-12-11T18:05:41.478412+0000) 2023-12-11T18:06:02.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:06:01 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:06:01.723+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2023-12-11T18:05:21.479017+0000 front 2023-12-11T18:05:21.479156+0000 (oldest deadline 2023-12-11T18:05:47.879527+0000) 2023-12-11T18:06:02.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:06:01 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:06:01.723+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2023-12-11T18:05:30.180654+0000 front 2023-12-11T18:05:30.180627+0000 (oldest deadline 2023-12-11T18:05:54.880677+0000) 2023-12-11T18:06:02.654 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:06:02 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:06:02.287+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:58.934598+0000 front 2023-12-11T18:04:58.934735+0000 (oldest deadline 2023-12-11T18:05:21.834661+0000) 2023-12-11T18:06:02.654 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:06:02 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:06:02.287+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:06.536080+0000 front 2023-12-11T18:05:06.535905+0000 (oldest deadline 2023-12-11T18:05:31.835855+0000) 2023-12-11T18:06:02.654 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:06:02 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:06:02.287+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.181:6816 osd.2 since back 2023-12-11T18:05:16.536946+0000 front 2023-12-11T18:05:16.537012+0000 (oldest deadline 2023-12-11T18:05:42.436997+0000) 2023-12-11T18:06:02.654 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:06:02 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:06:02.287+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2023-12-11T18:05:23.538131+0000 front 2023-12-11T18:05:23.538151+0000 (oldest deadline 2023-12-11T18:05:48.838365+0000) 2023-12-11T18:06:02.654 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:06:02 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:06:02.287+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2023-12-11T18:05:28.838946+0000 front 2023-12-11T18:05:28.838926+0000 (oldest deadline 2023-12-11T18:05:54.139025+0000) 2023-12-11T18:06:03.032 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:06:02 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:06:02.704+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:59.875115+0000 front 2023-12-11T18:04:59.874965+0000 (oldest deadline 2023-12-11T18:05:22.775102+0000) 2023-12-11T18:06:03.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:06:02 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:06:02.704+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:02.775745+0000 front 2023-12-11T18:05:02.775784+0000 (oldest deadline 2023-12-11T18:05:33.376735+0000) 2023-12-11T18:06:03.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:06:02 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:06:02.704+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.181:6816 osd.2 since back 2023-12-11T18:05:16.778632+0000 front 2023-12-11T18:05:16.778462+0000 (oldest deadline 2023-12-11T18:05:41.478412+0000) 2023-12-11T18:06:03.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:06:02 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:06:02.704+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2023-12-11T18:05:21.479017+0000 front 2023-12-11T18:05:21.479156+0000 (oldest deadline 2023-12-11T18:05:47.879527+0000) 2023-12-11T18:06:03.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:06:02 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:06:02.704+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2023-12-11T18:05:30.180654+0000 front 2023-12-11T18:05:30.180627+0000 (oldest deadline 2023-12-11T18:05:54.880677+0000) 2023-12-11T18:06:03.654 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:06:03 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:06:03.244+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:58.934598+0000 front 2023-12-11T18:04:58.934735+0000 (oldest deadline 2023-12-11T18:05:21.834661+0000) 2023-12-11T18:06:03.654 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:06:03 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:06:03.244+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:06.536080+0000 front 2023-12-11T18:05:06.535905+0000 (oldest deadline 2023-12-11T18:05:31.835855+0000) 2023-12-11T18:06:03.654 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:06:03 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:06:03.244+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.181:6816 osd.2 since back 2023-12-11T18:05:16.536946+0000 front 2023-12-11T18:05:16.537012+0000 (oldest deadline 2023-12-11T18:05:42.436997+0000) 2023-12-11T18:06:03.654 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:06:03 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:06:03.244+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2023-12-11T18:05:23.538131+0000 front 2023-12-11T18:05:23.538151+0000 (oldest deadline 2023-12-11T18:05:48.838365+0000) 2023-12-11T18:06:03.655 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:06:03 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:06:03.244+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2023-12-11T18:05:28.838946+0000 front 2023-12-11T18:05:28.838926+0000 (oldest deadline 2023-12-11T18:05:54.139025+0000) 2023-12-11T18:06:03.655 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:06:03 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:06:03.244+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.181:6817 osd.5 since back 2023-12-11T18:05:40.541039+0000 front 2023-12-11T18:05:40.541215+0000 (oldest deadline 2023-12-11T18:06:02.840912+0000) 2023-12-11T18:06:04.032 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:06:03 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:06:03.702+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:59.875115+0000 front 2023-12-11T18:04:59.874965+0000 (oldest deadline 2023-12-11T18:05:22.775102+0000) 2023-12-11T18:06:04.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:06:03 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:06:03.702+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:02.775745+0000 front 2023-12-11T18:05:02.775784+0000 (oldest deadline 2023-12-11T18:05:33.376735+0000) 2023-12-11T18:06:04.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:06:03 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:06:03.702+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.181:6816 osd.2 since back 2023-12-11T18:05:16.778632+0000 front 2023-12-11T18:05:16.778462+0000 (oldest deadline 2023-12-11T18:05:41.478412+0000) 2023-12-11T18:06:04.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:06:03 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:06:03.702+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2023-12-11T18:05:21.479017+0000 front 2023-12-11T18:05:21.479156+0000 (oldest deadline 2023-12-11T18:05:47.879527+0000) 2023-12-11T18:06:04.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:06:03 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:06:03.702+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2023-12-11T18:05:30.180654+0000 front 2023-12-11T18:05:30.180627+0000 (oldest deadline 2023-12-11T18:05:54.880677+0000) 2023-12-11T18:06:04.654 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:06:04 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:06:04.233+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:58.934598+0000 front 2023-12-11T18:04:58.934735+0000 (oldest deadline 2023-12-11T18:05:21.834661+0000) 2023-12-11T18:06:04.654 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:06:04 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:06:04.233+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:06.536080+0000 front 2023-12-11T18:05:06.535905+0000 (oldest deadline 2023-12-11T18:05:31.835855+0000) 2023-12-11T18:06:04.654 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:06:04 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:06:04.233+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.181:6816 osd.2 since back 2023-12-11T18:05:16.536946+0000 front 2023-12-11T18:05:16.537012+0000 (oldest deadline 2023-12-11T18:05:42.436997+0000) 2023-12-11T18:06:04.654 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:06:04 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:06:04.233+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2023-12-11T18:05:23.538131+0000 front 2023-12-11T18:05:23.538151+0000 (oldest deadline 2023-12-11T18:05:48.838365+0000) 2023-12-11T18:06:04.654 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:06:04 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:06:04.233+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2023-12-11T18:05:28.838946+0000 front 2023-12-11T18:05:28.838926+0000 (oldest deadline 2023-12-11T18:05:54.139025+0000) 2023-12-11T18:06:04.654 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:06:04 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:06:04.233+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.181:6817 osd.5 since back 2023-12-11T18:05:40.541039+0000 front 2023-12-11T18:05:40.541215+0000 (oldest deadline 2023-12-11T18:06:02.840912+0000) 2023-12-11T18:06:05.032 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:06:04 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:06:04.734+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:59.875115+0000 front 2023-12-11T18:04:59.874965+0000 (oldest deadline 2023-12-11T18:05:22.775102+0000) 2023-12-11T18:06:05.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:06:04 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:06:04.734+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:02.775745+0000 front 2023-12-11T18:05:02.775784+0000 (oldest deadline 2023-12-11T18:05:33.376735+0000) 2023-12-11T18:06:05.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:06:04 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:06:04.734+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.181:6816 osd.2 since back 2023-12-11T18:05:16.778632+0000 front 2023-12-11T18:05:16.778462+0000 (oldest deadline 2023-12-11T18:05:41.478412+0000) 2023-12-11T18:06:05.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:06:04 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:06:04.734+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2023-12-11T18:05:21.479017+0000 front 2023-12-11T18:05:21.479156+0000 (oldest deadline 2023-12-11T18:05:47.879527+0000) 2023-12-11T18:06:05.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:06:04 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:06:04.734+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2023-12-11T18:05:30.180654+0000 front 2023-12-11T18:05:30.180627+0000 (oldest deadline 2023-12-11T18:05:54.880677+0000) 2023-12-11T18:06:05.654 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:06:05 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:06:05.224+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:58.934598+0000 front 2023-12-11T18:04:58.934735+0000 (oldest deadline 2023-12-11T18:05:21.834661+0000) 2023-12-11T18:06:05.654 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:06:05 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:06:05.224+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:06.536080+0000 front 2023-12-11T18:05:06.535905+0000 (oldest deadline 2023-12-11T18:05:31.835855+0000) 2023-12-11T18:06:05.654 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:06:05 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:06:05.224+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.181:6816 osd.2 since back 2023-12-11T18:05:16.536946+0000 front 2023-12-11T18:05:16.537012+0000 (oldest deadline 2023-12-11T18:05:42.436997+0000) 2023-12-11T18:06:05.654 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:06:05 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:06:05.224+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2023-12-11T18:05:23.538131+0000 front 2023-12-11T18:05:23.538151+0000 (oldest deadline 2023-12-11T18:05:48.838365+0000) 2023-12-11T18:06:05.655 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:06:05 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:06:05.224+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2023-12-11T18:05:28.838946+0000 front 2023-12-11T18:05:28.838926+0000 (oldest deadline 2023-12-11T18:05:54.139025+0000) 2023-12-11T18:06:05.655 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:06:05 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:06:05.224+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.181:6817 osd.5 since back 2023-12-11T18:05:40.541039+0000 front 2023-12-11T18:05:40.541215+0000 (oldest deadline 2023-12-11T18:06:02.840912+0000) 2023-12-11T18:06:06.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:06:05 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:06:05.769+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:59.875115+0000 front 2023-12-11T18:04:59.874965+0000 (oldest deadline 2023-12-11T18:05:22.775102+0000) 2023-12-11T18:06:06.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:06:05 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:06:05.769+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:02.775745+0000 front 2023-12-11T18:05:02.775784+0000 (oldest deadline 2023-12-11T18:05:33.376735+0000) 2023-12-11T18:06:06.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:06:05 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:06:05.769+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.181:6816 osd.2 since back 2023-12-11T18:05:16.778632+0000 front 2023-12-11T18:05:16.778462+0000 (oldest deadline 2023-12-11T18:05:41.478412+0000) 2023-12-11T18:06:06.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:06:05 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:06:05.769+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2023-12-11T18:05:21.479017+0000 front 2023-12-11T18:05:21.479156+0000 (oldest deadline 2023-12-11T18:05:47.879527+0000) 2023-12-11T18:06:06.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:06:05 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:06:05.769+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2023-12-11T18:05:30.180654+0000 front 2023-12-11T18:05:30.180627+0000 (oldest deadline 2023-12-11T18:05:54.880677+0000) 2023-12-11T18:06:06.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:06:05 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:06:05.769+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.181:6817 osd.5 since back 2023-12-11T18:05:35.981809+0000 front 2023-12-11T18:05:35.981821+0000 (oldest deadline 2023-12-11T18:06:04.782070+0000) 2023-12-11T18:06:06.654 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:06:06 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:06:06.212+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:58.934598+0000 front 2023-12-11T18:04:58.934735+0000 (oldest deadline 2023-12-11T18:05:21.834661+0000) 2023-12-11T18:06:06.654 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:06:06 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:06:06.212+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:06.536080+0000 front 2023-12-11T18:05:06.535905+0000 (oldest deadline 2023-12-11T18:05:31.835855+0000) 2023-12-11T18:06:06.654 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:06:06 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:06:06.212+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.181:6816 osd.2 since back 2023-12-11T18:05:16.536946+0000 front 2023-12-11T18:05:16.537012+0000 (oldest deadline 2023-12-11T18:05:42.436997+0000) 2023-12-11T18:06:06.654 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:06:06 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:06:06.212+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2023-12-11T18:05:23.538131+0000 front 2023-12-11T18:05:23.538151+0000 (oldest deadline 2023-12-11T18:05:48.838365+0000) 2023-12-11T18:06:06.654 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:06:06 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:06:06.212+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2023-12-11T18:05:28.838946+0000 front 2023-12-11T18:05:28.838926+0000 (oldest deadline 2023-12-11T18:05:54.139025+0000) 2023-12-11T18:06:06.654 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:06:06 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:06:06.212+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.181:6817 osd.5 since back 2023-12-11T18:05:40.541039+0000 front 2023-12-11T18:05:40.541215+0000 (oldest deadline 2023-12-11T18:06:02.840912+0000) 2023-12-11T18:06:07.017 DEBUG:teuthology.orchestra.run.smithi181:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d0a07eaa-984b-11ee-95a3-87774f69a715@osd.8.service' 2023-12-11T18:06:07.032 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:06:06 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:06:06.768+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:59.875115+0000 front 2023-12-11T18:04:59.874965+0000 (oldest deadline 2023-12-11T18:05:22.775102+0000) 2023-12-11T18:06:07.032 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:06:06 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:06:06.768+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:02.775745+0000 front 2023-12-11T18:05:02.775784+0000 (oldest deadline 2023-12-11T18:05:33.376735+0000) 2023-12-11T18:06:07.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:06:06 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:06:06.768+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.181:6816 osd.2 since back 2023-12-11T18:05:16.778632+0000 front 2023-12-11T18:05:16.778462+0000 (oldest deadline 2023-12-11T18:05:41.478412+0000) 2023-12-11T18:06:07.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:06:06 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:06:06.768+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2023-12-11T18:05:21.479017+0000 front 2023-12-11T18:05:21.479156+0000 (oldest deadline 2023-12-11T18:05:47.879527+0000) 2023-12-11T18:06:07.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:06:06 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:06:06.768+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2023-12-11T18:05:30.180654+0000 front 2023-12-11T18:05:30.180627+0000 (oldest deadline 2023-12-11T18:05:54.880677+0000) 2023-12-11T18:06:07.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:06:06 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:06:06.768+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.181:6817 osd.5 since back 2023-12-11T18:05:35.981809+0000 front 2023-12-11T18:05:35.981821+0000 (oldest deadline 2023-12-11T18:06:04.782070+0000) 2023-12-11T18:06:07.057 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-11T18:06:07.057 INFO:tasks.cephadm.osd.8:Stopped osd.8 2023-12-11T18:06:07.057 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2023-12-11T18:06:07.057 DEBUG:teuthology.orchestra.run.smithi059:> sudo systemctl stop ceph-d0a07eaa-984b-11ee-95a3-87774f69a715@osd.9 2023-12-11T18:06:07.404 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:06:07 smithi059 systemd[1]: Stopping Ceph osd.9 for d0a07eaa-984b-11ee-95a3-87774f69a715... 2023-12-11T18:06:07.405 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:06:07 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:06:07.199+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:58.934598+0000 front 2023-12-11T18:04:58.934735+0000 (oldest deadline 2023-12-11T18:05:21.834661+0000) 2023-12-11T18:06:07.405 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:06:07 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:06:07.200+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:06.536080+0000 front 2023-12-11T18:05:06.535905+0000 (oldest deadline 2023-12-11T18:05:31.835855+0000) 2023-12-11T18:06:07.405 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:06:07 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:06:07.200+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.181:6816 osd.2 since back 2023-12-11T18:05:16.536946+0000 front 2023-12-11T18:05:16.537012+0000 (oldest deadline 2023-12-11T18:05:42.436997+0000) 2023-12-11T18:06:07.405 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:06:07 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:06:07.200+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2023-12-11T18:05:23.538131+0000 front 2023-12-11T18:05:23.538151+0000 (oldest deadline 2023-12-11T18:05:48.838365+0000) 2023-12-11T18:06:07.405 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:06:07 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:06:07.200+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2023-12-11T18:05:28.838946+0000 front 2023-12-11T18:05:28.838926+0000 (oldest deadline 2023-12-11T18:05:54.139025+0000) 2023-12-11T18:06:07.405 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:06:07 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:06:07.200+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.181:6817 osd.5 since back 2023-12-11T18:05:40.541039+0000 front 2023-12-11T18:05:40.541215+0000 (oldest deadline 2023-12-11T18:06:02.840912+0000) 2023-12-11T18:06:07.904 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:06:07 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:06:07.406+0000 7f8627ade700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-11T18:06:07.904 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:06:07 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:06:07.406+0000 7f8627ade700 -1 osd.9 123 *** Got signal Terminated *** 2023-12-11T18:06:07.904 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:06:07 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:06:07.406+0000 7f8627ade700 -1 osd.9 123 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-12-11T18:06:08.032 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:06:07 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:06:07.743+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:59.875115+0000 front 2023-12-11T18:04:59.874965+0000 (oldest deadline 2023-12-11T18:05:22.775102+0000) 2023-12-11T18:06:08.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:06:07 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:06:07.743+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:02.775745+0000 front 2023-12-11T18:05:02.775784+0000 (oldest deadline 2023-12-11T18:05:33.376735+0000) 2023-12-11T18:06:08.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:06:07 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:06:07.743+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.181:6816 osd.2 since back 2023-12-11T18:05:16.778632+0000 front 2023-12-11T18:05:16.778462+0000 (oldest deadline 2023-12-11T18:05:41.478412+0000) 2023-12-11T18:06:08.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:06:07 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:06:07.743+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2023-12-11T18:05:21.479017+0000 front 2023-12-11T18:05:21.479156+0000 (oldest deadline 2023-12-11T18:05:47.879527+0000) 2023-12-11T18:06:08.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:06:07 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:06:07.743+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2023-12-11T18:05:30.180654+0000 front 2023-12-11T18:05:30.180627+0000 (oldest deadline 2023-12-11T18:05:54.880677+0000) 2023-12-11T18:06:08.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:06:07 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:06:07.743+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.181:6817 osd.5 since back 2023-12-11T18:05:35.981809+0000 front 2023-12-11T18:05:35.981821+0000 (oldest deadline 2023-12-11T18:06:04.782070+0000) 2023-12-11T18:06:08.654 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:06:08 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:06:08.171+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:58.934598+0000 front 2023-12-11T18:04:58.934735+0000 (oldest deadline 2023-12-11T18:05:21.834661+0000) 2023-12-11T18:06:08.654 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:06:08 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:06:08.171+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:06.536080+0000 front 2023-12-11T18:05:06.535905+0000 (oldest deadline 2023-12-11T18:05:31.835855+0000) 2023-12-11T18:06:08.654 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:06:08 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:06:08.171+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.181:6816 osd.2 since back 2023-12-11T18:05:16.536946+0000 front 2023-12-11T18:05:16.537012+0000 (oldest deadline 2023-12-11T18:05:42.436997+0000) 2023-12-11T18:06:08.654 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:06:08 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:06:08.171+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2023-12-11T18:05:23.538131+0000 front 2023-12-11T18:05:23.538151+0000 (oldest deadline 2023-12-11T18:05:48.838365+0000) 2023-12-11T18:06:08.654 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:06:08 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:06:08.171+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2023-12-11T18:05:28.838946+0000 front 2023-12-11T18:05:28.838926+0000 (oldest deadline 2023-12-11T18:05:54.139025+0000) 2023-12-11T18:06:08.654 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:06:08 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:06:08.171+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.181:6817 osd.5 since back 2023-12-11T18:05:40.541039+0000 front 2023-12-11T18:05:40.541215+0000 (oldest deadline 2023-12-11T18:06:02.840912+0000) 2023-12-11T18:06:09.032 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:06:08 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:06:08.704+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:59.875115+0000 front 2023-12-11T18:04:59.874965+0000 (oldest deadline 2023-12-11T18:05:22.775102+0000) 2023-12-11T18:06:09.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:06:08 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:06:08.704+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:02.775745+0000 front 2023-12-11T18:05:02.775784+0000 (oldest deadline 2023-12-11T18:05:33.376735+0000) 2023-12-11T18:06:09.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:06:08 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:06:08.704+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.181:6816 osd.2 since back 2023-12-11T18:05:16.778632+0000 front 2023-12-11T18:05:16.778462+0000 (oldest deadline 2023-12-11T18:05:41.478412+0000) 2023-12-11T18:06:09.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:06:08 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:06:08.704+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2023-12-11T18:05:21.479017+0000 front 2023-12-11T18:05:21.479156+0000 (oldest deadline 2023-12-11T18:05:47.879527+0000) 2023-12-11T18:06:09.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:06:08 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:06:08.704+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2023-12-11T18:05:30.180654+0000 front 2023-12-11T18:05:30.180627+0000 (oldest deadline 2023-12-11T18:05:54.880677+0000) 2023-12-11T18:06:09.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:06:08 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:06:08.704+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.181:6817 osd.5 since back 2023-12-11T18:05:35.981809+0000 front 2023-12-11T18:05:35.981821+0000 (oldest deadline 2023-12-11T18:06:04.782070+0000) 2023-12-11T18:06:09.654 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:06:09 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:06:09.220+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:58.934598+0000 front 2023-12-11T18:04:58.934735+0000 (oldest deadline 2023-12-11T18:05:21.834661+0000) 2023-12-11T18:06:09.654 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:06:09 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:06:09.220+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:06.536080+0000 front 2023-12-11T18:05:06.535905+0000 (oldest deadline 2023-12-11T18:05:31.835855+0000) 2023-12-11T18:06:09.654 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:06:09 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:06:09.220+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.181:6816 osd.2 since back 2023-12-11T18:05:16.536946+0000 front 2023-12-11T18:05:16.537012+0000 (oldest deadline 2023-12-11T18:05:42.436997+0000) 2023-12-11T18:06:09.654 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:06:09 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:06:09.220+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2023-12-11T18:05:23.538131+0000 front 2023-12-11T18:05:23.538151+0000 (oldest deadline 2023-12-11T18:05:48.838365+0000) 2023-12-11T18:06:09.654 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:06:09 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:06:09.220+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2023-12-11T18:05:28.838946+0000 front 2023-12-11T18:05:28.838926+0000 (oldest deadline 2023-12-11T18:05:54.139025+0000) 2023-12-11T18:06:09.654 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:06:09 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:06:09.220+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.181:6817 osd.5 since back 2023-12-11T18:05:40.541039+0000 front 2023-12-11T18:05:40.541215+0000 (oldest deadline 2023-12-11T18:06:02.840912+0000) 2023-12-11T18:06:10.032 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:06:09 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:06:09.675+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:59.875115+0000 front 2023-12-11T18:04:59.874965+0000 (oldest deadline 2023-12-11T18:05:22.775102+0000) 2023-12-11T18:06:10.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:06:09 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:06:09.675+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:02.775745+0000 front 2023-12-11T18:05:02.775784+0000 (oldest deadline 2023-12-11T18:05:33.376735+0000) 2023-12-11T18:06:10.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:06:09 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:06:09.675+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.181:6816 osd.2 since back 2023-12-11T18:05:16.778632+0000 front 2023-12-11T18:05:16.778462+0000 (oldest deadline 2023-12-11T18:05:41.478412+0000) 2023-12-11T18:06:10.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:06:09 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:06:09.675+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2023-12-11T18:05:21.479017+0000 front 2023-12-11T18:05:21.479156+0000 (oldest deadline 2023-12-11T18:05:47.879527+0000) 2023-12-11T18:06:10.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:06:09 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:06:09.675+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2023-12-11T18:05:30.180654+0000 front 2023-12-11T18:05:30.180627+0000 (oldest deadline 2023-12-11T18:05:54.880677+0000) 2023-12-11T18:06:10.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:06:09 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:06:09.675+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.181:6817 osd.5 since back 2023-12-11T18:05:35.981809+0000 front 2023-12-11T18:05:35.981821+0000 (oldest deadline 2023-12-11T18:06:04.782070+0000) 2023-12-11T18:06:10.654 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:06:10 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:06:10.208+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:58.934598+0000 front 2023-12-11T18:04:58.934735+0000 (oldest deadline 2023-12-11T18:05:21.834661+0000) 2023-12-11T18:06:10.654 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:06:10 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:06:10.208+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:06.536080+0000 front 2023-12-11T18:05:06.535905+0000 (oldest deadline 2023-12-11T18:05:31.835855+0000) 2023-12-11T18:06:10.654 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:06:10 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:06:10.208+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.181:6816 osd.2 since back 2023-12-11T18:05:16.536946+0000 front 2023-12-11T18:05:16.537012+0000 (oldest deadline 2023-12-11T18:05:42.436997+0000) 2023-12-11T18:06:10.654 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:06:10 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:06:10.208+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2023-12-11T18:05:23.538131+0000 front 2023-12-11T18:05:23.538151+0000 (oldest deadline 2023-12-11T18:05:48.838365+0000) 2023-12-11T18:06:10.654 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:06:10 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:06:10.208+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2023-12-11T18:05:28.838946+0000 front 2023-12-11T18:05:28.838926+0000 (oldest deadline 2023-12-11T18:05:54.139025+0000) 2023-12-11T18:06:10.654 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:06:10 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:06:10.208+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.181:6817 osd.5 since back 2023-12-11T18:05:40.541039+0000 front 2023-12-11T18:05:40.541215+0000 (oldest deadline 2023-12-11T18:06:02.840912+0000) 2023-12-11T18:06:11.032 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:06:10 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:06:10.666+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:59.875115+0000 front 2023-12-11T18:04:59.874965+0000 (oldest deadline 2023-12-11T18:05:22.775102+0000) 2023-12-11T18:06:11.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:06:10 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:06:10.666+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:02.775745+0000 front 2023-12-11T18:05:02.775784+0000 (oldest deadline 2023-12-11T18:05:33.376735+0000) 2023-12-11T18:06:11.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:06:10 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:06:10.666+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.181:6816 osd.2 since back 2023-12-11T18:05:16.778632+0000 front 2023-12-11T18:05:16.778462+0000 (oldest deadline 2023-12-11T18:05:41.478412+0000) 2023-12-11T18:06:11.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:06:10 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:06:10.666+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2023-12-11T18:05:21.479017+0000 front 2023-12-11T18:05:21.479156+0000 (oldest deadline 2023-12-11T18:05:47.879527+0000) 2023-12-11T18:06:11.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:06:10 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:06:10.666+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2023-12-11T18:05:30.180654+0000 front 2023-12-11T18:05:30.180627+0000 (oldest deadline 2023-12-11T18:05:54.880677+0000) 2023-12-11T18:06:11.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:06:10 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:06:10.666+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.181:6817 osd.5 since back 2023-12-11T18:05:35.981809+0000 front 2023-12-11T18:05:35.981821+0000 (oldest deadline 2023-12-11T18:06:04.782070+0000) 2023-12-11T18:06:11.654 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:06:11 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:06:11.188+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:58.934598+0000 front 2023-12-11T18:04:58.934735+0000 (oldest deadline 2023-12-11T18:05:21.834661+0000) 2023-12-11T18:06:11.654 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:06:11 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:06:11.188+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:06.536080+0000 front 2023-12-11T18:05:06.535905+0000 (oldest deadline 2023-12-11T18:05:31.835855+0000) 2023-12-11T18:06:11.654 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:06:11 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:06:11.188+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.181:6816 osd.2 since back 2023-12-11T18:05:16.536946+0000 front 2023-12-11T18:05:16.537012+0000 (oldest deadline 2023-12-11T18:05:42.436997+0000) 2023-12-11T18:06:11.654 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:06:11 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:06:11.188+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2023-12-11T18:05:23.538131+0000 front 2023-12-11T18:05:23.538151+0000 (oldest deadline 2023-12-11T18:05:48.838365+0000) 2023-12-11T18:06:11.654 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:06:11 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:06:11.188+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2023-12-11T18:05:28.838946+0000 front 2023-12-11T18:05:28.838926+0000 (oldest deadline 2023-12-11T18:05:54.139025+0000) 2023-12-11T18:06:11.654 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:06:11 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:06:11.188+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.181:6817 osd.5 since back 2023-12-11T18:05:40.541039+0000 front 2023-12-11T18:05:40.541215+0000 (oldest deadline 2023-12-11T18:06:02.840912+0000) 2023-12-11T18:06:12.032 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:06:11 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:06:11.694+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:59.875115+0000 front 2023-12-11T18:04:59.874965+0000 (oldest deadline 2023-12-11T18:05:22.775102+0000) 2023-12-11T18:06:12.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:06:11 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:06:11.694+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:02.775745+0000 front 2023-12-11T18:05:02.775784+0000 (oldest deadline 2023-12-11T18:05:33.376735+0000) 2023-12-11T18:06:12.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:06:11 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:06:11.694+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.181:6816 osd.2 since back 2023-12-11T18:05:16.778632+0000 front 2023-12-11T18:05:16.778462+0000 (oldest deadline 2023-12-11T18:05:41.478412+0000) 2023-12-11T18:06:12.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:06:11 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:06:11.694+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2023-12-11T18:05:21.479017+0000 front 2023-12-11T18:05:21.479156+0000 (oldest deadline 2023-12-11T18:05:47.879527+0000) 2023-12-11T18:06:12.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:06:11 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:06:11.694+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2023-12-11T18:05:30.180654+0000 front 2023-12-11T18:05:30.180627+0000 (oldest deadline 2023-12-11T18:05:54.880677+0000) 2023-12-11T18:06:12.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:06:11 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:06:11.694+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.181:6817 osd.5 since back 2023-12-11T18:05:35.981809+0000 front 2023-12-11T18:05:35.981821+0000 (oldest deadline 2023-12-11T18:06:04.782070+0000) 2023-12-11T18:06:12.654 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:06:12 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:06:12.159+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:58.934598+0000 front 2023-12-11T18:04:58.934735+0000 (oldest deadline 2023-12-11T18:05:21.834661+0000) 2023-12-11T18:06:12.654 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:06:12 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:06:12.159+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:06.536080+0000 front 2023-12-11T18:05:06.535905+0000 (oldest deadline 2023-12-11T18:05:31.835855+0000) 2023-12-11T18:06:12.654 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:06:12 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:06:12.159+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.181:6816 osd.2 since back 2023-12-11T18:05:16.536946+0000 front 2023-12-11T18:05:16.537012+0000 (oldest deadline 2023-12-11T18:05:42.436997+0000) 2023-12-11T18:06:12.654 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:06:12 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:06:12.159+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2023-12-11T18:05:23.538131+0000 front 2023-12-11T18:05:23.538151+0000 (oldest deadline 2023-12-11T18:05:48.838365+0000) 2023-12-11T18:06:12.654 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:06:12 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:06:12.159+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2023-12-11T18:05:28.838946+0000 front 2023-12-11T18:05:28.838926+0000 (oldest deadline 2023-12-11T18:05:54.139025+0000) 2023-12-11T18:06:12.654 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:06:12 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9[142839]: 2023-12-11T18:06:12.159+0000 7f86220d5700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.181:6817 osd.5 since back 2023-12-11T18:05:40.541039+0000 front 2023-12-11T18:05:40.541215+0000 (oldest deadline 2023-12-11T18:06:02.840912+0000) 2023-12-11T18:06:13.032 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:06:12 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:06:12.653+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:59.875115+0000 front 2023-12-11T18:04:59.874965+0000 (oldest deadline 2023-12-11T18:05:22.775102+0000) 2023-12-11T18:06:13.032 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:06:12 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:06:12.653+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:02.775745+0000 front 2023-12-11T18:05:02.775784+0000 (oldest deadline 2023-12-11T18:05:33.376735+0000) 2023-12-11T18:06:13.032 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:06:12 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:06:12.653+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.181:6816 osd.2 since back 2023-12-11T18:05:16.778632+0000 front 2023-12-11T18:05:16.778462+0000 (oldest deadline 2023-12-11T18:05:41.478412+0000) 2023-12-11T18:06:13.032 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:06:12 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:06:12.653+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2023-12-11T18:05:21.479017+0000 front 2023-12-11T18:05:21.479156+0000 (oldest deadline 2023-12-11T18:05:47.879527+0000) 2023-12-11T18:06:13.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:06:12 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:06:12.653+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2023-12-11T18:05:30.180654+0000 front 2023-12-11T18:05:30.180627+0000 (oldest deadline 2023-12-11T18:05:54.880677+0000) 2023-12-11T18:06:13.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:06:12 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:06:12.653+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.181:6817 osd.5 since back 2023-12-11T18:05:35.981809+0000 front 2023-12-11T18:05:35.981821+0000 (oldest deadline 2023-12-11T18:06:04.782070+0000) 2023-12-11T18:06:13.397 INFO:journalctl@ceph.osd.9.smithi059.stdout:Dec 11 18:06:13 smithi059 bash[177941]: ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-9 2023-12-11T18:06:14.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:06:13 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:06:13.702+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:59.875115+0000 front 2023-12-11T18:04:59.874965+0000 (oldest deadline 2023-12-11T18:05:22.775102+0000) 2023-12-11T18:06:14.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:06:13 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:06:13.702+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:02.775745+0000 front 2023-12-11T18:05:02.775784+0000 (oldest deadline 2023-12-11T18:05:33.376735+0000) 2023-12-11T18:06:14.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:06:13 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:06:13.702+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.181:6816 osd.2 since back 2023-12-11T18:05:16.778632+0000 front 2023-12-11T18:05:16.778462+0000 (oldest deadline 2023-12-11T18:05:41.478412+0000) 2023-12-11T18:06:14.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:06:13 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:06:13.702+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2023-12-11T18:05:21.479017+0000 front 2023-12-11T18:05:21.479156+0000 (oldest deadline 2023-12-11T18:05:47.879527+0000) 2023-12-11T18:06:14.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:06:13 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:06:13.702+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2023-12-11T18:05:30.180654+0000 front 2023-12-11T18:05:30.180627+0000 (oldest deadline 2023-12-11T18:05:54.880677+0000) 2023-12-11T18:06:14.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:06:13 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:06:13.702+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.181:6817 osd.5 since back 2023-12-11T18:05:35.981809+0000 front 2023-12-11T18:05:35.981821+0000 (oldest deadline 2023-12-11T18:06:04.782070+0000) 2023-12-11T18:06:14.764 DEBUG:teuthology.orchestra.run.smithi059:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d0a07eaa-984b-11ee-95a3-87774f69a715@osd.9.service' 2023-12-11T18:06:14.804 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-11T18:06:14.805 INFO:tasks.cephadm.osd.9:Stopped osd.9 2023-12-11T18:06:14.805 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2023-12-11T18:06:14.805 DEBUG:teuthology.orchestra.run.smithi115:> sudo systemctl stop ceph-d0a07eaa-984b-11ee-95a3-87774f69a715@osd.10 2023-12-11T18:06:15.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:06:14 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:06:14.714+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:59.875115+0000 front 2023-12-11T18:04:59.874965+0000 (oldest deadline 2023-12-11T18:05:22.775102+0000) 2023-12-11T18:06:15.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:06:14 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:06:14.714+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:02.775745+0000 front 2023-12-11T18:05:02.775784+0000 (oldest deadline 2023-12-11T18:05:33.376735+0000) 2023-12-11T18:06:15.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:06:14 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:06:14.714+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.181:6816 osd.2 since back 2023-12-11T18:05:16.778632+0000 front 2023-12-11T18:05:16.778462+0000 (oldest deadline 2023-12-11T18:05:41.478412+0000) 2023-12-11T18:06:15.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:06:14 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:06:14.714+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2023-12-11T18:05:21.479017+0000 front 2023-12-11T18:05:21.479156+0000 (oldest deadline 2023-12-11T18:05:47.879527+0000) 2023-12-11T18:06:15.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:06:14 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:06:14.714+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2023-12-11T18:05:30.180654+0000 front 2023-12-11T18:05:30.180627+0000 (oldest deadline 2023-12-11T18:05:54.880677+0000) 2023-12-11T18:06:15.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:06:14 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:06:14.714+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.181:6817 osd.5 since back 2023-12-11T18:05:35.981809+0000 front 2023-12-11T18:05:35.981821+0000 (oldest deadline 2023-12-11T18:06:04.782070+0000) 2023-12-11T18:06:15.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:06:14 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:06:14.714+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.59:6822 osd.6 since back 2023-12-11T18:05:48.783643+0000 front 2023-12-11T18:05:48.783662+0000 (oldest deadline 2023-12-11T18:06:14.683616+0000) 2023-12-11T18:06:15.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:06:14 smithi115 systemd[1]: Stopping Ceph osd.10 for d0a07eaa-984b-11ee-95a3-87774f69a715... 2023-12-11T18:06:15.532 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:06:15 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:06:15.163+0000 7f1ec0c84700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-11T18:06:15.532 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:06:15 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:06:15.163+0000 7f1ec0c84700 -1 osd.10 123 *** Got signal Terminated *** 2023-12-11T18:06:15.532 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:06:15 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:06:15.163+0000 7f1ec0c84700 -1 osd.10 123 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-12-11T18:06:16.032 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:06:15 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:06:15.724+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:59.875115+0000 front 2023-12-11T18:04:59.874965+0000 (oldest deadline 2023-12-11T18:05:22.775102+0000) 2023-12-11T18:06:16.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:06:15 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:06:15.724+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:02.775745+0000 front 2023-12-11T18:05:02.775784+0000 (oldest deadline 2023-12-11T18:05:33.376735+0000) 2023-12-11T18:06:16.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:06:15 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:06:15.724+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.181:6816 osd.2 since back 2023-12-11T18:05:16.778632+0000 front 2023-12-11T18:05:16.778462+0000 (oldest deadline 2023-12-11T18:05:41.478412+0000) 2023-12-11T18:06:16.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:06:15 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:06:15.724+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2023-12-11T18:05:21.479017+0000 front 2023-12-11T18:05:21.479156+0000 (oldest deadline 2023-12-11T18:05:47.879527+0000) 2023-12-11T18:06:16.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:06:15 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:06:15.724+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2023-12-11T18:05:30.180654+0000 front 2023-12-11T18:05:30.180627+0000 (oldest deadline 2023-12-11T18:05:54.880677+0000) 2023-12-11T18:06:16.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:06:15 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:06:15.724+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.181:6817 osd.5 since back 2023-12-11T18:05:35.981809+0000 front 2023-12-11T18:05:35.981821+0000 (oldest deadline 2023-12-11T18:06:04.782070+0000) 2023-12-11T18:06:16.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:06:15 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:06:15.724+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.59:6822 osd.6 since back 2023-12-11T18:05:48.783643+0000 front 2023-12-11T18:05:48.783662+0000 (oldest deadline 2023-12-11T18:06:14.683616+0000) 2023-12-11T18:06:17.032 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:06:16 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:06:16.699+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:59.875115+0000 front 2023-12-11T18:04:59.874965+0000 (oldest deadline 2023-12-11T18:05:22.775102+0000) 2023-12-11T18:06:17.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:06:16 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:06:16.699+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:02.775745+0000 front 2023-12-11T18:05:02.775784+0000 (oldest deadline 2023-12-11T18:05:33.376735+0000) 2023-12-11T18:06:17.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:06:16 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:06:16.699+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.181:6816 osd.2 since back 2023-12-11T18:05:16.778632+0000 front 2023-12-11T18:05:16.778462+0000 (oldest deadline 2023-12-11T18:05:41.478412+0000) 2023-12-11T18:06:17.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:06:16 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:06:16.699+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2023-12-11T18:05:21.479017+0000 front 2023-12-11T18:05:21.479156+0000 (oldest deadline 2023-12-11T18:05:47.879527+0000) 2023-12-11T18:06:17.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:06:16 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:06:16.699+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2023-12-11T18:05:30.180654+0000 front 2023-12-11T18:05:30.180627+0000 (oldest deadline 2023-12-11T18:05:54.880677+0000) 2023-12-11T18:06:17.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:06:16 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:06:16.699+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.181:6817 osd.5 since back 2023-12-11T18:05:35.981809+0000 front 2023-12-11T18:05:35.981821+0000 (oldest deadline 2023-12-11T18:06:04.782070+0000) 2023-12-11T18:06:17.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:06:16 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:06:16.699+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.59:6822 osd.6 since back 2023-12-11T18:05:48.783643+0000 front 2023-12-11T18:05:48.783662+0000 (oldest deadline 2023-12-11T18:06:14.683616+0000) 2023-12-11T18:06:18.032 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:06:17 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:06:17.749+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:59.875115+0000 front 2023-12-11T18:04:59.874965+0000 (oldest deadline 2023-12-11T18:05:22.775102+0000) 2023-12-11T18:06:18.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:06:17 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:06:17.749+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:02.775745+0000 front 2023-12-11T18:05:02.775784+0000 (oldest deadline 2023-12-11T18:05:33.376735+0000) 2023-12-11T18:06:18.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:06:17 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:06:17.749+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.181:6816 osd.2 since back 2023-12-11T18:05:16.778632+0000 front 2023-12-11T18:05:16.778462+0000 (oldest deadline 2023-12-11T18:05:41.478412+0000) 2023-12-11T18:06:18.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:06:17 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:06:17.749+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2023-12-11T18:05:21.479017+0000 front 2023-12-11T18:05:21.479156+0000 (oldest deadline 2023-12-11T18:05:47.879527+0000) 2023-12-11T18:06:18.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:06:17 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:06:17.749+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2023-12-11T18:05:30.180654+0000 front 2023-12-11T18:05:30.180627+0000 (oldest deadline 2023-12-11T18:05:54.880677+0000) 2023-12-11T18:06:18.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:06:17 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:06:17.749+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.181:6817 osd.5 since back 2023-12-11T18:05:35.981809+0000 front 2023-12-11T18:05:35.981821+0000 (oldest deadline 2023-12-11T18:06:04.782070+0000) 2023-12-11T18:06:18.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:06:17 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:06:17.749+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.59:6822 osd.6 since back 2023-12-11T18:05:48.783643+0000 front 2023-12-11T18:05:48.783662+0000 (oldest deadline 2023-12-11T18:06:14.683616+0000) 2023-12-11T18:06:19.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:06:18 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:06:18.733+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:59.875115+0000 front 2023-12-11T18:04:59.874965+0000 (oldest deadline 2023-12-11T18:05:22.775102+0000) 2023-12-11T18:06:19.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:06:18 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:06:18.733+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:02.775745+0000 front 2023-12-11T18:05:02.775784+0000 (oldest deadline 2023-12-11T18:05:33.376735+0000) 2023-12-11T18:06:19.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:06:18 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:06:18.733+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.181:6816 osd.2 since back 2023-12-11T18:05:16.778632+0000 front 2023-12-11T18:05:16.778462+0000 (oldest deadline 2023-12-11T18:05:41.478412+0000) 2023-12-11T18:06:19.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:06:18 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:06:18.733+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2023-12-11T18:05:21.479017+0000 front 2023-12-11T18:05:21.479156+0000 (oldest deadline 2023-12-11T18:05:47.879527+0000) 2023-12-11T18:06:19.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:06:18 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:06:18.733+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2023-12-11T18:05:30.180654+0000 front 2023-12-11T18:05:30.180627+0000 (oldest deadline 2023-12-11T18:05:54.880677+0000) 2023-12-11T18:06:19.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:06:18 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:06:18.733+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.181:6817 osd.5 since back 2023-12-11T18:05:35.981809+0000 front 2023-12-11T18:05:35.981821+0000 (oldest deadline 2023-12-11T18:06:04.782070+0000) 2023-12-11T18:06:19.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:06:18 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:06:18.733+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.59:6822 osd.6 since back 2023-12-11T18:05:48.783643+0000 front 2023-12-11T18:05:48.783662+0000 (oldest deadline 2023-12-11T18:06:14.683616+0000) 2023-12-11T18:06:20.032 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:06:19 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:06:19.765+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2023-12-11T18:04:59.875115+0000 front 2023-12-11T18:04:59.874965+0000 (oldest deadline 2023-12-11T18:05:22.775102+0000) 2023-12-11T18:06:20.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:06:19 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:06:19.765+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.115:6804 osd.1 since back 2023-12-11T18:05:02.775745+0000 front 2023-12-11T18:05:02.775784+0000 (oldest deadline 2023-12-11T18:05:33.376735+0000) 2023-12-11T18:06:20.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:06:19 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:06:19.765+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.181:6816 osd.2 since back 2023-12-11T18:05:16.778632+0000 front 2023-12-11T18:05:16.778462+0000 (oldest deadline 2023-12-11T18:05:41.478412+0000) 2023-12-11T18:06:20.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:06:19 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:06:19.765+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2023-12-11T18:05:21.479017+0000 front 2023-12-11T18:05:21.479156+0000 (oldest deadline 2023-12-11T18:05:47.879527+0000) 2023-12-11T18:06:20.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:06:19 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:06:19.765+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.115:6812 osd.4 since back 2023-12-11T18:05:30.180654+0000 front 2023-12-11T18:05:30.180627+0000 (oldest deadline 2023-12-11T18:05:54.880677+0000) 2023-12-11T18:06:20.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:06:19 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:06:19.765+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.181:6817 osd.5 since back 2023-12-11T18:05:35.981809+0000 front 2023-12-11T18:05:35.981821+0000 (oldest deadline 2023-12-11T18:06:04.782070+0000) 2023-12-11T18:06:20.033 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:06:19 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10[132680]: 2023-12-11T18:06:19.765+0000 7f1ebb27b700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.59:6822 osd.6 since back 2023-12-11T18:05:48.783643+0000 front 2023-12-11T18:05:48.783662+0000 (oldest deadline 2023-12-11T18:06:14.683616+0000) 2023-12-11T18:06:21.032 INFO:journalctl@ceph.osd.10.smithi115.stdout:Dec 11 18:06:20 smithi115 bash[147861]: ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-osd-10 2023-12-11T18:06:22.274 DEBUG:teuthology.orchestra.run.smithi115:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d0a07eaa-984b-11ee-95a3-87774f69a715@osd.10.service' 2023-12-11T18:06:22.314 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-11T18:06:22.314 INFO:tasks.cephadm.osd.10:Stopped osd.10 2023-12-11T18:06:22.314 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2023-12-11T18:06:22.314 DEBUG:teuthology.orchestra.run.smithi181:> sudo systemctl stop ceph-d0a07eaa-984b-11ee-95a3-87774f69a715@osd.11 2023-12-11T18:06:29.813 DEBUG:teuthology.orchestra.run.smithi181:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d0a07eaa-984b-11ee-95a3-87774f69a715@osd.11.service' 2023-12-11T18:06:29.852 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-11T18:06:29.852 INFO:tasks.cephadm.osd.11:Stopped osd.11 2023-12-11T18:06:29.852 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2023-12-11T18:06:29.853 DEBUG:teuthology.orchestra.run.smithi059:> sudo systemctl stop ceph-d0a07eaa-984b-11ee-95a3-87774f69a715@mds.a 2023-12-11T18:06:30.155 INFO:journalctl@ceph.mds.a.smithi059.stdout:Dec 11 18:06:29 smithi059 systemd[1]: Stopping Ceph mds.a for d0a07eaa-984b-11ee-95a3-87774f69a715... 2023-12-11T18:06:30.654 INFO:journalctl@ceph.mds.a.smithi059.stdout:Dec 11 18:06:30 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mds-a[151476]: 2023-12-11T18:06:30.218+0000 7f288c648700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-11T18:06:30.654 INFO:journalctl@ceph.mds.a.smithi059.stdout:Dec 11 18:06:30 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mds-a[151476]: 2023-12-11T18:06:30.218+0000 7f288c648700 -1 mds.a *** got signal Terminated *** 2023-12-11T18:06:33.384 INFO:journalctl@ceph.mds.a.smithi059.stdout:Dec 11 18:06:33 smithi059 bash[178201]: ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mds-a 2023-12-11T18:06:33.580 DEBUG:teuthology.orchestra.run.smithi059:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d0a07eaa-984b-11ee-95a3-87774f69a715@mds.a.service' 2023-12-11T18:06:33.616 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-11T18:06:33.616 INFO:tasks.cephadm.mds.a:Stopped mds.a 2023-12-11T18:06:33.616 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2023-12-11T18:06:33.616 DEBUG:teuthology.orchestra.run.smithi059:> sudo systemctl stop ceph-d0a07eaa-984b-11ee-95a3-87774f69a715@mds.d 2023-12-11T18:06:33.904 INFO:journalctl@ceph.mds.d.smithi059.stdout:Dec 11 18:06:33 smithi059 systemd[1]: Stopping Ceph mds.d for d0a07eaa-984b-11ee-95a3-87774f69a715... 2023-12-11T18:06:34.404 INFO:journalctl@ceph.mds.d.smithi059.stdout:Dec 11 18:06:34 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mds-d[150478]: 2023-12-11T18:06:34.033+0000 7fa47891f700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-11T18:06:34.404 INFO:journalctl@ceph.mds.d.smithi059.stdout:Dec 11 18:06:34 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mds-d[150478]: 2023-12-11T18:06:34.033+0000 7fa47891f700 -1 mds.d *** got signal Terminated *** 2023-12-11T18:06:37.655 INFO:journalctl@ceph.mds.d.smithi059.stdout:Dec 11 18:06:37 smithi059 bash[178342]: ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mds-d 2023-12-11T18:06:37.809 DEBUG:teuthology.orchestra.run.smithi059:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d0a07eaa-984b-11ee-95a3-87774f69a715@mds.d.service' 2023-12-11T18:06:37.847 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-11T18:06:37.847 INFO:tasks.cephadm.mds.d:Stopped mds.d 2023-12-11T18:06:37.847 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2023-12-11T18:06:37.848 DEBUG:teuthology.orchestra.run.smithi059:> sudo systemctl stop ceph-d0a07eaa-984b-11ee-95a3-87774f69a715@mds.g 2023-12-11T18:06:38.154 INFO:journalctl@ceph.mds.g.smithi059.stdout:Dec 11 18:06:37 smithi059 systemd[1]: Stopping Ceph mds.g for d0a07eaa-984b-11ee-95a3-87774f69a715... 2023-12-11T18:06:38.654 INFO:journalctl@ceph.mds.g.smithi059.stdout:Dec 11 18:06:38 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mds-g[151089]: 2023-12-11T18:06:38.227+0000 7f38bb9eb700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-11T18:06:38.654 INFO:journalctl@ceph.mds.g.smithi059.stdout:Dec 11 18:06:38 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mds-g[151089]: 2023-12-11T18:06:38.227+0000 7f38bb9eb700 -1 mds.g *** got signal Terminated *** 2023-12-11T18:06:42.654 INFO:journalctl@ceph.mds.g.smithi059.stdout:Dec 11 18:06:42 smithi059 bash[178496]: ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mds-g 2023-12-11T18:06:42.735 DEBUG:teuthology.orchestra.run.smithi059:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d0a07eaa-984b-11ee-95a3-87774f69a715@mds.g.service' 2023-12-11T18:06:42.769 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-11T18:06:42.770 INFO:tasks.cephadm.mds.g:Stopped mds.g 2023-12-11T18:06:42.770 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2023-12-11T18:06:42.770 DEBUG:teuthology.orchestra.run.smithi059:> sudo systemctl stop ceph-d0a07eaa-984b-11ee-95a3-87774f69a715@mds.j 2023-12-11T18:06:43.153 INFO:journalctl@ceph.mds.j.smithi059.stdout:Dec 11 18:06:42 smithi059 systemd[1]: Stopping Ceph mds.j for d0a07eaa-984b-11ee-95a3-87774f69a715... 2023-12-11T18:06:43.404 INFO:journalctl@ceph.mds.j.smithi059.stdout:Dec 11 18:06:43 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mds-j[150136]: 2023-12-11T18:06:43.150+0000 7f4453cf0700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-11T18:06:43.404 INFO:journalctl@ceph.mds.j.smithi059.stdout:Dec 11 18:06:43 smithi059 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mds-j[150136]: 2023-12-11T18:06:43.150+0000 7f4453cf0700 -1 mds.j *** got signal Terminated *** 2023-12-11T18:06:47.424 INFO:journalctl@ceph.mds.j.smithi059.stdout:Dec 11 18:06:47 smithi059 bash[178645]: ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mds-j 2023-12-11T18:06:47.642 DEBUG:teuthology.orchestra.run.smithi059:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d0a07eaa-984b-11ee-95a3-87774f69a715@mds.j.service' 2023-12-11T18:06:47.678 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-11T18:06:47.679 INFO:tasks.cephadm.mds.j:Stopped mds.j 2023-12-11T18:06:47.679 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2023-12-11T18:06:47.679 DEBUG:teuthology.orchestra.run.smithi115:> sudo systemctl stop ceph-d0a07eaa-984b-11ee-95a3-87774f69a715@mds.b 2023-12-11T18:06:48.032 INFO:journalctl@ceph.mds.b.smithi115.stdout:Dec 11 18:06:47 smithi115 systemd[1]: Stopping Ceph mds.b for d0a07eaa-984b-11ee-95a3-87774f69a715... 2023-12-11T18:06:48.532 INFO:journalctl@ceph.mds.b.smithi115.stdout:Dec 11 18:06:48 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mds-b[139691]: 2023-12-11T18:06:48.038+0000 7f42d15a6700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-11T18:06:48.532 INFO:journalctl@ceph.mds.b.smithi115.stdout:Dec 11 18:06:48 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mds-b[139691]: 2023-12-11T18:06:48.038+0000 7f42d15a6700 -1 mds.b *** got signal Terminated *** 2023-12-11T18:06:52.109 INFO:journalctl@ceph.mds.b.smithi115.stdout:Dec 11 18:06:52 smithi115 bash[148136]: ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mds-b 2023-12-11T18:06:52.297 DEBUG:teuthology.orchestra.run.smithi115:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d0a07eaa-984b-11ee-95a3-87774f69a715@mds.b.service' 2023-12-11T18:06:52.334 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-11T18:06:52.334 INFO:tasks.cephadm.mds.b:Stopped mds.b 2023-12-11T18:06:52.334 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2023-12-11T18:06:52.334 DEBUG:teuthology.orchestra.run.smithi115:> sudo systemctl stop ceph-d0a07eaa-984b-11ee-95a3-87774f69a715@mds.e 2023-12-11T18:06:52.723 INFO:journalctl@ceph.mds.e.smithi115.stdout:Dec 11 18:06:52 smithi115 systemd[1]: Stopping Ceph mds.e for d0a07eaa-984b-11ee-95a3-87774f69a715... 2023-12-11T18:06:53.032 INFO:journalctl@ceph.mds.e.smithi115.stdout:Dec 11 18:06:52 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mds-e[139018]: 2023-12-11T18:06:52.720+0000 7fa343601700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-11T18:06:53.032 INFO:journalctl@ceph.mds.e.smithi115.stdout:Dec 11 18:06:52 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mds-e[139018]: 2023-12-11T18:06:52.720+0000 7fa343601700 -1 mds.e *** got signal Terminated *** 2023-12-11T18:06:56.081 INFO:journalctl@ceph.mds.e.smithi115.stdout:Dec 11 18:06:56 smithi115 bash[148281]: ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mds-e 2023-12-11T18:06:56.274 DEBUG:teuthology.orchestra.run.smithi115:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d0a07eaa-984b-11ee-95a3-87774f69a715@mds.e.service' 2023-12-11T18:06:56.308 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-11T18:06:56.308 INFO:tasks.cephadm.mds.e:Stopped mds.e 2023-12-11T18:06:56.308 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2023-12-11T18:06:56.309 DEBUG:teuthology.orchestra.run.smithi115:> sudo systemctl stop ceph-d0a07eaa-984b-11ee-95a3-87774f69a715@mds.h 2023-12-11T18:06:56.532 INFO:journalctl@ceph.mds.h.smithi115.stdout:Dec 11 18:06:56 smithi115 systemd[1]: Stopping Ceph mds.h for d0a07eaa-984b-11ee-95a3-87774f69a715... 2023-12-11T18:06:57.032 INFO:journalctl@ceph.mds.h.smithi115.stdout:Dec 11 18:06:56 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mds-h[139354]: 2023-12-11T18:06:56.693+0000 7f311ff1d700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-11T18:06:57.033 INFO:journalctl@ceph.mds.h.smithi115.stdout:Dec 11 18:06:56 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mds-h[139354]: 2023-12-11T18:06:56.693+0000 7f311ff1d700 -1 mds.h *** got signal Terminated *** 2023-12-11T18:07:00.242 DEBUG:teuthology.orchestra.run.smithi115:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d0a07eaa-984b-11ee-95a3-87774f69a715@mds.h.service' 2023-12-11T18:07:00.277 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-11T18:07:00.277 INFO:tasks.cephadm.mds.h:Stopped mds.h 2023-12-11T18:07:00.277 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2023-12-11T18:07:00.277 DEBUG:teuthology.orchestra.run.smithi115:> sudo systemctl stop ceph-d0a07eaa-984b-11ee-95a3-87774f69a715@mds.k 2023-12-11T18:07:00.663 INFO:journalctl@ceph.mds.k.smithi115.stdout:Dec 11 18:07:00 smithi115 systemd[1]: Stopping Ceph mds.k for d0a07eaa-984b-11ee-95a3-87774f69a715... 2023-12-11T18:07:01.032 INFO:journalctl@ceph.mds.k.smithi115.stdout:Dec 11 18:07:00 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mds-k[138680]: 2023-12-11T18:07:00.660+0000 7f10fe747700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-11T18:07:01.032 INFO:journalctl@ceph.mds.k.smithi115.stdout:Dec 11 18:07:00 smithi115 ceph-d0a07eaa-984b-11ee-95a3-87774f69a715-mds-k[138680]: 2023-12-11T18:07:00.660+0000 7f10fe747700 -1 mds.k *** got signal Terminated *** 2023-12-11T18:07:04.227 DEBUG:teuthology.orchestra.run.smithi115:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d0a07eaa-984b-11ee-95a3-87774f69a715@mds.k.service' 2023-12-11T18:07:04.261 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-11T18:07:04.261 INFO:tasks.cephadm.mds.k:Stopped mds.k 2023-12-11T18:07:04.261 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2023-12-11T18:07:04.261 DEBUG:teuthology.orchestra.run.smithi181:> sudo systemctl stop ceph-d0a07eaa-984b-11ee-95a3-87774f69a715@mds.c 2023-12-11T18:07:08.002 DEBUG:teuthology.orchestra.run.smithi181:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d0a07eaa-984b-11ee-95a3-87774f69a715@mds.c.service' 2023-12-11T18:07:08.037 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-11T18:07:08.038 INFO:tasks.cephadm.mds.c:Stopped mds.c 2023-12-11T18:07:08.038 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2023-12-11T18:07:08.038 DEBUG:teuthology.orchestra.run.smithi181:> sudo systemctl stop ceph-d0a07eaa-984b-11ee-95a3-87774f69a715@mds.f 2023-12-11T18:07:10.860 DEBUG:teuthology.orchestra.run.smithi181:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d0a07eaa-984b-11ee-95a3-87774f69a715@mds.f.service' 2023-12-11T18:07:10.893 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-11T18:07:10.894 INFO:tasks.cephadm.mds.f:Stopped mds.f 2023-12-11T18:07:10.894 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2023-12-11T18:07:10.894 DEBUG:teuthology.orchestra.run.smithi181:> sudo systemctl stop ceph-d0a07eaa-984b-11ee-95a3-87774f69a715@mds.i 2023-12-11T18:07:15.783 DEBUG:teuthology.orchestra.run.smithi181:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d0a07eaa-984b-11ee-95a3-87774f69a715@mds.i.service' 2023-12-11T18:07:15.815 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-11T18:07:15.815 INFO:tasks.cephadm.mds.i:Stopped mds.i 2023-12-11T18:07:15.815 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2023-12-11T18:07:15.815 DEBUG:teuthology.orchestra.run.smithi181:> sudo systemctl stop ceph-d0a07eaa-984b-11ee-95a3-87774f69a715@mds.l 2023-12-11T18:07:19.014 DEBUG:teuthology.orchestra.run.smithi181:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d0a07eaa-984b-11ee-95a3-87774f69a715@mds.l.service' 2023-12-11T18:07:19.047 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-11T18:07:19.047 INFO:tasks.cephadm.mds.l:Stopped mds.l 2023-12-11T18:07:19.048 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid d0a07eaa-984b-11ee-95a3-87774f69a715 --force --keep-logs 2023-12-11T18:07:19.400 INFO:teuthology.orchestra.run.smithi059.stdout:Deleting cluster with fsid: d0a07eaa-984b-11ee-95a3-87774f69a715 2023-12-11T18:07:21.053 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid d0a07eaa-984b-11ee-95a3-87774f69a715 --force --keep-logs 2023-12-11T18:07:21.439 INFO:teuthology.orchestra.run.smithi115.stdout:Deleting cluster with fsid: d0a07eaa-984b-11ee-95a3-87774f69a715 2023-12-11T18:07:23.110 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid d0a07eaa-984b-11ee-95a3-87774f69a715 --force --keep-logs 2023-12-11T18:07:23.489 INFO:teuthology.orchestra.run.smithi181.stdout:Deleting cluster with fsid: d0a07eaa-984b-11ee-95a3-87774f69a715 2023-12-11T18:07:25.096 DEBUG:teuthology.orchestra.run.smithi059:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-12-11T18:07:25.134 DEBUG:teuthology.orchestra.run.smithi115:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-12-11T18:07:25.170 DEBUG:teuthology.orchestra.run.smithi181:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-12-11T18:07:25.199 INFO:tasks.cephadm:Archiving crash dumps... 2023-12-11T18:07:25.201 DEBUG:teuthology.misc:Transferring archived files from smithi059:/var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/crash to /home/teuthworker/archive/rishabh-2023-12-11_15:37:57-fs-rishabh-2023dec11-testing-default-smithi/7487144/remote/smithi059/crash 2023-12-11T18:07:25.202 DEBUG:teuthology.orchestra.run.smithi059:> sudo tar c -f - -C /var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/crash -- . 2023-12-11T18:07:25.234 INFO:teuthology.orchestra.run.smithi059.stderr:tar: /var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/crash: Cannot open: No such file or directory 2023-12-11T18:07:25.234 INFO:teuthology.orchestra.run.smithi059.stderr:tar: Error is not recoverable: exiting now 2023-12-11T18:07:25.236 DEBUG:teuthology.misc:Transferring archived files from smithi115:/var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/crash to /home/teuthworker/archive/rishabh-2023-12-11_15:37:57-fs-rishabh-2023dec11-testing-default-smithi/7487144/remote/smithi115/crash 2023-12-11T18:07:25.237 DEBUG:teuthology.orchestra.run.smithi115:> sudo tar c -f - -C /var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/crash -- . 2023-12-11T18:07:25.282 INFO:teuthology.orchestra.run.smithi115.stderr:tar: /var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/crash: Cannot open: No such file or directory 2023-12-11T18:07:25.282 INFO:teuthology.orchestra.run.smithi115.stderr:tar: Error is not recoverable: exiting now 2023-12-11T18:07:25.283 DEBUG:teuthology.misc:Transferring archived files from smithi181:/var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/crash to /home/teuthworker/archive/rishabh-2023-12-11_15:37:57-fs-rishabh-2023dec11-testing-default-smithi/7487144/remote/smithi181/crash 2023-12-11T18:07:25.284 DEBUG:teuthology.orchestra.run.smithi181:> sudo tar c -f - -C /var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/crash -- . 2023-12-11T18:07:25.316 INFO:teuthology.orchestra.run.smithi181.stderr:tar: /var/lib/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/crash: Cannot open: No such file or directory 2023-12-11T18:07:25.316 INFO:teuthology.orchestra.run.smithi181.stderr:tar: Error is not recoverable: exiting now 2023-12-11T18:07:25.317 INFO:tasks.cephadm:Checking cluster log for badness... 2023-12-11T18:07:25.317 DEBUG:teuthology.orchestra.run.smithi059:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2023-12-11T18:07:25.353 INFO:teuthology.orchestra.run.smithi059.stderr:grep: /var/log/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/ceph.log: No such file or directory 2023-12-11T18:07:25.355 INFO:tasks.cephadm:Compressing logs... 2023-12-11T18:07:25.355 DEBUG:teuthology.orchestra.run.smithi059:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-12-11T18:07:25.400 DEBUG:teuthology.orchestra.run.smithi115:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-12-11T18:07:25.402 DEBUG:teuthology.orchestra.run.smithi181:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-12-11T18:07:25.428 INFO:teuthology.orchestra.run.smithi059.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2023-12-11T18:07:25.428 INFO:teuthology.orchestra.run.smithi059.stderr:: No such file or directory 2023-12-11T18:07:25.428 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2023-12-11T18:07:25.428 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/ceph-osd.0.log 2023-12-11T18:07:25.431 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.log: gzip -5 --verbose -- /var/log/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/ceph-mds.g.log 2023-12-11T18:07:25.431 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/ceph-mds.a.log 2023-12-11T18:07:25.431 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/ceph-osd.6.log 2023-12-11T18:07:25.431 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/ceph-mds.g.log: /var/log/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/ceph-osd.3.log 2023-12-11T18:07:25.433 INFO:teuthology.orchestra.run.smithi181.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2023-12-11T18:07:25.433 INFO:teuthology.orchestra.run.smithi181.stderr:: No such file or directory 2023-12-11T18:07:25.433 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/ceph-osd.5.log 2023-12-11T18:07:25.433 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/ceph-osd.2.log 2023-12-11T18:07:25.433 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/ceph-osd.11.log 2023-12-11T18:07:25.433 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/ceph-osd.5.log: /var/log/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/ceph-mgr.z.log 2023-12-11T18:07:25.433 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/ceph-osd.8.log 2023-12-11T18:07:25.433 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/ceph-mds.f.log 2023-12-11T18:07:25.433 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/ceph-mds.c.log 2023-12-11T18:07:25.433 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/ceph-mds.i.log 2023-12-11T18:07:25.434 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/ceph-mon.c.log 2023-12-11T18:07:25.434 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/ceph-volume.log 2023-12-11T18:07:25.434 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/ceph-mds.l.log 2023-12-11T18:07:25.434 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/ceph-osd.6.log: /var/log/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/ceph-osd.3.log: 93.9% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2023-12-11T18:07:25.434 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/ceph-mds.d.log 2023-12-11T18:07:25.435 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/ceph-mds.d.log: 94.3% -- replaced with /var/log/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/ceph-mds.g.log.gz 2023-12-11T18:07:25.435 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/ceph-mgr.x.log 2023-12-11T18:07:25.436 INFO:teuthology.orchestra.run.smithi059.stderr: 93.3% -- replaced with /var/log/ceph/cephadm.log.gz 94.3% 2023-12-11T18:07:25.436 INFO:teuthology.orchestra.run.smithi059.stderr: -- replaced with /var/log/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/ceph-mds.a.log.gz 2023-12-11T18:07:25.438 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/ceph-mgr.x.log: 94.3%gzip -- replaced with /var/log/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/ceph-mds.d.log.gz-5 2023-12-11T18:07:25.439 INFO:teuthology.orchestra.run.smithi059.stderr:--verbose -- /var/log/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/ceph-osd.9.log 2023-12-11T18:07:25.439 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/ceph-volume.log 2023-12-11T18:07:25.439 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/ceph-mds.j.log 2023-12-11T18:07:25.444 INFO:teuthology.orchestra.run.smithi115.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2023-12-11T18:07:25.444 INFO:teuthology.orchestra.run.smithi115.stderr:: No such file or directory 2023-12-11T18:07:25.446 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/ceph-volume.log: /var/log/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/ceph-mds.j.log: 94.3% -- replaced with /var/log/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/ceph-mds.j.log.gz 2023-12-11T18:07:25.447 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/ceph-mon.a.log 2023-12-11T18:07:25.453 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/ceph-osd.10.log 2023-12-11T18:07:25.453 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/ceph-mds.h.log 2023-12-11T18:07:25.454 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/ceph-mds.e.log 2023-12-11T18:07:25.454 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/ceph-osd.4.log 2023-12-11T18:07:25.454 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/ceph-mds.h.log: /var/log/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/ceph-osd.1.log 2023-12-11T18:07:25.454 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/ceph-mds.k.log 2023-12-11T18:07:25.454 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/ceph-volume.log 2023-12-11T18:07:25.455 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/ceph-mon.a.log: 91.2% -- replaced with /var/log/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/ceph-volume.log.gz 2023-12-11T18:07:25.456 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/ceph-osd.7.log 2023-12-11T18:07:25.456 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/ceph-mon.b.log 2023-12-11T18:07:25.456 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/ceph-mds.b.log 2023-12-11T18:07:25.456 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/ceph-mgr.y.log 2023-12-11T18:07:25.462 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/ceph-mds.b.log: /var/log/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/ceph-mgr.y.log: 92.1% -- replaced with /var/log/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/ceph-mgr.y.log.gz 2023-12-11T18:07:25.505 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/ceph-volume.log: /var/log/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/ceph-mds.l.log: 94.2% -- replaced with /var/log/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/ceph-mds.l.log.gz 2023-12-11T18:07:25.515 INFO:teuthology.orchestra.run.smithi181.stderr: 91.9% -- replaced with /var/log/ceph/cephadm.log.gz 2023-12-11T18:07:25.576 INFO:teuthology.orchestra.run.smithi181.stderr: 91.4% -- replaced with /var/log/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/ceph-volume.log.gz 2023-12-11T18:07:25.683 INFO:teuthology.orchestra.run.smithi115.stderr: 91.9% -- replaced with /var/log/ceph/cephadm.log.gz 2023-12-11T18:07:25.712 INFO:teuthology.orchestra.run.smithi059.stderr: 90.2% -- replaced with /var/log/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/ceph-mgr.x.log.gz 2023-12-11T18:07:25.897 INFO:teuthology.orchestra.run.smithi115.stderr: 91.2% -- replaced with /var/log/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/ceph-volume.log.gz 2023-12-11T18:07:26.977 INFO:teuthology.orchestra.run.smithi181.stderr: 91.5% -- replaced with /var/log/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/ceph-mgr.z.log.gz 2023-12-11T18:07:27.020 INFO:teuthology.orchestra.run.smithi059.stderr: 90.4% -- replaced with /var/log/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/ceph-mon.a.log.gz 2023-12-11T18:07:27.947 INFO:teuthology.orchestra.run.smithi181.stderr: 92.2% -- replaced with /var/log/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/ceph-mon.c.log.gz 2023-12-11T18:07:28.213 INFO:teuthology.orchestra.run.smithi115.stderr: 92.0% -- replaced with /var/log/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/ceph-mon.b.log.gz 2023-12-11T18:07:55.775 INFO:teuthology.orchestra.run.smithi115.stderr: 95.1% -- replaced with /var/log/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/ceph-mds.e.log.gz 2023-12-11T18:08:06.212 INFO:teuthology.orchestra.run.smithi181.stderr: 95.4% -- replaced with /var/log/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/ceph-mds.f.log.gz 2023-12-11T18:08:08.015 INFO:teuthology.orchestra.run.smithi181.stderr: 95.3% -- replaced with /var/log/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/ceph-mds.c.log.gz 2023-12-11T18:08:10.006 INFO:teuthology.orchestra.run.smithi059.stderr: 91.3% -- replaced with /var/log/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/ceph-osd.0.log.gz 2023-12-11T18:08:13.901 INFO:teuthology.orchestra.run.smithi059.stderr: 91.3% -- replaced with /var/log/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/ceph-osd.3.log.gz 2023-12-11T18:08:19.000 INFO:teuthology.orchestra.run.smithi059.stderr: 91.2% -- replaced with /var/log/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/ceph-osd.6.log.gz 2023-12-11T18:08:25.960 INFO:teuthology.orchestra.run.smithi059.stderr: 91.2% -- replaced with /var/log/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/ceph-osd.9.log.gz 2023-12-11T18:08:25.962 INFO:teuthology.orchestra.run.smithi059.stderr: 2023-12-11T18:08:25.962 INFO:teuthology.orchestra.run.smithi059.stderr:real 1m0.547s 2023-12-11T18:08:25.962 INFO:teuthology.orchestra.run.smithi059.stderr:user 3m22.603s 2023-12-11T18:08:25.962 INFO:teuthology.orchestra.run.smithi059.stderr:sys 0m6.073s 2023-12-11T18:08:42.539 INFO:teuthology.orchestra.run.smithi181.stderr: 91.6% -- replaced with /var/log/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/ceph-osd.8.log.gz 2023-12-11T18:08:43.125 INFO:teuthology.orchestra.run.smithi115.stderr: 91.7% -- replaced with /var/log/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/ceph-osd.10.log.gz 2023-12-11T18:08:47.605 INFO:teuthology.orchestra.run.smithi181.stderr: 91.5% -- replaced with /var/log/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/ceph-osd.11.log.gz 2023-12-11T18:08:52.013 INFO:teuthology.orchestra.run.smithi181.stderr: 91.5% -- replaced with /var/log/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/ceph-osd.2.log.gz 2023-12-11T18:09:07.718 INFO:teuthology.orchestra.run.smithi115.stderr: 91.4% -- replaced with /var/log/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/ceph-osd.1.log.gz 2023-12-11T18:09:17.620 INFO:teuthology.orchestra.run.smithi181.stderr: 91.4% -- replaced with /var/log/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/ceph-osd.5.log.gz 2023-12-11T18:09:30.266 INFO:teuthology.orchestra.run.smithi115.stderr: 91.3% -- replaced with /var/log/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/ceph-osd.7.log.gz 2023-12-11T18:09:54.438 INFO:teuthology.orchestra.run.smithi115.stderr: 93.4% -- replaced with /var/log/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/ceph-mds.k.log.gz 2023-12-11T18:09:57.636 INFO:teuthology.orchestra.run.smithi115.stderr: 91.4% -- replaced with /var/log/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/ceph-osd.4.log.gz 2023-12-11T18:10:17.436 INFO:teuthology.orchestra.run.smithi181.stderr: 93.7% -- replaced with /var/log/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/ceph-mds.i.log.gz 2023-12-11T18:10:17.438 INFO:teuthology.orchestra.run.smithi181.stderr: 2023-12-11T18:10:17.438 INFO:teuthology.orchestra.run.smithi181.stderr:real 2m52.020s 2023-12-11T18:10:17.438 INFO:teuthology.orchestra.run.smithi181.stderr:user 5m18.372s 2023-12-11T18:10:17.438 INFO:teuthology.orchestra.run.smithi181.stderr:sys 0m11.818s 2023-12-11T18:10:56.539 INFO:teuthology.orchestra.run.smithi115.stderr: 93.5% -- replaced with /var/log/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/ceph-mds.h.log.gz 2023-12-11T18:11:05.685 INFO:teuthology.orchestra.run.smithi115.stderr: 93.6% -- replaced with /var/log/ceph/d0a07eaa-984b-11ee-95a3-87774f69a715/ceph-mds.b.log.gz 2023-12-11T18:11:05.705 INFO:teuthology.orchestra.run.smithi115.stderr: 2023-12-11T18:11:05.705 INFO:teuthology.orchestra.run.smithi115.stderr:real 3m40.287s 2023-12-11T18:11:05.705 INFO:teuthology.orchestra.run.smithi115.stderr:user 6m27.529s 2023-12-11T18:11:05.705 INFO:teuthology.orchestra.run.smithi115.stderr:sys 0m14.827s 2023-12-11T18:11:05.705 INFO:tasks.cephadm:Archiving logs... 2023-12-11T18:11:05.706 DEBUG:teuthology.misc:Transferring archived files from smithi059:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-12-11_15:37:57-fs-rishabh-2023dec11-testing-default-smithi/7487144/remote/smithi059/log 2023-12-11T18:11:05.707 DEBUG:teuthology.orchestra.run.smithi059:> sudo tar c -f - -C /var/log/ceph -- . 2023-12-11T18:11:40.499 DEBUG:teuthology.misc:Transferring archived files from smithi115:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-12-11_15:37:57-fs-rishabh-2023dec11-testing-default-smithi/7487144/remote/smithi115/log 2023-12-11T18:11:40.559 DEBUG:teuthology.orchestra.run.smithi115:> sudo tar c -f - -C /var/log/ceph -- . 2023-12-11T18:12:44.302 DEBUG:teuthology.misc:Transferring archived files from smithi181:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-12-11_15:37:57-fs-rishabh-2023dec11-testing-default-smithi/7487144/remote/smithi181/log 2023-12-11T18:12:44.304 DEBUG:teuthology.orchestra.run.smithi181:> sudo tar c -f - -C /var/log/ceph -- . 2023-12-11T18:13:39.080 INFO:tasks.cephadm:Removing cluster... 2023-12-11T18:13:39.081 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid d0a07eaa-984b-11ee-95a3-87774f69a715 --force 2023-12-11T18:13:39.442 INFO:teuthology.orchestra.run.smithi059.stdout:Deleting cluster with fsid: d0a07eaa-984b-11ee-95a3-87774f69a715 2023-12-11T18:13:40.268 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid d0a07eaa-984b-11ee-95a3-87774f69a715 --force 2023-12-11T18:13:41.015 INFO:teuthology.orchestra.run.smithi115.stdout:Deleting cluster with fsid: d0a07eaa-984b-11ee-95a3-87774f69a715 2023-12-11T18:13:41.977 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid d0a07eaa-984b-11ee-95a3-87774f69a715 --force 2023-12-11T18:13:42.327 INFO:teuthology.orchestra.run.smithi181.stdout:Deleting cluster with fsid: d0a07eaa-984b-11ee-95a3-87774f69a715 2023-12-11T18:13:43.207 INFO:tasks.cephadm:Removing cephadm ... 2023-12-11T18:13:43.207 DEBUG:teuthology.orchestra.run.smithi059:> rm -rf /home/ubuntu/cephtest/cephadm 2023-12-11T18:13:43.229 DEBUG:teuthology.orchestra.run.smithi115:> rm -rf /home/ubuntu/cephtest/cephadm 2023-12-11T18:13:43.543 DEBUG:teuthology.orchestra.run.smithi181:> rm -rf /home/ubuntu/cephtest/cephadm 2023-12-11T18:13:43.560 INFO:tasks.cephadm:Teardown complete 2023-12-11T18:13:43.561 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-12-11T18:13:43.571 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-12-11T18:13:43.571 DEBUG:teuthology.orchestra.run.smithi059:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-12-11T18:13:43.574 DEBUG:teuthology.orchestra.run.smithi115:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-12-11T18:13:43.576 DEBUG:teuthology.orchestra.run.smithi181:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-12-11T18:13:43.712 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-12-11T18:13:43.722 INFO:teuthology.task.clock:Checking final clock skew... 2023-12-11T18:13:43.722 DEBUG:teuthology.orchestra.run.smithi059:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-12-11T18:13:43.725 DEBUG:teuthology.orchestra.run.smithi115:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-12-11T18:13:43.727 DEBUG:teuthology.orchestra.run.smithi181:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-12-11T18:13:43.742 INFO:teuthology.orchestra.run.smithi059.stderr:bash: ntpq: command not found 2023-12-11T18:13:43.743 INFO:teuthology.orchestra.run.smithi181.stderr:bash: ntpq: command not found 2023-12-11T18:13:43.743 INFO:teuthology.orchestra.run.smithi115.stderr:bash: ntpq: command not found 2023-12-11T18:13:43.747 INFO:teuthology.orchestra.run.smithi181.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-12-11T18:13:43.747 INFO:teuthology.orchestra.run.smithi181.stdout:=============================================================================== 2023-12-11T18:13:43.747 INFO:teuthology.orchestra.run.smithi181.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 7 +1362us[+1362us] +/- 91ms 2023-12-11T18:13:43.747 INFO:teuthology.orchestra.run.smithi181.stdout:^* hv02.front.sepia.ceph.com 3 6 377 8 -609us[ -589us] +/- 61ms 2023-12-11T18:13:43.747 INFO:teuthology.orchestra.run.smithi181.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 8 -285us[ -285us] +/- 71ms 2023-12-11T18:13:43.747 INFO:teuthology.orchestra.run.smithi181.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-12-11T18:13:43.748 INFO:teuthology.orchestra.run.smithi059.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-12-11T18:13:43.748 INFO:teuthology.orchestra.run.smithi059.stdout:=============================================================================== 2023-12-11T18:13:43.749 INFO:teuthology.orchestra.run.smithi059.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 18 +1357us[+1357us] +/- 91ms 2023-12-11T18:13:43.749 INFO:teuthology.orchestra.run.smithi059.stdout:^* hv02.front.sepia.ceph.com 3 6 377 20 -609us[ -576us] +/- 61ms 2023-12-11T18:13:43.749 INFO:teuthology.orchestra.run.smithi059.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 21 -282us[ -249us] +/- 71ms 2023-12-11T18:13:43.749 INFO:teuthology.orchestra.run.smithi059.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-12-11T18:13:43.758 INFO:teuthology.orchestra.run.smithi115.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-12-11T18:13:43.758 INFO:teuthology.orchestra.run.smithi115.stdout:=============================================================================== 2023-12-11T18:13:43.758 INFO:teuthology.orchestra.run.smithi115.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 24 +1304us[+1304us] +/- 91ms 2023-12-11T18:13:43.758 INFO:teuthology.orchestra.run.smithi115.stdout:^* hv02.front.sepia.ceph.com 3 6 377 24 -625us[ -628us] +/- 61ms 2023-12-11T18:13:43.758 INFO:teuthology.orchestra.run.smithi115.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 24 -322us[ -322us] +/- 71ms 2023-12-11T18:13:43.758 INFO:teuthology.orchestra.run.smithi115.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-12-11T18:13:43.759 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-12-11T18:13:43.772 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-12-11T18:13:43.772 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-12-11T18:13:43.784 DEBUG:teuthology.orchestra.run.smithi059:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-12-11T18:13:43.901 DEBUG:teuthology.orchestra.run.smithi115:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-12-11T18:13:44.112 DEBUG:teuthology.orchestra.run.smithi181:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-12-11T18:13:44.283 DEBUG:teuthology.orchestra.run.smithi059:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-12-11T18:13:44.319 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-11T18:13:44.320 DEBUG:teuthology.orchestra.run.smithi115:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-12-11T18:13:44.354 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-11T18:13:44.355 DEBUG:teuthology.orchestra.run.smithi181:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-12-11T18:13:44.386 INFO:teuthology.orchestra.run.smithi181.stdout:type=AVC msg=audit(1702317480.627:45): avc: denied { node_bind } for pid=2918 comm="ping" saddr=172.21.15.181 scontext=system_u:system_r:ping_t:s0 tcontext=system_u:object_r:node_t:s0 tclass=icmp_socket permissive=1 2023-12-11T18:13:44.387 DEBUG:teuthology.task.selinux:ubuntu@smithi181.front.sepia.ceph.com has 1 denials 2023-12-11T18:13:44.387 ERROR:teuthology.run_tasks:Manager failed: selinux Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/task/__init__.py", line 136, in __exit__ self.teardown() File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/task/selinux.py", line 166, in teardown self.get_new_denials() File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/task/selinux.py", line 215, in get_new_denials raise SELinuxError(node=remote, teuthology.exceptions.SELinuxError: SELinux denials found on ubuntu@smithi181.front.sepia.ceph.com: ['type=AVC msg=audit(1702317480.627:45): avc: denied { node_bind } for pid=2918 comm="ping" saddr=172.21.15.181 scontext=system_u:system_r:ping_t:s0 tcontext=system_u:object_r:node_t:s0 tclass=icmp_socket permissive=1'] 2023-12-11T18:13:44.387 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-12-11T18:13:44.402 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-12-11T18:13:44.417 INFO:teuthology.task.internal:Duration was 2996.989902 seconds 2023-12-11T18:13:44.418 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-12-11T18:13:44.433 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-12-11T18:13:44.434 DEBUG:teuthology.orchestra.run.smithi059:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-12-11T18:13:44.436 DEBUG:teuthology.orchestra.run.smithi115:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-12-11T18:13:44.439 DEBUG:teuthology.orchestra.run.smithi181:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-12-11T18:13:44.487 INFO:teuthology.orchestra.run.smithi059.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-12-11T18:13:44.521 INFO:teuthology.orchestra.run.smithi115.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-12-11T18:13:44.532 INFO:teuthology.orchestra.run.smithi181.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-12-11T18:13:44.810 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-12-11T18:13:44.810 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi059.front.sepia.ceph.com 2023-12-11T18:13:44.810 DEBUG:teuthology.orchestra.run.smithi059:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-12-11T18:13:44.836 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi115.front.sepia.ceph.com 2023-12-11T18:13:44.837 DEBUG:teuthology.orchestra.run.smithi115:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-12-11T18:13:44.868 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi181.front.sepia.ceph.com 2023-12-11T18:13:44.868 DEBUG:teuthology.orchestra.run.smithi181:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-12-11T18:13:44.890 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-12-11T18:13:44.890 DEBUG:teuthology.orchestra.run.smithi059:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-12-11T18:13:44.892 DEBUG:teuthology.orchestra.run.smithi115:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-12-11T18:13:44.910 DEBUG:teuthology.orchestra.run.smithi181:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-12-11T18:13:44.993 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-12-11T18:13:44.993 DEBUG:teuthology.orchestra.run.smithi059:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-12-11T18:13:44.996 DEBUG:teuthology.orchestra.run.smithi115:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-12-11T18:13:44.998 DEBUG:teuthology.orchestra.run.smithi181:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-12-11T18:13:45.112 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-12-11T18:13:45.124 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-12-11T18:13:45.124 DEBUG:teuthology.orchestra.run.smithi059:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-12-11T18:13:45.152 DEBUG:teuthology.orchestra.run.smithi115:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-12-11T18:13:45.191 DEBUG:teuthology.orchestra.run.smithi181:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-12-11T18:13:45.218 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-12-11T18:13:45.230 DEBUG:teuthology.orchestra.run.smithi059:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-12-11T18:13:45.233 DEBUG:teuthology.orchestra.run.smithi115:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-12-11T18:13:45.235 DEBUG:teuthology.orchestra.run.smithi181:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-12-11T18:13:45.258 INFO:teuthology.orchestra.run.smithi059.stdout:kernel.core_pattern = core 2023-12-11T18:13:45.266 INFO:teuthology.orchestra.run.smithi115.stdout:kernel.core_pattern = core 2023-12-11T18:13:45.291 INFO:teuthology.orchestra.run.smithi181.stdout:kernel.core_pattern = core 2023-12-11T18:13:45.317 DEBUG:teuthology.orchestra.run.smithi059:> test -e /home/ubuntu/cephtest/archive/coredump 2023-12-11T18:13:45.333 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-11T18:13:45.333 DEBUG:teuthology.orchestra.run.smithi115:> test -e /home/ubuntu/cephtest/archive/coredump 2023-12-11T18:13:45.349 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-11T18:13:45.349 DEBUG:teuthology.orchestra.run.smithi181:> test -e /home/ubuntu/cephtest/archive/coredump 2023-12-11T18:13:45.373 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-11T18:13:45.374 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-12-11T18:13:45.388 INFO:teuthology.task.internal:Transferring archived files... 2023-12-11T18:13:45.388 DEBUG:teuthology.misc:Transferring archived files from smithi059:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-12-11_15:37:57-fs-rishabh-2023dec11-testing-default-smithi/7487144/remote/smithi059 2023-12-11T18:13:45.389 DEBUG:teuthology.orchestra.run.smithi059:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-12-11T18:13:45.432 DEBUG:teuthology.misc:Transferring archived files from smithi115:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-12-11_15:37:57-fs-rishabh-2023dec11-testing-default-smithi/7487144/remote/smithi115 2023-12-11T18:13:45.432 DEBUG:teuthology.orchestra.run.smithi115:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-12-11T18:13:45.472 DEBUG:teuthology.misc:Transferring archived files from smithi181:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-12-11_15:37:57-fs-rishabh-2023dec11-testing-default-smithi/7487144/remote/smithi181 2023-12-11T18:13:45.472 DEBUG:teuthology.orchestra.run.smithi181:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-12-11T18:13:45.511 INFO:teuthology.task.internal:Removing archive directory... 2023-12-11T18:13:45.511 DEBUG:teuthology.orchestra.run.smithi059:> rm -rf -- /home/ubuntu/cephtest/archive 2023-12-11T18:13:45.513 DEBUG:teuthology.orchestra.run.smithi115:> rm -rf -- /home/ubuntu/cephtest/archive 2023-12-11T18:13:45.515 DEBUG:teuthology.orchestra.run.smithi181:> rm -rf -- /home/ubuntu/cephtest/archive 2023-12-11T18:13:45.561 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-12-11T18:13:45.577 INFO:teuthology.task.internal:Not uploading archives. 2023-12-11T18:13:45.577 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-12-11T18:13:45.593 INFO:teuthology.task.internal:Tidying up after the test... 2023-12-11T18:13:45.594 DEBUG:teuthology.orchestra.run.smithi059:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-12-11T18:13:45.596 DEBUG:teuthology.orchestra.run.smithi115:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-12-11T18:13:45.598 DEBUG:teuthology.orchestra.run.smithi181:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-12-11T18:13:45.610 INFO:teuthology.orchestra.run.smithi059.stdout: 262307 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Dec 11 18:13 /home/ubuntu/cephtest 2023-12-11T18:13:45.612 INFO:teuthology.orchestra.run.smithi115.stdout: 262181 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Dec 11 18:13 /home/ubuntu/cephtest 2023-12-11T18:13:45.617 INFO:teuthology.orchestra.run.smithi181.stdout: 262181 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Dec 11 18:13 /home/ubuntu/cephtest 2023-12-11T18:13:45.618 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-12-11T18:13:45.632 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-12-11T18:13:45.647 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: SELinuxError: SELinux denials found on ubuntu@smithi181.front.sepia.ceph.com: ['type=AVC msg=audit(1702317480.627:45): avc: denied { node_bind } for pid=2918 comm="ping" saddr=172.21.15.181 scontext=system_u:system_r:ping_t:s0 tcontext=system_u:object_r:node_t:s0 tclass=icmp_socket permissive=1'] 2023-12-11T18:13:45.647 INFO:teuthology.nuke:Checking targets against current locks 2023-12-11T18:13:45.673 DEBUG:teuthology.nuke:shortname: smithi059 2023-12-11T18:13:45.673 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-12-11T18:13:45.697 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi059.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-12-11_15:37:57-fs-rishabh-2023dec11-testing-default-smithi/7487144', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-11 17:18:22.640935', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-12-11T18:13:45.701 DEBUG:teuthology.nuke:shortname: smithi115 2023-12-11T18:13:45.701 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-12-11T18:13:45.725 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi115.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-12-11_15:37:57-fs-rishabh-2023dec11-testing-default-smithi/7487144', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-11 17:18:22.643994', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-12-11T18:13:45.730 DEBUG:teuthology.nuke:shortname: smithi181 2023-12-11T18:13:45.731 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-12-11T18:13:45.749 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi181.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-12-11_15:37:57-fs-rishabh-2023dec11-testing-default-smithi/7487144', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-11 17:18:22.642543', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-12-11T18:13:45.757 INFO:teuthology.orchestra.console.smithi059:Power off 2023-12-11T18:13:45.757 DEBUG:teuthology.orchestra.console.smithi059:pexpect command: ipmitool -H smithi059.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-12-11T18:13:45.773 INFO:teuthology.orchestra.console.smithi115:Power off 2023-12-11T18:13:45.773 DEBUG:teuthology.orchestra.console.smithi115:pexpect command: ipmitool -H smithi115.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-12-11T18:13:45.786 DEBUG:teuthology.orchestra.console.smithi059:power off output: Chassis Power Control: Down/Off 2023-12-11T18:13:45.786 DEBUG:teuthology.orchestra.console.smithi059:pexpect command: ipmitool -H smithi059.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-11T18:13:45.796 DEBUG:teuthology.orchestra.console.smithi115:power off output: Chassis Power Control: Down/Off 2023-12-11T18:13:45.796 DEBUG:teuthology.orchestra.console.smithi115:pexpect command: ipmitool -H smithi115.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-11T18:13:45.808 DEBUG:teuthology.orchestra.console.smithi059:check power output: Chassis Power is on 2023-12-11T18:13:45.820 INFO:teuthology.orchestra.console.smithi181:Power off 2023-12-11T18:13:45.820 DEBUG:teuthology.orchestra.console.smithi181:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-12-11T18:13:45.830 DEBUG:teuthology.orchestra.console.smithi115:check power output: Chassis Power is on 2023-12-11T18:13:45.843 DEBUG:teuthology.orchestra.console.smithi181:power off output: Chassis Power Control: Down/Off 2023-12-11T18:13:45.843 DEBUG:teuthology.orchestra.console.smithi181:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-11T18:13:45.865 DEBUG:teuthology.orchestra.console.smithi181:check power output: Chassis Power is on 2023-12-11T18:13:49.809 DEBUG:teuthology.orchestra.console.smithi059:pexpect command: ipmitool -H smithi059.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-11T18:13:49.830 DEBUG:teuthology.orchestra.console.smithi115:pexpect command: ipmitool -H smithi115.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-11T18:13:49.870 DEBUG:teuthology.orchestra.console.smithi181:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-11T18:13:49.921 DEBUG:teuthology.orchestra.console.smithi059:check power output: Chassis Power is on 2023-12-11T18:13:49.941 DEBUG:teuthology.orchestra.console.smithi115:check power output: Chassis Power is on 2023-12-11T18:13:49.981 DEBUG:teuthology.orchestra.console.smithi181:check power output: Chassis Power is on 2023-12-11T18:13:53.926 DEBUG:teuthology.orchestra.console.smithi059:pexpect command: ipmitool -H smithi059.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-11T18:13:53.942 DEBUG:teuthology.orchestra.console.smithi115:pexpect command: ipmitool -H smithi115.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-11T18:13:53.982 DEBUG:teuthology.orchestra.console.smithi181:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-11T18:13:54.037 DEBUG:teuthology.orchestra.console.smithi059:check power output: Chassis Power is on 2023-12-11T18:13:54.050 DEBUG:teuthology.orchestra.console.smithi115:check power output: Chassis Power is on 2023-12-11T18:13:54.092 DEBUG:teuthology.orchestra.console.smithi181:check power output: Chassis Power is on 2023-12-11T18:13:58.043 DEBUG:teuthology.orchestra.console.smithi059:pexpect command: ipmitool -H smithi059.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-11T18:13:58.051 DEBUG:teuthology.orchestra.console.smithi115:pexpect command: ipmitool -H smithi115.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-11T18:13:58.093 DEBUG:teuthology.orchestra.console.smithi181:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-11T18:13:58.153 DEBUG:teuthology.orchestra.console.smithi059:check power output: Chassis Power is off 2023-12-11T18:13:58.165 DEBUG:teuthology.orchestra.console.smithi115:check power output: Chassis Power is off 2023-12-11T18:13:58.204 DEBUG:teuthology.orchestra.console.smithi181:check power output: Chassis Power is off 2023-12-11T18:13:58.254 INFO:teuthology.orchestra.console.smithi059:Power off completed 2023-12-11T18:13:58.266 INFO:teuthology.orchestra.console.smithi115:Power off completed 2023-12-11T18:13:58.304 INFO:teuthology.orchestra.console.smithi181:Power off completed 2023-12-11T18:13:58.409 INFO:teuthology.run:Summary data: description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/no 5-workunit/suites/blogbench}} duration: 2996.9899022579193 failure_reason: '[Errno 104] Connection reset by peer' flavor: default owner: scheduled_rishabh@teuthology status: fail success: false 2023-12-11T18:13:58.410 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-11T18:13:58.490 INFO:teuthology.run:FAIL